SE Labs

Special Edition
Computer security testing comment and analysis from SE LABS Ⓡ

Cyber Security DE:CODED – Full attack chain testing

“Because we test realistically, sometimes bad guys come onto our test network and mess with us”

SUBSCRIBE! Use one of the ‘Listen on’ links below to keep updated using your favourite podcast platform.

Listen on Apple Podcasts Listen on Spotify

Series 1 | Series 2 | Series 3 (in production)

Other ways to listen: YouTube | Google Podcasts | Stitcher | RSS

Show notes for series 2, episode 9 (final episode of series 2)

What is the attack chain? Why is it good to test using full attack chains? And what are some of the alternative approaches, with their pros and cons? We’ll try to answer all of these questions and more in this special presentation episode recorded at the AVAR conference in Singapore in December 2022.

Full attack chain testing

Hear our CEO talk about realistic and useful security testing at the premium Asian security conference, AVAR. This special presentation episode is the last in the series! If you enjoy it please check out the previous episodes and sign up to our newsletter to receive notification about when the next series is scheduled.

Please subscribe and join the discussions. Use one of the ‘Listen On’ links above to subscribe using your favourite podcast platform.

This is the final episode of series 2. We’ll be back! In the meantime, email us with suggestions about what you like, dislike and want more of in series 3!


Sign up to our newsletter!

Other resources


(Generated automatically)

Simon Edwards 0:02
Welcome to DE:CODED, providing in depth insight into cybersecurity. What is the attack chain? Why is it good to test using full attack chains? And what are some of the alternative approaches with the pros and cons? We’ll try to answer all of these questions and more in this special presentation episode recorded at the AVAR conference in Singapore in December 2022. Show notes, including any links mentioned in the show, are available at

Okay, good afternoon, everybody. Thank you for being in the room to listen to my fascinating talk about targeted attacks and testing with targeted attacks. Now we’ve heard the gentleman talking about ransomware. In my opinion, ransomware threat is very important. But ransomware is a payload. So we talked again, about hygiene, cyber hygiene, attackers have to go all the way through the systems in order to deploy the ransomware. So if you patch your systems, if you do all this other good things, MFA, that kind of thing, it does, in my opinion, massively reduced the chances of a ransomware attack. But let’s not get ahead of ourselves. But my name is Simon Edwards, I’m the CEO of SE Labs, you’ll see our logo on your wonderful, great T shirts and your plaques. And we’re a security testing organisation. And we test lots of different kinds of technologies, you probably if you know about us, you’ll know that we test antivirus kind of stuff. But we also do massive firewalls, you know, 100 200,000, pound firewalls, network intrusion systems, that kind of thing. And if you’ve got artificial intelligence built in even better, because we’d really love to know if that actually works or not.

Now, we test for a number of reasons. One of the reasons is the cybersecurity industry is worth many, many billions of dollars, people are buying this stuff, by the pile, it’s extremely expensive. And the consequence of not buying it is getting hacked, having ransomware, that kind of thing. So as a result, there’s a big competition for sell security products to us all and to the rest of the world. And some of the marketing claims that we see are, quite honestly not believable. And some of them are fairly reasonable. So we try and cut through the unsubstantiated claims. And we say, Well, does this firewall really stop the threats? Is it really that fast? Can it be faster and stop the threats? Because that is an issue as well? Do these endpoint protect protection systems? Detect, do they protect, they do both? Let’s let’s find out. And, and maybe they do detect some kinds of threats, maybe the ones that we get in our junk email boxes every day, but maybe they’re not so good with targeted attacks. And as we’re seeing targeted attacks are very serious and very likely to end up with a ransomware situation. And so we’re trying to help the big companies that are spending millions of dollars on security, make educated decisions about which things they’re choosing to buy. But we’re also on the side of the security vendors as well. Strangely, we’re not just out there saying this is good, this is bad. If we find a product doesn’t do what it says. We will talk to the companies that make it and we will explain in detail what’s gone wrong, because we don’t want to say well done, you get a badge, bad luck, you’re rubbish. We want to be able to say look, you’ve got a problem. Here’s how you can fix it. And at the end of it, people need to buy the good products and fixed products that aren’t so good and then the world’s slightly more secure. Now, I’ve always been told that to explain the joke makes it funnier in this situation where Got some crash test dummies, and they’ve been told not to fasten their seat belts, seat belts. And the reason for that is you want to see what happens when something goes wrong. And things aren’t patched in the software context. So when we test, we don’t necessarily patch everything properly, we don’t necessarily use the very latest version of Windows. Because we’re trying to reflect reality, what really happens in the real world, people probably aren’t using the very latest Intel processors, for example, they possibly are using default settings, maybe they shouldn’t. So we try and look at sub optimal conditions as well as optimal conditions. And as we go through the rest of this talk, and we start talking about using targeted attacks, this is quite important. Because if you have Windows seven, fully patched on the latest Intel systems, you actually are quite well secured against many of the attacks that we’re seeing in the real world. And I’m sure as the antivirus world you’ll be seeing as well. So I’m pretty sure most of you know what the attack chain is. But I’m going to really dig into that. Later on in this talk, we’re going to say, do you test at the beginning of an attack? At the end? If there is such a thing is NAND, or somewhere in the middle? What do you do the whole thing. And we’re also going to talk about atomic testing, nothing to do with nuclear technology. It’s more about getting into the detail of things. And finally, we’re going to look into what you actually want to know. So if I’m buying a firewall or an endpoint products or something like that, am I trying to stop all threats? Am I trying to plug a gap in my security stack, maybe I’m really happy that I’ve got a signature based AV. And I just want to make sure that my users can’t go to certain websites, for example, or maybe I just want to comply with some rules, some regulation that says I have to have a V on my network. So anything we’ll do, people have different reasons for, for deploying security. And it may be that I’m willing to choose something that isn’t on paper quite as good because it’s compatible with other products I have, or my team is already trained in certain types of products. So just plucking a name out of the air Fortinet, for example. They’ve got products across the range. And if my team is really well trained on Fortinet firewalls, maybe that will make me more inclined to go to a fortunate email gateway or endpoint or something like that. And, you know, touched on this earlier, why do you want to know these answers? If you’re an AV company, you might your marketing department will want to know if you get a badge. But you guys probably want to know, why didn’t we detect that? How can we detect it, or maybe it doesn’t matter that we didn’t protect it, because we were blocking it anyway. Now the reason that we test the way that we do, which is to say not automated, we’ve got a team of hackers, good hackers, I hasten to add, although we might be Richard, if we weren’t good hackers in the UK who constantly test these products, we do it in a way that you can repeat it. And the reason that’s important is why tell you that your product doesn’t work properly, I need to be able to prove that to you. And you want to be able to play with it in the lab as well and try things and try and fix it and and repeat it. So I can prove that I’m not lying about my results. But I can also help you improve the product as well. And then there is the marketing element where if I produce a report and say nortonlifelock is great, or AVG is terrible, or whatever it happens to be. People have to understand why that’s important. And if I say I tested with some malware, that doesn’t really mean anything to anyone else in the real world. But if I say I did this targeted attack, that was the same as was used against the Democratic Party in the US or against a nuclear facility in Ukraine or whatever, people can understand that it’s a lot easier to get your head around. But this isn’t a penetration test. So penetration test is where I’d come to your company and say, let me break into your systems. What we’re doing is we’re breaking into our own systems, and using these technologies to try and detect what we’re doing maybe to block us.

Now, Tomic testing, I’m sure a lot of you know about the myth of the elephant and the blind man or the blindfolded people. If you only look at one part of a thing, you don’t really understand it. So, you know, you might feel the trunk and think, Well, I’ve got a snake here, or I might feel it makes you go, this feels like a treat to me. You’ve got to open your eyes and say, What is this thing I’m looking at? And so to put it into a security context, you could say, well, this product I’ve got here, it says it’s got anti exploits on it. Is that all it does? Well, it’s got AI, great AI for what? And you know, maybe it’s just an anti virus product with lots of cool marketing terms attached to it. So taking a broader look at test and testing it fully from beginning to end. That can be very useful. So the attack chain in the context of this talk simply means the full story of an attack from the very beginning to the end. And as the gentleman here alluded to earlier, actually, not a lot has changed in the last 22 years, if you were to read a copy of hacking exposed, Second Edition from about the year 2000, don’t say, you’re gonna see the hackers doing the same thing today, as they were, then they’re doing reconnaissance, there may be running some exploits that may be fooling people with social engineering. And then they’re achieving persistence. They’re stealing data, they’re causing damage, it’s largely the same picture. Ransomware is just a payload at the end of this attack chain. Now, I’m sure you’re familiar with mitre, for those who aren’t, mitre take certain attack groups, and they explain what happens generally, the they list the tactics and the other ways that things work. So there’s a website that MITRE has called be the Navigator. And this is what an attack chain looks like, generally. So it starts off with some reconnaissance at the beginning, on the far left, and it works its way across to the right, where they get further access, steal stuff, break stuff, and we can zoom in. And you can see a bit closer there. This attack starts with a spear phishing link. So not a fight, there’s a link to another website, you go there, there’s an exploit some PowerShell gets involved, they steal some credentials, and they get into the registry and achieve persistence. Now the bad guys can do this, but so can the good guys, we can test any of us in this room probably could test like this, just copy a PT three. And you’ll know what would happen if the same guys did the same thing again in the future. And like, obviously, you can do variations because they’re not going to be using exactly the same executable file, don’t use the stuff you get from VirusTotal, do something similar. And you’ll get a similar result, and you can almost predict the future. So here’s the general approach to how we do the testing, you’ve got to deliver the threat in some way, if I just dropped the threat onto the desktop, I have bypassed those products, different methods of detection, I’ve just gone straight past any email filtering, and nothing else like that. It’s not really fair, unless I’m only interested in one pacifist. So you’ve got the delivery, you’ve then got the execution of the threat. It could be PowerShell. It could be an xe could be an HTS file, whatever. And then something happens in most cases, as we seen in one of the other talks, I think was the checkpoint one, they start looking at what’s going on on the network, what system Am I on? What systems can I connect to, if you’re a tester and you don’t do that you’re not being very realistic, you’ve got to generate some realistic noise, give the products a chance to detect what would actually happen. To do a lot of the cool stuff like keylogging, and things like that you need further privileges, if you’re going to steal credentials as well, you kind of need to be like the systems administrator, you can’t just be a user. So escalating privileges is another common stage in an attack that testers can do. It also increases chances of being detected. But that’s, that’s fine. That’s what we’re looking for. And then after that, you can do your credential, stealing and keylogging, that kind of business. And then again, as was mentioned earlier, by the chairman of the panel, you’ve got SMB shares lateral movement, if you can block those in a real world great. But as a test that you probably don’t want to because you want to try and move through. And then if CrowdStrike or other products go hang on a minute, I’ve seen that I want to stop it, you’re giving them a chance to demonstrate their features. And finally, lateral movements. If I break into someone’s desktop, I’m probably not going to ransomware their desktop, I’m going to get further into the network and deploy the ransomware throughout. Again, a tester might choose not to go that far. But you’re you’re limiting your conclusions to the report if you don’t do the full attack chain. This is how we present it in our reports. So much like you have the spearfishing and the PowerShell in the registry. We have spear phishing PowerShell registry just done in a slightly more friendly way for the grannies to read our reports. Now this is how we could do a PT three otherwise known as Gothic Panda, spear phishing link, Windows PowerShell, sorry, Windows command shell. And then we look for files, we look for processes. We look for other parts of the systems information. Then we escalate privileges. We want to steal some credentials, so it’s necessary. And then we start keylogging. We achieve persistence as testers we don’t need to be persistent, but it’s the right thing to do. And then finally we move through some admin shares and by doing that we’ve copied what Gothic panda a PT three did. And we’re not using the same malware we’re doing the same activities. So if Have a product or set of products because you can combine. If that detects most of that some of that whatever, you can feel fairly confident that if similar attackers come to your network, you’re going to detect them. It’s not fair to test all products thoroughly like that, though. So imagine you’ve got a firewall, the firewall can detect the initial entry of the attack. And it can maybe can detect some of the exfiltration people connecting back to a see to stealing information. But when you escalate privileges, the firewalls got no way of seeing that. So you shouldn’t, if you’re a tester, condemn the firewall fade into the tech things happening just on endpoints, for example. And finally, you can detect that you can test things in protection or detection mode. If you’re familiar with mitre, they genuinely do things in detection mode. And that might be what you want, you want to know, well, this endpoint acts as like a CCTV system on my network. I don’t want it to stop stuff, I just wanted to tell me, or you might want it to block things. But if you’re a tester and you test in detection mode, what you’re doing is you’re allowing the product to demonstrate your abilities all the way down the attack chain. Whereas if you test and protection mode, it might stop right at the beginning of the attack. And then that’s the end of the test. And you’ll never know if it could have detected that the things. False positive testing is super boring, but super necessary. I could Witter on for an hour about that I’m not going to. But if you want to know more, please come and find me. And these are the kinds of reports if you do you’ve got advanced firewalls, you’ve got breach detection systems, you’ve got, yeah, all sorts of things. I’m going to hurry up slightly. But so I’m going to finish with a two minute story. Because we test realistically, sometimes bad guys come onto our test network can mess with us. So I’m going to explain what happens in an attack I’m going to call grumpy sloth. So in this case, it was a couple of years ago, now we’re running an endpoint test. And we visited a bad URL with a regular PC, we don’t need VMs, we try and keep everything as real as possible. And some guy exploited our TeamViewer with a DLL side load, very cool, whatever. And we waited for him to start doing bad stuff. And normally this stuff is automated. So we see Process Monitor lights up, and it’s all very exciting. But in this case, actually nothing happened. And six hours later, a real person connected to our system, uploaded some tools and tried to protect them, but not well enough for us. We uploaded his tools to virus total, they weren’t detected by anything. So that was cool. That’s cool for the for the whole community. But unfortunately for us, he noticed because it’s a real person who could see that we’re monitoring him. And it was a him, I’m pretty sure. And then he destroyed all of his evidence, but we’d already saved it. So that’s funny. And he then created a very rude message for us in the directory telling us to do something to ourselves. And that’s why I’m assuming he’s a male. And then he disconnected and went away. So we then put all the IOCs on our website, and you can download those. And that’s it. So my point is test realistically be prepared for really horrible people to jump on your network. And I probably don’t have time for questions, maybe 30 seconds. Well, I’m around for the next day and a half. So if you want to come and talk to me about testing, and hacking, please do thank you very much.

Now, just before we finish it security life hack time. At the end of each episode, we give a special security tip that works for real people in the real world, for work and in personal lives. And this episode’s life hacker is well me. I’m Simon Edwards, founder and CEO of SE Labs, co chair of the anti malware testing standards organisation, and writer of motorbikes, player of electric guitars and possibly not someone you should play cards with. The security life hack I’d like to pass on is to use at least two bank accounts. First, use an account with one of the newer so called challenger banks for your day to day business, you know things like grocery shopping, grabbing a coffee and paying for parking. They make life easy when it comes to managing your money. They bring banking into the 21st century with their apps and other things. And if you still queuing at a branch to pay in cheques now, it’s time to start catching up because both of those things checks and branches will they’re becoming increasingly rare. And you can often deposit checks from home using an app.

But some people are worried about losing money to hackers and scammers and this high tech approach these new accounts, they can be intimidating. So my tip is to use this newer bank account to spend money on a day to day basis. Get used to the contactless payment

Please subscribe, and if you enjoyed this episode, please send a link to just one of your close colleagues. We also have a free email newsletter. Sign up on our website, where you’ll also find this episodes, show notes, and bonus episodes featuring full length interviews with our guests. Just visit And that’s it. Thank you for listening. This is the last episode of the series. So make sure you do sign up for the newsletter and you’ll find out when we’re back. Hope to see you again soon.


Please send your comments, questions and concerns to


SE Labs Ltd is a private, independently-owned and run testing company that assesses security products and services. The main laboratory is located in Wimbledon, South London. It has excellent local and international travel connections. The lab is open for prearranged client visits.


SE Labs Ltd
Hill Place House
55A High Street
SW19 5BA