Special Edition is the blog for security testing business SE Labs. It explains how we test security products, reports on the internet threats we find and provides security tips for businesses, other organisations and home users.

Monday, 5 February 2018

Hacked! Will your anti-malware protect you from targeted attacks?

The news isn't good. Discover your best options in our latest reports.

Latest reports now online.

Criminals routinely create ingenious scams and indiscriminate attacks designed to compromise the unlucky and, occasionally, foolish. But sometimes they focus on a specific target rather than casting a net wide in the hope of landing something interesting.

Targeted attacks can range from basic, like an email simply asking you to send some money to an account, through to extremely devious and technical. If you received an email from your accountant with an attached PDF or Excel spreadsheet would you open it?

Most would and all that then stands between them and a successful hack (because the email was a trick and contained a dodgy document that gives remote control to the attacker) is the security software running on their PC.

In this test we've included indiscriminate, public attacks that come at victims from the web and via email, but we've also included some devious targeted attacks to see how well-protected potential victims would be.

We've not created any new types of threat and we've not discovered and used 'zero day' attacks. Instead we took tools that are freely distributed online and are well-known to penetration testers and criminals alike. We used these to generate threats that are realistic representations of what someone could quite easily put together to attack you or your business.

The results are extremely worrying. While a few products were excellent at detecting and protecting against these threats many more were less useful. We will continue this work and report any progress that these companies make in improving their products.

Our latest reports, for enterprise, small business and home users are now available for free from our website. Please download them and follow us on Twitter and/or Facebook to receive updates and future reports.

Tuesday, 12 December 2017

Network appliances vs. targeted attacks

There have been so many publicised data breaches in 2017 that we didn't even have enough space in our latest report to provide a basic summary. In many cases a business network was breached. Business networks comprise endpoints (usually Windows PCs), servers, Point of Sale computers and a range of other devices.

Latest reports now online.

In this report (PDF) we explore the effectiveness of network appliances designed to detect and block attacks against endpoint systems.

One approach to compromising a business is to hack an endpoint (PC) and then to use it as a platform from which to launch further attacks into the network. For example, rather than going straight for a company's main servers why not trick a user into infecting his/ her computer with malware? We can then scan and infect the entire network, stealing information, causing damage and generally behaving in ways contrary to the business' best interests.

There is some really good endpoint software available, as we see in our regular Endpoint Protection tests, but nothing is perfect and any extra layers of security are welcome. If one layer fails, others exist to mitigate the threat. In this report we explore the effectiveness of network appliances designed to detect and protect against attacks against endpoint systems.

The systems we have tested here are popular appliances designed to sit between your endpoints and the internet router. They are designed to detect, and often protect against, threats coming in from the internet or passing through the local network. Their role is to stop threats before they reach the endpoints. If they fail to stop a threat, they might learn that an attack has happened and generate an alert, while subsequently blocking future, similar attacks.

There are no guarantees that technology will always protect you from attackers, but our results show that adding layers of security is an effective way to improve your prospects when facing general and more targeted attacks.

Tuesday, 17 October 2017

100% Certifiable

Whether you're in the market for a car, hamburger or computer security product, certifications are useful. They don't tell you how smooth the car drives, how tasty the sandwich is or how completely accurate the anti-virus software will be, but certifications indicate a general level of competence.

Latest reports now online.

In the UK new cars must be certified by the Vehicle Certification Agency (VCA), restaurants are checked for hygiene by the Food Standards Agency (FSA) and various independent testing organisations, including SE Labs, test IT security products for basic functionality.

A certification emphatically does not indicate the overall quality of a product, though. The FSA specifically states that, "The food hygiene rating is not a guide to food quality." In other words, the food won't make you ill, but you might not like it! Similarly, the VCA cares more about cars being made according to specification rather than how nice they look.

SE Labs has a range of available testing services. We consider certification to be the most basic type of testing. If a product claims to be able to detect malware then we can test that, but we don't claim it can detect all types. For a higher level of understanding about a product's capabilities so-called 'real-world' testing is necessary.

The report you are reading now is based on our more advanced testing, which exposes real products to live threats in a realistic environment, running on real computers on an internet-connected network.

But how can you be sure that we're really doing that, and not just making up the figures or giving some products an unfair advantage? After all, some companies contribute financially to supporting the tests, while others do not.

To go some way to addressing this concern, as well as to improve generally and continue to evolve the business, SE Labs has achieved ISO 9001:2015 certification for "The Provision of IT Security Product Testing". We think it's fair for the testers to be tested and we're very proud to have passed!
If you spot a detail in this report that you don't understand, or would like to discuss, please contact us via our Twitter or Facebook accounts.

SE Labs uses current threat intelligence to make our tests as realistic as possible. To learn more about how we test, how we define 'threat intelligence' and how we use it to improve our tests please visit our website and follow us on Twitter.

Our latest reports, for enterprise, small business and home users are now available for free from our website. Please download them and follow us on Twitter and/or Facebook to receive updates and future reports.

Tuesday, 10 October 2017

Anatomy of a Phishing Attack

Who attacked a couple of Internet pressure groups earlier this year? Jon Thompson examines the evidence.

For those on those of us engaged in constructing carefully-crafted tests against client email filtering services, the public details of an unusually high-quality spear-phishing attack against a low value target make for interesting reading.

In this case, there were two targets: Free Press, and Fight for the Future. The attack, dubbed "Phish for the Future" in a brief analysis by the Electronic Frontier Foundation, is curious for several reasons.

Free Press is a pressure group campaigning for an open internet, fighting media consolidation by large corporations, and defending press freedom. Fight for the Future works to protect people's basic online freedoms. Objectively, they're working for a better online future, which makes the whole affair stand out like a pork buffet at a bar mitzvah.

The first thing that struck me was that the emails were apparently all sent during office hours. The time zones place the senders anywhere between Finland and India, but apparently resolve to office hours when normalised to a single zone.

Another interesting aspect is that even though the emails were sent on 23 active days, the attackers didn't work weekends. This immediately marks them out as unusual. Anyone who's run an email honeypot knows that commodity spam flows 24 hours a day.

The attackers first tried generic phishing expeditions, but quickly cranked up their targeting and psychological manipulation. This begs an interesting question: If you're an experienced, professional, disciplined crew, why jeopardise the operation by beginning with less convincing samples that may alert the target to be on the lookout? Why didn't they simply start with the good stuff, get the job done, and move on?

One possible explanation is that the attackers were trainees on a course, authorised to undertake a carefully controlled "live fire" exercise. Psychologically manipulative techniques such as pretending to be a target's husband sending family photos, or a fan checking a URL to someone's music, imply a level of confident duplicity normally associated with spying scandals.

The level of sophistication and persistence on display forms a shibboleth. It looks and smells somehow "wrong". The published report reveals an attention to detail and target reconnaissance usually reserved for high value commercial targets. Either the attackers learn at a tremendous rate
through sheer interest alone, or they're methodically being taught increasingly sophisticated techniques to a timetable. If it was part of a course, then maybe the times the emails were sent show a break for morning coffee, lunch and afternoon tea, or fall into patterns of tuition followed by practical exercises.

The timing of the complete attack also stands out. It began on 7th July, ended on 8th August, and straddled the Net Neutrality Day of Action (12th July). With a lot happening at both targets during that time, and one assumes a lot of email flying about, perhaps the attackers believed they stood a better chance when the staff were busiest.

So, to recap, it looks like highly motivated yet disciplined attackers were operating with uncommonly sophisticated confidence against two small online freedom groups. Neither target has the business acumen of a large corporation, which rules out criminal gain, and yet an awful lot of effort was ranged against them.

The product of phishing is access, either to abuse directly or to be sold to others. Who would want secret access to organisations campaigning for online freedom? Both targets exist to change minds and therefore policy, which makes them political. They're interesting not only to governments, but also to media companies seeking to control the internet.

I'm speculating wildly, of course. The whole thing could very easily have been perpetrated by an under-worked individual at a large company, using their office computer and keeping regular hours to avoid suspicion. The rest is down to ingenuity and personal motivation.

We'll never know the truth, but the supporting infrastructure detailed in the EFF report certainly points to some considerable effort over a long period of time. If it was an individual, he's out there, he'll strike again, and he learns fast. In many ways, I'd prefer it to have been a security service training new recruits.

Thursday, 28 September 2017

Who certifies the certifiers?

At SE Labs we test security software and services methodically, realistically and in great detail. Or, at least, we claim to. But how does anyone really know?

Testing can be a very process-driven task. If you are going to be fair to every product undergoing a test you need to be consistent with how you run the test as a whole and how you test each individual product. It's probably best carried out by well-qualified people, then?

You don't need to be certified to work here…

We figured that as we certify, so should we be certified. As such, for the last few months we have worked towards having our business certified to an international level for providing consistent security testing services.

Another purpose of quality management is improvement. There is always room for improvement in testing, and we constantly strive to make things more realistic, useful and fair for everyone involved.

As such I am extremely proud to announce that SE Labs has now achieved compliance with the ISO 9001:2015 standard for quality management systems, specifically relating to "The Provision of IT Security Product Testing".

That means we do what we say we do, and strive to improve.

Tuesday, 19 September 2017

Email hosted protection tested

Our first cloud-based email protection report is now available.

Email provides a route right into the heart of our computers, phones and other devices. As such, it is frequently abused to perform a variety of attacks against potential victims of cybercrime.

Latest report now online.

The sophistication of attacks vary but many rely on our almost unbreakable instinct to open, read and interact with messages sent to work and personal email accounts. Businesses rely on email security services to filter out large numbers of such attacks.

The range of attack types in the real world is wide, but in general we consider there to be two main categories: targeted attacks, in which the attacker attempts to target a specific individual; and public attacks, which spread wide and far in an attempt to compromise as many people as possible.
Many of the same techniques are used in public and targeted attacks. The least technically sophisticated include requests for a money transfer or banking login credentials. More credible attempts include professionally-formatted emails and links to fake websites designed to trick users into entering their valuable details.

Attackers with more resources may use malware to achieve their goals, either in the form of attached files or by linking to websites that exploit visiting computers.
SE Labs monitors email threats in real-time, analysing large  numbers of messages and extracting samples that represent  large groups of those threats. Human testers then manually verify that any malware included works properly before re-sending these threats to our own accounts through the tested services.

We also generate targeted attacks using the same tools and techniques used by advanced attackers. In gathering threats this way we achieve a realistic and relevant coverage of existing threats in a small set of test samples.

Our latest reports, for enterprise, small business and home users are now available for free from our website. Please download them and follow us on Twitter and/or Facebook to receive updates and future reports.

Thursday, 7 September 2017

Can Microsoft Solve Security?

Windows is becoming increasingly secure. Does this spell the end of third-party security products and services?

Follow the herd
There is a well-regarded theory that, when a certain number of a community is immunised against a contagious disease, the rest of the community benefits from the reduced opportunity of an outbreak. Enough people are immune that a disease can't run rampant through the general population. It can't get enough of a foothold because too many people it encounters are immune. Some people get sick but the spread is limited and the community, including many who are not immune, avoids an epidemic.

Some do not subscribe to the 'herd immunity' theory , noting that diseases still exist in communities that vaccinate and that vaccinations carry their own risks. They also observe that there is a financial cost to vaccination. There are pros and cons to both sides of the debate. Some positions are rational while others are based on emotion, supposition and misinformation. Much the same happens in the world of cyber security and, specifically, issues surrounding anti-virus (aka anti-malware) products and services.

Hygiene issues
Biological disease does not knowingly seek out its victims, unless it is being used in the context of biological weapons (in which case it is an agent of the attacker rather than the attacker itself). In some respects we can compare 'mindless' internet worms with in-the-wild viral or bacterial contagions. All flow where the animals, air, water, computer networks, USB drives or other vectors take them.

In the 1990s and early 00s personal computer security was relatively limited in the general population, including in the business world. You were doing well if you installed a firewall, anti-virus software and maybe used some form of file or disk encryption. If you had a network appliance or two, and backed up your files, you were in exceptionally good shape – relatively speaking.

Even in those days some recognised the benefits of diversity. If you ran Symantec anti-virus on your endpoints, Trend Micro's product on your servers and Kaspersky Lab's on your email server you had pretty good coverage. If Symantec and Kaspersky missed a threat, maybe Trend would pick it up as it hit the servers.

Businesses and other organisations, including those who definitely should have known better (e.g. the UK's MI5 and SIS (MI6) intelligence services), often gave the impression that they were practising good information hygiene. Press reports from the time, and even now, suggest that things are not as we might assume.

Unencrypted laptops containing secret information were stolen, left in taxis and lost at train stations and airports. Databases were exposed to the internet and plundered using basic techniques. Malware files were received in email messages and executed. Sensible IT management, including a rapid patching cycle and decent anti-virus, was probably good enough to mitigate most of the issues. Maybe not enough of our herd was fully protected (immune), but the world did not fall apart every time a new worm was released.

But then attackers evolved their methods and things became more challenging for the defenders. Penetration testing became increasingly popular and sometimes mandatory. The contagions were no longer flowing randomly through the internet. They were being directed by criminals and spies to their targets.

Targeted attacks
While general-purpose worms and other 'viruses' still exist, targeted attacks are occurring more often. Or, at least, we are more aware of more targeted attacks today than previously. In fact corporate espionage at a nation-state level goes back as least as far as the early 18th century, so it's unlikely that targeted internet-based spying started in 2012, when FireEye started to highlight the existence of 'APTs'.

In the early 1700s a French Jesuit priest called Fran├žois Xavier d'Entrecolles was working in Beijing when he discovered the secret of its world class porcelain manufacturing. He sent this information back to Europe and single-handedly sent the famous Chinese ceramics industry into decline. (Perhaps appropriately, given the theme of this article, he also discovered that the Chinese also used oral vaccination against smallpox.)

Persistent advanced threats
Criminals and spies do not give up when faced with a locked door, closed border or uncooperative opponent. It is their job to achieve a mission and they will do all that is required to complete their goals for money, glory or duty. That said, we don't have to make things easy for them.

Imagine that there was only one lock manufacturer in the world. And only one company in charge of the world's border controls. Imagine that every person thought and behaved in exactly the same way. The bad guys would find life very easy. They need only to be able to pick one style of lock; find weak points in one organisation's procedures; and learn how to influence one type of person.

In reality there is much more diversity. There are dozens of lock manufacturers, hundreds of computer security companies and 7.5 billion human perceptions of the world. Those who wish to subvert the existing order face a lot of challenges. While a defender's life is also hard (attackers can be almost limitlessly unpredictable), attackers are constantly having to discover the lay of the land, create or adjust their approaches to problems and test their attacks to improve their chances of success. They can never assume that an attack will work. They can only try to stack the odds in their favour.

This is true not only of computer security but any security system. From muggers to terrorists, petty thieves to corrupt politicians, you have under-resourced defenders and imaginative attackers. But the diverse ways in which we can protect ourselves always puts the attackers in some position of doubt.

The ideal monolith
If you could create an operating system from scratch, predicting accurately all of the future threats and making no mistakes in your implementation, then maybe a monolithic approach to security would work. You could manage memory in ways that made it extremely hard for attackers to exploit vulnerabilities. You could create network stack implementations resistant to denial of service attacks. You could ensure that included applications were clean of legacy issues, such as trying to load DLLs that no longer exist. You could deny your users, and their applications, access to the deeper recesses of the system. You, the vendor, own the system while your users just borrow it.

Of course, you'd probably want to write or, at least, vet every compatible application before it was allowed to be installed on your system too, or risk vulnerabilities entering your fortress environment. And you'd have to make no mistakes in your coding. History does not offer much hope, but theoretically it's possible.

To a certain extent this is what Apple and Google have achieved with their iOS and Android operating systems. Users do not have low-level access and neither do applications. Anti-malware products cannot automatically remove malware and users have to run exploits against their own devices if they want to be able to perform certain functions, like remove worthless, annoying and space-occupying pre-installed applications. Which might be anti-malware applications! However, the existence of security updates demonstrates that neither Apple nor Google can predict future threats or code 100 per cent without error.

Back to healthcare
Let's think back to the objections to vaccination. There are costs involved and possible impacts on patients' health, although that last concern is extremely controversial. But this is similar to the concerns about anti-malware software. It costs money and, analogous to causing harm, it has some degree of impact on system performance. It might not be written very well, or interact with the system in such a way that it creates some instability. It may even introduce vulnerabilities to the system that did not exist before.

There may be little credible evidence that vaccination of humans creates a vulnerability to autism, but security vulnerabilities introduced by anti-malware products have been proved in the past and will no doubt appear again in the future.

From Microsoft's perspective, it wants to own the world's best operating system, which should be usable, stable and secure. It has a reputation for providing none of these attributes, but in reality it is doing much better than in previous decades.

Windows 10 is harder to hack than Windows XP and Windows 7. It includes anti-exploit technologies and forces users to update, taking the first step towards lending the system to users rather than allowing them to own it. It has its own anti-malware product built in, which it will activate automatically if the user doesn't install an alternative (or if the alternative becomes out of date). Third-party anti-malware products that impact system performance too heavily will be removed at certain stages.

From a user's perspective this could be a welcome move. Responsibility for securing computer systems is delegated to Microsoft, which is assuming part of the role of a managed services company. It handles anti-virus, encryption, web filtering and, if you use OneDrive, even secure storage of your data. If you just care about using your computer, and not administering it, then Microsoft's vision is attractive.

Vive la difference
No company is going to be able to create the perfect operating system, which must balance usability, stability and security. There will always be vulnerabilities in the operating system, its applications and its users. If we find ourselves in a world in which everyone is forced to run a particular version of Windows, with a restricted set of security applications, such as anti-malware, file encryption and network security, then the attackers have very much less work to do in order to achieve their goals.

Currently there are dozens of brands anti-malware products running on systems throughout the world. Attackers can make no assumptions and must work hard to evade and/ or disable many of these products to intrude into any number of systems. Even if some of these products are little better than Microsoft's Windows Defender (and some will be much worse – Windows Defender is a competent application), the very fact that an attacker does not know for sure what s/he will encounter is a good enough reason for diversity to exist in our community. It protects everyone by limiting the attackers' options.

If Windows Defender was the only anti-malware product in the world it would receive far more interest from criminals, penetration testers and mischievous researchers than it enjoys today.

Real world protection
So far we've implied that all anti-malware products are at least roughly as capable as each other. This is not a fair reflection of reality, though. Some very well established brands of anti-malware are extremely limited in their capabilities, while others are exceptionally advanced and wide-ranging in their approaches to the malware problem. Newer entrants to the market have made interesting and ambitious claims about their technologies and some of these have merit. But none of them provide the panacea that they imply or state outright.

A common approach is to use layers of security to achieve a useful level of protection. For example, Vendor A might choose to emphasise website address (URL) reputation, building a world-class, accurate URL filtering system. It might also include some malware signatures and a basic behavioural engine, but its emphasis is on URLs.

Another, Vendor B, might not care at all about URLs but really focusses on characteristics of file content (maybe using machine learning to achieve its goal). It does include some general malware signatures too, though, because it's unwise to ignore what you already know to be bad.

Customers might choose to deploy a really good URL reputation application in addition to a useful machine learning-based file analysis tool. The diversity in the market allows users to choose one or more approaches. Or they might choose a product that claims to offer the full stack. Managing more than one tool is usually harder work than installing one and forgetting about it, but choice is a good thing. As we noted earlier, there are those who argue in favour of running different anti-malware products on different parts of the network. Some large companies even choose different products for different business divisions in the hope that this could compartmentalise a breach.

Test results
Security testing organisations compare products, assessing how effective they are in stopping different types of threats. This might include URLs, malicious JavaScript email attachments and malicious documents. For the purposes of illustration we examined how effective a few of Microsoft's security measures were compared to third-party alternatives.

Hosted Email Services
Microsoft provides anti-malware protection in the cloud for those who use its Office 365 email services. We have found this to be quite effective at stopping known public threats, preventing around (25/75) 33 per cent from arriving into the inbox and sending a further 64 per cent into the Junk folder. However, it was much less effective with targeted attacks, allowing 52 per cent of the threats into the inbox. There were some false positives, but only a few.

Let's compare these figures to a third-party vendor of hosted email protection services. One industry leading competitor stopped nearly all of the public threats from entering either the Junk folder or the inbox. It also blocked all but one of the targeted attacks. There is clearly a benefit to the user when chaining these products together. The third-party product was more accurate and stopped threats from entering the endpoint, whereas although Microsoft's product did flag many threats as being 'Junk', these still ended up within reach of the user. What could go wrong? We have information that one very high profile breach in recent times was the result of a user pulling a message out of the Junk folder and executing its contents…

Email Sandbox Results
Not everyone wants to run their email protection in the cloud. There are technical and legal reasons why some businesses need to keep their data in-house and processed by on-premises security systems. We were approached by one such company to compare 'on-prem' email sandboxing solutions from the very largest vendors in that space.

There was a vast difference in the effectiveness of these products, even when we used some threats that everyone should know about. We found that 20 – 25 per cent of threats went undetected. Clearly there is some level of benefit to using this type of technology, but it's far from perfect and not cheap. They did, however, boost overall security when combined with Microsoft's email security product.

Endpoint Test Results
We regularly test endpoint security solutions with live in-the-wild threats to the public and targeted attacks that we craft ourselves using well-known, easily used tools. We look at products designed for enterprises, small to medium businesses and consumers. We always see Microsoft's products at or near the bottom of the list.

That is not to say that Microsoft Security Essentials (for Windows 7) or Windows Defender (for Windows 8 – 10) is no good. It is vastly better than it was a year or two ago, according to our test results and those from some other well-known testers. But it's definitely not the best and never has been. The idea that users are pushed towards this product is discomforting. The thought that alternatives might disappear due to market forces is frightening.

Diversity for the win
Maybe in the future there will be a useable, locked-down desktop operating system as secure as the mobile versions we use on our Apple and Google phones, tablets and 'netbooks'. While Microsoft is clearly moving towards a position in which it takes more direct responsibility for its users' security, the security gaps are so large, and the attackers' resources so huge, that now is not the time to embrace a walled-garden approach.

The attack landscape is wide and defenders are challenged for resources. The idea of a single product that can be managed easily by a well-trained staff and that defeats all threats is a lovely thing. But it is neither realistic nor desirable. It suggests a single point of failure upon which attackers will be focussing to the exclusion of all else. Once the monolith is broken (and it will be), there are no other layers of protection. You can't download and deploy an anti-malware recovery tool from a third party if there are no third parties still in business.

Diversity in security has always been essential. Just as disease evolves to succeed so do attackers. While we are lucky to have a large group of very clever and dedicated people in our community, producing tools and techniques to defend against the attackers, there are vast numbers of would-be opponents who can continually fail, day after day, until they succeed. Their job will be much easier if they have a limited number of things to break. Let's not make their job any easier than it already is.