SE Labs

Special Edition
Computer security testing comment and analysis from SE LABS Ⓡ

Can Microsoft Solve Security?

Can Microsoft Solve Security?

Windows is becoming increasingly secure. Does this spell the end of third-party security products and services? Can Microsoft solve security on its own?

Follow the herd

There is a well-regarded theory that, when a certain number of a community is immunised against a contagious disease, the rest of the community benefits from the reduced opportunity of an outbreak. Enough people are immune that a disease can’t run rampant through the general population. It can’t get enough of a foothold because too many people it encounters are immune. Some people get sick but the spread is limited and the community, including many who are not immune, avoids an epidemic.

Some do not subscribe to the ‘herd immunity‘ theory , noting that diseases still exist in communities that vaccinate and that vaccinations carry their own risks. They also observe that there is a financial cost to vaccination. There are pros and cons to both sides of the debate. Some positions are rational while others are based on emotion, supposition and misinformation. Much the same happens in the world of cyber security and, specifically, issues surrounding anti-virus (aka anti-malware) products and services.

Hygiene issues

Biological disease does not knowingly seek out its victims, unless it is being used in the context of biological weapons. In the latter case it is an agent of the attacker rather than the attacker itself. In some respects we can compare ‘mindless’ internet worms with in-the-wild viral or bacterial contagions. All flow where the animals, air, water, computer networks, USB drives or other vectors take them.

In the 1990s and early 00s personal computer security was relatively limited in the general population. This included the business world. You were doing well if you installed a firewall, anti-virus software and maybe used some form of file or disk encryption. If you had a network appliance or two, and backed up your files, you were in exceptionally good shape. This is all relatively speaking.

Defence in depth

Even in those days some recognised the benefits of diversity. If you ran Symantec anti-virus on your endpoints, Trend Micro’s product on your servers and Kaspersky Lab’s on your email server you had pretty good coverage. If Symantec and Kaspersky missed a threat, maybe Trend would pick it up as it hit the servers.

Businesses and other organisations, including those who definitely should have known better (e.g. the UK’s MI5 and SIS (MI6) intelligence services), often gave the impression that they were practising good information hygiene. Press reports from the time, and even now, suggest that things are not as we might assume.

Sign up to our monthly business and personal security newsletters.

Unencrypted laptops containing secret information were stolen, left in taxis and lost at train stations and airports. Databases were exposed to the internet and plundered using basic techniques. Malware files were received in email messages and executed. Sensible IT management, including a rapid patching cycle and decent anti-virus, was probably good enough to mitigate most of the issues. Maybe not enough of our herd was fully protected (immune), but the world did not fall apart every time a new worm was released.

But then attackers evolved their methods and things became more challenging for the defenders. Penetration testing became increasingly popular and sometimes mandatory. The contagions were no longer flowing randomly through the internet. They were being directed by criminals and spies to their targets.

Targeted attacks

While general-purpose worms and other ‘viruses’ still exist, targeted attacks are occurring more often. Or, at least, we are more aware of more targeted attacks today than previously. In fact corporate espionage at a nation-state level goes back as least as far as the early 18th century. As such, it’s unlikely that targeted internet-based spying started in 2012, when FireEye started to highlight the existence of ‘APTs’.

In the early 1700s a French Jesuit priest called François Xavier d’Entrecolles was working in Beijing when he discovered the secret of its world class porcelain manufacturing. He sent this information back to Europe and single-handedly sent the famous Chinese ceramics industry into decline. (Perhaps appropriately, given the theme of this article, he also discovered that the Chinese also used oral vaccination against smallpox.)

Persistent advanced threats

Criminals and spies do not give up when faced with a locked door, closed border or uncooperative opponent. It is their job to achieve a mission and they will do all that is required to complete their goals for money, glory or duty. That said, we don’t have to make things easy for them.

Imagine that there was only one lock manufacturer in the world. And only one company in charge of the world’s border controls. Imagine that every person thought and behaved in exactly the same way. The bad guys would find life very easy. They need only to be able to pick one style of lock; find weak points in one organisation’s procedures; and learn how to influence one type of person.

Strength in diversity

In reality there is much more diversity. There are dozens of lock manufacturers, hundreds of computer security companies and 7.5 billion human perceptions of the world. Those who wish to subvert the existing order face a lot of challenges.

While a defender’s life is also hard (attackers can be almost limitlessly unpredictable), attackers are constantly having to discover the lay of the land, create or adjust their approaches to problems and test their attacks to improve their chances of success. They can never assume that an attack will work. They can only try to stack the odds in their favour.

This is true not only of computer security but any security system. From muggers to terrorists, petty thieves to corrupt politicians, you have under-resourced defenders and imaginative attackers. But the diverse ways in which we can protect ourselves always puts the attackers in some position of doubt.

The ideal monolith

If you could create an operating system from scratch, predicting accurately all of the future threats and making no mistakes in your implementation, then maybe a monolithic approach to security would work.

You could manage memory in ways that made it extremely hard for attackers to exploit vulnerabilities. Maybe you could create network stack implementations resistant to denial of service attacks. You could ensure that included applications were clean of legacy issues, such as trying to load DLLs that no longer exist. How about denying your users, and their applications, access to the deeper recesses of the system? You, the vendor, own the system while your users just borrow it. It makes you wonder – can Microsoft solve security on its own?

Of course, you’d probably want to write or, at least, vet every compatible application before it was allowed to be installed on your system too, or risk vulnerabilities entering your fortress environment. And you’d have to make no mistakes in your coding. History does not offer much hope, but theoretically it’s possible.

Mobile strengths and weaknesses

To a certain extent this is what Apple and Google have achieved with their iOS and Android operating systems. Users do not have low-level access and neither do applications. Anti-malware products cannot automatically remove malware and users have to run exploits against their own devices if they want to be able to perform certain functions, like remove worthless, annoying and space-occupying pre-installed applications. Which might be anti-malware applications! However, the existence of security updates demonstrates that neither Apple nor Google can predict future threats or code 100 per cent without error.

Back to healthcare

Let’s think back to the objections to vaccination. There are costs involved and possible impacts on patients’ health, although that last concern is extremely controversial. But this is similar to the concerns about anti-malware software. It costs money and, analogous to causing harm, it has some degree of impact on system performance. AV software might not be written very well, or interact with the system in such a way that it creates some instability. It may even introduce vulnerabilities to the system that did not exist before.

There may be little credible evidence that vaccination of humans creates a vulnerability to autism, but anti-malware products have contained security vulnerabilities in the past and will no doubt again in the future.

Constant improvement

From Microsoft’s perspective, it wants to own the world’s best operating system, which should be usable, stable and secure. It has a reputation for providing none of these attributes, but in reality it is doing much better than in previous decades.

Windows 10 is harder to hack than Windows XP and Windows 7. It includes anti-exploit technologies and forces users to update, taking the first step towards lending the system to users rather than allowing them to own it. It has its own anti-malware product built in, which it will activate automatically if the user doesn’t install an alternative (or if the alternative becomes out of date). Third-party anti-malware products that impact system performance too heavily will be removed at certain stages.

From a user’s perspective this could be a welcome move. Microsoft takes on responsibility for securing computer systems, assuming part of the role of a managed services company. It handles anti-virus, encryption, web filtering and, if you use OneDrive, even secure storage of your data. If you just care about using your computer, and not administering it, then Microsoft’s vision is attractive.

Vive la difference

No company is going to be able to create the perfect operating system, which must balance usability, stability and security. There will always be vulnerabilities in the operating system, its applications and its users. If we find ourselves in a world in which everyone is forced to run a particular version of Windows, with a restricted set of security applications, such as anti-malware, file encryption and network security, then the attackers have very much less work to do in order to achieve their goals.

Currently there are dozens of brands anti-malware products running on systems throughout the world. Attackers can make no assumptions and must work hard to evade and/ or disable many of these products to intrude into any number of systems. Even if some of these products are little better than Microsoft’s Windows Defender (and some will be much worse – Windows Defender is a competent application), the very fact that an attacker does not know for sure what s/he will encounter is a good enough reason for diversity to exist in our community. It protects everyone by limiting the attackers’ options.

If Windows Defender was the only anti-malware product in the world it would receive far more interest from criminals, penetration testers and mischievous researchers than it enjoys today.

Real world protection

So far we’ve implied that all anti-malware products are at least roughly as capable as each other. This is not a fair reflection of reality, though. Some very well established brands of anti-malware are extremely limited in their capabilities. Others are exceptionally advanced and wide-ranging in their approaches to the malware problem. Newer entrants to the market have made interesting and ambitious claims about their technologies and some of these have merit. But none of them provide the panacea that they imply or state outright.

A common approach is to use layers of security to achieve a useful level of protection. For example, Vendor A might choose to emphasise website address (URL) reputation, building a world-class, accurate URL filtering system. It might also include some malware signatures and a basic behavioural engine, but its emphasis is on URLs.

Another, Vendor B, might not care at all about URLs but really focusses on characteristics of file content (maybe using machine learning to achieve its goal). It does include some general malware signatures too, though, because it’s unwise to ignore what you already know to be bad.

Multiple choice

Customers might choose to deploy a really good URL reputation application in addition to a useful machine learning-based file analysis tool. The diversity in the market allows users to choose one or more approaches. Or they might choose a product that claims to offer the full stack. Managing more than one tool is usually harder work than installing one and forgetting about it, but choice is a good thing. As we noted earlier, there are those who argue in favour of running different anti-malware products on different parts of the network. Some large companies even choose different products for different business divisions in the hope that this could compartmentalise a breach.

Test results

Security testing organisations compare products, assessing how effective they are in stopping different types of threats. This might include URLs, malicious JavaScript email attachments and malicious documents. For the purposes of illustration we examined how effective a few of Microsoft’s security measures were, compared to third-party alternatives.

Hosted Email Services

Microsoft provides anti-malware protection in the cloud for those who use its Office 365 email services. We have found this to be quite effective at stopping known public threats, preventing around (25/75) 33 per cent from arriving into the inbox and sending a further 64 per cent into the Junk folder. However, it was much less effective with targeted attacks, allowing 52 per cent of the threats into the inbox. There were some false positives, but only a few.

Let’s compare these figures to a third-party vendor of hosted email protection services. One industry leading competitor stopped nearly all of the public threats from entering either the Junk folder or the inbox. It also blocked all but one of the targeted attacks. There is clearly a benefit to the user when chaining these products together. The third-party product was more accurate and stopped threats from entering the endpoint, whereas although Microsoft’s product did flag many threats as being ‘Junk’, these still ended up within reach of the user. What could go wrong? We have information that one very high profile breach in recent times was the result of a user pulling a message out of the Junk folder and executing its contents…

Email Sandbox Results

Not everyone wants to run their email protection in the cloud. There are technical and legal reasons why some businesses need to keep their data in-house and processed by on-premises security systems. We were approached by one such company to compare ‘on-prem’ email sandboxing solutions from the very largest vendors in that space.

There was a vast difference in the effectiveness of these products, even when we used some threats that everyone should know about. We found that 20 – 25 per cent of threats went undetected. Clearly there is some level of benefit to using this type of technology, but it’s far from perfect and not cheap. They did, however, boost overall security when combined with Microsoft’s email security product.

Endpoint Test Results

We regularly test endpoint security solutions with live in-the-wild threats to the public and targeted attacks that we craft ourselves using well-known, easily used tools. Our testers look at products designed for enterprises, small to medium businesses and consumers. We always see Microsoft’s products at or near the bottom of the list.

That is not to say that Microsoft Security Essentials (for Windows 7) or Windows Defender (for Windows 8 – 10) is no good. It is vastly better than it was a year or two ago, according to our test results and those from some other well-known testers. But it’s definitely not the best and never has been. The idea that users are pushed towards this product is discomforting. The thought that alternatives might disappear due to market forces is frightening.

Diversity for the win

Maybe in the future there will be a useable, locked-down desktop operating system as secure as the mobile versions we use on our Apple and Google phones, tablets and ‘netbooks’. While Microsoft is clearly moving towards a position in which it takes more direct responsibility for its users’ security, the security gaps are so large, and the attackers’ resources so huge, that now is not the time to embrace a walled-garden approach.

Can Microsoft solve security? Probably not. And it shouldn’t try. Not on its own.

The attack landscape is wide and defenders don’t have infinite resources. The idea of a single product, that well-trained staff manage easily and that defeats all threats, is a lovely thing. But it is neither realistic nor desirable. It suggests a single point of failure upon which attackers will be focussing to the exclusion of all else. Once attackers break the monolith (and they will ), there are no other layers of protection. You can’t download and deploy an anti-malware recovery tool from a third party if there are no third parties still in business.

Diversity in security has always been essential. Just as disease evolves to succeed so do attackers. While we are lucky to have a large group of very clever and dedicated people in our community, producing tools and techniques to defend against the attackers, there are vast numbers of would-be opponents who can continually fail, day after day, until they succeed. Their job will be much easier if they have a limited number of things to break. Let’s not make their job any easier than it already is.

Featured podcast:

Find out more

Free security test reports

Stay in touch

Get tested

Discover how we work with large organisations and security vendors.

  • Do you run a large organisation’s security infrastructure and want an assessment?
  • Are you a security vendor that needs certification?
  • SE Labs anti-virus certification can help security vendors access Windows Early Launch Antimalware (ELAM).

Please contact us now.


SE Labs Ltd is a private, independently-owned and run testing company that assesses security products and services. The main laboratory is located in Wimbledon, South London. It has excellent local and international travel connections. The lab is open for prearranged client visits.


SE Labs Ltd
Hill Place House
55A High Street
SW19 5BA