Special Edition is the blog for security testing business SE Labs. It explains how we test security products, reports on the internet threats we find and provides security tips for businesses, other organisations and home users.

Thursday, 28 September 2017

Who certifies the certifiers?

At SE Labs we test security software and services methodically, realistically and in great detail. Or, at least, we claim to. But how does anyone really know?

Testing can be a very process-driven task. If you are going to be fair to every product undergoing a test you need to be consistent with how you run the test as a whole and how you test each individual product. It's probably best carried out by well-qualified people, then?

You don't need to be certified to work here…

We figured that as we certify, so should we be certified. As such, for the last few months we have worked towards having our business certified to an international level for providing consistent security testing services.

Another purpose of quality management is improvement. There is always room for improvement in testing, and we constantly strive to make things more realistic, useful and fair for everyone involved.

As such I am extremely proud to announce that SE Labs has now achieved compliance with the ISO 9001:2015 standard for quality management systems, specifically relating to "The Provision of IT Security Product Testing".

That means we do what we say we do, and strive to improve.

Tuesday, 19 September 2017

Email hosted protection tested

Our first cloud-based email protection report is now available.

Email provides a route right into the heart of our computers, phones and other devices. As such, it is frequently abused to perform a variety of attacks against potential victims of cybercrime.

Latest report now online.

The sophistication of attacks vary but many rely on our almost unbreakable instinct to open, read and interact with messages sent to work and personal email accounts. Businesses rely on email security services to filter out large numbers of such attacks.

The range of attack types in the real world is wide, but in general we consider there to be two main categories: targeted attacks, in which the attacker attempts to target a specific individual; and public attacks, which spread wide and far in an attempt to compromise as many people as possible.
Many of the same techniques are used in public and targeted attacks. The least technically sophisticated include requests for a money transfer or banking login credentials. More credible attempts include professionally-formatted emails and links to fake websites designed to trick users into entering their valuable details.

Attackers with more resources may use malware to achieve their goals, either in the form of attached files or by linking to websites that exploit visiting computers.
SE Labs monitors email threats in real-time, analysing large  numbers of messages and extracting samples that represent  large groups of those threats. Human testers then manually verify that any malware included works properly before re-sending these threats to our own accounts through the tested services.

We also generate targeted attacks using the same tools and techniques used by advanced attackers. In gathering threats this way we achieve a realistic and relevant coverage of existing threats in a small set of test samples.

Our latest reports, for enterprise, small business and home users are now available for free from our website. Please download them and follow us on Twitter and/or Facebook to receive updates and future reports.

Thursday, 7 September 2017

Can Microsoft Solve Security?

Windows is becoming increasingly secure. Does this spell the end of third-party security products and services?

Follow the herd
There is a well-regarded theory that, when a certain number of a community is immunised against a contagious disease, the rest of the community benefits from the reduced opportunity of an outbreak. Enough people are immune that a disease can't run rampant through the general population. It can't get enough of a foothold because too many people it encounters are immune. Some people get sick but the spread is limited and the community, including many who are not immune, avoids an epidemic.

Some do not subscribe to the 'herd immunity' theory , noting that diseases still exist in communities that vaccinate and that vaccinations carry their own risks. They also observe that there is a financial cost to vaccination. There are pros and cons to both sides of the debate. Some positions are rational while others are based on emotion, supposition and misinformation. Much the same happens in the world of cyber security and, specifically, issues surrounding anti-virus (aka anti-malware) products and services.

Hygiene issues
Biological disease does not knowingly seek out its victims, unless it is being used in the context of biological weapons (in which case it is an agent of the attacker rather than the attacker itself). In some respects we can compare 'mindless' internet worms with in-the-wild viral or bacterial contagions. All flow where the animals, air, water, computer networks, USB drives or other vectors take them.

In the 1990s and early 00s personal computer security was relatively limited in the general population, including in the business world. You were doing well if you installed a firewall, anti-virus software and maybe used some form of file or disk encryption. If you had a network appliance or two, and backed up your files, you were in exceptionally good shape – relatively speaking.

Even in those days some recognised the benefits of diversity. If you ran Symantec anti-virus on your endpoints, Trend Micro's product on your servers and Kaspersky Lab's on your email server you had pretty good coverage. If Symantec and Kaspersky missed a threat, maybe Trend would pick it up as it hit the servers.

Businesses and other organisations, including those who definitely should have known better (e.g. the UK's MI5 and SIS (MI6) intelligence services), often gave the impression that they were practising good information hygiene. Press reports from the time, and even now, suggest that things are not as we might assume.

Unencrypted laptops containing secret information were stolen, left in taxis and lost at train stations and airports. Databases were exposed to the internet and plundered using basic techniques. Malware files were received in email messages and executed. Sensible IT management, including a rapid patching cycle and decent anti-virus, was probably good enough to mitigate most of the issues. Maybe not enough of our herd was fully protected (immune), but the world did not fall apart every time a new worm was released.

But then attackers evolved their methods and things became more challenging for the defenders. Penetration testing became increasingly popular and sometimes mandatory. The contagions were no longer flowing randomly through the internet. They were being directed by criminals and spies to their targets.

Targeted attacks
While general-purpose worms and other 'viruses' still exist, targeted attacks are occurring more often. Or, at least, we are more aware of more targeted attacks today than previously. In fact corporate espionage at a nation-state level goes back as least as far as the early 18th century, so it's unlikely that targeted internet-based spying started in 2012, when FireEye started to highlight the existence of 'APTs'.

In the early 1700s a French Jesuit priest called Fran├žois Xavier d'Entrecolles was working in Beijing when he discovered the secret of its world class porcelain manufacturing. He sent this information back to Europe and single-handedly sent the famous Chinese ceramics industry into decline. (Perhaps appropriately, given the theme of this article, he also discovered that the Chinese also used oral vaccination against smallpox.)

Persistent advanced threats
Criminals and spies do not give up when faced with a locked door, closed border or uncooperative opponent. It is their job to achieve a mission and they will do all that is required to complete their goals for money, glory or duty. That said, we don't have to make things easy for them.

Imagine that there was only one lock manufacturer in the world. And only one company in charge of the world's border controls. Imagine that every person thought and behaved in exactly the same way. The bad guys would find life very easy. They need only to be able to pick one style of lock; find weak points in one organisation's procedures; and learn how to influence one type of person.

In reality there is much more diversity. There are dozens of lock manufacturers, hundreds of computer security companies and 7.5 billion human perceptions of the world. Those who wish to subvert the existing order face a lot of challenges. While a defender's life is also hard (attackers can be almost limitlessly unpredictable), attackers are constantly having to discover the lay of the land, create or adjust their approaches to problems and test their attacks to improve their chances of success. They can never assume that an attack will work. They can only try to stack the odds in their favour.

This is true not only of computer security but any security system. From muggers to terrorists, petty thieves to corrupt politicians, you have under-resourced defenders and imaginative attackers. But the diverse ways in which we can protect ourselves always puts the attackers in some position of doubt.

The ideal monolith
If you could create an operating system from scratch, predicting accurately all of the future threats and making no mistakes in your implementation, then maybe a monolithic approach to security would work. You could manage memory in ways that made it extremely hard for attackers to exploit vulnerabilities. You could create network stack implementations resistant to denial of service attacks. You could ensure that included applications were clean of legacy issues, such as trying to load DLLs that no longer exist. You could deny your users, and their applications, access to the deeper recesses of the system. You, the vendor, own the system while your users just borrow it.

Of course, you'd probably want to write or, at least, vet every compatible application before it was allowed to be installed on your system too, or risk vulnerabilities entering your fortress environment. And you'd have to make no mistakes in your coding. History does not offer much hope, but theoretically it's possible.

To a certain extent this is what Apple and Google have achieved with their iOS and Android operating systems. Users do not have low-level access and neither do applications. Anti-malware products cannot automatically remove malware and users have to run exploits against their own devices if they want to be able to perform certain functions, like remove worthless, annoying and space-occupying pre-installed applications. Which might be anti-malware applications! However, the existence of security updates demonstrates that neither Apple nor Google can predict future threats or code 100 per cent without error.

Back to healthcare
Let's think back to the objections to vaccination. There are costs involved and possible impacts on patients' health, although that last concern is extremely controversial. But this is similar to the concerns about anti-malware software. It costs money and, analogous to causing harm, it has some degree of impact on system performance. It might not be written very well, or interact with the system in such a way that it creates some instability. It may even introduce vulnerabilities to the system that did not exist before.

There may be little credible evidence that vaccination of humans creates a vulnerability to autism, but security vulnerabilities introduced by anti-malware products have been proved in the past and will no doubt appear again in the future.

From Microsoft's perspective, it wants to own the world's best operating system, which should be usable, stable and secure. It has a reputation for providing none of these attributes, but in reality it is doing much better than in previous decades.

Windows 10 is harder to hack than Windows XP and Windows 7. It includes anti-exploit technologies and forces users to update, taking the first step towards lending the system to users rather than allowing them to own it. It has its own anti-malware product built in, which it will activate automatically if the user doesn't install an alternative (or if the alternative becomes out of date). Third-party anti-malware products that impact system performance too heavily will be removed at certain stages.

From a user's perspective this could be a welcome move. Responsibility for securing computer systems is delegated to Microsoft, which is assuming part of the role of a managed services company. It handles anti-virus, encryption, web filtering and, if you use OneDrive, even secure storage of your data. If you just care about using your computer, and not administering it, then Microsoft's vision is attractive.

Vive la difference
No company is going to be able to create the perfect operating system, which must balance usability, stability and security. There will always be vulnerabilities in the operating system, its applications and its users. If we find ourselves in a world in which everyone is forced to run a particular version of Windows, with a restricted set of security applications, such as anti-malware, file encryption and network security, then the attackers have very much less work to do in order to achieve their goals.

Currently there are dozens of brands anti-malware products running on systems throughout the world. Attackers can make no assumptions and must work hard to evade and/ or disable many of these products to intrude into any number of systems. Even if some of these products are little better than Microsoft's Windows Defender (and some will be much worse – Windows Defender is a competent application), the very fact that an attacker does not know for sure what s/he will encounter is a good enough reason for diversity to exist in our community. It protects everyone by limiting the attackers' options.

If Windows Defender was the only anti-malware product in the world it would receive far more interest from criminals, penetration testers and mischievous researchers than it enjoys today.

Real world protection
So far we've implied that all anti-malware products are at least roughly as capable as each other. This is not a fair reflection of reality, though. Some very well established brands of anti-malware are extremely limited in their capabilities, while others are exceptionally advanced and wide-ranging in their approaches to the malware problem. Newer entrants to the market have made interesting and ambitious claims about their technologies and some of these have merit. But none of them provide the panacea that they imply or state outright.

A common approach is to use layers of security to achieve a useful level of protection. For example, Vendor A might choose to emphasise website address (URL) reputation, building a world-class, accurate URL filtering system. It might also include some malware signatures and a basic behavioural engine, but its emphasis is on URLs.

Another, Vendor B, might not care at all about URLs but really focusses on characteristics of file content (maybe using machine learning to achieve its goal). It does include some general malware signatures too, though, because it's unwise to ignore what you already know to be bad.

Customers might choose to deploy a really good URL reputation application in addition to a useful machine learning-based file analysis tool. The diversity in the market allows users to choose one or more approaches. Or they might choose a product that claims to offer the full stack. Managing more than one tool is usually harder work than installing one and forgetting about it, but choice is a good thing. As we noted earlier, there are those who argue in favour of running different anti-malware products on different parts of the network. Some large companies even choose different products for different business divisions in the hope that this could compartmentalise a breach.

Test results
Security testing organisations compare products, assessing how effective they are in stopping different types of threats. This might include URLs, malicious JavaScript email attachments and malicious documents. For the purposes of illustration we examined how effective a few of Microsoft's security measures were compared to third-party alternatives.

Hosted Email Services
Microsoft provides anti-malware protection in the cloud for those who use its Office 365 email services. We have found this to be quite effective at stopping known public threats, preventing around (25/75) 33 per cent from arriving into the inbox and sending a further 64 per cent into the Junk folder. However, it was much less effective with targeted attacks, allowing 52 per cent of the threats into the inbox. There were some false positives, but only a few.

Let's compare these figures to a third-party vendor of hosted email protection services. One industry leading competitor stopped nearly all of the public threats from entering either the Junk folder or the inbox. It also blocked all but one of the targeted attacks. There is clearly a benefit to the user when chaining these products together. The third-party product was more accurate and stopped threats from entering the endpoint, whereas although Microsoft's product did flag many threats as being 'Junk', these still ended up within reach of the user. What could go wrong? We have information that one very high profile breach in recent times was the result of a user pulling a message out of the Junk folder and executing its contents…

Email Sandbox Results
Not everyone wants to run their email protection in the cloud. There are technical and legal reasons why some businesses need to keep their data in-house and processed by on-premises security systems. We were approached by one such company to compare 'on-prem' email sandboxing solutions from the very largest vendors in that space.

There was a vast difference in the effectiveness of these products, even when we used some threats that everyone should know about. We found that 20 – 25 per cent of threats went undetected. Clearly there is some level of benefit to using this type of technology, but it's far from perfect and not cheap. They did, however, boost overall security when combined with Microsoft's email security product.

Endpoint Test Results
We regularly test endpoint security solutions with live in-the-wild threats to the public and targeted attacks that we craft ourselves using well-known, easily used tools. We look at products designed for enterprises, small to medium businesses and consumers. We always see Microsoft's products at or near the bottom of the list.

That is not to say that Microsoft Security Essentials (for Windows 7) or Windows Defender (for Windows 8 – 10) is no good. It is vastly better than it was a year or two ago, according to our test results and those from some other well-known testers. But it's definitely not the best and never has been. The idea that users are pushed towards this product is discomforting. The thought that alternatives might disappear due to market forces is frightening.

Diversity for the win
Maybe in the future there will be a useable, locked-down desktop operating system as secure as the mobile versions we use on our Apple and Google phones, tablets and 'netbooks'. While Microsoft is clearly moving towards a position in which it takes more direct responsibility for its users' security, the security gaps are so large, and the attackers' resources so huge, that now is not the time to embrace a walled-garden approach.

The attack landscape is wide and defenders are challenged for resources. The idea of a single product that can be managed easily by a well-trained staff and that defeats all threats is a lovely thing. But it is neither realistic nor desirable. It suggests a single point of failure upon which attackers will be focussing to the exclusion of all else. Once the monolith is broken (and it will be), there are no other layers of protection. You can't download and deploy an anti-malware recovery tool from a third party if there are no third parties still in business.

Diversity in security has always been essential. Just as disease evolves to succeed so do attackers. While we are lucky to have a large group of very clever and dedicated people in our community, producing tools and techniques to defend against the attackers, there are vast numbers of would-be opponents who can continually fail, day after day, until they succeed. Their job will be much easier if they have a limited number of things to break. Let's not make their job any easier than it already is.

Friday, 1 September 2017

Review: ImmuniWeb On-Demand Application Security Testing

What do a start-up, small business and enterprise have in common?

They all have one or more websites.

That's not a very humorous punchline, but the security implications of managing business websites aren't funny either.

In an age when extremely large organisations are being hacked, as well as specialist security companies, website security could not be a more serious business. Throw into the mix regulations such as the data protection act and the incoming GDPR legislation and being the person responsible for the company website just became positively horrible.

A website is a business' public face, whether it be a local taxi company or a global pharmaceutical giant. It is virtually impossible to do business these days without a website and maintain credibility, but a website hack instantly harms any company's standing.

How do websites get hacked? Sometimes the attackers will focus on compromising the site's administrator, but more often than not (in our experience) the site itself is attacked directly by means of an exploit.

Such an exploit could be a aimed at a vulnerability in the platform, such as WordPress, or the server's operating system. Sometimes the hosting company itself is targeted: a good value-for-money proposition for an attacker who wants to run one attack and gain access to thousands of websites.

Will AI save our sites?

Artificial intelligence is great but people are often necessary for some tasks. ImmuniWeb understands that. Assessing the security of a website is non-trivial and, while automated tools exist to test for the presence of various vulnerabilities, often it takes a human brain to really get to the bottom of a problem. Much in the same way that SE Labs uses people to enhance security testing, ImmuniWeb adds the personal touch to checking the quality of a website's security.

The service provides testing for vulnerabilities listed in the OWASP Top Ten Vulnerabilities list, PCI DSS vulnerabilities and a range of other sensible criteria, including predictable CAPTCHA protections and open directory listings.

Wizard setup

Setting up the initial test was a very simple task. Enter a few relevant details into  ImmuniWeb's Wizard-driven website, pay the fee and the work starts. A couple of days later a report is made available and you have around three months to download it before it is deleted automatically. You will receive warnings about the impending deletion.

The report is detailed. The first pages give an overview of the risk level based on how many vulnerabilities have been found, certain administration configuration issues that might exist and even an indication of other websites that might be impersonating yours.

Who is hosting?

The data in the reports is interesting and some of the issues brought to light could be easily solved. It does depend on how you have your web hosting organised, though. For example, if you run your own servers you can follow advice on upgrading certain services, such as Apache or SSH.

However, if your site runs on a hosting platform provided by a third-party, such as GoDaddy, 1&1, 123Reg or a thousand others then you have a choice: You could contact the company and request that they upgrade; or move to another host and hope that they do a better job with updates.

In this review we discovered that the hosting company we use for the SE Labs website was a little behind with some updates. We used the ImmuniWeb report as evidence that there was a potential problem and, to our surprise, the company responded fast and claimed to fix the issues.

While we could verify the changes ourselves (after all, we test security systems ourselves) we understand that for most businesses a second test would be warranted. We ran a second test for this review and were pleased to see that the previous issues had indeed been fixed.

How much?

This is where things could get expensive, though. An on-demand small business (SMB) test costs $1,499. If you are a start-up and want to have your site assessed then this is a reasonable business expense. Multiple verification tests add up, though. A faster 'Express' test is less expensive, coming in at $499. If you expect your site to change frequently then continuous assessments are available, with prices starting at $999 per month.

Total Cost of Reassurance

But while your site might not change, knowledge about security vulnerabilities does. New vulnerabilities are being discovered at a frightening rate and updates for popular web server components, such as MySQL, appear often. When testing our own website ImmuniWeb noted out of date software, which was updated accordingly.

By the time we ran the second test the same, updated software was again out of date. If the same issues happen to you, it might be worth learning how to test the versions of the services running at your web hosting company and give them a prod to update as and when necessary. Paying over $1,000 to assess something they should be taking care of seems unnecessary.

Monitoring the weak link

Losing control of your website is a situation no business wants to contemplate, whether it’s a start-up looking for funding or a massively profitable public company. Web application vulnerabilities are a significant weak point that can and should be assessed regularly. ImmuniWeb provides just such a service but because people are involved, as well as machine learning-equipped systems, there is a significant cost to the system, as well as an advantage over free website scanning sites and tools.

While, on the face of it, using ImmuniWeb's service might appear expensive, compared to training your own team of penetration testers, or sub-contracting a company to do the work for you, it is good value for money.