Special Edition is the blog for security testing business SE Labs. It explains how we test security products, reports on the internet threats we find and provides security tips for businesses, other organisations and home users.

Tuesday, 24 July 2018

Detected, blocked, quarantined, cleaned?

What happens when your choice of security software handles an attack?

Latest reports now online.

It should be simple. You've clicked on the wrong link, opened a malicious email or installed something inadvisable. A threat is now attacking your PC and it's up to your choice of anti-malware product to handle things.

But what does it actually do under the hood?

Detection is important. The product should recognise that a threat exists, even if it can't fully handle it. At least you can receive an alert and seek help (or an alternative anti-malware program!)
Blocking threats is also very important. Ideally the protection system will prevent the malware from running. Sometimes that doesn't happen and the malware runs. In that case one hopes that the security software would recognise that bad things are happening and stop them. This is what we call 'neutralisation'.

Following a neutralisation your computer might not be completely clean. There could be some rogue code still on your hard disk, possibly even on your Desktop. There might also be entries in the Registry and elsewhere that will try to run this code (or code that has been deleted or quarantined).
You probably want your system to be protected by having threats blocked and, in cases where they are not, that they be removed as fast as possible and all significant traces removed. We call this happy state 'complete remediation'.

In SE Labs tests we measure all of these outcomes, including the worst one: compromise.

If you want to know how the different products tested in this report handled threats in detail, check out the Protection Details table and graph on page 10 of our reports. We don't show details of which products completely remediated threats and which did not when neutralising but the Protection Ratings on page eight take these into account.

If you spot a detail in this report that you don't understand, or would like to discuss, please contact us via our Twitter or Facebook accounts.

SE Labs uses current threat intelligence to make our tests as realistic as possible. To learn more about how we test, how we define 'threat intelligence' and how we use it to improve our tests please visit our website and follow us on Twitter.

Our latest reports, for enterprise, small business and home users are now available for free from our website. Please download them and follow us on Twitter and/or Facebook to receive updates and future reports.

Tuesday, 26 June 2018

What's the difference between SE Labs and a cyber-criminal?

As we prepared this network security appliance report for publication we were also getting ready to present at BT's internal security conference Snoopcon.

We had been asked to talk about security products and how they might not do what you assume they will.

Reports like this (PDF) provide an interesting insight into how security products actually work. Marketing messages will inevitably claim world-beating levels of effectiveness, while basic tests might well support these selling points. But when you actually hack target systems through security appliances you sometimes get a very different picture.

Some vendors will support the view that testing using a full attack chain (from a malicious URL pushing an exploit, which in turn delivers a payload that finally provides us with remote access to the system) is the right way to test. Others may point out that the threats we are using don't exactly exist in the real world of criminality because we created them in the lab and are not using them to break into systems worldwide.

We think that is a weak argument. If we can obtain access to certain popular, inexpensive tools online and create threats then these (or variants extremely close to them) are just as likely to exist in the 'real world' of the bad guys as in a legitimate, independent test lab. Not only that, but we don't keep creating new threats until we break in, which is what the criminals (and penetration testers) do. We create a set and, without bias, expose all of the tested products to these threats.

But in some ways we have evolved from being anti-malware testers to being penetration testers, because we don't just scan malware, execute scripts or visit URLs. Once we gain access to a target we perform the same tasks as a criminal would do: escalating privileges, stealing password hashes and installing keyloggers. The only difference between us and the bad guys is that we're hacking our own systems and helping the security vendors plug the gaps.

Latest report (PDF) now online.

Thursday, 24 May 2018

Big Time Crooks


When an online scam becomes too successful, the results can be farcical.

In the movie Small Time Crooks, Woody Allen leads an inept gang of would-be robbers who rent a store next to a bank. They plan to tunnel into the vault. As a cover, Allen's girlfriend (played by Tracey Ullman) sets up a cookie business in the store. Ullman's business takes off, and to maintain the cover the gang must set up production facilities, hire staff, find distributors, and so on.

Why is this relevant? Well, rewind to 2002. The internet had already taken off in a big way and people were pouring online as new opportunities exploded into the public consciousness. Also exploding was cybercrime, as the internet presented a new breed of tech savvy crooks with their own set of opportunities. For one gang, an Allenesque adventure was about to begin.

Humble Beginnings
How many times have you browsed a web page that suddenly throws up an alarming warning that your computer is infected and the only thing that can save you is to immediately buy a special program or call a special number? If you're up to date with system patches and use a reputable anti-virus solution, you're rarely in danger from such sites these days.

It was not always so.

For millions of internet users back in the day, who were running without protection, the apparent authority of such "scareware" sites made them act. They downloaded free "anti-virus" software that infected them with real malware, they parted with real cash, and many also paid again to have their computers cleaned by professionals.

Look through the history of scareware, and one company repeatedly appears: Innovative Marketing Inc (to give it the name used in US Federal Trade Commission paperwork but also known by a wide
range of other names). Innovative was registered in Belize in 2002. Despite the appearance of being a legitimate business, its initial products were dodgy: pirated music, porn and illicit Viagra, along with sales of "grey" versions of real anti-virus products.

After Symantec and McAfee both put pressure on the company to stop those software sales in 2003, Innovative tried to write its own. The resulting Computershield wasn't effective as anti-virus protection, but the company sold it anyway as a defence against the MyDoom worm. Innovative aggressively marketed its new product, and according to press reports, it was soon raking in $1 million per month. As the threat from MyDoom receded, so too did profits.

The company initially turned to adware as a new revenue source. This enabled so-called "affiliates" to use malicious web sites to silently install the adware on vulnerable Windows computers. Getting victims to visit those sites was achieved by placing what looked like legitimate adverts on real sites. Click them, and you became infected. The affiliates then pocketed a fee of 10 cents per infection, but it's through that Innovative made between $2 and $5 from sales of the advertised products.

Meanwhile, development of completely fake anti-virus software snowballed at the company's Kiev office. A classic example is "XP Antivirus 2008", though it also went by a large number of pseudonyms and evolved through many versions. A video of it trashing an XP machine can be found here. Its other major names include Winfixer, WinAntivirus, Drivecleaner, and SystemDoctor.

In many ways, Innovative's scareware was, well, innovative. It disabled any legitimate protection and told you the machine was heavily infected, even going to the trouble of creating fake blue screens of death. At the time, some antivirus companies had trouble keeping up with the rate of development.


Attempts to access Windows internet or security settings were blocked. The only way of "cleaning" the machine was to register the software and pay the fee. Millions of people did just that. The FTC estimates that between 2004 and 2008, the company and its subsidiaries raked in $163 million.

In 2008, a hacker with the handle NeoN found a database belonging to one of the developers, revealing that in a single week one affiliate made over $158,000 from infections.

The Problem of Success
Initially, Innovative used banks in Canada to process the credit card transactions of its victims, but problems quickly mounted as disgruntled cardholders began raising chargebacks. These are claims made to credit card companies about shoddy goods or services.

With Canadian banks beginning to refuse Innovative's business, it created subsidiary companies to hide its true identity, and approached the Bank of Kuwait and Bahrain. Trouble followed, and in 2005 this bank also stopped handling Innovative's business due to the high number of chargebacks. Eventually, the company found a Singaporean bank called DBS Bank to handle the mounting backlog of credit card transactions.

The only solution to the chargeback problem was to keep customers happy. So, in true Allenesque style, Innovative began to invest in call centres to help customers through their difficulties. It quickly opened facilities in Ukraine, India and the USA. Operatives would talk the customers through the steps needed for the software to miraculously declare their systems free of malware. It seems that enough customers were satisfied to allow the company to keep on raking in the cash.

But people did complain, not to the company but to the authorities. The FTC received over 3,000 complaints in all and launched an investigation. Marc D'Souza has been convicted of his role in the company and ordered to pay £8.2 million, along with his father who received some of the money. The case of Kristy Ross for her part in the scam is still going through the US courts, with lawyers arguing that she was merely an employee.

Several others, including Shaileshkumar "Sam" Jain and Bjorn Daniel Sundin, are still at large, and have had a $163 million judgement entered against them in their absence. Jain and Sundin remain on the FBI's Most Wanted Cyber Criminal list with rewards for their arrests totalling $40,000.

 


An Evergreen Scam
Scareware is a business model that rewards creativity while skirting the bounds of legality. Unlike ransomware, where criminal gangs must cover their tracks with a web of bank accounts and Bitcoin wallets, scareware can operate quite openly from countries with under-developed law enforcement and rife corruption. However, the gap between scareware and ransomware is rapidly closing.

Take the case of Latvian hacker Peteris Sahurovs, AKA "Piotrek" AKA "Sagade". He was arrested on an international arrest warrant in Latvia in 2011 for his part in a scareware scam, but he fled to Poland where he was subsequently detained in 2016.

He was extradited to the US and pled guilty in February this year to making $150,000 - $200,000.  US authorities claim the total made by Sahurovs' gang was closer to $2 million. He's due to be sentenced in June.

According to the Department of Justice, the Sahurovs gang set up a fake advertising agency that claimed to represent a US hotel chain. Once adverts were purchased on the Minneapolis Star Tribune's website, they were quickly swapped out for ones that infected vulnerable visitors with their malware. This made computers freeze and produce pop-ups explaining that victims needed to purchase special antivirus software to restore proper functionality. This case is interesting as it shows a clear cross over from scareware to ransomware. All data on the machines was scrambled until the software was purchased.

The level of sophistication and ingenuity displayed by scareware gangs is increasing, as is their boldness. You have probably been called by someone from India claiming to be from Microsoft, expressing concern that your computer is badly infected and offering to fix it. Or they may have posed as someone from your phone company telling you that they need to take certain steps to restore your internet connection to full health. There are many variations on the theme. Generally, they want you to download software that confirms their diagnosis. Once done, you must pay them to fix the problem. This has led to a plethora of amusing examples of playing the attackers at their own game.

It's easy to see the people who call you as victims of poverty with no choice but to scam, but string them along for a while and the insults soon fly. They know exactly what they're doing, and from the background chatter on such calls, so do hundreds of others. Scareware in all its forms is a crime that continues to bring in a lot of money for its perpetrators and will remain a threat for years to come.

Wednesday, 9 May 2018

Are you buying solid protection or snake oil?


Sometimes testers need to be tested too. We're always up for a challenge!

Latest reports now online.

How do you know which security products to buy? Many rely on independent tests to help in the decision-making process. But how do you know if a test is any good or not?

The Anti-Malware Testing Standards Organization (AMTSO) has been working to create a Standard that will give you, the customer, some assurance that the test was conducted fairly.

Earlier this year AMTSO has been trying out its Standard, which it has been working on for many months. SE Labs is proud to be involved in this initiative and the testing for this report has been assessed for compliance with the Standard.

If that sounds a bit dry, what it means is that there are experimental rules about how a tester should behave and we have put ourselves up for judgment by AMTSO.

Did participating in this process change the way we worked? Yes, but not in the technical ways that we test. Instead we turned the testing world's business model on its head.

Many testers charge vendors money to be tested. Some will test regardless, but charge money if the vendors want to see their results before publication (and have the opportunity to make requests for corrections).

We think that the dispute process should be free for all. SE Labs has not charged any vendor for its participation in this test and we provided a free dispute process to any vendor that requested it. In this way every vendor is treated as equally as possible, for the fairest possible test.

UPDATE (10th May 2018): We are extremely proud to announce that our 2018 Q1 reports have been judged compliant (PDF) with the AMTSO Draft Standard v6.1 – 2018-05-10.

If you spot a detail in this report that you don't understand, or would like to discuss, please contact us via our Twitter or Facebook accounts.

SE Labs uses current threat intelligence to make our tests as realistic as possible. To learn more about how we test, how we define 'threat intelligence' and how we use it to improve our tests please visit our website and follow us on Twitter.

Our latest reports, for enterprise, small business and home users are now available for free from our website. Please download them and follow us on Twitter and/or Facebook to receive updates and future reports.

Thursday, 3 May 2018

Tough test for email security services

Our latest email cloud security test really challenged the services under evaluation.

Latest report now online.

Last summer we launched our first email cloud security test and, while it was very well received by our readers and the security industry as a whole, we felt that there was still work to do on the methodology.

This report shows the results of six months of further development, and a much clearer variation in the capabilities of the services under test.

The most significant change to the way we conducted this test lies in the selection of threats we used to challenge the security services: we increased the number and broadened the sophistication.

Whereas we might have used one fake FBI blackmail email previously, in this test we sent 10, each created using a different level of sophistication. Maybe a service will detect the easier versions but allow more convincing examples through to the inbox?

We wanted to test the breaking point.

We also used a much larger number of targeted attacks. There was one group of public 'commodity' attacks, such as anyone on the internet might receive at random, but also three categories of crafted, targeted attacks including phishing, social engineering (e.g. fraud) and targeted malware (e.g. malicious PDFs).

Each individual attack was recreated 10 times in subtly different but important ways.

Attackers have a range of capabilities, from poor to extremely advanced. We used our "zero to Neo" approach to include basic, medium, advanced and very advanced threats to see what would be detected, stopped or allowed through.

The result was an incredibly tough test.

We believe that a security product that misses a threat should face significant penalties, while blocking legitimate activity is even more serious.

If you're paying for protection threats should be stopped and your computing experience shouldn't be hindered. As such, services that allowed threats through, and blocked legitimate messages, faced severe reductions to their accuracy ratings and, subsequently, their chances of winning an award.

Intelligence-Led Testing

We pay close attention to how criminals attempt to attack victims over email. The video below shows a typically convincing attack that starts with a text message and ends stealing enough information to clean out a bank account.


SE Labs uses current threat intelligence to make our tests as realistic as possible. To learn more about how we test, how we define 'threat intelligence' and how we use it to improve our tests please visit our website and follow us on Twitter.

Wednesday, 11 April 2018

Predictably Evil

A common criticism of computer security products is that they can only protect against known threats. When new attacks are detected and analysed security companies produce updates based on this new knowledge. It's a reactive approach that can provide attackers with a significant window of opportunity.

It's why anti-virus has been declared dead on more than one occasion.

Latest report now online.

Security companies have, for some years, developed advanced detection systems, often labelled as using 'AI', 'machine learning' or some other technical-sounding term. The basic idea is that past threats are analysed in deep ways to identify what future threats might look like. Ideally the result will be a product that can detect potentially bad files or behaviour before the attack is successful.

(We wrote a basic primer to understanding machine learning a couple of years ago.)

So does this AI stuff really work? Is it possible to predict new types of evil software? Certainly investors in tech companies believe so, piling hundreds of millions of funding dollars into new start-ups in the cyber defence field.

We prefer lab work to Silicon Valley speculation, though, and built a test designed to challenge the often magical claims made by 'next-gen' anti-malware companies.

With support from Cylance, we took four of its AI models and exposed them to threats that were seen in well-publicised attacks (e.g. WannaCry; Petya) months and even years later than the training that created the models.

It’s the equivalent of sending an old product forward in time and seeing how well it works with future threats. To find out how the Cylance AI models fared, and to discover more about how we tested, please download our report for free from our website.

Follow us on Twitter and/ or Facebook to receive updates and future reports.

Monday, 5 February 2018

Hacked! Will your anti-malware protect you from targeted attacks?

The news isn't good. Discover your best options in our latest reports.

Latest reports now online.

Criminals routinely create ingenious scams and indiscriminate attacks designed to compromise the unlucky and, occasionally, foolish. But sometimes they focus on a specific target rather than casting a net wide in the hope of landing something interesting.

Targeted attacks can range from basic, like an email simply asking you to send some money to an account, through to extremely devious and technical. If you received an email from your accountant with an attached PDF or Excel spreadsheet would you open it?

Most would and all that then stands between them and a successful hack (because the email was a trick and contained a dodgy document that gives remote control to the attacker) is the security software running on their PC.

In this test we've included indiscriminate, public attacks that come at victims from the web and via email, but we've also included some devious targeted attacks to see how well-protected potential victims would be.

We've not created any new types of threat and we've not discovered and used 'zero day' attacks. Instead we took tools that are freely distributed online and are well-known to penetration testers and criminals alike. We used these to generate threats that are realistic representations of what someone could quite easily put together to attack you or your business.

The results are extremely worrying. While a few products were excellent at detecting and protecting against these threats many more were less useful. We will continue this work and report any progress that these companies make in improving their products.

Our latest reports, for enterprise, small business and home users are now available for free from our website. Please download them and follow us on Twitter and/or Facebook to receive updates and future reports.