SE Labs

Posts tagged 'anti-virus'

Latest security tests introduce attack chain scoring

When is a security breach serious, less serious or not a breach at all?

Latest reports now online.

UPDATE (29/10/2018): This set of reports are confirmed to be compliant with AMTSO Standard v1.0 by the Anti-Malware Testing Standards Organization.


Our endpoint protection tests have always included targeted attacks.

These allow us to gauge how effectively anti-malware products, in use by millions of customers, can stop hackers from breaching your systems.

We penalise products heavily for allowing partial or full breaches and, until now, that penalisation has been the same regardless of how deeply we’ve been able to penetrate into the system. Starting with this report we have updated our scoring to take varying levels of ‘success’ by us, the attackers, into account.

The new scores only apply to targeted attacks and the scoring system is listed in detail on page eight of each of the reports.

If the attackers are able to gain basic access to a target, which means they are able to run basic commands that, for example, allow them to explore the file system, then the score is -1.

The next stage is to attempt to steal a file. If successful there is a further -1 penalty.

At this stage the attackers want to take much greater control of the system. This involves increasing their account privileges – so-called privilege escalation. Success here turns a bad situation worse for the target and, if achieved, there is an additional -2 penalty.

Finally, if escalation is achieved, certain post-escalation steps are attempted, such as running a key logger or stealing passwords. A final -1 penalty is imposed if these stages are completed, making possible scores for a breach range between -1 and -5 depending on how many attack stages are possible to complete.

We have decided not to publish exact details of where in the attack chain each product stands or falls, but have provided that detailed information to the companies who produce the software tested in this report and who have asked for it.

If you spot a detail in this report that you don’t understand, or would like to discuss, please contact us via our Twitter or Facebook accounts.

SE Labs uses current threat intelligence to make our tests as realistic as possible. To learn more about how we test, how we define ‘threat intelligence’ and how we use it to improve our tests please visit our website and follow us on Twitter.
Our latest reports, for enterprisesmall business and home users are now available for free from our website. Please download them and follow us on Twitter and/or Facebook to receive updates and future reports.

Detected, blocked, quarantined, cleaned?

2018q2-1157895

What happens when your choice of security software handles an attack?

Latest reports now online.

It should be simple. You’ve clicked on the wrong link, opened a malicious email or installed something inadvisable. A threat is now attacking your PC and it’s up to your choice of anti-malware product to handle things.

But what does it actually do under the hood?

Detection is important. The product should recognise that a threat exists, even if it can’t fully handle it. At least you can receive an alert and seek help (or an alternative anti-malware program!)
Blocking threats is also very important. Ideally the protection system will prevent the malware from running. Sometimes that doesn’t happen and the malware runs. In that case one hopes that the security software would recognise that bad things are happening and stop them. This is what we call ‘neutralisation’.

Following a neutralisation your computer might not be completely clean. There could be some rogue code still on your hard disk, possibly even on your Desktop. There might also be entries in the Registry and elsewhere that will try to run this code (or code that has been deleted or quarantined).
You probably want your system to be protected by having threats blocked and, in cases where they are not, that they be removed as fast as possible and all significant traces removed. We call this happy state ‘complete remediation’.

In SE Labs tests we measure all of these outcomes, including the worst one: compromise.

If you want to know how the different products tested in this report handled threats in detail, check out the Protection Details table and graph on page 10 of our reports. We don’t show details of which products completely remediated threats and which did not when neutralising but the Protection Ratings on page eight take these into account.

If you spot a detail in this report that you don’t understand, or would like to discuss, please contact us via our Twitter or Facebook accounts.

SE Labs uses current threat intelligence to make our tests as realistic as possible. To learn more about how we test, how we define ‘threat intelligence’ and how we use it to improve our tests please visit our website and follow us on Twitter.

Our latest reports, for enterprise, small business and home users are now available for free from our website. Please download them and follow us on Twitter and/or Facebook to receive updates and future reports.

Big Time Crooks

federal_bureau_of_investigation_seal-8620436
When an online scam becomes too successful, the results can be farcical.

In the movie Small Time Crooks, Woody Allen leads an inept gang of would-be robbers who rent a store next to a bank. They plan to tunnel into the vault. As a cover, Allen’s girlfriend (played by Tracey Ullman) sets up a cookie business in the store. Ullman’s business takes off, and to maintain the cover the gang must set up production facilities, hire staff, find distributors, and so on.

Why is this relevant? Well, rewind to 2002. The internet had already taken off in a big way and people were pouring online as new opportunities exploded into the public consciousness. Also exploding was cybercrime, as the internet presented a new breed of tech savvy crooks with their own set of opportunities. For one gang, an Allenesque adventure was about to begin.

Humble Beginnings
How many times have you browsed a web page that suddenly throws up an alarming warning that your computer is infected and the only thing that can save you is to immediately buy a special program or call a special number? If you’re up to date with system patches and use a reputable anti-virus solution, you’re rarely in danger from such sites these days.

It was not always so.

For millions of internet users back in the day, who were running without protection, the apparent authority of such “scareware” sites made them act. They downloaded free “anti-virus” software that infected them with real malware, they parted with real cash, and many also paid again to have their computers cleaned by professionals.

computer-health-alert-large-6338481Look through the history of scareware, and one company repeatedly appears: Innovative Marketing Inc (to give it the name used in US Federal Trade Commission paperwork but also known by a wide
range of other names). Innovative was registered in Belize in 2002. Despite the appearance of being a legitimate business, its initial products were dodgy: pirated music, porn and illicit Viagra, along with sales of “grey” versions of real anti-virus products.

After Symantec and McAfee both put pressure on the company to stop those software sales in 2003, Innovative tried to write its own. The resulting Computershield wasn’t effective as anti-virus protection, but the company sold it anyway as a defence against the MyDoom worm. Innovative aggressively marketed its new product, and according to press reports, it was soon raking in $1 million per month. As the threat from MyDoom receded, so too did profits.

The company initially turned to adware as a new revenue source. This enabled so-called “affiliates” to use malicious web sites to silently install the adware on vulnerable Windows computers. Getting victims to visit those sites was achieved by placing what looked like legitimate adverts on real sites. Click them, and you became infected. The affiliates then pocketed a fee of 10 cents per infection, but it’s through that Innovative made between $2 and $5 from sales of the advertised products.

Meanwhile, development of completely fake anti-virus software snowballed at the company’s Kiev office. A classic example is “XP Antivirus 2008”, though it also went by a large number of pseudonyms and evolved through many versions. A video of it trashing an XP machine can be found here. Its other major names include Winfixer, WinAntivirus, Drivecleaner, and SystemDoctor.

In many ways, Innovative’s scareware was, well, innovative. It disabled any legitimate protection and told you the machine was heavily infected, even going to the trouble of creating fake blue screens of death. At the time, some antivirus companies had trouble keeping up with the rate of development.

xp2bantivirus2b2008-7843602

Attempts to access Windows internet or security settings were blocked. The only way of “cleaning” the machine was to register the software and pay the fee. Millions of people did just that. The FTC estimates that between 2004 and 2008, the company and its subsidiaries raked in $163 million.

In 2008, a hacker with the handle NeoN found a database belonging to one of the developers, revealing that in a single week one affiliate made over $158,000 from infections.

The Problem of Success
Initially, Innovative used banks in Canada to process the credit card transactions of its victims, but problems quickly mounted as disgruntled cardholders began raising chargebacks. These are claims made to credit card companies about shoddy goods or services.

With Canadian banks beginning to refuse Innovative’s business, it created subsidiary companies to hide its true identity, and approached the Bank of Kuwait and Bahrain. Trouble followed, and in 2005 this bank also stopped handling Innovative’s business due to the high number of chargebacks. Eventually, the company found a Singaporean bank called DBS Bank to handle the mounting backlog of credit card transactions.

The only solution to the chargeback problem was to keep customers happy. So, in true Allenesque style, Innovative began to invest in call centres to help customers through their difficulties. It quickly opened facilities in Ukraine, India and the USA. Operatives would talk the customers through the steps needed for the software to miraculously declare their systems free of malware. It seems that enough customers were satisfied to allow the company to keep on raking in the cash.

But people did complain, not to the company but to the authorities. The FTC received over 3,000 complaints in all and launched an investigation. Marc D’Souza has been convicted of his role in the company and ordered to pay £8.2 million, along with his father who received some of the money. The case of Kristy Ross for her part in the scam is still going through the US courts, with lawyers arguing that she was merely an employee.

Several others, including Shaileshkumar “Sam” Jain and Bjorn Daniel Sundin, are still at large, and have had a $163 million judgement entered against them in their absence. Jain and Sundin remain on the FBI’s Most Wanted Cyber Criminal list with rewards for their arrests totalling $40,000.

shaileshkumar-p-jain-3887344 bjorn-daniel-sundin-8778038

An Evergreen Scam
Scareware is a business model that rewards creativity while skirting the bounds of legality. Unlike ransomware, where criminal gangs must cover their tracks with a web of bank accounts and Bitcoin wallets, scareware can operate quite openly from countries with under-developed law enforcement and rife corruption. However, the gap between scareware and ransomware is rapidly closing.

peteris-sahurovs-in-us-federal-court-for-cybercrime-5312054Take the case of Latvian hacker Peteris Sahurovs, AKA “Piotrek” AKA “Sagade”. He was arrested on an international arrest warrant in Latvia in 2011 for his part in a scareware scam, but he fled to Poland where he was subsequently detained in 2016.

He was extradited to the US and pled guilty in February this year to making $150,000 – $200,000.  US authorities claim the total made by Sahurovs’ gang was closer to $2 million. He’s due to be sentenced in June.

According to the Department of Justice, the Sahurovs gang set up a fake advertising agency that claimed to represent a US hotel chain. Once adverts were purchased on the Minneapolis Star Tribune’s website, they were quickly swapped out for ones that infected vulnerable visitors with their malware. This made computers freeze and produce pop-ups explaining that victims needed to purchase special antivirus software to restore proper functionality. This case is interesting as it shows a clear cross over from scareware to ransomware. All data on the machines was scrambled until the software was purchased.

The level of sophistication and ingenuity displayed by scareware gangs is increasing, as is their boldness. You have probably been called by someone from India claiming to be from Microsoft, expressing concern that your computer is badly infected and offering to fix it. Or they may have posed as someone from your phone company telling you that they need to take certain steps to restore your internet connection to full health. There are many variations on the theme. Generally, they want you to download software that confirms their diagnosis. Once done, you must pay them to fix the problem. This has led to a plethora of amusing examples of playing the attackers at their own game.

It’s easy to see the people who call you as victims of poverty with no choice but to scam, but string them along for a while and the insults soon fly. They know exactly what they’re doing, and from the background chatter on such calls, so do hundreds of others. Scareware in all its forms is a crime that continues to bring in a lot of money for its perpetrators and will remain a threat for years to come.

Are you buying solid protection or snake oil?

2018q1epp-9559512
Sometimes testers need to be tested too. We’re always up for a challenge!
Latest reports now online.
How do you know which security products to buy? Many rely on independent tests to help in the decision-making process. But how do you know if a test is any good or not?
The Anti-Malware Testing Standards Organization (AMTSO) has been working to create a Standard that will give you, the customer, some assurance that the test was conducted fairly.
Earlier this year AMTSO has been trying out its Standard, which it has been working on for many months. SE Labs is proud to be involved in this initiative and the testing for this report has been assessed for compliance with the Standard.
If that sounds a bit dry, what it means is that there are experimental rules about how a tester should behave and we have put ourselves up for judgment by AMTSO.
Did participating in this process change the way we worked? Yes, but not in the technical ways that we test. Instead we turned the testing world’s business model on its head.
Many testers charge vendors money to be tested. Some will test regardless, but charge money if the vendors want to see their results before publication (and have the opportunity to make requests for corrections).
We think that the dispute process should be free for all. SE Labs has not charged any vendor for its participation in this test and we provided a free dispute process to any vendor that requested it. In this way every vendor is treated as equally as possible, for the fairest possible test.

UPDATE (10th May 2018): We are extremely proud to announce that our 2018 Q1 reports have been judged compliant (PDF) with the AMTSO Draft Standard v6.1 – 2018-05-10.

If you spot a detail in this report that you don’t understand, or would like to discuss, please contact us via our Twitter or Facebook accounts.
SE Labs uses current threat intelligence to make our tests as realistic as possible. To learn more about how we test, how we define ‘threat intelligence’ and how we use it to improve our tests please visit our website and follow us on Twitter.
Our latest reports, for enterprise, small business and home users are now available for free from our website. Please download them and follow us on Twitter and/or Facebook to receive updates and future reports.

Predictably Evil

pmr-1176337

A common criticism of computer security products is that they can only protect against known threats. When new attacks are detected and analysed security companies produce updates based on this new knowledge. It’s a reactive approach that can provide attackers with a significant window of opportunity.

It’s why anti-virus has been declared dead on more than one occasion.

Latest report now online.

Security companies have, for some years, developed advanced detection systems, often labelled as using ‘AI’, ‘machine learning’ or some other technical-sounding term. The basic idea is that past threats are analysed in deep ways to identify what future threats might look like. Ideally the result will be a product that can detect potentially bad files or behaviour before the attack is successful.

(We wrote a basic primer to understanding machine learning a couple of years ago.)

So does this AI stuff really work? Is it possible to predict new types of evil software? Certainly investors in tech companies believe so, piling hundreds of millions of funding dollars into new start-ups in the cyber defence field.

We prefer lab work to Silicon Valley speculation, though, and built a test designed to challenge the often magical claims made by ‘next-gen’ anti-malware companies.

With support from Cylance, we took four of its AI models and exposed them to threats that were seen in well-publicised attacks (e.g. WannaCry; Petya) months and even years later than the training that created the models.

It’s the equivalent of sending an old product forward in time and seeing how well it works with future threats. To find out how the Cylance AI models fared, and to discover more about how we tested, please download our report for free from our website.

Follow us on Twitter and/ or Facebook to receive updates and future reports.

Hacked! Will your anti-malware protect you from targeted attacks?

2017q4-4717048

The news isn’t good. Discover your best options in our latest reports.

Latest reports now online.

Criminals routinely create ingenious scams and indiscriminate attacks designed to compromise the unlucky and, occasionally, foolish. But sometimes they focus on a specific target rather than casting a net wide in the hope of landing something interesting.

Targeted attacks can range from basic, like an email simply asking you to send some money to an account, through to extremely devious and technical. If you received an email from your accountant with an attached PDF or Excel spreadsheet would you open it?

Most would and all that then stands between them and a successful hack (because the email was a trick and contained a dodgy document that gives remote control to the attacker) is the security software running on their PC.

In this test we’ve included indiscriminate, public attacks that come at victims from the web and via email, but we’ve also included some devious targeted attacks to see how well-protected potential victims would be.

We’ve not created any new types of threat and we’ve not discovered and used ‘zero day’ attacks. Instead we took tools that are freely distributed online and are well-known to penetration testers and criminals alike. We used these to generate threats that are realistic representations of what someone could quite easily put together to attack you or your business.

The results are extremely worrying. While a few products were excellent at detecting and protecting against these threats many more were less useful. We will continue this work and report any progress that these companies make in improving their products.

Our latest reports, for enterprise, small business and home users are now available for free from our website. Please download them and follow us on Twitter and/or Facebook to receive updates and future reports.

100% Certifiable

food-hygiene-ratings-5244024

Whether you’re in the market for a car, hamburger or computer security product, certifications are useful. They don’t tell you how smooth the car drives, how tasty the sandwich is or how completely accurate the anti-virus software will be, but certifications indicate a general level of competence.

Latest reports now online.

In the UK new cars must be certified by the Vehicle Certification Agency (VCA), restaurants are checked for hygiene by the Food Standards Agency (FSA) and various independent testing organisations, including SE Labs, test IT security products for basic functionality.

A certification emphatically does not indicate the overall quality of a product, though. The FSA specifically states that, “The food hygiene rating is not a guide to food quality.” In other words, the food won’t make you ill, but you might not like it! Similarly, the VCA cares more about cars being made according to specification rather than how nice they look.

SE Labs has a range of available testing services. We consider certification to be the most basic type of testing. If a product claims to be able to detect malware then we can test that, but we don’t claim it can detect all types. For a higher level of understanding about a product’s capabilities so-called ‘real-world’ testing is necessary.

The report you are reading now is based on our more advanced testing, which exposes real products to live threats in a realistic environment, running on real computers on an internet-connected network.

But how can you be sure that we’re really doing that, and not just making up the figures or giving some products an unfair advantage? After all, some companies contribute financially to supporting the tests, while others do not.

To go some way to addressing this concern, as well as to improve generally and continue to evolve the business, SE Labs has achieved ISO 9001:2015 certification for “The Provision of IT Security Product Testing”. We think it’s fair for the testers to be tested and we’re very proud to have passed!
If you spot a detail in this report that you don’t understand, or would like to discuss, please contact us via our Twitter or Facebook accounts.

SE Labs uses current threat intelligence to make our tests as realistic as possible. To learn more about how we test, how we define ‘threat intelligence’ and how we use it to improve our tests please visit our website and follow us on Twitter.

Our latest reports, for enterprise, small business and home users are now available for free from our website. Please download them and follow us on Twitter and/or Facebook to receive updates and future reports.

Can anti-malware be 100 per cent effective?

2017q2-4940659

You can probably guess the answer, but we’ll explore how products can score very well in tough tests, and which are the best.

Latest reports now online

There are a lot of threats on the web, and going online without protection is very risky. We need good, consistently effective anti-malware products to reduce our risk of infection.

And the ones included in these reports look great – in fact, some score 100 per cent. That means they stopped all the threats that we exposed them to and didn’t block anything legitimate.

But wait a minute! Those in the security industry know full well that there is no such thing as 100 per cent security. There is always a way past every security measure, and this is as true in the anti-malware world as with any other measures for threat protection.

This test includes some of the very best anti-malware products in the world, and pits them against prevalent threats, be they ones that affect hundreds of thousands of users worldwide, or those that could be used to target individuals and organisations. It’s a tough test, but a fair one.

You could argue that any anti-malware product worth its salt would score 100 per cent or thereabouts.

Products can score 100 per cent in our tests because we’re not choosing thousands of weird and wonderful rare pieces of malware to test. Regular users are extremely unlikely to encounter those in the real world.

We’re looking at the threats that could affect you.

Our mission is to help improve computer security through testing, both publicly and privately. We also want to help customers choose the best products by publishing some of those test results.

But don’t forget that success today is not a guarantee of success tomorrow. It’s important to keep monitoring test results.

Our latest reports, for enterprise, small business and home users are now available for free from our website. Please download them and follow us on Twitter and/or Facebook to receive updates and future reports.

Testing anti-malware’s protection layers

layers-3683923

Our first set of anti-malware test results for 2017 are now available.

Endpoint security is an important component of computer security, whether you are a home user, a small business or running a massive company. But it’s just one layer.

Latest reports now online

Using multiple layers of security, including a firewall, anti-exploit technologies built into the operating system and virtual private networks (VPNs) when using third-party WiFi is very important too.

What many people don’t realise is that anti-malware software often actually contains its own different layers of protection. Threats can come at you from many different angles, which is why security vendors try to block and stop them using a whole chain of approaches.

A fun video we created to show how anti-malware tries to stop threats in different ways

How layered protection works

For example, let’s consider a malicious website that will infect victims automatically when they visit the site. Such ‘drive-by’ threats are common and make up about one third of this test’s set of attacks. You visit the site with your web browser and it exploits some vulnerable software on your computer, before installing malware – possibly ransomware, a type of malware that also features prominently in this test.

browser-8901457

Here’s how the layers of endpoint security can work. The URL (web link) filter might block you from visiting the dangerous website. If that works you are safe and nothing else need be done.

But let’s say this layer of security crumbles, and the system is exposed to the exploit.

toaster-9827773Maybe the product’s anti-exploit technology prevents the exploit from running or, at least, running fully? If so, great. If not, the threat will likely download the ransomware and try to run it.

At this stage file signatures may come into play. Additionally, the malware’s behaviour can be analysed. Maybe it is tested in a virtual sandbox first. Different vendors use different approaches.

Ultimately the threat has to move down through a series of layers of protection in all but the most basic of ‘anti-virus’ products.

The way we test endpoint security is realistic and allows all layers of its protection to be tested.

Our latest reports, for enterprisesmall business and home users are now available for free from our website. Please download them and follow us on Twitter and/or Facebook to receive updates and future reports.

Inside the CIA…

cia-ioc-9786148

Who is behind the CIA’s hacking tools? Surprisingly ordinary geeks, it seems.

At the start of March came the first part of yet another Wikileaks document dump, this time detailing the CIA’s hacking capabilities. The world suddenly feared spooks watching them through their TVs and smartphones. It all made for great headlines.

The Agency has developed scores of interesting projects, not to mention a stash of hitherto unknown zero day vulnerabilities. The dump also gives notes on how to create well-behaved, professional malware that stands the least chance of detection, analysis and attribution to Langley. We’ve also learned some useful techniques for defeating antivirus software, which the Agency calls Personal Security Products (PSPs).

There’s also a deeper tale to tell. It’s about the personalities behind the redacted names working on these tools and techniques. They don’t seem so different from anyone else working in infosec.

User #524297 says he is a “Coffee addict, Connoisseur of International Barbecues, and Varied Malt Beverage Enthusiast.” Thanks to his comments, we know an ex-boss (nicknamed “Panty-Raider”) was considered “really odd”. Another had a large, carved wooden desk that went with him from job to job.

User #524297 also maintains a page dedicated to some interesting ideas. One is to use the OpenDNS DNSCrypt service to hide DNS requests emanating from a compromised host.

Another fun-loving User is #71473. He has a page called “List of ideas for fun and interesting ways to kill/crash a process“, which enumerates a dozen homebrew techniques and variations. Most are still at the concept stage, but under the list of uses to which they may be put, he includes “Knockover (sic) PSPs” and “Troll people”.

He also describes several proof-of-concept tools for his process crashing techniques. One is called DisorderlyShutdown, which waits a programmable amount of time (plus a random offset to make things seem natural) to select a random process to crash in the hope of leading to “data loss and gnashing of teeth”. Another is WarheadsToForeheads, which attempts to crash processes. About this tool, he says: “Considering making this an infinite enumeration to squash all user processes and make the user experience especially horrific.”

Revealingly, User #71473 also likes to hack the home pages of other Users: ” Its 11:30… time to deface people’s unprotected user pages…”

User #11628962 was deeply impressed by Subramaniam and Hunt’s “Practices of an Agile Developer”, and went to great lengths to enumerate the principles behind the work for others in his group. 

Meanwhile, we learn that User # 71475 loves to listen to music online and lists several streaming services and YouTube channels. He’s also an avid collector of ASCII-based emoticons. Everyone needs a hobby, right? ¯_(ツ)_/¯

Amusingly, User #20873595 is keen for people understand that his last name does not begin with C, implying that it is in fact Hunt. There was also some debate about what User #72907’s office nickname should be. “Monster Lite” was the apparent front runner.

hearthstone_screenshot-1364834

We also learned from the dump that some of the Users are heavily into the online card game Hearthstone, which unfriendly foreign state actors are likely now feverishly trying to hack.

The public at large has moved on, and the first of the vulnerabilities highlighted in the dump has been patched, but the industrious CIA hackers who originally found them are still beavering away, creating new tools to replace the old ones, finding new zero-days, thinking up new nicknames, trolling each other, and of course playing Hearthstone.

About

SE Labs Ltd is a private, independently-owned and run testing company that assesses security products and services. The main laboratory is located in Wimbledon, South London. It has excellent local and international travel connections. The lab is open for prearranged client visits.

Contact

SE Labs Ltd
Hill Place House
55A High Street
Wimbledon
SW19 5BA

020 3875 5000

info@selabs.uk

Press