2010-07-01
Abstract
‘Some of the new testing labs that have appeared recently mimic the tactics of rogue AV products.’ Costin Raiu, Kaspersky Lab
Copyright © 2010 Virus Bulletin
Recently, I was sitting with some colleagues, discussing everybody’s favourite subject (OK, second favourite subject after the malware naming mess): the state of AV testing. During the discussion, somebody brought up the name of a new, obscure testing organization in the Far East. Nobody else had ever heard of it, so my colleague jokingly dubbed it a ‘rogue Andreas Marx’.
It then occurred to us that some of the new testing labs that have appeared recently mimic the tactics of rogue AV products. As we know, the rogue AV business model is based on selling a false sense of security; we professionals know it is fake, but the victims don’t. People buy a rogue AV product because they believe it will solve their security problems, but at best the products don’t do anything, and at worst, they install additional malware.
Rogue AV testers display similar behaviour. In this case, the business model is not based on a false sense of security, but instead on a false sense of insecurity. So, how do they operate? Well, it seems to start with a number of tests which look legitimate, and which mimic real-world conditions. The tests then slowly become more ‘complicated’, and security products perform increasingly poorly. Finally, the main idea emerges: that all security products are useless.
Hence, the false sense of insecurity is promoted through the tests: you are insecure, the money you paid for AV software was misspent. Rogue AV testers also often fail to disclose product names in published test results and attempt to sell their results for significant sums of money.
The following are some characteristics we identified as being specific to rogue AV testers:
They are not affiliated with any serious testing organization, such as AMTSO. Rogue AV testers may also show fake affiliations or even falsely display, say, the AMTSO logo on their website.
They publish free public reports, but charge for the ‘full’ reports. In general, the public reports are made to look as bad as possible for all participants, to maximize the profits from selling the full reports.
The public reports are full of charts that look complicated and clever, but which sometimes reveal amusing mistakes. Although exact numbers are not usually available, the charts can provide useful information about the errors in the tests.
They claim that all AV products are useless. This is the foundation stone of any business based on the ‘false sense of insecurity’.
They charge (usually large sums of money) for samples and methodology to make sure the flawed methodology and samples cannot be reviewed externally. Reputable testers will make samples and methodology freely available to the developers of the products they test.
Should a company or individual agree to pay the large sums to obtain the methodology, the fees escalate, revealing new, previously hidden costs. The main idea here is that the rogue AV testers do not want to provide access to samples and methodology, because it would reveal gross errors in their tests – by escalating their prices they hope that many will be deterred or prevented from accessing them.
There are other characteristics, but I think everybody gets the point.
Just as rogue AV products exploded and became one of the most profitable categories of crimeware, I suspect rogue AV testers will follow. In the process, they will also become extremely profitable and have a negative impact on the industry.
So, if you are trying to compare security solutions, I recommend sticking to established testing organizations such as Virus Bulletin, AV-test.org and AV-Comparatives or reputable magazines with a good history behind them.
Do not become a victim of the rogue AV testers!
