Testing patience

Posted by   Virus Bulletin on   Sep 1, 2006

Consumer Reports does it again.

After having come in for a great deal of criticism in recent weeks over its AV testing methodology (see VB, September 2006, p.2), Consumer Reports has - amazingly - damaged its credibility further after having confirmed that, during its testing of anti-spyware applications, CR did not test against any spyware.

CR's review of anti-spyware products was based on running the applications against the Spycar set of applications that mimic spyware behaviour. While it would be understandable (if not entirely forgivable) that testers lacking experience in the anti-malware field could make such a gaffe, what makes this more astounding is that the creators of Spycar state clearly and specifically that Spycar is not suitable (as a sole test method) for anti-spyware testing.

The EULA states '...Spycar ... is intended to be used to see how anti-spyware tools cope with new spyware for which they didn't have a signature. It is not intended to provide perfect anti-spyware tests, or to act as a substitute for any other form of evaluation.' And the Spycar website reads: 'Is Spycar a comprehensive test of anti-spyware tools? No ... Spycar does not evaluate the signature base, the user interface, and other vital aspects of an anti-spyware tool. Thus, Spycar alone cannot be used to determine how good or bad an anti-spyware product is.'

After such a controversial performance CR will need to work hard if it is to recoup its credibility in the anti-malware field.

Posted on 01 September 2006 by Virus Bulletin

 Tags

twitter.png
fb.png
linkedin.png
hackernews.png
reddit.png

 

Latest posts:

VBSpam tests to be executed under the AMTSO framework

VB is excited to announce that, starting from the Q3 test, all VBSpam tests of email security products will be executed under the AMTSO framework.

In memoriam: Prof. Ross Anderson

We were very sorry to learn of the passing of Professor Ross Anderson a few days ago.

In memoriam: Dr Alan Solomon

We were very sorry to learn of the passing of industry pioneer Dr Alan Solomon earlier this week.

New paper: Nexus Android banking botnet – compromising C&C panels and dissecting mobile AppInjects

In a new paper, researchers Aditya K Sood and Rohit Bansal provide details of a security vulnerability in the Nexus Android botnet C&C panel that was exploited in order to gather threat intelligence, and present a model of mobile AppInjects.

New paper: Collector-stealer: a Russian origin credential and information extractor

In a new paper, F5 researchers Aditya K Sood and Rohit Chaturvedi present a 360 analysis of Collector-stealer, a Russian-origin credential and information extractor.

We have placed cookies on your device in order to improve the functionality of this site, as outlined in our cookies policy. However, you may delete and block all cookies from this site and your use of the site will be unaffected. By continuing to browse this site, you are agreeing to Virus Bulletin's use of data as outlined in our privacy policy.