Posted by Virus Bulletin on Jul 19, 2007
Interactive system rated best blocker in questioned survey.
A study of anti-spam systems, using a bespoke scale to rate protection offered, has found the challenge-response method most effective, with a massive lead over managed service systems, rated second place, and with ISP-based filters trailing far behind the rest of the field. However, with the results receiving widespread coverage, both the methodology of the study and the neutrality of the researchers has been called into question by anti-spam experts.
The study was carried out by research and consultancy firm Brockmann & Company, who developed a 'spam index' system to rate the effectiveness of spam filters by measuring factors such as the amount of spam hitting inboxes and the amount of time spent dealing with each unblocked spam and each false positive. Their system gave challenge-response systems a rating of 160, with hosted services such as MessageLabs and Postini rated at 316, appliance solutions 349, software-based gateway filters 366, real-time black-listing 367 and ISP-based filtering worst at 442.
The surprising results have picked up considerable media attention, but have been questioned by some commentators, including blogger and SpamAssassin developer Justin Mason, who criticised the relative importance given to false positives, which were given about the same weight in the study as unblocked spam, rather than being counted as a much more significant problem as they are by many spam analysts. He also points out apparent links between the head of the research firm behind the tests and challenge-response vendor Sendio, and discusses the problems of extra traffic created by challenge-response systems contacting non-existent or unconnected addresses to query spam origins.
Mason's detailed analysis of the study is here, while a lengthy press release from Brockmann & Company is carried here and the full study can be accessed (after a registration process) here.
Posted on 19 July 2007 by Virus Bulletin
