Vulnerability test raises hackles

Posted by   Virus Bulletin on   Oct 17, 2008

Secunia suite trial slates lack of PoC detection, but test methods called into question.

Vulnerability specialist Secunia published the results of a trial of internet security suites this week, with stark findings showing very poor detection rates of the selection of exploits and proof-of-concept overruns used. However, the test has run into considerable criticism from several angles.

Secunia's test focused on 300 known vulnerabilities, around half of them based on file types and the other half based on web pages. These were represented mostly by payload-free proof-of-concept attacks designed to trigger vulnerabilities, and in some cases prove that code execution could be achieved, while some fully fledged exploits were also included. A dozen 'Internet Security' suites were lined up and set to scan the files and web pages in question, and of them only Symantec's Norton product put in any kind of a performance, achieving just over 20% of all samples and just over 30% of those deemed 'important'. No other product detected more than 2.5% of the full testset, or better than 4% of the most dangerous items.

In the wake of the test's publication, a range of criticisms have been levelled at Secunia, mostly focusing on the test methodology, which many claim failed to properly exercise any of the protection features included in these suites beyond the basic anti-virus scanner. Many product developers have claimed that if the vulnerabilities had actually been triggered, rather than scanned statically, other technologies would have kicked in to block any resulting exploits, and that in many cases payloads would be more likely to be blocked than the first stages of exploitation.

Others have criticised the lack of clarity and openness about how the testing was carried out, and some have seen the whole business as a stunt to promote Secunia's own vulnerability scanning products and services. A blogger at Kaspersky Lab points out that it is a far better policy to patch vulnerabilities as soon as they are discovered, rather than leaving holes open and relying on other software to block anything that tries to wriggle through, an angle which Kaspersky's integrated vulnerability scanner, like those offered by eEye Blink and BitDefender, attempts to cover. Secunia representatives themselves claim, in a blog entry announcing their results, that there is a serious lack of proper and timely patching among home users and businesses alike.

Full details of the Secunia test are published in a PDF here, with some comment from the company here. Further comment is in The Register here, on the Sunbelt blog here or at Panda here.

Posted on 17 October 2008 by Virus Bulletin



Latest posts:

New paper: Behind the scenes of GandCrab's operation

The GandCrab ransomware regularly updated itself to newer versions to stay ahead of decryptors released by security researchers, and regularly included taunts, jokes and references to security organizations in its code. In a new paper, the AhnLab…

VB2019 paper: King of the hill: nation-state counterintelligence for victim deconfliction

At VB2019 Juan Andres Guerrero-Saade looked at nation-state actors using threat intelligence for victim deconfliction. Today we publish both his paper and the recording of his presentation.

The VB2020 call for papers - how it works

With the VB2020 Call for Papers now open, we explain how the selection procedure works, which may help you during your abstract submission.

VB2019 presentation: Targeted attacks through ISPs

In 2019 we saw a rise in the number of targeted malware infections spread via ISPs and service providers. In a last-minute paper presented at VB2019 in London, Kaspersky researcher Denis Legezo discussed the details of a number of such cases. Today…

VB2019 presentation: A deep dive into iPhone exploit chains

In a last-minute presentation at VB2019 in London, John Bambenek of the University of Illinois at Urbana-Champaign discussed details of campaigns that used advanced iOS and Android exploit chains against China’s Uighur minority. Today we release the…

We have placed cookies on your device in order to improve the functionality of this site, as outlined in our cookies policy. However, you may delete and block all cookies from this site and your use of the site will be unaffected. By continuing to browse this site, you are agreeing to Virus Bulletin's use of data as outlined in our privacy policy.