VB100 certification report - December 2020

VB Test Team

Virus Bulletin

Copyright © 2020 Virus Bulletin

Introduction

The VB100 certification scheme provides a stamp of quality and competence for anti-malware products that satisfy a minimum standard of detecting malicious executables that have recently been seen in the wild, while blocking few to no legitimate programs.

This report details the VB100 certification results of 41 of such products from 36 different vendors during November and December 2020.

The VB100 set-up

In the VB100 test, a copy of the product to be tested is installed on two platforms: Windows 10 and Windows 7. On each platform, and at three different times in the test, the product is asked to scan both the latest version of the WildList¹ and a selection of clean files taken from Virus Bulletin’s own set of files belonging to widely used legitimate software.

A legitimate file that is blocked at least once is considered a false positive, while a WildList file that isn’t blocked is considered a miss.

A product achieves a VB100 certification if:

• No more than 0.5% of WildList samples are missed

and

• No more than 0.01% of legitimate files are blocked

For full details, we refer to the VB100 methodology on the Virus Bulletin website: https://www.virusbulletin.com/testing/vb100/vb100-methodology/vb100-methodology-ver1-1/. This test used version 1.1 of the VB100 methodology.

Diversity Test

The malware part of the VB100 certification uses the WildList, a regularly updated list of extremely well-vetted malware samples, guaranteed to have been spotted in the wild multiple times. This makes them very suitable for a certification test like VB100.

The ‘Diversity Test’ looks at products’ detection of another set of recent malware samples, to acknowledge the fact that products detect malware samples beyond a standard set of samples, and provides a measure of that detection.

Upcoming test changes

As part of planned updates to the VB100 test in 2021, we will be retiring testing on the legacy Windows 7 platform, effective from January 2021.

We are introducing this change because the relevance of Windows 7 has diminished greatly in recent years, and the platform reaches end-of-life in January 2020 – as a consequence of which, a growing number of tested products either lack support for this platform, or struggle to perform properly on the legacy operating system. We expect the retirement to have negligible impact on the relevance of the VB100 reports.

Products & results

Products were allowed to download updates during the course of the test. The version numbers listed in the results that follows refer to those at the start of the test.

Acronis Cyber Protect

Windows 7 version	15.0.24600
Windows 10 version	15.0.24600
WildList detection	100.0%
False positive rate	0.000%
Diversity Test rate	99.90%

Acronis True Image 2021

Windows 7 version	Version 2021 build 32010
Windows 10 version	Version 2021 build 32010
WildList detection	99.9%
False positive rate	0.000%
Diversity Test rate	99.80%

Adaware Antivirus Free

Windows 7 version	12.10.55.0
Windows 10 version	12.10.55.0
WildList detection	99.9%
False positive rate	0.000%
Diversity Test rate	99.50%

Adaware Antivirus Pro

Windows 7 version	12.10.55.0
Windows 10 version	12.10.55.0
WildList detection	99.9%
False positive rate	0.000%
Diversity Test rate	99.50%

Ad Spider

Windows 7 version	2020.11.10
Windows 10 version	2020.11.10
WildList detection	99.9%
False positive rate	0.000%
Diversity Test rate	99.50%

AhnLab V3 Endpoint Security

Windows 7 version	9.0.63.3 (b 1614)
Windows 10 version	9.0.63.3 (b 1614)
WildList detection	100.0%
False positive rate	0.000%
Diversity Test rate	100.00%

Arcabit AntiVirus

Windows 7 version	2020.11.10
Windows 10 version	2020.11.10
WildList detection	100.0%
False positive rate	0.000%
Diversity Test rate	99.80%

Avast Free Antivirus

Windows 7 version	20.9.2437
Windows 10 version	20.8.2432
WildList detection	100.0%
False positive rate	0.000%
Diversity Test rate	100.00%

AVG Internet Security

Windows 7 version	20.9.3152
Windows 10 version	20.8.3147
WildList detection	100.0%
False positive rate	0.000%
Diversity Test rate	100.00%

CMC Malware Detection and Defense

Windows 7 version	v1.8.2020 build 80
Windows 10 version	v1.8.2020 build 80
WildList detection	100.0%
False positive rate	0.000%
Diversity Test rate	100.00%

CORE Antivirus

(see notes in Appendix 2)

Windows 7 version	N/A
Windows 10 version	1.0.57.0
WildList detection	100.0%
False positive rate	0.000%
Diversity Test rate	100.00%

Cynet 360

Windows 7 version	5.4
Windows 10 version	5.4
WildList detection	100.0%
False positive rate	0.000%
Diversity Test rate	100.00%

CyRadar Endpoint Detection and Response

Windows 7 version	1.0.0.56
Windows 10 version	1.0.0.56
WildList detection	100.0%
False positive rate	0.000%
Diversity Test rate	99.70%

Defenx Security Suite

Windows 7 version	1.7.3.1
Windows 10 version	1.7.3.1
WildList detection	99.9%
False positive rate	0.000%
Diversity Test rate	100.00%

Emsisoft Anti-Malware

Windows 7 version	2020.11.0.10501
Windows 10 version	2020.11.0.10501
WildList detection	100.0%
False positive rate	0.000%
Diversity Test rate	99.80%

eScan Internet Security Suite for Windows

Windows 7 version	14.0.1400.2228
Windows 10 version	14.0.1400.2228
WildList detection	100.0%
False positive rate	0.000%
Diversity Test rate	99.80%

ESTsecurity ALYac

Windows 7 version	4.0.2.23116
Windows 10 version	4.0.2.23116
WildList detection	99.9%
False positive rate	0.000%
Diversity Test rate	99.50%

Exosphere Endpoint Protection

Windows 7 version	1.5.22.1
Windows 10 version	1.5.22.1
WildList detection	100.0%
False positive rate	0.000%
Diversity Test rate	100.00%

Faronics Anti-Virus

Windows 7 version	4.21.3102.484
Windows 10 version	4.21.3102.484
WildList detection	100.0%
False positive rate	0.000%
Diversity Test rate	99.80%

FireEye Endpoint Security

Windows 7 version	32.30.13
Windows 10 version	32.30.13
WildList detection	99.9%
False positive rate	0.000%
Diversity Test rate	99.50%

Fortinet FortiClient

Windows 7 version	6.2.7.0984
Windows 10 version	6.2.7.0984
WildList detection	100.0%
False positive rate	0.000%
Diversity Test rate	100.00%

G DATA Antivirus

Windows 7 version	25.5.8.14
Windows 10 version	25.5.8.14
WildList detection	100.0%
False positive rate	0.000%
Diversity Test rate	100.00%

IKARUS anti.virus

Windows 7 version	3.2.4
Windows 10 version	3.2.4
WildList detection	100.0%
False positive rate	0.000%
Diversity Test rate	100.00%

Intego AV

Windows 7 version	1.0.1.7
Windows 10 version	1.0.1.7
WildList detection	100.0%
False positive rate	0.000%
Diversity Test rate	100.00%

K7 Total Security

Windows 7 version	16.0.0658
Windows 10 version	16.0.0658
WildList detection	100.0%
False positive rate	0.000%
Diversity Test rate	100.00%

PCProtect

Windows 7 version	5.5.83
Windows 10 version	5.5.83
WildList detection	100.0%
False positive rate	0.000%
Diversity Test rate	100.00%

Private Internet Antivirus

Windows 7 version	1.0.1.6
Windows 10 version	1.0.1.6
WildList detection	100.0%
False positive rate	0.000%
Diversity Test rate	100.00%

Qi-ANXIN Tianqing Endpoint Security Management System

Windows 7 version	6.6.0.4086
Windows 10 version	6.6.0.4086
WildList detection	100.0%
False positive rate	0.000%
Diversity Test rate	99.80%

Rising Enterprise Security Management System

Windows 7 version	3.0.97
Windows 10 version	3.0.97
WildList detection	100.0%
False positive rate	0.001%
Diversity Test rate	99.10%

Scanguard

Windows 7 version	5.5.83
Windows 10 version	5.5.83
WildList detection	100.0%
False positive rate	0.000%
Diversity Test rate	100.00%

SecureAge SecureAPlus Pro

Windows 7 version	6.4.0
Windows 10 version	6.4.0
WildList detection	99.5%
False positive rate	0.010%
Diversity Test rate	100.00%

Systweak Anti-virus

(see notes in Appendix 2)

Windows 7 version	N/A
Windows 10 version	1.0.1000.10853
WildList detection	100.0%
False positive rate	0.000%
Diversity Test rate	99.90%

TACHYON Endpoint Security

Windows 7 version	5.0.1.33
Windows 10 version	5.0.1.33
WildList detection	99.9%
False positive rate	0.001%
Diversity Test rate	99.50%

TeamViewer Endpoint Protection

Windows 7 version	20.9.1
Windows 10 version	20.9.1
WildList detection	99.9%
False positive rate	0.000%
Diversity Test rate	99.50%

Tencent PC Manager

Windows 7 version	12.3.26609.901
Windows 10 version	12.3.26609.901
WildList detection	100.0%
False positive rate	0.000%
Diversity Test rate	99.80%

TotalAV

Windows 7 version	5.5.83
Windows 10 version	5.5.83
WildList detection	100.0%
False positive rate	0.000%
Diversity Test rate	100.00%

Total Defense Premium

Windows 7 version	12.0.0.298 SP3
Windows 10 version	12.0.0.298
WildList detection	99.9%
False positive rate	0.000%
Diversity Test rate	99.50%

TUXGUARD Endpoint Protection

(see notes in Appendix 2)

Windows 7 version	N/A
Windows 10 version	1.0.1.60
WildList detection	100.0%
False positive rate	0.000%
Diversity Test rate	N/A

United Endpoint Protector

(see notes in Appendix 2)

Windows 7 version	N/A
Windows 10 version	1.2
WildList detection	100.0%
False positive rate	0.000%
Diversity Test rate	100.00%

VIPRE Endpoint Cloud Business

Windows 7 version	12.0.7874
Windows 10 version	12.0.7874
WildList detection	99.9%
False positive rate	0.000%
Diversity Test rate	99.80%

VirIT eXplorer PRO

Windows 7 version	9.3
Windows 10 version	9.3
WildList detection	100.0%
False positive rate	0.000%
Diversity Test rate	71.50%

Appendix 1: products not certified

All products achieved VB100 certification in this test.

Appendix 2: testing notes

• CORE Antivirus gained VB100 certification based on measurements taken on Windows 10 only.
• Systweak Anti-virus gained VB100 certification based on measurements taken on Windows 10 only.
• Tabidus Technology’s United Endpoint Protector gained VB100 certification based on measurements taken on Windows 10 only.
• TUXGUARD Endpoint Protection gained VB100 certification based on measurements taken on Windows 10 only and, due to a technical failure, the Diversity Test results for this product were invalidated.

Appendix 3: sample set sizes

The Certification Set contained 1,401 malicious samples. The set of clean samples used for the false positive test contained 100,000 files, of which 29,168 were portable executable (PE) files. The set used for the Diversity Test contained 1000 malicious samples.

Footnotes

¹ The WildList is an extremely well-vetted set of malware recently observed in the wild by researchers: http://www.wildlist.org/.

Latest reviews: