VB100 certification report - June 2020

VB Test Team

Virus Bulletin

Copyright © 2020 Virus Bulletin

---

 

Introduction

Though in the ever-changing threat landscape anti-malware products have long ceased to be the only thing needed to keep devices secure, they remain as important as ever, as both a first and a last line of defence: to scan potentially malicious files before they make it onto a device, and to block them from running if all other defences have failed.

For more than two decades, the VB100 certification scheme has provided an easy to recognize (but not always easy to obtain) award that affirms that a product satisfies the minimum requirements one should expect of an anti-malware product.

This report details the VB100 certification results of 44 anti‑malware products from 38 different vendors during May and June 2020.

 

The VB100 set-up

In the VB100 test, a copy of the product to be tested is installed on two platforms: Windows 10 and Windows 7. On each platform, and at three different times in the test, the product is asked to scan both the latest version of the WildList¹ and a selection of clean files taken from Virus Bulletin’s own set of files belonging to widely used legitimate software.

A legitimate file that is blocked at least once is considered a false positive, while a WildList file that isn’t blocked is considered a miss.

A product achieves a VB100 certification if:

• No more than 0.5% of WildList samples are missed

and

• No more than 0.01% of legitimate files are blocked

For full details, we refer to the VB100 methodology on the Virus Bulletin website: https://www.virusbulletin.com/testing/vb100/vb100-methodology/vb100-methodology-ver1-1/. This test used version 1.1 of the VB100 methodology.

 

Diversity Test

The malware part of the VB100 certification uses the WildList, a regularly updated list of extremely well-vetted malware samples, guaranteed to have been spotted in the wild multiple times. This makes them very suitable for a certification test like VB100.

The ‘Diversity Test’ looks at products’ detection of another set of recent malware samples, to acknowledge the fact that products detect malware samples beyond a standard set of samples, and provides a measure of that detection.

 

Products & results

Products were allowed to download updates during the course of the test. The version numbers listed in the results that follows refer to those at the start of the test.

 

Acronis Cyber Protect

Windows 7 version	12.5.22410
Windows 10 version	12.5.22410
WildList detection	100.0%
False positive rate	0.000%
Diversity Test rate	98.35%

 

Adaware Antivirus Free

Windows 7 version	12.7.1055.0
Windows 10 version	12.7.1055.0
WildList detection	99.7%
False positive rate	0.000%
Diversity Test rate	98.66%

 

Adaware Antivirus Pro

Windows 7 version	12.7.1055.0
Windows 10 version	12.7.1055.0
WildList detection	99.7%
False positive rate	0.000%
Diversity Test rate	99.59%

 

Ad-Spider

Windows 7 version	2020.05.04 - 7.0.84460
Windows 10 version	2020.05.04 - 7.0.84460
WildList detection	99.7%
False positive rate	0.000%
Diversity Test rate	99.59%

 

AhnLab V3 Endpoint Security

Windows 7 version	9.0.61.6
Windows 10 version	9.0.61.6
WildList detection	100.0%
False positive rate	0.000%
Diversity Test rate	99.79%

 

Arcabit AntiVirus

Windows 7 version	2020.05.06
Windows 10 version	2020.05.05
WildList detection	100.0%
False positive rate	0.000%
Diversity Test rate	100.00%

 

Avast Free Antivirus

Windows 7 version	20.3.2405
Windows 10 version	20.3.2405
WildList detection	100.0%
False positive rate	0.000%
Diversity Test rate	99.69%

 

AVG Internet Security

Windows 7 version	20.3.3120
Windows 10 version	20.3.3120
WildList detection	100.0%
False positive rate	0.000%
Diversity Test rate	99.69%

 

BitSentry Elite

Windows 7 version	1.0.5.255
Windows 10 version	1.0.5.255
WildList detection	100.0%
False positive rate	0.000%
Diversity Test rate	98.55%

 

CORE Antivirus

(See notes in Appendix 2)

Windows 7 version	1.0.20.0
Windows 10 version	1.0.20.0
WildList detection	99.9%
False positive rate	0.000%
Diversity Test rate	100.00%

 

Cynet 360

(See notes in Appendix 2)

Windows 7 version	8.3.60.18
Windows 10 version	8.3.60.18
WildList detection	N/A
False positive rate	N/A
Diversity Test rate	N/A

 

Defenx Security Suite

(See notes in Appendix 2)

Windows 7 version	1.6.1.3
Windows 10 version	1.6.1.3
WildList detection	N/A
False positive rate	N/A
Diversity Test rate	N/A

 

Emsisoft Anti-Malware

Windows 7 version	2020.5.0.10149
Windows 10 version	2020.5.0.10149
WildList detection	99.9%
False positive rate	0.000%
Diversity Test rate	100.00%

 

eScan Internet Security Suite for Windows

Windows 7 version	14.0.1400.2228
Windows 10 version	14.0.1400.2228
WildList detection	100.0%
False positive rate	0.000%
Diversity Test rate	100.00%

 

ESET Endpoint Security

Windows 7 version	7.1.2053.0
Windows 10 version	7.1.2053.0
WildList detection	100.0%
False positive rate	0.000%
Diversity Test rate	100.00%

 

ESET Internet Security

Windows 7 version	13.1.21.0
Windows 10 version	13.1.23.0
WildList detection	100.0%
False positive rate	0.000%
Diversity Test rate	100.00%

 

ESET Smart Security Premium

Windows 7 version	13.1.21.0
Windows 10 version	13.1.21.0
WildList detection	100.0%
False positive rate	0.000%
Diversity Test rate	100.00%

 

ESTsecurity ALYac

Windows 7 version	4.0.2.23116
Windows 10 version	4.0.2.23116
WildList detection	99.7%
False positive rate	0.000%
Diversity Test rate	99.59%

 

Exosphere Endpoint Protection

(See notes in Appendix 1)

Windows 7 version	8.3.60.18
Windows 10 version	8.3.60.18
WildList detection	N/A
False positive rate	N/A
Diversity Test rate	N/A

 

Faronics Anti-Virus

Windows 7 version	4.21.3102.484
Windows 10 version	4.21.3102.484
WildList detection	100.0%
False positive rate	0.000%
Diversity Test rate	100.00%

 

FireEye Endpoint Security

Windows 7 version	31.28.8
Windows 10 version	31.28.8
WildList detection	99.6%
False positive rate	0.000%
Diversity Test rate	99.59%

 

Fortinet FortiClient

Windows 7 version	6.2.3.0912
Windows 10 version	6.2.3.0912
WildList detection	100.0%
False positive rate	0.000%
Diversity Test rate	98.97%

 

G DATA Antivirus

Windows 7 version	25.5.6.20
Windows 10 version	25.5.6.20
WildList detection	100.0%
False positive rate	0.000%
Diversity Test rate	99.90%

 

IKARUS anti.virus

Windows 7 version	3.1.16
Windows 10 version	3.1.16
WildList detection	100.0%
False positive rate	0.001%
Diversity Test rate	99.17%

 

Intego AV

Windows 7 version	0.3.1.1
Windows 10 version	0.3.1.1
WildList detection	99.9%
False positive rate	0.000%
Diversity Test rate	99.90%

 

ITL Total Security

(See notes in Appendix 2)

Windows 7 version	1.0.0.15
Windows 10 version	1.0.0.15
WildList detection	N/A
False positive rate	N/A
Diversity Test rate	N/A

 

K7 Total Security

Windows 7 version	16.0.0637
Windows 10 version	16.0.0636
WildList detection	100.0%
False positive rate	0.000%
Diversity Test rate	99.07%

 

PCProtect

(See notes in Appendix 2)

Windows 7 version	5.5.83
Windows 10 version	5.5.83
WildList detection	N/A
False positive rate	N/A
Diversity Test rate	N/A

 

Qi-ANXIN Tianqing Endpoint Security Management System

Windows 7 version	v10.0 build: 6.6.0.4084
Windows 10 version	v10.0 build: 6.6.0.4084
WildList detection	99.8%
False positive rate	0.000%
Diversity Test rate	99.69%

 

Rising Enterprise Security Management System

Windows 7 version	3.0.0.96
Windows 10 version	3.0.0.96
WildList detection	100.0%
False positive rate	0.002%
Diversity Test rate	78.70%

 

Scanguard

(See notes in Appendix 2)

Windows 7 version	5.5.83
Windows 10 version	5.5.83
WildList detection	N/A
False positive rate	N/A
Diversity Test rate	N/A

 

SecureAge SecureAPlus Pro

Windows 7 version	6.2.0
Windows 10 version	6.2.0
WildList detection	99.7%
False positive rate	0.001%
Diversity Test rate	99.90%

 

SentinelOne Endpoint Security Platform

Windows 7 version	3.7.2.45
Windows 10 version	3.7.2.45
WildList detection	99.9%
False positive rate	0.001%
Diversity Test rate	100.00%

 

Systweak Anti-virus

Windows 7 version	1.0.1000.10204
Windows 10 version	1.0.1000.10204
WildList detection	99.7%
False positive rate	0.000%
Diversity Test rate	100.00%

 

Tabidus United Endpoint Protector

(See notes in Appendix 2)

Windows 7 version	Not tested
Windows 10 version	1.0
WildList detection	100.0%
False positive rate	0.002%
Diversity Test rate	99.17%

 

TACHYON Endpoint Security

Windows 7 version	5.0.0.57
Windows 10 version	5.0.0.57
WildList detection	100.0%
False positive rate	0.000%
Diversity Test rate	99.79%

 

TeamViewer Endpoint Protection

Windows 7 version	1.5.238835
Windows 10 version	1.5.238835
WildList detection	99.7%
False positive rate	0.000%
Diversity Test rate	99.48%

 

Tencent PC Manager

Windows 7 version	12.3.26609.901
Windows 10 version	12.3.26609.901
WildList detection	100.0%
False positive rate	0.000%
Diversity Test rate	99.90%

 

Total Defense Premium

Windows 7 version	12.0.0.251 SP1
Windows 10 version	12.0.0.251 SP1
WildList detection	99.7%
False positive rate	0.000%
Diversity Test rate	98.97%

 

TotalAV

Windows 7 version	5.5.83
Windows 10 version	5.5.83
WildList detection	100.0%
False positive rate	0.000%
Diversity Test rate	100.00%

 

TUXGUARD Endpoint Protection

Windows 7 version	1.0.1.41
Windows 10 version	1.0.1.41
WildList detection	99.9%
False positive rate	0.000%
Diversity Test rate	100.00%

 

VIPRE Endpoint Cloud Business

Windows 7 version	11.0.7633
Windows 10 version	11.0.7633
WildList detection	99.7%
False positive rate	0.000%
Diversity Test rate	99.59%

 

VirIT eXplorer PRO

Windows 7 version	9.1.78
Windows 10 version	9.1.77
WildList detection	100.0%
False positive rate	0.000%
Diversity Test rate	55.33%

 

Appendix 1: products not certified

The following products did not receive VB100 certification in this test:

• Exosphere Endpoint Protection. A recurring product crash on both test platforms prevented us from taking precise measurements. For this reason, no certification is issued for this product in this test cycle.
• SecureAge SecureAPlus Pro with offline settings. This product failed to meet the requirements for VB100 certification.

 

Appendix 2: testing notes

• The CORE Antivirus product tested was a pre-release version.
• Tabidus Technology’s United Endpoint Protector was tested only on Windows 10 as there is no Windows 7 version of the product.
• Multiple products (Cynet 360, Defenx Security Suite, ITL Total Security, PCProtect and ScanGuard) were affected for a short period of time by a technical issue impacting the backend infrastructure they share. VB has received reasonable assurance that this technical issue would not have impacted end‑users. However, within the scope of the VB100 test, the products would have failed to meet the certification parameters. As the extent of the issue was confined to the VB100 test model, with no real-world impact, and was outside the control of the affected products, VB has decided that it would not be relevant to publish the resultant test data.

 

Appendix 3: sample set sizes

The Certification Set contained 2,029 malicious samples. The set of clean samples used for the false positive test contained 99,990 files, of which 29,107 were portable executable (PE) files. The set used for the Diversity Test contained 967 malicious samples.

 

Footnotes

¹ The WildList is an extremely well-vetted set of malware recently observed in the wild by researchers: http://www.wildlist.org/.

 

Latest reviews: