2016-01-29
Abstract
The latest VB100 test on Windows 7 Pro resulted in a pleasingly high pass rate, with most products having no problems with the WildList or a set of clean files. John Hawes has the details.
Copyright © 2016 Virus Bulletin
This test on Windows 7 began shortly before the release of Windows 10, the platform version designed to replace and supplant Windows 7. However, it seems likely that many users will stick with Windows 7 for at least some time to come. The latest figures available at the time of writing this report put Windows 7 on just under half of all desktops and laptops, with non-Windows platforms accounting for little more than 10%. With Windows still the biggest target for malware creators, it is vital that users are protected, and as ever we endeavour to detail which security solutions are providing the best service.
A typically large and diverse selection of solutions were submitted for testing, with most of our regular participants present, as well as a handful of occasional ones and a few new faces. For the most part these new faces integrate established scanning engines into their products, but a few brave souls continue to try to break into the market – some of them achieving impressive results. The long-rumoured consolidation of the anti-malware industry shows no signs of occurring any time soon, with the market apparently supporting a broad diversity to supply the needs of the huge range of users around the world.
Our standard Windows 7 test images were used for this test, with a few minor tweaks and updates, mainly to the tools we use to facilitate our tests. Set-up was thus fairly straightforward, aided by several years of practice and experience, leaving plenty of time to prepare the sample sets.
The set of clean files used for false-positive testing was given a standard update, with older items retired and a batch of new packages added, including both installers and installed content. The detection sets were also built using our standard process, incorporating all samples first seen by our labs no more than two weeks prior to running each part of the test.
The malware certification sets were based on the latest WildList set available on the test deadline – version 4.018 released on 17 June 2015, a week prior to the submission deadline of 24 June, thus giving all participants ample time to ensure their detection was up to scratch.
With everything in place we got down to business, with a long list of products to work through.
Main version: 15.0.11.574
Update versions: 8.11.242.104, 15.0.11.579/8.11.246.10, 8.11.247.212, 8.11.253.186
Last 6 tests: 5 passed, 0 failed, 1 no entry
Last 12 tests: 10 passed, 0 failed, 2 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Fair
First up this month is Avira’s ‘Pro’ edition, which has a strong record in our tests of late, with passes in all but our Linux tests (which were not entered) over the last couple of years. Under Windows 7, the product installs in decent time and looks sober and sensible with a wide range of featured modules listed in the main menu.
Stability was a little shaky at times, with a number of scan jobs freezing or crashing out, but in more normal use, away from large swathes of malicious samples, things were much more stable. Performance was decent, with memory use nice and low, CPU use a little high, and impact on our set of standard activities around average for this month’s batch of business products.
Detection was excellent as usual, with good coverage across the sets, and Avira’s tally of VB100 passes is extended with another VB100 award.
Main version: 1.5.0.32
Update versions: 8320, 8640, 8734, 9035
Last 6 tests: 3 passed, 0 failed, 3 no entry
Last 12 tests: 4 passed, 0 failed, 8 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Stable
BluePex has become a fairly familiar face in our tests over the last year or so, with the Brazilian product, which uses the VIPRE engine, achieving a good run of passes after some initial teething problems. It took quite some time to complete the initial installation and set-up, but this was perhaps thanks to the distance of our lab from the company’s main market regions. Once up, the interface features a fairly bright colour scheme with a spectral image of what appears to be a big cat adorning the background. These touches do little to disguise the basic controls, which are easy to access.
Stability was good, with only some minor issues noted, including a few missed translations and an oddity with the updates, which appeared to announce completion while still downloading data on some occasions. Performance was good too, all our measures falling pretty close to this month’s averages, and detection was decent – strong in the reactive sets but dropping off noticeably into the later parts of the RAP test.
There were no problems in the core sets, with full coverage of our WildList set and no false alarms, thus BluePex earns another VB100 award.
Main version: 9.1
Update versions: 4560.701.1951 build 104, build 117, build 123, build 143
Last 6 tests: 5 passed, 0 failed, 1 no entry
Last 12 tests: 9 passed, 0 failed, 3 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Solid
Defenx has managed a clean sweep in our Windows tests in the last year, with several more passes going further back. The latest product version takes a few minutes to set up, but not too long, and the GUI is clean and crisp with a big, reassuring check-mark on the main page to confirm all is well. Configuration is provided in reasonable if not comprehensive depth, and is sensibly laid out and intuitive to navigate.
Stability was flawless with no issues to report. Performance was a mixed bag, with memory use on the low side, CPU use fairly high and our set of tasks taking quite some time to run through.
Detection was a little below par, with a rather steep decline through the sections of the RAP sets, but the WildList set was handled well, and with no problems in the clean sets a VB100 award is earned by Defenx.
Main version: 3.0.0.4
Update versions: 13.3.21.1/554335.2015062415/7.61221/5706574.20150624, 13.3.21.1/561903.2015071016/7.61465/5778196.20150710, 13.3.21.1/562510.2015071716/7.61567/5795770.20150717, 13.3.21.1/563823.2015080316/7.61855/5877962.20150803
Last 6 tests: 5 passed, 0 failed, 1 no entry
Last 12 tests: 10 passed, 0 failed, 2 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Stable
Another product boasting a full house in our Windows tests over the last two years, ESTsoft’s ALYac uses the Bitdefender engine alongside its own technologies and is mainly active in the Korean market region. Installation takes a very long time indeed, but users geographically closer to the company’s headquarters may find their experience somewhat better than ours. The interface is bright to the point of luminosity, and provides a good basic set of controls.
Stability was pretty good, with only a single minor incident noted, in which a scan job failed to produce the expected completion dialog at the end. Performance impact was pretty light, with memory use barely detectable. CPU use was well below average and our set of tasks completed in good time.
Detection was strong too, with very good scores in the reactive sets and not too sharp a drop into the offline reactive sets. The core certification sets were dealt with without problems, and a VB100 award is easily earned by ESTsoft.
Main version: 5.0.9.1349
Update versions: 5.220/26.259, 26.507, 26.671, 27.159
Last 6 tests: 5 passed, 0 failed, 1 no entry
Last 12 tests: 9 passed, 1 failed, 2 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Fair
Fortinet’s products are rarely absent from the VB100 test bench, and have been doing well of late, with passes in all Windows tests in the past year. Set-up doesn’t take too long and the interface presented is simple and basic with minimal options available to the end-user.
Stability was dented rather heavily by a single incident in which the machine restarted unexpectedly, but this only occurred once and only when operating under an unusually heavy load. Performance was good, with low resource usage and speedy completion of our set of standard activities.
Detection was good, with a strong showing in the reactive sets and not too severe a decline into the proactive sets. The certification sets revealed no unexpected misses or false alarms, and another VB100 award goes to Fortinet.
Main version: 2.8.20
Update versions: 1.9.5/92281, 92439, 92565, 2.8.31/1.9.5/92854
Last 6 tests: 3 passed, 1 failed, 2 no entry
Last 12 tests: 6 passed, 2 failed, 4 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Stable
Our test history for Ikarus shows a fairly steady pattern with passes comfortably outweighing fails, and entries in the majority of our comparatives. The overall look and feel of the main interface has changed little in several years, sticking to a tried-and-trusted formula in terms of both appearance and underlying layout; it presents a decent basic set of configuration options, and seemed generally responsive.
Stability was pretty good in all aspects, with only a single incident noted: the app crashing when switching rapidly between high-stress tasks. Performance impact was decent too, with RAM use a little higher than some other products this month but CPU usage very low and our set of activities completing in good time.
Detection was strong in the reactive sets, a little below the leaders in the proactive parts of the RAP test but still reasonable. The WildList and clean sets were dealt with appropriately, and Ikarus is deserving of another VB100 award.
Main version: 5.6.0.0.605
Update versions: 5.6.0/3.71, 5.6.2.0704, 5.6.3.0716
Last 6 tests: 5 passed, 0 failed, 1 no entry
Last 12 tests: 10 passed, 0 failed, 2 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Solid
Another product which includes the popular Bitdefender engine in its arsenal, iSheriff has also managed to pass all Windows comparatives in the last two years, skipping only our Linux tests. As usual, installation involved visiting an online portal to acquire the correct package, but with some experience using the system it was all fairly simple and speedy. The interface is also web-based, with a cloud-side portal providing much of the configuration. Some controls are also available locally, but again a browser is used to access them.
Stability was very good indeed, with no problems noted throughout the testing period. Performance impact was pretty light, with RAM use spiking somewhat during busy periods but other resource measures remaining low and our set of tasks barely affected.
Detection was a little below what we might expect from the engine but still very decent across the board, and with no issues in the WildList or clean sets, iSheriff earns another VB100 award.
Main version: 10.2.2.10535(mr1)
Update versions: N/A
Last 6 tests: 5 passed, 0 failed, 1 no entry
Last 12 tests: 10 passed, 0 failed, 2 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Stable
Kaspersky’s history in our tests is more complex than most, with a wide range of solutions submitted to most of our comparatives; they generally fare well, with a strong pass rate over the last few years. The current enterprise endpoint product takes some time to set up, but once active presents a highly polished interface with a complete set of controls and fine-tuning options presented in a sensible and user-friendly fashion.
Stability was good, but once again we hit some issues with the company’s cloud system, which did not fit well with the specialist set-up of our test lab; as a result much of our detection data was rendered unusable. Performance was good with a light footprint and minimal interruption of our set of tasks.
As mentioned above, some of our detection measures were rendered unreliable by issues with the cloud lookup system, but there were no problems handling the WildList and clean sets and a VB100 award is easily earned.
Main version: 15.0.2.361
Update versions: 15.0.2.361(c)
Last 6 tests: 5 passed, 0 failed, 1 no entry
Last 12 tests: 7 passed, 0 failed, 5 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Stable
The second business offering from Kaspersky this month is aimed at smaller firms. Once again it takes a good while to install and presents a slick and stylish interface with a full and detailed set of controls within easy reach.
Stability was generally sound, with the only issue once again a problem with the interaction of the KSN cloud system with our test set-up. RAM use was low but CPU use was fairly high, and our set of activities also showed a significant slowdown.
Detection figures are absent once again, but with no problems to report in the certification sets a VB100 award is earned.
Main version: 4.7.209.0
Update versions: 1.1.11701.0/1.199.2954.0, 1.11804.0/1.201.1054.0, 1.201.1609.0, 1.11903.0/1.203.1086.0
Last 6 tests: 5 passed, 0 failed, 1 no entry
Last 12 tests: 10 passed, 0 failed, 2 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Solid
Microsoft’s business solution has become a regular fixture in the VB100 tests of late, showing slow but steady improvements and a very dependable pass rate, with awards in all Windows tests over the last two years (our annual Linux comparatives, of course, being skipped).
Stability was rock-solid once again, and performance impact was good too, with minimal RAM use and use of CPU cycles, and slowdown in our set of activities not excessive either.
Detection was decent, with a marked decline into the proactive sets, but the core certification sets were handled impeccably and another VB100 award is well deserved.
Main version: 7.61221
Update versions: 5706574, 7.61458/577574, 7.61563/5791931, 7.61861/5881437
Last 6 tests: 5 passed, 0 failed, 1 no entry
Last 12 tests: 8 passed, 2 failed, 2 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Stable
Norman’s fortunes over the last few years have been in flux, with the company dividing up, being acquired, the flagship product being overhauled to use a third-party engine from Bitdefender, and its own venerable technology being retired. Of late, at least, it has been a regular participant in our testing, with a clean sheet of passes in the past year.
Stability was good this month, with only a few minor issues related to the browser-based interface. Performance was good too, with very low resource use and barely perceptible effect on the runtime of our set of activities.
Detection was strong too, with good scores across the board, and with a similarly strong performance in the core certification sets Norman comfortably achieves a VB100 award.
Main version: 7.20.0
Update versions: 7.20.0/7.20.2
Last 6 tests: 5 passed, 0 failed, 1 no entry
Last 12 tests: 9 passed, 1 failed, 2 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Stable
Panda’s products have been doing well in our tests of late, with a good run of passes in the last year. The vendor’s business product is simple to set up with the usual minimalist local GUI; more complex configuration is available in an online portal system, which looked slick and sensibly designed.
Stability was decent, and although we did have a couple of issues with big scans crashing out, these did not occur during normal everyday operations. Performance was good too, with very low resource usage and not too much interference with our set of standard activities.
Detection was decent in the reactive sets, with no proactive data thanks to the product’s heavy reliance on the cloud. The core sets were handled ably though, with no issues to report, and a VB100 award is well deserved.
Main version: 16.00(9.0.0.4) 64Bit
Update versions: N/A
Last 6 tests: 2 passed, 0 failed, 4 no entry
Last 12 tests: 2 passed, 0 failed, 10 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Stable
Quick Heal’s Seqrite business product is a relative newcomer to our tests, with only one other appearance under its belt, which resulted in a pass. The product interface, available after a lengthy install process, will be familiar to anyone who knows the company’s standard product lines, with much the same layout but a more businesslike monochrome colour scheme, reverting to the usual green-and-grey on a few pop-ups.
Stability was OK, and although we did see a few scans crashing out, this only happened under heavy stress. Performance impact was reasonable too, with resource usage a little above average and our set of activities slowed down just a little.
Detection was very strong in the reactive sets with a steep drop into the offline proactive sets, and the core certification sets were dealt with well, earning Quick Heal another VB100 award.
Main version: 2015(15.0.2.5430)
Update versions: 2015(15.0.2.5431), 2015(15.0.3.5432)
Last 6 tests: 3 passed, 0 failed, 3 no entry
Last 12 tests: 7 passed, 0 failed, 5 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Stable
There are two entries from TrustPort this month, with similar products entered in both the business and consumer categories.
The business-focused product set up nice and quickly, offering a pleasant, clear interface with large, informative buttons and icons; configuration is provided in detail with a sensible, easy-to-navigate design.
Stability was good too, with just some very minor issues with updating on some runs. Performance impact was also pleasingly small, with low resource use and minimal slowdown of our set of tasks.
Detection was excellent, as one would expect from a multi-engine solution, with superb scores in all sets. With impeccable handling of the core certification sets as well, TrustPort easily earns a VB100 award for its business solution.
(Click for a larger version of the table)
(Click for a larger version of the table)
(Click for a larger version of the table)
.
(Click for a larger version of the graph)
(Click for a larger version of the table)
(Click for a larger version of the chart)
(Click for a larger version of the chart)
Main version: 9.1
Update versions: 4652.701.1951 build 104, build 117, build 123, build 143
Last 6 tests: 5 passed, 0 failed, 1 no entry
Last 12 tests: 10 passed, 0 failed, 2 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Solid
There has been a very strong run of late from Agnitum, with passes in all Windows comparatives over the last two years. The install process is a little drawn out, as usual, but the multi-component solution is slick and attractive with a good simple layout and easy access to a basic set of controls.
Stability was once again impeccable with no problems noted, even under the heaviest of stress. Performance impact was a little wild, with resource usage low, but our set of tasks slowed down significantly – this may have impacted our CPU usage measure which is averaged out over the full period of testing, implying that much of the extra time was spent idly waiting for a response from a remote source.
Detection was reasonable, with a steady decline through the sets, but the core sets were handled well, and Agnitum earns another VB100 award.
Main version: 14.8.2015
Update versions: 2.0.0.2242/1.1.4776.0, 13.7.2016/2.0.0.2272, 28.7.2015/2.0.0.2284, 13.7.2016/2.0.0.2325
Last 6 tests: 0 passed, 1 failed, 5 no entry
Last 12 tests: 0 passed, 1 failed, 11 no entry
ItW on demand: 83.30%
ItW on access: 82.44%
False positives: 25
Stability: Fair
Making its first appearance in the VB100 test this month, ALLIT Service hails from Ukraine, where it claims to be the only AV lab and boasts over 3 million users. The set-up process was reasonably smooth and speedy, and the interface is glossy and clear with a nice intuitive layout and plenty of controls available.
Stability was shaken by a couple of issues, including one which seemed to lock up the system while trying to export a log to file, but for the most part things ran smoothly. Performance measures showed resource use a little above average, but our set of activities was completed in a pleasingly rapid time.
Detection was a little below par but still pretty impressive for a fairly young product; the WildList was reasonably well covered but there were a good few items not detected. A number of items in the clean sets were flagged as malicious too, including packages from Dell, Sony, Google, Microsoft and nVidia. These issues mean that, despite a generally strong showing, the team behind Zillya! still have a little more work to do to earn their first VB100 award.
Main version: 2015.10.2.2218
Update versions: 150623-7, 150706-1, 150713-0, 10.3.225/150803-1
Last 6 tests: 6 passed, 0 failed, 0 no entry
Last 12 tests: 9 passed, 2 failed, 1 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Stable
Avast has picked up an unbroken set of passes in our tests in the last year, a noteworthy achievement. The free product installs in decent time and presents its glossy and professional interface, once noted for its beauty but which has lost a little elegance in the latest redesign which squares up the edges to fit the stylings of the latest versions of Windows.
Stability was good, with just a very minor issue with the logging system, and performance impact was good too, with low use of resources and our set of tasks running through in good time.
Detection was very strong, with just a slight decline into the proactive sets. The WildList and clean sets presented no difficulties, and Avast picks up yet another VB100 award.
Main version: 2015.0.6037
Update versions: 4365/10089, 10174, 2015.0.6081/10219, 2015.0.6.122/4392/10364
Last 6 tests: 6 passed, 0 failed, 0 no entry
Last 12 tests: 12 passed, 0 failed, 0 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Stable
AVG maintains a flawless recent record in our tests, with passes in all comparatives in the last two years, and an excellent record of success in the long term too – no more than a handful of fails in the last decade of testing. The latest suite edition installs in reasonable time and presents the usual dark and brooding interface, lightened on the home screen by some nice clear buttons covering the main components. Configuration options are in good depth and easy to access.
Stability was decent, with just a single issue noted, and that only under heavy stress; performance was good too, with pleasingly low resource usage and little impact on our set of tasks.
Detection was decent, with respectable scores across the board, and with another clean sweep of our certification sets AVG extends its string of success with another VB100 award.
Main version: 15.0.11.574
Update versions: 8.11.242.104, 15.0.11.579/8.11.246.10, 8.11.247.212, 8.11.253.180
Last 6 tests: 3 passed, 0 failed, 3 no entry
Last 12 tests: 6 passed, 0 failed, 6 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Fair
Avira’s free solution usually only appears in our desktop tests, but has picked up a strong run of passes where it has participated. The current edition is quick to install and as ever looks glossy and professional with a sensible layout and a good set of controls.
Stability was mostly decent, but we did encounter a number of crashes during scanning. The developers seem to have pinned these down based on our feedback. Performance impact was minimal, with very low use of resources and a zippy time taken to complete our set of activities.
Detection was excellent, with very high numbers in our reactive sets, the proactive sets a little lower but still very respectable. The core sets were handled very efficiently with no issues to report, and Avira picks up another VB100 award.
Main version: 18.22.0.1521
Update versions: 7.61207/5709137, 7.61458/5775574, 7.61561/5790293, 7.61807/5867639
Last 6 tests: 6 passed, 0 failed, 0 no entry
Last 12 tests: 12 passed, 0 failed, 0 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Solid
Bitdefender also boasts an unbroken string of passes in our comparatives, dating back more than five years now. As its name suggests, the current ‘AV+’ product does indeed feature a range of components beyond the basics of malware detection, including a batch of browser add-ons, but set-up was swift and simple nevertheless.
Stability was splendid, with not the slightest wobble even under heavy pressure; performance impact was excellent too, with RAM use around average but CPU use and impact on our set of tasks barely discernible.
Detection was superb too, with very good scores across the board. With no issues in the certification sets, Bitdefender comfortably notches up yet another VB100 award.
Main version: 15.1.305
Update versions: 7.61221, 15.1.306.1/7.61458, 7.61561, 15.1.307.2/7.61807
Last 6 tests: 3 passed, 0 failed, 3 no entry
Last 12 tests: 8 passed, 0 failed, 4 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Solid
BullGuard’s test history has been confined to our desktop tests of late, where the vendor’s home-user product has fared very well indeed for some time. The current iteration takes a fair while to install and run initial updates, but once complete provides an interface with plenty of space for additional components not included in the basic package.
Stability was once again excellent with no issues to report, and performance was good too, with RAM use fairly standard, CPU use low and our set of tasks barely affected.
Detection was very strong indeed, helped along as usual by the Bitdefender engine, and BullGuard proves more than worthy of another VB100 award.
Main version: 13.4.263.000
Update versions: 8.5.1.88
Last 6 tests: 3 passed, 0 failed, 3 no entry
Last 12 tests: 6 passed, 0 failed, 6 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Solid
Another product mainly featured in our desktop tests, Check Point’s iconic ZoneAlarm has managed to pass all tests entered in the last two years. Set-up is reasonably speedy and straightforward, and the product GUI is much as it has been for many years now – a little wordy and with only a basic set of options, but sensibly laid out and simple to navigate.
Stability was impeccable, with no problems noted. Performance impact measures showed RAM use a little on the high side, and our set of tasks noticeably slowed down, which may account for the low CPU use figure which is averaged out over the time taken to complete the set of activities.
Detection was decent, if not stellar, with a distinct decline into the later test sets, but the core certification sets were well handled and Check Point picks up another VB100 award.
Main version: 15.3
Update versions: 7.61220/5707819, 7.61458/5775574, 7.61561/5790293, 7.61807/5867639
Last 6 tests: 1 passed, 0 failed, 5 no entry
Last 12 tests: 1 passed, 0 failed, 11 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Solid
Another new face making its first appearance this month, CompuClever provides a range of PC optimization tools and other software alongside its anti-malware offering, which is based on the Bitdefender engine. Installation wasn’t too slow, and the interface presented proved only cosmetically different from Bitdefender’s own, even including the same suite of extras such as the browser plug-in wallet system. The design is slick and attractive, with a good range of options available and intuitive deployment of controls.
Stability was perfect throughout testing, with not the slightest shake even under heavy stress. Performance impact was decent too, with RAM use perhaps a little above average but CPU usage and impact on our set of tasks both pretty low.
Detection was as excellent as one would expect, with good scores everywhere. The WildList and clean sets were handled with pinpoint accuracy, earning CompuClever its first VB100 award on its debut.
Main version: 5.1.35
Update versions: 5.4.20
Last 6 tests: 1 passed, 4 failed, 1 no entry
Last 12 tests: 2 passed, 7 failed, 3 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 2
Stability: Solid
CYREN’s Command product has suffered from issues with false positives of late, with problems cropping up in a number of recent tests. The product itself remains minimally changed over the last several years, with a very compact install package which runs through very quickly and presents the usual pared-down, simple interface with a decent basic set of configuration options within easy reach.
Stability was great, with no problems at all, and performance impact numbers show low RAM use, CPU use a little high, and slowdown of our set of tasks a little on the high side too.
Detection was excellent in the reactive sets where CYREN’s cloud system is in play, and a little less distinguished in the offline proactive test. The WildList sets were properly covered, but once again in the clean sets we spotted a couple of unexpected alerts: one on an install package for the widely deployed Java Runtime Environment and another on an updater tool shipped with Lenovo laptops. This, of course, meant that once again there was no VB100 award for CYREN.
Main version: 10.0.0.5409
Update versions: 10.0.0.5532, 10.0.0.5532, 10.0.0.5561
Last 6 tests: 5 passed, 0 failed, 1 no entry
Last 12 tests: 9 passed, 1 failed, 2 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Fair
Emsisoft has managed a pass in all our tests in the last year, and in most of them in the last two years. The company’s flagship anti-malware solution takes a while to install, but once up and running presents a very bright and cheery interface with loud colours providing instant information and a good range of configuration options.
Stability was shaken somewhat by some problems scanning large sets of files – this issue has since been pinned down and fixed by the developers and is unlikely to affect normal users in the real world. We also saw a single instance of an update failing to complete, but running fine on the second attempt.
Performance was good, with very low use of resources and a good time taken to complete our set of tasks. Detection, aided as usual by that Bitdefender engine, was pretty strong across the board, even with the issues completing some scans, and with an excellent showing in the certification sets Emsisoft earns another VB100 award.
Main version: 14.0.1400.1766 DB
Update versions: 14.0.1400.1771 DB, 14.0.1400.1771 DB, 14.0.1400.1780 DB
Last 6 tests: 6 passed, 0 failed, 0 no entry
Last 12 tests: 11 passed, 1 failed, 0 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Fair
Another of the band of vendors who routinely participate in all our tests, eScan boasts a perfect six out of six VB100s in the last year and a pretty excellent history in general. This month it took a little while to get things set up. The interface presented was fairly colourful in the main sections, remaining dark and shadowy around the edges. Configuration options are provided in complete depth, with a clear and simple layout.
Stability was shaken on this occasion by a number of issues, including a couple of scans crashing out, some strange behaviour with the desktop, and on one occasion an unexpected restart which may have been due to the product re-enabling Windows Update. Performance measures showed fairly high RAM use but quite low CPU, at least when spread out over the longer-than-usual time taken to complete our set of activities.
Detection was very good though, with good coverage throughout, and with no issues in the core certification sets eScan earns another VB100 award.
Main version: 8.0.312.0
Update versions: 11836, 11896, 11931, 8.0.319.0/12035
Last 6 tests: 6 passed, 0 failed, 0 no entry
Last 12 tests: 12 passed, 0 failed, 0 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Solid
ESET is the longest-standing member of the elite group of vendors with passes in all our comparatives in recent years – in this case the run of passes stretching way back into the distant past. This time we once again saw the venerable NOD32 product in its latest incarnation, which installs in good time and presents the usual very slick and attractive interface, laden with detailed configuration controls but simple to navigate and operate.
Stability was rock-solid once again, with no problems at all, and our performance data shows RAM use barely perceptible at idle, rising to around average during intensive use, with a very minimal impact on our set of standard activities.
Detection was solid and reliable too, and with yet another clean sweep of the certification sets, ESET’s collection of VB100 awards continues to grow.
Main version: 25.1.0.3
Update versions: AVA 25.2167/GD 25.5274, AVA 25.2437/GD 25.5341, AVA 25.2540/GD 25.5362, AVA 25.2788/GD 25.5415
Last 6 tests: 5 passed, 0 failed, 1 no entry
Last 12 tests: 9 passed, 0 failed, 3 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Stable
G Data has also been building up an impressive run of results of late, with passes in all tests entered in the last two years, including all Windows tests in the last 12 months. The latest edition looks crisp and fresh, with good clear information and the usual wealth of fine-tuning options.
Stability was good, but we did have a couple of minor annoyances, including a scan refusing to run thanks to loss of connection to the engine and an update not completing properly. Performance impact was a little heavy at times, with RAM use around average but CPU use very high, and our set of tasks slowed down somewhat.
Detection was strong, as usual, with good scores in all sets, and the certification sets were easily dealt with, earning G Data another VB100 award.
Main version: 4.5.2.687
Update versions: N/A
Last 6 tests: 0 passed, 1 failed, 5 no entry
Last 12 tests: 1 passed, 1 failed, 10 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 2
Stability: Stable
We don’t have a lot of historical data for iolo, with only a handful of entries over the last few years, the most recent being a pass in late 2013. The product seems little changed from previous appearances though, presenting a clear and simple interface after its fairly rapid set-up process; configuration is in reasonable depth and easy to find.
Stability was good, with just a single scan crashing out, and performance impact figures show fairly low use of memory. CPU use was on the high side despite a pretty long time spent processing our set of activities.
Detection was reasonable, the CYREN engine revealing its heavy reliance on cloud technology, which is not available to this product. The WildList sets were properly dealt with, but in the clean sets the same items as in CYREN’s case were erroneously flagged as malicious, meaning iolo’s System Shield does not quite make the grade for a VB100 award this time.
Main version: 14.2.0271
Update versions: 9.205.16344, 9.205.16472, 15.1.0275/9.205.16540, 9.207.16763
Last 6 tests: 5 passed, 0 failed, 1 no entry
Last 12 tests: 7 passed, 1 failed, 4 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Solid
K7 has a good string of passes over the last 12 months, combined with some excellent stability ratings. This time, installation was speedy. The interface remains ruggedly appealing with easy access to a good selection of controls, and stability was once again impeccable, with not the slightest sign of shakiness even under the heaviest of pressure.
Performance was good too, with RAM use around average, CPU use just a touch on the high side, and a good speed through our set of tasks.
Detection was reasonable, a little behind the very best but still pretty decent, and with a perfect showing in the certification sets K7 comfortably earns another VB100 award.
Main version: 15.0.2.361(a)
Update versions: 15.0.2.361(c), 15.0.2.361(c), 16.0.0.614(a)
Last 6 tests: 5 passed, 0 failed, 1 no entry
Last 12 tests: 8 passed, 0 failed, 4 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Stable
Alongside the pair of business products this month we have a consumer solution from Kaspersky, the company’s flagship suite product, which has become something of a fixture on the VB test bench of late with a clean sweep of passes in the last year. The set-up includes a number of extras including several browser plug-ins, which adds to the time required to complete the job. The interface is as glossy and professional as ever, with a comprehensive set of configuration options within easy reach of the user.
Stability was decent, with a few minor oddities and some network problems which have once again meant we have no usable RAP data. Performance data shows RAM use a little below average, CPU use on the high side, but our set of activities zipping through in excellent time.
Detection measures were too unreliable to report, but the core certification sets were handled nicely in a performance that easily earns the product a VB100 award.
Main version: 11.6.306.7947
Update versions: 11.7.485.8398
Last 6 tests: 3 passed, 0 failed, 3 no entry
Last 12 tests: 6 passed, 0 failed, 6 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Fair
Entering all our desktop tests over the last few years, Lavasoft’s Ad-Aware has managed passes every time it has appeared of late. The current set-up process is rather lengthy, but once finally up, the interface is bright, clean and modern looking, with good access to a basic set of configuration options.
Stability was shaken by some scans crashing out and an unexpected restart, all under heavy stress. Performance impact was once again a little odd, with RAM use a little high and our set of activities slowed down very significantly; as the very low CPU average indicates, much of this time was spent idly waiting for input from elsewhere – we continue to investigate this anomaly.
Detection, aided by the Bitdefender engine, was strong across the board, and the certification sets presented no difficulties either, earning Lavasoft another VB100 award.
Main version: 2.0.75.134
Update versions: 2.0.75.134, 2.0.77.136, 2.0.79.138
Last 6 tests: 0 passed, 1 failed, 5 no entry
Last 12 tests: 0 passed, 4 failed, 8 no entry
ItW on demand: 23.10%
ItW on access: 10.89%
False positives: 12
Stability: Buggy
Maya Software has submitted a product for VB100 testing a few times over the last few years, as yet without achieving the required standard for certification. The product installs rapidly, and presents a somewhat garish interface which looks reasonably professional, if a little unfinished in places.
Stability was somewhat shaky, with a number of scans crashing out, issues with the logging system, and the on-access component refusing to work on several installs. Performance impact was fairly heavy too, with high use of RAM and a noticeable slowdown of our set of tasks.
Detection was mediocre and a little unpredictable, with a much better showing in the offline proactive sets than elsewhere, fairly poor coverage of the WildList sets, and a handful of false positives too. Maya clearly still has some way to go before it earns its first VB100 award.
Main version: 0.30.24.67802 Beta
Update versions: 0.14.23.5668, 0.30.24.68023/0.14.23.5725, 0.14.23.5758, 0.30.24.68532/0.14.23.5888
Last 6 tests: 0 passed, 1 failed, 5 no entry
Last 12 tests: 0 passed, 2 failed, 10 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 28
Stability: Stable
Another relatively new name, NANO has only a single VB100 entry under its belt so far, and remains in Beta at the time of testing. Set-up was not too complex and the interface presented was clean and clear with a decent set of controls.
Stability was pretty good, with just one instance of a scan not covering all areas requested. Performance impact was reasonable, with resource usage a little above average but slowdown of our set of activities not excessive at all.
Detection was pretty good too, keeping pace with the majority of products this month, and the WildList was covered perfectly, but a few false positives in our clean sets mean there is no VB100 award for the product just yet, despite a generally excellent performance from NANO.
Main version: 15.1.0
Update versions: N/A
Last 6 tests: 3 passed, 0 failed, 3 no entry
Last 12 tests: 4 passed, 0 failed, 8 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Stable
Panda’s second product this month is its home-user suite, which has become a regular in our desktop tests lately, racking up a full set of passes in the last year. As usual, set-up is quick and simple, and the interface looks very glossy and modern, divided into sections for the multiple components bundled in. Configuration is fairly simple and minimal, but easy to access.
Stability was generally very good, with just a single instance of an install process failing to complete properly. Performance impact figures show minimal RAM use, CPU use a little higher than most, and our set of tasks running through in good time.
Detection figures are pretty similar to those achieved by the business version – a little below Panda’s usual high standards but still very respectable – and with a perfect showing in the certification sets Panda’s consumer offering also earns a VB100 award.
Main version: 1.0.0.55
Update versions: N/A
Last 6 tests: 1 passed, 2 failed, 3 no entry
Last 12 tests: 2 passed, 4 failed, 6 no entry
ItW on demand: 99.41%
ItW on access: 100.00%
False positives: 982
Stability: Fair
PC Pitstop’s test history has been somewhat patchy over the last few years, but some impressive detection rates have been garnered thanks to the whitelisting component that is included in the product and exercised in most of our tests. Set-up isn’t too lengthy or complex and, once up and running, the product interface is rather different from most, focusing as it does on optimization as well as security; configuration is only available at a basic level.
Stability was knocked by a number of unexpected restarts, which are actively being investigated by the product’s developers. Performance was decent, with memory use around average, CPU use just a touch high, and our set of tasks not too severely impacted.
Detection was pretty superb, with excellent numbers across the RAP sets, but there were some misses in the WildList on-demand where the whitelisting component is unavailable, and in the clean sets quite a number of items were flagged too, meaning there is no VB100 award for PC Pitstop this month.
Main version: 16.00(9.0.0.28) 64Bit
Update versions: N/A
Last 6 tests: 2 passed, 0 failed, 4 no entry
Last 12 tests: 7 passed, 1 failed, 4 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Stable
Quick Heal products have been absent from a few of our comparatives lately, but they generally do well, with a strong record of passes in the last few years. The current ‘Total’ version takes quite some time to set up, the time mainly being spent performing initial updates. The interface presented is clean and well laid out with the usual wealth of configuration options.
Stability was generally good, but a single scan failed to complete cleanly. Performance impact was OK too, with RAM use a touch high, CPU use average, and our set of tasks not too slowed down.
Detection was decent, with a fairly big drop into the proactive sets, but the certification sets were handled well and another VB100 award is earned by Quick Heal.
Main version: 8.0
Update versions: 5774, 5787, 5795, 5817
Last 6 tests: 0 passed, 3 failed, 3 no entry
Last 12 tests: 0 passed, 3 failed, 9 no entry
ItW on demand: 19.97%
ItW on access: 7.08%
False positives: 5
Stability: Buggy
Another fairly recent arrival on the VB100 test bench, SmartCOP’s previous appearances haven’t been great and we look forward to some improvements. The set-up process is reasonably speedy, the interface a rather lurid orange colour but otherwise nicely designed with a decent set of configuration options.
Stability was not great: we had several scans crash out or freeze up, and even a couple of dreaded Blue Screen incidents, but most of the issues only occurred under heavy stress and are less likely to impact users in everyday life.
Performance figures show minimal RAM use but distinctly high use of CPU cycles, while our set of activities ran through in very good time.
Detection was fairly dismal, again rather better in the proactive parts of the RAP sets, somewhat surprisingly. The WildList was not well covered and there were issues in the clean sets too, meaning there is still no VB100 award for SmartCOP.
Main version: 10.0.4.3879
Update versions: 10.0.4.3879, 10.0.45471
Last 6 tests: 4 passed, 0 failed, 2 no entry
Last 12 tests: 4 passed, 0 failed, 8 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Stable
ITbrain is another relatively new name in the VB100 tests, but has picked up a number of awards in the past year. The product integrates nicely with the popular TeamViewer package produced by the same company. Installation is fairly zippy and simple, the interface a combination of an online portal for setting up and applying policies and some local controls plugged into the standard TeamViewer app. Configuration is fairly detailed, and simple to apply.
Stability was good, with just a couple of high-stress scans not making it all the way through and, at one point, a complaint about a shortage of memory on the test machine (which is supplied with a fairly standard 4GB).
Performance impact figures show resource usage a fraction higher than this month’s average, with a small but perceptible slowdown through our set of standard activities.
Detection, mainly provided by the Bitdefender engine, was pretty decent in the reactive sets, with no proactive data thanks to a reliance on Internet connectivity for operation of the product. The core certification sets were handled well though, and ITBrain picks up another VB100 award.
Main version: 10.9.25828.501
Update versions: N/A
Last 6 tests: 3 passed, 0 failed, 3 no entry
Last 12 tests: 8 passed, 0 failed, 4 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Solid
Having been absent from the last few tests after some issues with the configuration of submitted products, Tencent returns to pick up where it left off, with a good string of passes over several years. The product includes the Avira engine alongside the company’s own technologies, and is very quick indeed to install. The GUI is clean and simple with a good set of configuration options available, and stability was impeccable throughout our testing.
Performance impact was good too, with all stats hovering around this month’s average numbers and our set of tasks not too heavily impeded.
Detection was decent, dropping slightly into the proactive sets, and the core certification sets were covered perfectly, heralding a return to VB100 award winning for Tencent.
Main version: 9.0.0.355
Update versions: 3.0.2.1015/2015.6.24.5, 2015.7.7.10, 2015.7.15.7, 2015.8.5.12
Last 6 tests: 2 passed, 0 failed, 4 no entry
Last 12 tests: 3 passed, 1 failed, 8 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Stable
Total Defense hasn’t been the most regular of participants in our tests of late, but has picked up passes in its last few appearances. The latest version is adorned with large boxes to denote the various components and provide nice clear information, and has a decent basic set of configuration options.
Stability was decent, with only a few very minor issues, and performance measures showed slightly high use of memory but very low CPU use and barely any impact on our set of tasks.
Detection was pretty strong across the board, and the core certification sets were dealt with well, thus adding another VB100 award to Total Defense’s collection.
Main version: 2015(15.0.2.5430)
Update versions: 2015(15.0.2.5431), 2015(15.0.2.5432)
Last 6 tests: 3 passed, 0 failed, 3 no entry
Last 12 tests: 7 passed, 0 failed, 5 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Stable
Our final product this month is a second entry from TrustPort, essentially identical to the business edition. Installation took a little while but not too long. The interface is fairly nicely laid out with a good set of configuration options available, and stability was only shaken by a very minor error during one update run.
Performance impact was minimal, with low use of resources and only a very slight slowdown in our set of tasks.
Detection was superb once again, with very high scores everywhere, and with no problems in the core sets TrustPort comfortably earns a second VB100 award this month.
(Click for a larger version of the table)
(Click for a larger version of the table)
(Click for a larger version of the table)
(Click for a larger version of the graph)
(Click for a larger version of the table)
(Click for a larger version of the chart)
(Click for a larger version of the chart)
Several other products were submitted for testing this month but were not included in this report for one reason or another; these included solutions from Kyrol Security, MSecure Data Labs and SecureaLive.
Once again this month, the pass rate is pleasingly high, with most products having no problems with the WildList or our set of clean files. This is as it should be – the WildList is intended to be a set of very widespread threats which all respectable products should routinely pick up, while our clean set is similarly constructed to contain only fairly recent and relatively common files which should not be flagged by most products, most of the time. Of those entrants that did not quite make the grade, the majority are relative newcomers to the market, several of them bravely building up whole new detection technologies to compete with the existing long-established players.
Stability ratings were also pleasingly good this month, with all products in the business category managing a ‘stable’ rating at the very least, a good sign that more demanding corporate users are being well served with well-tested solutions. We hope to see a similarly high standard in the next test, on Windows Server 2008.
Test environment. All tests were run on identical systems with AMD A6-3670K Quad Core 2.7GHz processors, 4GB DUAL DDR3 1600MHz RAM, dual 500GB and 1TB SATA hard drives and gigabit networking, running Microsoft Windows 7 Professional, 64-bit edition with SP1.
Any developers interested in submitting products for VB's comparative reviews, or anyone with any comments or suggestions on the test methodology, should contact [email protected]. The current schedule for the publication of VB comparative reviews can be found at http://www.virusbtn.com/vb100/about/schedule.xml.
