2015-02-17
Abstract
The latest VB100 comparative on the evergreen Windows 7 resulted in a pleasingly high success rate with just a few products failing to make the grade for certification - John Hawes has the details.
Copyright © 2015 Virus Bulletin
This month’s VB100 is on Windows 7, and as usual when we visit the leading desktop platforms we expected a large field of products to put through their paces. As in the last desktop test, in order to make it easier to compare similar product types the products were split into business-focused products and those aimed at the home-user market.
With testing starting late thanks to previous delays, and once again running longer than expected, this report is long overdue and should be followed shortly by the next comparative: a mercifully compact Linux test for which the bulk of the actual testing work has already been completed. This will help get us back on schedule, but in addition, we are looking at a number of ways to speed up the testing process, at the same time allowing us to produce more accurate, complete and inclusive data for our readers. It is likely that our desktop and server tests will continue to diverge in content, but we plan to keep the core aims of the test process aligned for the two spheres.
As this report was being compiled, Windows 7 reached the end of its ‘mainstream’ support period and entered a five-year ‘extended’ phase. In the case of Windows XP, of course, it was the expiry of this ‘extended’ phase that caught so many people by surprise and left them stuck with unpatched and vulnerable systems, so anyone planning any long-term installations may best be advised to consider other alternatives. For the time being though, the platform remains the most widely deployed on the planet, with most estimates putting it on between 30% and 45% of all endpoint systems – at least three times as many as its successor Windows 8 (and now 8.1) at the time of writing.
Preparing for a Windows 7 comparative proved to be a reasonably painless affair thanks to the good chunk of history the platform has behind it and the fact that it tends to be in use on most systems we encounter. Well‑worn system images were dusted off and updated with our latest preferences in terms of handy tools and system settings but with no OS or software updates beyond those included on the original install media. These tweaks were deployed to the full suite of standard test machines, which were then also updated with the latest versions of our test sets.
With the test deadline set for 22 October, the certification sets included the October (v4.010) edition of the WildList, released a week earlier, alongside a raft of updates to our clean sets. Other sample sets, including those used for speed and performance measures as well as the RAP/response sets compiled over the days preceding each round of testing, were put together in the standard manner.
On the deadline day itself, products flooded in as usual, although there were not quite as many as we had expected. The total was well over 40 products, but several of these were clearly unlikely to make it very far through the testing process. Alongside the familiar faces we once again saw a selection of new names to keep things interesting. The testing process began almost immediately in late October, and ran well into December with the final double-checking and verification spilling over into 2015.
Main version: 141022050139
Update versions: 141107103628,141113090040,141121084332
Last 6 tests: 4 passed, 0 failed, 2 no entry
Last 12 tests: 4 passed, 0 failed, 8 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Stable
First up in the corporate category this month is Arcabit. After an absence of several years, Arcabit returned to the VB100 test bench in 2014 with a new look and the Bitdefender engine under the hood, putting in a string of strong performances.
The installation process for the current product version isn’t too lengthy and no reboot is required. The interface is simple and angular with a good selection of configuration options. Stability was decent for the most part with just a single scan crashing out.
Scanning speeds were reasonable, overheads not too high, and our set of activities completed quickly, with only CPU use a little on the high side.
Detection was very strong in the reactive sets, only dropping a little into the proactive weeks, and our certification sets were covered well, earning Arcabit another VB100 award to extend its recent run of passes.
Main version: 14.0.7.306
Update versions: 8.11.180.122, 8.11.183.52, 8.11.184.204, 8.11.187.66
Last 6 tests: 5 passed, 0 failed, 1 no entry
Last 12 tests: 10 passed, 0 failed, 2 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Solid
Avira’s ‘Pro’ version has an excellent history in our VB100 comparatives, with a long string of passes and very few missed tests.
Installation was quick with very speedy updates. The interface is clean and attractive with an excellent range of controls, and it remained very stable throughout testing.
Speeds were good and file access lag times were minimal thanks to an absence of full on-read protection these days. Our set of tasks ran through very quickly too, with low RAM use but a notable increase in use of CPU cycles.
Detection was very strong once again, dropping just a little into the proactive sets, and the certification sets were dealt with properly too, the product earning a VB100 award quite comfortably.
Main version: 9.1
Update versions: 4557.690.1951 build 948, build 957, build 695, build 971
Last 6 tests: 5 passed, 0 failed, 1 no entry
Last 12 tests: 6 passed, 0 failed, 6 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Solid
Sister product to Agnitum’s Outpost and sharing the same core engine, Defenx has an established history in our comparatives dating back to 2010, with a good rate of passes.
Installation took a little while to complete and a reboot was required at the end. The interface is clean and uncluttered, providing easy access to a good range of controls, and it maintained stability reliably throughout our set of tests.
Scanning speeds were a little on the slow side initially but very rapid later on, while overheads started a little high but improved greatly after initial settling in. RAM use was somewhat above average, CPU use not bad at all, and impact on our set of activities was noticeable but not too heavy.
Detection was no more than respectable in the reactive sets, dropping rather low into the proactive weeks, but the WildList and clean sets presented no difficulties and a VB100 award is earned by Defenx.
Main version: 3.0.0.4
Update versions: 13.3.21.1/539358.2014102215/7.57342/5665768.20141022, 13.3.21.1/542890.2014111100/7.57657/5729147.20141110, 13.3.21.1/543158.2014111401/7.57713/5753427.20141113, 13.3.21.1/543660.2014112111/7.57858/5816551.20141121
Last 6 tests: 5 passed, 0 failed, 1 no entry
Last 12 tests: 9 passed, 0 failed, 3 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Stable
Hailing from Korea, ESTsoft first entered our tests in 2011 and has generally done well ever since. The product integrates the Bitdefender engine along with some in-house technologies; during installation we are also informed that the Sophos engine is available, but not on 64-bit platforms.
Installation of the base product takes only a few minutes but updates took rather longer – pushing 20 minutes on occasion. However, these delays may not affect users based in the company’s native regions. The interface is busy with lots of information. Navigation is reasonably straightforward, although hampered in parts by some less-than-clear descriptions. Configuration options are provided in slightly more than minimal depth. Stability was knocked very slightly by some odd variations in archive detection from run to run.
Scanning speeds were slow first time around and a little better in the warm runs, while overheads were not too bad initially and barely perceptible later on. Resource use was around average, with a lowish impact on our set of activities.
Detection was strong, with a fairly gradual decline through the RAP sets. With no issues in the certification sets, ESTsoft earns another VB100 award.
Main version: 5.0.9.0350
Update versions: 5.158/23.039, 23.123, 23.171, 23.201
Last 6 tests: 4 passed, 1 failed, 1 no entry
Last 12 tests: 9 passed, 1 failed, 2 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Fair
Fortinet’s products have been taking part in our comparatives for many years, and the vendor’s recent test history shows a decent ratio of passes with only our annual Linux tests not entered.
The installation process is fairly speedy, the interface fairly simple with almost no controls at the user level. Stability was once again knocked by a couple of major system crashes – these have been under investigation by the developers for some time now and should have been completely resolved by the time this report is published.
Scanning speeds were OK, if a little slow over binaries, while overheads were fairly low in most areas, at least with the default settings. Resource consumption was fairly average, with a reasonably low hit on the speed of our set of activities.
Detection was once again very strong in the response sets, dropping away a little into the proactive sets but remaining respectable. The core certification sets were handled well, and Fortinet earns another VB100 award.
Main version: 2.7.30
Update versions: 1.7.5/89433, 1.8.3/89564, 89658, 89722
Last 6 tests: 4 passed, 0 failed, 2 no entry
Last 12 tests: 6 passed, 3 failed, 3 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Solid
Our test history for Ikarus shows some strong improvements lately, with a nice run of passes building up over the last year or so.
Installation is pretty speedy, and the .NET interface looks clear and simple without too many options to confuse things. Stability was impeccable, with no issues to report.
Scanning speeds were slow in the first runs but blazed through the repeat jobs very quickly indeed. Overheads were distinctly heavy though, with some slight signs of improvement in the warm runs, in some areas. However, our set of activities ran through very quickly indeed, without any major drain on system resources.
Detection was very good in the response sets with the usual drop into the proactive sets, and with good coverage of the WildList and no false alarms in the clean sets a VB100 award is easily earned.
Main version: 15.2.0000
Update versions: 5.2.2/12.163, 5.2.1.0000
Last 6 tests: 5 passed, 0 failed, 1 no entry
Last 12 tests: 10 passed, 0 failed, 2 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Stable
Our test history for iSheriff is complicated by several changes in ownership and rebrandings, yet is pleasingly simple in its nice neat run of passes over the last few years.
Set-up is fairly speedy, although this may have been helped by the lab team’s familiarity with the complex online control system from which the installer is fetched. Updates proved rapid too. The controls are split between a cloud-based portal and a local console which is also accessed via a browser, and are reasonably simple to navigate and operate. Stability was mostly OK, but on a few occasions scan logs failed to materialize or were not shown properly.
Scanning speeds were reasonable, a little slow over binaries where more attention is required, and overheads were uniformly light. RAM use was perhaps a fraction above average but CPU use a touch lighter than most, while impact on our set of activities was unexceptional.
Detection from the integrated Bitdefender engine was very strong in the response sets. Sadly, a special install prepared for the proactive sets was lost in a major disk crash, and with no offline set-up to fall back on, we don’t have a full set of RAP results available – we would expect to see scores similar to others based on the same core engine though. Fortunately, all went smoothly in the certification sets, and iSheriff earns another VB100 award.
Main version: 13.0.4.233(a)
Update versions: 13.0.4.233(b)
Last 6 tests: 3 passed, 0 failed, 3 no entry
Last 12 tests: 4 passed, 0 failed, 8 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Stable
Kaspersky’s small business solution, SOS, has a limited history in our comparatives, but has done well whenever it has participated.
The set-up process is fairly rapid, but initial updates were very slow, with 20-minute waits not uncommon. The interface has the usual professional look: stylish and highly polished, the standard Kaspersky green colour scheme replaced with a more sober and business-like grey one. Configuration is available in impeccably comprehensive depth.
On one occasion we were informed that the activation server was out of action for maintenance and were asked to come back later to complete the set-up. Other than that, there were no serious stability problems, although we did note that one scan of the local C: partition had only reached 25% when we abandoned it after several hours.
Scanning started out reasonably quickly and soon became lightning-fast in the warm runs, with overheads also starting OK and becoming barely perceptible. Resource use and impact on our set of tasks were both reasonable too.
Detection was decent if not stellar, and with perfect handling of the WildList and clean sets, Kaspersky earns another VB100 award without much trouble.
Main version: 4.6.305.0
Update versions: 1.1.11104.0/1.187.58.0, 1.187.1285.0, 1.187.1884.0, 1.1.11202.0/1.189.439.0
Last 6 tests: 5 passed, 0 failed, 1 no entry
Last 12 tests: 8 passed, 0 failed, 4 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Solid
Microsoft’s business-oriented product, formerly known as Forefront, has managed to pass every VB100 comparative it has taken part in, becoming a reliable participant over the last few years.
The installation process is speedy with updates rolled in, and the interface fits in nicely with its surroundings (as one would expect with the same company providing both product and platform). Configuration options are limited, but stability was impeccable throughout our tests.
Scanning speeds were a little sluggish over binaries and archives but fast elsewhere, while overheads were reasonable on first sight of files and barely noticeable later on. RAM use was low, CPU use a little above average, and impact on our set of activities was not too severe at all.
Detection wasn’t great, but remained fairly respectable, and with no issues in the core certification sets, Microsoft once again proves worthy of a VB100 award.
Main version: 6.81.11
Update versions: 7.00.00
Last 6 tests: 5 passed, 0 failed, 1 no entry
Last 12 tests: 8 passed, 2 failed, 2 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Stable
Our test history for Panda covers both its free ‘Cloud AV’ product and this, a more business-oriented version. It shows a strong set of passes in the last year, with just a few minor blips in the last few years since the company’s return to VB100 testing.
The set-up process is a little more involved here than in the super-rapid home-user product, but it still doesn’t take too long to complete. The interface will be familiar to users of the home product: small and minimal with just a basic set of configuration options. Stability was dented only by a single incident – a minor freeze under heavy pressure.
Scanning was a little slow over archives, which are covered in-depth by default, but decent elsewhere. Overheads were light with the default settings, which do not check many file types on-read, and not too bad with more complete coverage enabled. Our performance measures showed very minimal use of system memory, not much CPU use either, but a noticeable impact on our set of tasks.
Detection was pretty good in the response sets, but no proactive data is available thanks to the product’s reliance on the cloud. The certification sets were well handled, and a VB100 award is easily earned by Panda.
Main version: 2.5.0.23
Update versions: 13.3.21.1/539358.2014.102215/7.57342/5665768.20141022, 13.3.21.1/542890.2014111100/7.57657/5729147.20141110, 13.3.21.1/543160.2014111418/7.57721/5757136.20141114, 13.3.21.1/543660.201412111/7.57858/5816551.20141121
Last 6 tests: 5 passed, 0 failed, 1 no entry
Last 12 tests: 9 passed, 0 failed, 3 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Fair
Roboscan is closely related to the ESTsoft solution, mirroring its performances in most of our tests and showing a good run of passes in the last few years.
Set-up is speedy but updates tend to be rather slow, topping 20 minutes on at least one occasion. The interface closely resembles that of ESTsoft’s ALYac, with a lot of business going on and a few of the controls obscured by unclear language. Nevertheless, a decent level of configuration is provided. Stability was knocked by a number of scans crashing out – mainly but not exclusively when handling large sets of malware samples – and occasional complaints about memory shortages.
Scanning speeds were very rapid and overheads not too bad at first and barely detectable in the warm runs. There was low use of RAM (at least in normal, everyday use), reasonable CPU use, and very little impact on our set of activities.
Detection from the underlying Bitdefender engine was of course excellent, and with no issues in the certification sets a VB100 award is earned.
Main version: 10.0.35352 Beta
Update versions: N/A
Last 6 tests: 1 passed, 0 failed, 5 no entry
Last 12 tests: 1 passed, 0 failed, 11 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Stable
A newcomer to our tests, TeamViewer will doubtless be familiar to many of our readers for its flagship remote-access software – a useful tool for anyone who is regularly called in to investigate issues on the systems of distant friends and relatives. The vendor’s anti-malware effort was still in beta at the time of testing, released to the public as this report was being finalized, and some of the bugs mentioned will doubtless have been mopped up by the time real-world users get their hands on it.
We encountered some complications during early installation attempts, but these can all be ascribed to the beta status of the product, and by the last few rounds of testing the process had become smooth and simple. The local interface is little different from the standard remote-control software, with the bulk of the anti-malware side operated from a web-based console which proved reasonably easy to operate. Stability was a little shaky, with a number of scans crashing out, and dependence on web access meant that no proactive RAP scores could be recorded.
Scanning speeds followed the common pattern of being slowish to start with, then very fast in the warm runs, showing some nice scanning optimization. File access lag times were light from the off with further improvements later on. Our activities measure showed minimal impact on everyday tasks, running through faster than with Windows Defender active, with low RAM use and CPU use also below average.
Detection, at least in the reactive sets, proved very strong indeed thanks to the Bitdefender engine underlying things, and the core certification sets were dealt with well, earning TeamViewer a VB100 award on its first attempt.
(Click for a larger version of the table)
(Click for a larger version of the table)
(Click for a larger version of the table)
(Click for a larger version of the table)
(Click for a larger version of the table)
(Click for a larger version of the table)
(Click for a larger version of the chart)
(Click for a larger version of the chart)
Main version: 9.1
Update versions: 4654701.1951 build 948, build 957, build 965, build 971
Last 6 tests: 5 passed, 0 failed, 1 no entry
Last 12 tests: 8 passed, 0 failed, 4 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Solid
Agnitum’s VB100 test history shows a good run of passes in the last couple of years, following a short absence, and in the long term a strong ratio of passes since our first look at Outpost in 2007.
Installation is a little lengthy and involved, but updates are nice and speedy. The product interface is sparse and uncluttered but provides the most important bits of information nice and clearly, with good access to a decent range of options. Stability was unshaken by any of our high-stress tests.
Scanning speeds were slow initially but super-fast later on. File access lag times were a touch high to start with over most types of files, but again thanks to some nice optimization they quickly became imperceptible. RAM use was slow, CPU use perhaps just a touch on the high side, and impact on our set of tasks was discernible but not excessive.
Detection was reasonable, a little way behind the leading pack but still not too bad, and there were no issues in the certification sets, earning Agnitum a VB100 award.
Main version: 2015.10.0.2206
Update versions: 141021-0, 141104-0, 141111-0, 141118-0
Last 6 tests: 3 passed, 2 failed, 1 no entry
Last 12 tests: 8 passed, 3 failed, 1 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Stable
Avast’s products have been a fixture in the VB100 line-up since the nineties, and over the years have built up some nice strings of consecutive successes. Things have been a little wobbly lately, but are hopefully returning to an even keel.
Installation of the 2015 offering includes some optional extra ‘free stuff’ as well as an initial quick scan, but the process completes in good time with rapid updating. The GUI is a thing of rare beauty, showing excellent attention to detail as well as nice design sensibilities and providing a comprehensive set of controls in a usable format. Stability was decent, although we did get a few memory errors at various points during the test.
Scanning speeds were around average for this month’s field and overheads were very light with the default settings (which do not monitor most file types on-read). Resource use and impact on our set of tasks were also around average.
Detection was OK, falling a little short of the heights reached by some this month, but still well within the bounds of respectability. There were no issues in our WildList or clean sets, and a VB100 award is duly earned by Avast.
Main version: 2015.0.5557
Update versions: 4189/8508, 4213/8554, 4213/8587
Last 6 tests: 6 passed, 0 failed, 0 no entry
Last 12 tests: 11 passed, 0 failed, 1 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Fair
It has been almost five years since AVG skipped a VB100 comparative, and over that time its success has been regular with only a tiny handful of problems.
Installing the 2015 edition of the product via the ‘express’ option was unchallenging and fairly zippy, with most updates only taking a couple of seconds to complete. The interface is dark and moody but provides nice large informative tiles and a good level of fine-tuning under the covers. Stability was mostly good, although we did see an unexpected reboot in the middle of a scan on one occasion.
Scanning speeds were pretty decent from the off and much better in the warm runs, while overheads were perhaps a touch high first time out of the box but barely perceptible in repeat runs. Resource use was reasonable, and our set of activities completed in decent time.
Detection was very strong indeed, right up with the best, and the certification sets presented no unwanted surprises either, easily earning AVG a VB100 award.
Main version: 14.0.7.306
Update versions: 8.11.180.122, 8.11.183.52, 8.11.184.204, 14.0.7.342/8.11.187.66
Last 6 tests: 3 passed, 0 failed, 3 no entry
Last 12 tests: 6 passed, 0 failed, 6 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Solid
Avira’s free home-user offerings generally only appear in our desktop tests every other cycle. Nevertheless, they have built up a good string of passes over the last five years or so.
Installation is completed in good time, although reboots are required on some updates. The interface is neat and tidy without too much clutter but provides a thorough set of options for those who need them. Stability was perfect with no issues noted.
Scanning speeds weren’t too bad and remained very consistent from run to run. File access lag times were minimal thanks to on-read protection being absent, and this stretched to our set of activities which zipped through slightly faster than our Windows Defender baseline measures.
Detection was excellent across the board and there were no issues in the core sets, thus Avira comfortably earns its second VB100 award this month.
Main version: 5.0.3.93878
Update versions: N/A
Last 6 tests: 1 passed, 0 failed, 5 no entry
Last 12 tests: 2 passed, 0 failed, 10 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Fair
Chinese web giant Baidu provides a range of anti-malware solutions, including a Chinese-only product for the domestic market and this, a free product for the global market integrating the high-performing Avira engine. This one has only appeared once before on the VB100 test bench, in 2013, when it earned certification with ease and picked up some highly favourable comments from the lab team.
Installation of the latest version proved extremely simple and speedy, completing in half a minute or less with no need to reboot. The interface is attractive and professional, proving simple to navigate and operate. Stability was decent for most purposes, but a few problems did crop up – a few scans crashed out or failed to report anything on completion, and there were a few incidents of logging failing to record or display properly, but nothing too severe.
Scanning speeds were fairly zippy to start with and got better later on, while overheads proved distinctly heavy initially but very light indeed after that first in-depth look at files. RAM use was on the low side, CPU use a little high, and impact on our set of activities distinctly high too.
Detection was very solid, dropping away a little into the proactive sets as one would expect. In the core certification sets there were no problems and Baidu’s global product earns another VB100 award without difficulty.
Main version: 18.17.0.1227
Update versions: 7.57341/5715264, 7.57661/5730290, 7.57718/5755645, 7.57851/5812241
Last 6 tests: 6 passed, 0 failed, 0 no entry
Last 12 tests: 12 passed, 0 failed, 0 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Fair
Bitdefender’s recent test history speaks for itself, with every test entered and passed for the last two years and then some.
Installation of the 2015 edition of the product includes downloading a fresh install package if the one you’re using is not fully up to date, but still doesn’t take too long to complete. The interface is clean and stark but manages to provide a good set of controls. Stability was dented by some problems with updating in one round of testing (it seemed the update server was out of action for a spell) and several big scans freezing or failing to produce results, but in everyday use it seemed fine.
Scanning speeds were a little slow at first but very fast in the warm runs, at least in most sets; somewhere in our ‘miscellaneous’ set things seemed to get stuck, with each scan abandoned after reaching the one-hour mark. Overheads, on the other hand, were very light indeed, at least with the default settings. RAM use was low, CPU use a little high, and once again our set of activities seemed to take an enormous amount of time to complete – we continue to search for the root cause of this oddity.
Detection, as ever, was splendid, and with the certification sets nicely dealt with a VB100 award is well earned.
Main version: 14.1.287.1
Update versions: 7.57342, 15.0.288.1/7.57601, 7.57716, 15.0.289/7.57851
Last 6 tests: 4 passed, 0 failed, 2 no entry
Last 12 tests: 9 passed, 0 failed, 3 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Solid
BullGuard’s VB100 test history is almost as impressive as that of its core engine provider, Bitdefender, with nothing but passes (and the occasional no-entry, mainly in our annual Linux tests) over the last four years or so.
Installation isn’t the quickest but includes some options to install beta features and run some system optimization tasks, with updates also taking a few minutes. The GUI has an informative tiled format on the front page, covering a wealth of features, and provides good configuration options under the covers. Stability was impeccable, with no issues encountered throughout the set of tests.
Scanning speeds were OK to start with and super-fast on repeat runs, with overheads also pleasingly low. Our set of tasks was completed in excellent time too, with CPU use perhaps a shade on the high side but RAM use fairly low.
Detection was excellent, with just a gentle decline through the test sets. With the certification sets handled without issues, BullGuard’s very strong performance this month easily earns VB100 certification.
Main version: 13.3.209.000
Update versions: 8.3.4.7
Last 6 tests: 3 passed, 0 failed, 3 no entry
Last 12 tests: 3 passed, 2 failed, 7 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Stable
Check Point’s venerable ZoneAlarm brand has a somewhat patchy history in our tests, but has started building up a nice string of good showings over the last year or so.
Installation of the latest ‘Extreme’ edition offered a ‘quick’ option, but this still took several minutes and included the offer of a toolbar. The interface hasn’t changed much for some time, but provides a good level of clear information on the home screen and a decent basic set of controls. Stability was reasonable, with a single freeze and some logging issues observed, but nothing too serious.
Scanning speeds were distinctly sluggish, with our set of binaries almost reaching the point at which we abandoned the job, but did at least show some signs of speeding up in repeat runs. Overheads were also a little on the high side, but again improved a little later on. Our main performance tests showed fairly average resource use and average impact on everyday tasks though.
Detection, aided by the Kaspersky engine, was decent in the reactive sets, with no score in the proactive sets as the product apparently relies heavily on the cloud. The WildList and clean sets were handled well, and a VB100 award goes to Check Point for its efforts.
Main version: 5.1.31
Update versions: 5.4.11
Last 6 tests: 1 passed, 3 failed, 2 no entry
Last 12 tests: 2 passed, 7 failed, 3 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 3
Stability: Solid
CYREN’s Command product managed a pass in the last comparative but has had a rather rough time of late, with false positives cropping up rather more often than we would like to see.
Installation of this compact product remains very rapid, and the interface has a very simple, unflashy and minimalist look, providing a decent basic set of controls. Stability was good throughout.
Scanning speeds were reasonable and nicely consistent, with lag times a little heavy. RAM use was a little higher than some, but CPU use not too bad, and while our set of tasks was noticeably slowed, the impact was not too significant.
Detection was very impressive indeed in the reactive sets, and not too bad in the proactive sets despite the lack of input from the company’s cloud resources, The WildList sets were handled well, but in the clean sets came the counterbalance to that super detection: a scattering of false alarms including several components from a VMware installation, flagged with heuristic rules. This was enough to deny CYREN a VB100 award this month.
Main version: 9.0.0.4570
Update versions: N/A
Last 6 tests: 4 passed, 1 failed, 1 no entry
Last 12 tests: 9 passed, 1 failed, 2 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Solid
Emsisoft’s recent test history shows a good cluster of passes over the last couple of years since it adopted the Bitdefender engine alongside the company’s own technology.
Installation of the latest version takes rather a long time, but updates are speedy. The product GUI has a very bright and clear tiling system on the front page and looks very clean and slick, with a decent set of controls. Stability was very good, with no problems noted.
Scanning speeds weren’t too bad and overheads were very light thanks to minimal on-read protection, while our set of tasks got through in super-quick time with normal resource usage.
Detection was strong with a bit of a drop into the proactive sets, and there were no issues in the certification sets, thus earning Emsisoft a VB100 award.
Main version: 14.0.1400.1657 DB
Update versions: N/A
Last 6 tests: 6 passed, 0 failed, 0 no entry
Last 12 tests: 11 passed, 1 failed, 0 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Stable
There are no gaps in our test history for eScan going back more than five years, and a strong rate of passes over that time.
Installation and updating takes some time – more than ten minutes on average. Once up, the product interface has a dark grey background and grey text in places, but seems to get brighter each time we see it, with tiles on the main screen now luminously white. Stability was good, but several large scans of malware sets did freeze up or crash out.
Scanning speeds were impressive from the off and got even better in repeat runs, while lag times were light across the board. RAM use was perhaps a fraction higher than the average this month, but CPU use wasn’t very high and our set of tasks got through in rapid time.
Detection, aided by the integrated Bitdefender engine, was as strong as one would expect, dipping down a little in the later parts of the RAP sets. There was perfect coverage of the WildList with no alerts in our clean sets, meaning that eScan earns another VB100 award.
Main version: 8.0.304.0
Update versions: 10603, 10669, 10706, 10737
Last 6 tests: 6 passed, 0 failed, 0 no entry
Last 12 tests: 12 passed, 0 failed, 0 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Solid
ESET’s test history speaks for itself, being uniformly excellent going back a very long time indeed.
Installation of the version 8 product doesn’t take long, with updates over in a flash. The interface is another looker: nicely uncluttered but with good information and easy access to a comprehensive set of controls. Stability was once again impeccable, with no problems even under heavy stress.
Scanning speeds were just OK initially but blazing fast later on, with very low overheads, lowish impact on our set of everyday tasks and reasonable resource usage.
Detection was decent, with a distinct tailing off into the later parts of the sets, and the certification sets were once again dealt with perfectly, earning ESET yet another VB100 award.
Main version: 17.21.0.925
Update versions: 7.57350/5652003, 7.57659/5729934, 7.57716/5755021, 7.57851/5812241
Last 6 tests: 1 passed, 0 failed, 5 no entry
Last 12 tests: 1 passed, 0 failed, 11 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Solid
FileMedic may be a new name to many readers, but the product does at least have some provenance, the company having evolved from old-timer MKS which has a couple of entries in our test records.
Installation of this all-new solution was pretty slow for some reason, much of the time being taken up with updates. The GUI looks attractive and professional, highly reminiscent of several recent Bitdefender products (the Bitdefender engine is also powering things). Stability was good, with no problems noted.
Scanning speeds were OK initially and very fast indeed in the warm runs, while overheads were very light. RAM use was very low, CPU use rather high and our set of tasks ran through very slowly indeed.
Detection was excellent though, remaining strong well into the proactive sets, and with a perfect score in the certification sets a VB100 award is easily earned by FileMedic.
Main version: 25.0.2.2
Update versions: AVA 24.4573/GD 25.4063, AVA 24.4827/GD 25.4136, AVA 24.4943/GD 25.4165, AVA 24.5080/GD 25.4193
Last 6 tests: 4 passed, 0 failed, 2 no entry
Last 12 tests: 9 passed, 0 failed, 3 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Stable
G DATA’s history in our tests shows a strong ratio of passes stretching back over a decade.
Installation followed standard lines and didn’t take too long. The product looks attractive and clear with plenty of information on the surface and a wealth of fine-tuning controls underneath it. Stability was good, but on one occasion we did get some ‘server busy’ messages during the performance tests.
Scanning speeds were reasonable and quite consistent, although the local system partition scan did take a fair while to complete. Overheads were uniformly light, with low RAM use, but CPU use was above average, as was impact on our set of tasks.
Detection was splendid as ever with the combination of the Bitdefender engine and in-house technologies, and there were no problems in the certification sets, earning G DATA another VB100 award.
Main version: 6.0.0.0
Update versions: 5715264, 5729520, 5750765, 5839394
Last 6 tests: 1 passed, 0 failed, 5 no entry
Last 12 tests: 2 passed, 2 failed, 8 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Fair
It’s a little hard to spot patterns in Hauri’s history in our tests, with periods of regular participation interspersed with quiet times, and passes similarly clustered between less successful periods.
Installation of the product, still labelled ‘2011’, is very slow indeed, regularly taking longer than 20 minutes from start to finish. The GUI is little changed, as one would expect, looking reasonably current though, with large tiles displaying the status of the major components and a reasonable set of controls. Stability was shaky, with a number of scans freezing, crashing out or failing to start, often showing unhelpful ‘unspecified error’ messages.
Scanning speeds were fairly average, if nicely consistent, with overheads and system resource consumption a touch on the heavy side, but our set of tasks were completed at a reasonable rate.
Detection, helped along by the Bitdefender engine, was very good, and with no surprises in the certification sets a VB100 award is earned.
Main version: 14.2.0253
Update versions: 9.185.13772, 14.2.0255/9.185.13901, 9.185.13992, 9.185.14055
Last 6 tests: 4 passed, 0 failed, 2 no entry
Last 12 tests: 6 passed, 1 failed, 5 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Solid
K7’s products appear with ever greater frequency in our tests, generally doing well and always popular with the lab team for ease of use and reliability.
Installation takes very little time to complete, and once up, the interface has a rugged efficiency to it, complete with military-style fonts. Under the covers, a good level of configuration options is provided. Stability was once again flawless, with no problems noted.
Scanning speeds were pretty good, overheads not bad at first and barely perceptible in the warm runs. Resource use was around average, with slowdown through our set of tasks nice and low.
Detection was pretty good in the earlier parts of the RAP sets, declining steadily into the later weeks. The core certification sets were all dealt with well though, earning K7 a VB100 award.
Main version: 15.0.1.415
Update versions: 15.0.1.415(a)
Last 6 tests: 4 passed, 0 failed, 2 no entry
Last 12 tests: 5 passed, 0 failed, 7 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Stable
Our second product from Kaspersky this month is a full IS suite, which has its own entry in our test history, filled sporadically when the company chooses to submit multiple entries with a good rate of success.
Installation is a little slow, with updates adding some time. The product interface has a pale and sensitive look, with the usual stylish, well-honed feel and thorough set of controls. Stability was good, but on one job the scanner did seem to freeze up for quite some time.
This incident notwithstanding, scanning speeds were mostly good, starting off a little slower over some file types but very fast indeed later on. Overheads similarly started low to medium and became very low indeed in the warm runs. Resource use was fairly standard, with a slightly high impact on the runtime of our set of standard activities.
Detection was decent – a little behind the leaders but still respectable, and with no unwanted surprises in the certification sets, a VB100 award is well deserved.
Main version: 11.4.6792.0
Update versions: 5665768, 5729520, 5775339, 5814257
Last 6 tests: 3 passed, 0 failed, 3 no entry
Last 12 tests: 6 passed, 0 failed, 6 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Stable
Lavasoft’s Ad-Aware has been taking part in our desktop tests for the last few years, with a steady run of passes built up.
Installation this month took rather a while, but updates were rapid. The product GUI had a major overhaul not so long ago and still looks fresh and clean – a little wordy in places but simple to navigate and providing decent controls. Stability was good for the most part, although we did note a few issues including messages complaining about loss of connection to a service on a couple of occasions.
Scanning speeds were OK first time round and very quick later. Overheads were similarly improved in the warm runs after a decent start, although a little slower with more thorough options enabled, as one would expect. RAM use was low, CPU use average, but once again our set of tasks took a long time to complete. Investigations have shown that this is somehow connected to the download phase fetching the sample files from a local intranet server over HTTP.
Detection, helped along by the Bitdefender engine, was very strong in the reactive sets. Sadly, the set-up required for the proactive test was damaged in a severe hard drive crash, so no data could be gathered there, but one would expect scores similar to others with the same technology underlying things. There were no issues in the core sets though, and a VB100 award is well earned.
Main version: 1.1.107.0
Update versions: 89410, 89564, 89676, 89722
Last 6 tests: 0 passed, 4 failed, 2 no entry
Last 12 tests: 0 passed, 7 failed, 5 no entry
ItW on demand: 100.00%
ItW on access: N/T
False positives: 0
Stability: Buggy
MSecure comes into this test on the back of a string of disappointing test results, most of them caused by a rather serious problem affecting on-access protection.
Installation was rapid and easy, bringing up a glossy and clean interface providing clear status info and a decent basic set of controls. Stability seemed mostly fine, with the new GUI blanking out occasionally, but no crashes or hangs.
Scanning speeds were very slow over binaries but decent elsewhere, with file access lag times looking low, although this may have been affected by an ongoing issue with the real-time protection. RAM use was low, CPU use around average, and our set of tasks ran through in good time.
Detection was pretty strong, very good indeed in the reactive sets, with the Ikarus engine powering things. The WildList sets were handled well on demand, and there were no issues scanning the clean sets either, but once again in the on-access test we saw barely anything being picked up. With the same problem coming up once again – that of many major file types being ignored despite being listed in the set of extensions to be covered – this is counted as a fairly major bug as well as reason to deny to MSecure a VB100 award once again.
Main version: 11.00
Update versions: 11.0.0
Last 6 tests: 4 passed, 1 failed, 1 no entry
Last 12 tests: 7 passed, 4 failed, 1 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Solid
Norman’s test history is complicated by a switch from entirely in-house technology to the Bitdefender engine not long ago, and looks likely to be further complicated in the near future.
Installation of the version 11 suite proved fairly straightforward and not too time-consuming, with updates completing very rapidly. The interface looks sparse and simple but provides clear basic information and some decent configuration controls. Stability was good, with nothing to report.
Scanning speeds were fairly zippy, overheads noticeable but not too bad, CPU use a little higher than some but RAM use and impact on our set of tasks pleasingly low.
Detection was strong with something of a dip into the proactive sets, and there were no issues in the core sets, thus earning Norman another VB100 award.
Main version: 15.0.4
Update versions: N/A
Last 6 tests: 2 passed, 0 failed, 4 no entry
Last 12 tests: 2 passed, 0 failed, 10 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Stable
Panda’s secondary entry in our test records shows only a single previous appearance, a pass, but on a few occasions products fairly similar to this have appeared in the company's main line, generally doing nicely.
Installation is pretty quick, as we have come to expect from Panda’s cloud-heavy solutions, but the interface is a little more complex than we’re used to, with a colourful tiled front page and plenty of information and options for a range of components. Stability was mostly good, with just a few temporary problems encountered in the activation system.
Scanning speeds were a little slow and overheads fairly light, helped by a lack of complete on-read protection. Resource use and impact on our set of activities were both low.
Detection was pretty good in the reactive sets, with again no proactive data thanks to reliance on the cloud. The core sets were nicely dealt with, and a second VB100 award goes to Panda this month.
Main version: 1.0.0.48
Update versions: 1.0.0.50, 1.0.0.46
Last 6 tests: 1 passed, 2 failed, 3 no entry
Last 12 tests: 4 passed, 2 failed, 6 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Buggy
PC Pitstop’s solutions made a fairly good start in our comparatives, with a string of passes in 2013 assisted by the VIPRE engine. In the last couple of tests we have, at the request of the developers, run our standard test suite in reverse – using the real-time mode for the main detection tests to bring the firm’s own whitelisting technology into play. Results of this approach have been mixed, with some very high detection scores indeed, counterbalanced by some fairly spectacular false positive rates, and even some incomplete coverage of the WildList. For this test we have returned to our normal approach to testing, using mainly the VIPRE technology referred to within the product as ‘industry standard’.
Installation of the product itself is pretty speedy, although the additional anti-malware components take some time to fill in. The interface is mainly informational, displaying lists of current vulnerabilities and threats with very little by way of controls. Stability was shaky, with numerous scans crashing out, real-time protection shutting down from time to time, and on one occasion an unexpected reboot.
Scanning speeds were decidedly slow, overheads a little on the high side in places, and impact on our set of tasks was fairly heavy too, with reasonably low resource use.
Detection was pretty good though, with decent scores in all sets and on this occasion coverage of the core certification sets was complete and accurate, earning PC Pitstop a VB100 award.
Main version: 5.0.0.5091(x64)
Update versions: N/A
Last 6 tests: 5 passed, 0 failed, 1 no entry
Last 12 tests: 9 passed, 1 failed, 2 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Solid
Qihoo has become a very consistent participant in our comparatives lately, rarely missing anything but Linux tests and generally doing well, with a clean sheet of passes in the last year.
Installation is very fast indeed, updates adding a little to the overall set-up time but not much at all. The product interface is clean and simple with nice large buttons, and provides a decent amount of configuration under the hood. Stability was perfect throughout our tests with no problems noted.
Scanning speeds were on the slow side, especially over binaries, with lag times looking minimal thanks to the real-time monitoring not actually intercepting file reads, but instead monitoring them on the way past. Resource use was fairly average, but our set of activities blasted through very quickly, largely unimpeded.
Detection was very strong, thanks to assistance from the Bitdefender engine, and with a clean run over the core certification sets, another VB100 award is earned by Qihoo.
Main version: 8.0
Update versions: 5530, 5549, 5552, 5560
Last 6 tests: 0 passed, 1 failed, 5 no entry
Last 12 tests: 0 passed, 1 failed, 11 no entry
ItW on demand: 21.50%
ItW on access: 12.36%
False positives: 3
Stability: Fair
Despite being another newcomer to our reports, we have been aware of SmartCOP for a couple of years already, having seen it arrive on the test bench but fail to make it to the full test on several previous occasions.
Installation is fairly straightforward and doesn’t take too long, bringing up a rather lurid orange-and-yellow GUI with a somewhat old-school look and feel but a decent set of controls. Stability was a little shaky, with several crashes observed and many scan jobs requiring repeat runs to complete.
Scanning speeds were very slow indeed, with several scans abandoned after running over the one-hour limit set, but our set of archives at least got through quickly. With no discernible detection of the standard EICAR test file, our archive measures don’t really indicate whether any of these files were being checked internally. File access lag times were imperceptible, thanks to the absence of on-read protection, and our set of activities was repeatedly aborted due to the product blocking various parts of the automation.
Detection was pretty mediocre to say the least, with pretty low scores across the board and, rather worryingly, the WildList handled less well on access than on demand. There were also a few False positives:, including the common wget tool and some items from Adobe. All this means that there is no VB100 award just yet for SmartCOP, and the firm still has some work to do to reach the required standard.
Main version: 8.10.25293.501
Update versions: N/A
Last 6 tests: 5 passed, 0 failed, 1 no entry
Last 12 tests: 10 passed, 0 failed, 2 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Solid
A clean sweep of all Windows comparatives for the last two years should give some indication of Tencent’s performance in our tests – reliably strong month after month, with the Avira engine running alongside the company’s own technologies.
Installation is very fast and updates similarly quick for the most part. With the interface all in Chinese, there’s not a great deal we can say about it other than that there seems to be a lot going on. Stability was very good, with no issues.
Scanning speeds were decent and overheads light, again thanks to there being no on-read protection by default. Our set of tasks ran through very rapidly, with RAM use a touch above average but CPU use on the low side.
Detection was good, with only the slightest dip into the later parts of the RAP sets, and there were no problems in the core sets, comfortably earning Tencent another VB100 award.
Main version: 8.10.25026.501
Update versions: N/A
Last 6 tests: 3 passed, 0 failed, 3 no entry
Last 12 tests: 3 passed, 0 failed, 9 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Solid
The second product from Tencent is something of an experimental edition containing only the company’s in-house detection capabilities.
Installation was again very rapid, with updates even quicker this time. Overall, the product seemed very similar to the mainline edition – a busy interface, rock-solid stability, decent scanning speeds and light system impact.
Detection was considerably lower than that of the mainline product, as one might expect, but the WildList was covered well, and with no False positives in the clean set, Tencent earns another VB100 award.
Main version: 8.0.5.3
Update versions: 34166, 8.0.4.3/34702, 34782, 35084
Last 6 tests: 2 passed, 0 failed, 4 no entry
Last 12 tests: 7 passed, 0 failed, 5 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Fair
The appearance of ThreatTrack Security’s VIPRE product in our tests has been something of a rarity of late, but in the past has been a regular participant with several good runs of passes.
Installation of the 2015 edition isn’t the fastest, but updates at least are speedy. The interface has had a facelift with boxy Windows 8-style tiling, and looks quite good; the control system remains limited but provides the basics. Stability was not perfect, with a number of crashes mostly happening during scans of large malware sets.
Scanning speeds were only OK initially but very quick indeed later on, while overheads were fairly low and also showed some signs of optimization in the warm runs. Our set of tasks ran through quickly, with lowish resource use.
Detection was excellent with good scores everywhere, and coverage of the certification sets was strong too with no issues to report and a VB100 award is well deserved.
(Click for a larger version of the table)
(Click for a larger version of the table)
(Click for a larger version of the table)
(Click for a larger version of the table)
(Click for a larger version of the table)
(Click for a larger version of the table)
(Click for a larger version of the chart)
(Click for a larger version of the chart)
Additional products that were submitted for testing this month, but which proved unsuitable thanks to instability or other problems included ULIS Adept AV.
At the end of another comparative on the evergreen Windows 7 platform we have a pleasingly high success rate with just a few products failing to make the grade for certification. Of these, we seem to have a full range of reasons for failure, with one scoring well for detection but tripped up by false alarms, one knocked out of contention by a rather serious bug crippling the detection, and another simply not yet up to the task.
Of the rest, many did excellently, with a good number of ‘Solid’ ratings in our stability table and not too many horrific bugs to cope with. It’s good to see several of our long-time regular products steadily building up long chains of success, indicating their ongoing dedication to quality.
Up next will be a Linux test with a far smaller field of participants but still plenty of challenges for the lab team, to be followed by another large desktop comparative on Windows 8.1. For that one we plan to make some more adjustments to our performance measuring systems, hopefully improving the completeness of the data provided while at the same time retiring some older and less relevant components.
As usual, we welcome any ideas, suggestions, complaints or criticisms, to the usual address ([email protected]).
Test environment. All tests were run on identical systems with AMD A6-3670K Quad Core 2.7GHz processors, 4GB DUAL DDR3 1600MHz RAM, dual 500GB and 1TB SATA hard drives and gigabit networking, running Microsoft Windows 7, 64-bit Professional edition, with SP1.
Any developers interested in submitting products for VB's comparative reviews, or anyone with any comments or suggestions on the test methodology, should contact [email protected]. The current schedule for the publication of VB comparative reviews can be found at http://www.virusbtn.com/vb100/about/schedule.xml.