2014-11-14
Abstract
The VB test team put 14 corporate products and 34 consumer products through their paces on Windows 8.1 - John Hawes has the details.
Copyright © 2014 Virus Bulletin
Our first look at Windows 8.1 arrives with its successor, Windows 10, not far off. This report is rather overdue, and with a lot of products to get through, the preamble will be kept to a minimum – suffice to say, the deadline was set for 25 June, and testing proper got under way in mid-July, giving developers ample time to prepare for the test. With a fair sprinkling of new names on the list, we reached a total of 48 products – some way short of a record, but still plenty to keep us busy.
Given the scale of the test, we decided to implement a division of products (which we had been planning for a while), dividing the product set into corporate and consumer-grade solutions. We asked vendors to specify into which category their products fell at submission time – some managed to do so, while others had to be chased up, and some left it entirely to us to determine what we felt was appropriate. As has been pointed out, this divide may not be very useful in our server tests, as few consumers are likely to be running server editions, but we plan to continue with the division of products in our desktop comparatives at least.
Windows 8.1 is really little more than a service pack for Windows 8, with not much to differentiate it during the install and set-up process, and not much beyond a slightly more usable desktop to distinguish it during testing. As usual, we kept additions to a minimum, installing only a few very basic tools, and locked down our test image on the deadline day with no updates beyond those included in the basic install media.
We had something of a clearout of our test sets, removing from the clean sets several swathes of software from sources which seemed to be including rather large numbers of ‘grey’ items, most of which were contaminated with unwanted ‘free extras’ such as toolbars. We added a bundle of new files to make up for this, and the set size remained reasonably close to that of previous months, with the 900,000 files weighing in at around 200GB. Other sets were built along standard lines, with the proactive parts of the RAP set put together in the weeks leading up to the test deadline and the reactive sections put together on the fly as testing proceeded. The WildList set was based on the ‘4.006’ list released a week or so prior to our deadline.
As usual, there were a number of products that could not be provided in a form that could be installed and updated offline, so these were given full installs on the deadline day, with updates sucked down in the usual manner and the systems frozen for later use in the proactive tests. For the rest of the tests, each product was installed and updated on the day of testing, with fresh installs on fresh systems to ensure that any bugs could be analysed properly and reproduced where possible.
Main version: 140625083609
Update versions: 140717142001,140806100310,140822083427
Last 6 tests: 3 passed, 0 failed, 3 no entry
Last 12 tests: 3 passed, 0 failed, 9 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Stable
After a number of sporadic and rather unconvincing performances, followed by a lengthy absence from our tests, Arcabit’s switch to the popular Bitdefender engine has proved a wise choice, with the product turning in a couple of good results in recent tests.
The angular interface fits in nicely with the styling of Windows 8, with big clear text and a reasonable set of controls. Stability was mainly good, although on one occasion a scan of part of our clean sets exited unexpectedly with no results reported.
Scanning speeds were decent too, and very stable across various runs, while file access lag times started a little high but became very slight on later runs. Resource use was low and our set of tasks completed in very good time.
Detection was very good indeed, and with no problems in the certification sets, a VB100 award is easily earned, getting this month’s comparative off to a good start.
Main version: 14.0.5.450
Update versions: 8.03.20.16/8.11.156.242, 8.03.20.34/8.11.160.212, 14.0.5.464/ 8.03.24.02/8.11.165.68, 14.0.6.552/ 8.03.24.16/ 8.11.168.126
Last 6 tests: 5 passed, 0 failed, 1 no entry
Last 12 tests: 9 passed, 0 failed, 3 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Solid
Avira’s ‘Pro’ solution has a very solid record in our tests, having had no serious problems in almost five years and very few tests not entered.
The product has looked fairly similar for some time, with a slight revamp to give it the appropriate boxiness for Windows 8, and is nicely laid out overall, with good access to a wide range of configuration options. Stability was very good throughout testing, with no problems to report.
Scanning speeds were decent and very stable too, with the very light overhead times recorded in our tables partly attributable to on-read protection being inactive by default. Resource use was very low and our set of activities – which gives a more accurate measure of system impact – also showed relatively little slowdown.
Detection was very strong indeed, even into the proactive sets, and the core sets were handled with precision, easily earning the product a VB100 award.
Main version: 1.5.0.15
Update versions: 2969, 3786, 1.5.0.18/ 4257
Last 6 tests: 1 passed, 0 failed, 5 no entry
Last 12 tests: 1 passed, 0 failed, 11 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Flaky
This is the first time BluePex has featured in a full VB100 report – although the product (which incorporates the VIPRE engine from ThreatTrack Security) has been submitted for testing on numerous occasions in the past few years, it has not proved sufficiently stable to produce a usable set of results until now.
The company hails from Brazil, but the product interface – which has a reasonably pleasant appearance and an acceptable set of basic controls – is available in English as well as its native Portuguese. At times, the interface seemed reluctant to accept this language adjustment though, switching back to Portuguese on a number of occasions after crashes.
Indeed, there were a number of crashes, and the fact that the product has finally made it to a full comparative review may say more about the tenacity of the test engineer tasked with wrangling the product than any great improvement in its stability. We saw a number of scans crashing out, often with no results to report, and on a number of occasions the app, or even the entire machine got stuck in a hang. On one occasion, an unexpected reboot occurred. All of this led to a rather dismal stability rating.
Scanning speeds were a little on the slow side, but showed some signs of improvement in the warm runs over some file types, while overheads were a little high. Resource use was OK though, and our set of activities got through in decent time.
Detection was excellent in the response sets, with a slight drop into the proactive sets, and the WildList and clean sets were handled well – although, as usual with products using the VIPRE engine, a number of files were not blocked in real time, instead being scanned in the background and alerted on some moments after being written to disk, which does not inspire great confidence. Nevertheless, and despite the rather poor stability, the product achieved the required basic standard for certification, and a VB100 award is granted to BluePex for the first time.
Main version: 9.1
Update versions: 4557.690.1951 build 828, build 860, build 870, build 882
Last 6 tests: 4 passed, 0 failed, 2 no entry
Last 12 tests: 4 passed, 0 failed, 8 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Stable
Defenx has built up a string of decent results in our tests of late, following an absence of a year or so.
The product has had a similar look for several years, although, as with several others, an extra bit of angularity has been added to it of late. Configuration options for the anti-malware component are fairly detailed for a suite product that provides a wide range of other protective layers. Stability was decent, with only a few minor wobbles under heavy stress.
Scanning speeds were rather slow to start with, but showed some good signs of optimization in the warm runs, and overheads were likewise decidedly heavy initially but improved greatly later on. Resource use wasn’t too bad, but our set of activities took a very long time to complete.
Detection was decent in the reactive sets, a little less impressive in the proactive areas, but the WildList set was well covered and there were no problems in the clean sets, earning Defenx another VB100 award.
Main version: 3.0.0.4
Update versions: 13.3.21.1/531240.2014062515/7.55534/11587836.20140625, 13.3.21.1/532393.2014071715/7.55903/10720898.20140717, 13.3.21.1/534759.2014080710/7.56242/9650047.20140807, 13.3.21.1/536470.2014082218/7.56468/7791660.20140822
Last 6 tests: 4 passed, 0 failed, 2 no entry
Last 12 tests: 7 passed, 1 failed, 4 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Stable
Another user of the popular Bitdefender engine, ESTsoft’s ALYac apparently now also has the option of using the Sophos engine – although a message informed us that this was not available on 64-bit platforms. The product has done well enough without it so far, building up a run of good performances over the last year.
The installation process is rather lengthy but not too laborious, and the interface is clear and friendly, with a large egg-shaped cartoon character adorning the main screen. Controls seem to be in decent depth, but a few oddities of language can make them less than clear to operate. Stability was decent too, with only a few minor issues noted.
Scanning speeds were no more than reasonable initially, but they increased noticeably in the warm runs, while overheads also started out fairly average but improved greatly later. Resource use was barely different from our Windows Defender baseline, and our set of tasks was not much slower.
Detection was very good, with excellent scores across the board, and the certification sets were well handled too, meaning that ALYac comfortably makes the grade for VB100 certification.
Main version: 3.42.2102.251
Update versions: 3.9.2592.2/30640,31776,32044,32556
Last 6 tests: 1 passed, 0 failed, 5 no entry
Last 12 tests: 1 passed, 0 failed, 11 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Stable
The first of several new names in this month’s comparative, Faronics is best known for its Deep Freeze range of system imaging tools. Based on the VIPRE SDK from ThreatTrack Security, the product is closely integrated with the Deep Freeze product line, operated from the same central management system.
The interface is fairly simple, unflashy and word-heavy, but provides some decent controls and remained fairly stable throughout testing, with just a single scan crashing out.
Scanning speeds were reasonable and showed signs of some decent optimization in later runs (over some file types at least), while overheads were a little heavy initially but again sped up nicely after settling in. Resource use and impact on our set of tasks were minimal, although as with other products using the same engine, there were indications that a number of files were not blocked in real time, instead being scanned in the background and alerted on some moments after being written to disk.
Detection was strong, with scores remaining high into the proactive sets. The WildList proved well handled, and with no problems in the clean sets either, Faronics joins the ranks of VB100 certified products on its first attempt.
Main version: 5.0.7.333
Update versions: 5.152/22.387, 5.0.9.347/ 22.472, 22.661
Last 6 tests: 4 passed, 1 failed, 1 no entry
Last 12 tests: 10 passed, 1 failed, 1 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Buggy
Fortinet rarely misses a VB100 comparative, and a strong record of passes, complemented by a steady improvement in detection over the last couple of years, have put it well up with the leaders in the last few tests. The FortiClient solution is fairly minimal, with a very basic GUI and controls restricted to the most obvious requirements, but it still manages to take a fair amount of time to install and update.
The interface was mostly stable and responsive, but once again we noted a number of unexpected shutdowns during intense scans, heavily denting the product’s stability rating; we are in discussion with the developers to try to pin down a cause for these problems.
Scanning was reasonably zippy, but a little slower over executable files, with overheads also a touch on the high side but improving in the warm runs. RAM use was low, CPU use a little high perhaps, but our set of activities ran through very quickly indeed – faster than with the baseline Windows Defender in place.
Detection was once again excellent in the reactive sets, not bad in the proactive sets either, and with no issues in the WildList or clean sets, Fortinet earns another a VB100 award.
Main version: 2.7.20
Update versions: 1.6.1/88019,88282, 2.7.29/88469, 2.7.30/1.7.5/88675
Last 6 tests: 3 passed, 1 failed, 2 no entry
Last 12 tests: 5 passed, 4 failed, 3 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Stable
Ikarus has had problems with false positives in the past but seems to be improving in this respect, with a good run of passes in the last year or so. The product remains little changed on the surface, with the .NET-based interface already appropriately boxy for the Windows 8 setting.
Stability was reasonable, with the GUI occasionally slow to respond at busy times and on a couple of occasions vanishing without warning, but protection remained in place and scans generally completed happily.
Scanning speeds were sluggish over archive files and very slow indeed on the first look at executables, but sped up considerably on repeat runs, while overheads were distinctly heavy throughout. Resource use was a little high too, and impact on our set of activities was fairly noticeable.
Detection rates were very good, only tailing off a little into the later parts of the sets, and with no issues in the certification sets, another VB100 award goes to Ikarus.
Main version: 5.1.0.0622
Update versions: 5.1.1/5.1.1/5.1.0/12.163, 5.1.0.0710/5.1.2/5.1.3, 5.1.0.0722/5.1.5/5.1.4, 5.1.1.0821/5.1.6/5.1.7
Last 6 tests: 5 passed, 0 failed, 1 no entry
Last 12 tests: 9 passed, 0 failed, 3 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Stable
Having branched out from the business side of Total Defense a year or so back, iSheriff inherited the cloud-based solution centred on the Bitdefender engine which we have been testing for some time – our records merge the old and new brands together, putting the cut-off at the point at which the cloud product came into play, rather than at the change of company name.
The set-up process is fairly simple and speedy, with rather quicker updates than we’re used to as well. The interface resides in the browser, and suffers from odd lag issues at times as a result, but it has become fairly easy to navigate after a little practice, and provides a reasonable set of fine-tuning options, split between local and central-management systems. Stability was mostly fine, with problems limited to the interface, mainly when dealing with large amounts of log data.
Scanning speeds were pretty decent – a little slower over executables, as one might expect, and with a little improvement in the warm runs. File access lag times weren’t too bad either, and remained very consistent over various measures. Resource use was very low and our set of tasks ran through very quickly.
Detection was very strong indeed, and with a good performance in the core sets, iSheriff comfortably earns a VB100 award.
Main version: 10.2.1.23
Update versions: 10.2.1.23(a)
Last 6 tests: 5 passed, 0 failed, 1 no entry
Last 12 tests: 10 passed, 1 failed, 1 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Fair
A trio of products were submitted by Kaspersky Lab this month, with the vendor’s enterprise desktop offering up first. This one is grouped with the company’s main line in our test records – one of the most complete, with only a handful of absences over many years and a number of healthy strings of passes. The setup process proved rather slow, with updates taking quite some time to complete, but once up and running the interface is very slick and attractive with an excellent selection of fine-tuning options.
Stability was mostly decent, but we did see a few instances of GUI freezes, and on one occasion the product simply refused to open – due to time restrictions, we had to reinstall on a fresh system to get things moving along.
Scanning was OK to start with, and once things had been checked out for the first time, proved very fast indeed, particularly over our sets of media and miscellaneous file types. File access lag times were barely perceptible in the warm runs, and not too bad the first time around either. Resource use was low and our set of activities got through nice and quickly.
Detection was decent, tailing off somewhat in the later parts of the sets where cloud lookups were not available, and the core sets were handled well, earning Kaspersky a VB100 award for its enterprise desktop product and boding well for the fortunes of the vendor’s other products on test this month.
Main version: 13.0.4.233(a)
Update versions: 13.0.4.233(b)
Last 6 tests: 2 passed, 0 failed, 4 no entry
Last 12 tests: 2 passed, 0 failed, 10 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Solid
Kaspersky’s second submission this month is a small business version, with an interface that closely resembles the vendor’s consumer solutions, but which has been given a makeover to a greyish colour scheme – perhaps considered more sober and suitable for business settings. Installation was again notable for rather slow update times.
Stability this time was perfect, with no problems to report.
As with the vendor’s first product, scanning speeds were OK initially and extremely fast in the warm runs, with lag times also showing strong improvement from a decent start. Resource use was again low, although our set of activities did take a little longer to get through.
Detection was decent, tailing off somewhat into the proactive sets, but there were no issues in the WildList or clean sets and another VB100 award is easily earned.
Main version: 4.6.205.0
Update versions: 1.1.10701.0/ 1.77.2074.0, 1.1.10802.0/1.179.179.0, 1.179.1871.0, 1.1.10903.0/ 1.181.75.0
Last 6 tests: 5 passed, 0 failed, 1 no entry
Last 12 tests: 6 passed, 0 failed, 6 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Stable
Microsoft’s business solution has become the company’s default submission lately, entering all of our Windows tests over the last year and doing well in all of them. Installation was very rapid, and although on a few occasions initial update attempts returned error messages, these were quickly overcome with a simple retry. The interface is very clean and slick, remaining stable throughout testing even under heavy pressure, and although configuration options are limited, all the basics are in place.
Scanning speeds were a little slow over archives and executables but decent elsewhere, and file access lag times were a little slower than we might expect, but sped up nicely into the warm runs. Resource use measures were low and our set of activities ran through a fraction faster than with Microsoft’s standard Windows Defender product in place.
Detection was impressive, improving considerably on recent performances, and the WildList and clean sets were admirably handled, easily earning Microsoft another VB100 award.
Main version: 2.5.0.23
Update versions: 13.3.21.1/528420.2014042215/7.54309/11657838.20140423, 13.3.21.1/532393.2014071715/7.55903/10720898.20140717, 13.3.21.1/534749.2014080615/7.56234/9674505.20140806, 13.3.21.1/536470.2014082218/7.56468/7791660.20140822
Last 6 tests: 5 passed, 0 failed, 1 no entry
Last 12 tests: 7 passed, 1 failed, 4 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Stable
Sibling to the ESTsoft product, Roboscan differs mainly in its branding, with the install process once again highlighted by a rather sluggish update time and the GUI similarly cartoony and reasonably simple to operate once a few oddities of language have been deciphered.
Stability was OK, although a few scans failed to complete or refused to produce results.
Scanning speeds were good, a little better in the warm runs, with overheads not bad to start with and barely perceptible later on. Resource use was fairly low, but our set of tasks took a little longer to complete.
Detection was excellent, as we have come to expect from the Bitdefender engine underlying things, and with no issues in the certification sets, a VB100 award is well deserved.
Main version: 14.0.3.5256
Update versions: N/A
Last 6 tests: 5 passed, 0 failed, 1 no entry
Last 12 tests: 10 passed, 0 failed, 2 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Solid
The last of this month’s business solutions is TrustPort, another product that uses the Bitdefender engine, this time in parallel with that of AVG. TrustPort’s test history is very strong, with passes in all Windows comparatives for the last two years.
The setup process is reasonably quick and simple. The interface is divided into modules, but is fairly simple to operate with a good range of options. Stability was impeccable, with no issues to report.
Detection was very strong, with scores remaining high well into the proactive sets, and with the WildList and clean sets properly dealt with, TrustPort earns another VB100 award to add to its excellent history in our tests.
(Click for a larger version of the table)
(Click for a larger version of the table)
(Click for a larger version of the table)
(Click for a larger version of the table)
(Click for a larger version of the table)
(Click for a larger version of the table)
(Click for a larger version of the table)
(Click for a larger version of the chart)
(Click for a larger version of the chart)
Main version: 9.1
Update versions: 4646.690.1951 build 828, build 860, build 870, build 882
Last 6 tests: 5 passed, 0 failed, 1 no entry
Last 12 tests: 6 passed, 0 failed, 6 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Fair
A familiar name, Agnitum’s Outpost has been a fairly regular participant in our tests since 2007, with a good run of success in the last year or so. The current version takes a little while to install and after a reboot presents a slick and attractive interface with clear controls and a decent level of configuration. Stability was mostly good throughout testing, but at one point we encountered an unexpected reboot.
Scanning speeds were slow to start with, but very fast in the warm runs; file access lag times were fairly high, but again better after initial exploration. RAM use wasn’t too high, and CPU use seems low thanks to a rather long time spent processing our set of activities.
Detection was good in the reactive sets but tailed off rather sharply into the later sets. There were no issues in the certification sets and a VB100 award is earned.
Main version: 2014.9.0.2018
Update versions: 140624-0, 2014.9.0.2021/140714-0, 140801-0, 140819-0
Last 6 tests: 3 passed, 2 failed, 1 no entry
Last 12 tests: 8 passed, 3 failed, 1 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 1
Stability: Stable
Avast’s history in our tests goes right back to the very beginning in the late 1990s, with several long unbroken spells of success built up over the years. Setup was very speedy indeed, and the design of the GUI is as ever very stylish and attractive with good access to a wide range of options and controls.
Stability was excellent in the main, and the only issue we noted – with logs not being created in certain circumstances – was apparently already known to the developers and fixed before we reported it to them.
Scanning speeds were reasonable, file access lag times very light thanks to only limited scanning of files on-read, and resource use was low with a good time taken to complete our set of activities.
Detection was very good across the board, dropping off just a little into the proactive sets, and the WildList was covered without problems. In the clean sets, alongside rather a large number of warnings about overly nested archives which could constitute ‘decompression bombs’, a single item was misclassified: a component of some virtualization software from Dell was labelled as Zbot malware. While this issue would be unlikely to affect many users, it is enough to deny Avast a VB100 award this month, despite an otherwise excellent performance.
Main version: 2.8.6
Update versions: 41933, 3.2.5/42304, 3.2.8/42668, 2.8.6/42933
Last 6 tests: 1 passed, 0 failed, 5 no entry
Last 12 tests: 1 passed, 0 failed, 11 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Fair
Another new name, Avetix is an Italian company based in Rome whose product integrates the widely deployed Bitdefender engine. The product requires the .NET platform, which may add a little time to the setup process for those who don’t have it installed already, and presents a bright and colourful interface which is clearly laid out and provides a decent set of options.
Stability was reasonable, but we did see quite a few scans failing to complete properly – most worryingly, with some of them claiming to have finished and found nothing, despite subsequent re runs of the same job reporting large numbers of infections.
Scanning speeds were pretty fast, and overheads look very light thanks to an absence of on-read protection by default. Some slowdown was noted in our activities test though, and RAM use was a little higher than many products this month.
Detection was very good indeed, and with no problems in the certification sets, Avetix can claim VB100-certified status after just a single attempt.
Main version: 2014.0.4714
Update versions: 3972/7740, 3986/7850, 2014.0.4716/3986/7961, 4007/8068
Last 6 tests: 6 passed, 0 failed, 0 no entry
Last 12 tests: 11 passed, 1 failed, 0 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Stable
Returning to one of the regulars, AVG hasn’t missed a VB100 comparative since 2010, and has passed all but a handful of tests in the last decade. The ‘CloudCare’ product seems to differ from previous editions in name only, with a low-interaction install process which completes reasonably quickly. The interface is dark and brooding with typical angular styling and a thorough set of configuration options in fairly easy reach.
Stability was mostly very good indeed, with the only issues observed being a couple of cases of updates failing to complete properly first time – on each occasion a second attempt was all that was needed to finish the job.
Scanning speeds were impressive to start with and even faster in the warm runs. Lag times were not bad either – a little high on executable files, but shrinking to almost nothing in later runs. RAM use was perhaps a touch higher than most, but CPU use was low, and impact on our set of tasks was noticeable but not extreme.
Detection was excellent, tailing off just a little into the proactive sets, and the certification sets were dealt with properly, earning AVG another VB100 award.
Main version: 14.0.5.450
Update versions: 8.03.20.16/8.11.156.242, 8.03.20.34/8.11.160.212, 14.0.5.464/8.03.24.02/ 8.11.165.70, 14.0.6.570/ 8.03.24.16/8.11.168.124
Last 6 tests: 3 passed, 0 failed, 3 no entry
Last 12 tests: 5 passed, 0 failed, 7 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Solid
Avira’s free personal edition appears in most of our desktop tests, and has achieved a pass on every appearance since it first took part almost five years ago. Installation is speedy, even with the automatic scan fired off as part of the process, and the interface is crisp and businesslike with a good set of controls.
Stability was flawless throughout testing, with no problems observed.
Scanning speeds were decent and very consistent, lag times negligible thanks to the absence of full on-read protection by default, and resource use and impact on our set of activities were also minimal.
Detection rates were excellent even in the later parts of the sets, and there were no issues in the core sets, earning Avira another VB100 award and much gratitude from the team for a fast and easy testing experience.
Main version: 17.28.0.1191
Update versions: 7.55540/11572668, 7.55905/10718763, 7.56237/9645100, 7.56470/7764989
Last 6 tests: 6 passed, 0 failed, 0 no entry
Last 12 tests: 12 passed, 0 failed, 0 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Stable
Bitdefender comes into this month’s test with a flawless record in recent years, having entered and passed every VB100 comparative since August 2010. With a number of other products using the engine already having notched up good performances, things looked good for a further extension of that record. The installation process was reasonably fast and simple, and the interface was sharp and glossy with a sensible layout and a good depth of options.
Stability was decent, with only a few minor problems noted, including one instance of a scan failing to complete properly and one rather odd event where a job was claimed to have taken a few seconds when it actually ran for over ten minutes.
Scanning speeds needed no such exaggeration, being fine to start with and very fast indeed later on. File access times were also not bad at all at first and very light indeed later, while RAM use was average and CPU use looks very low, thanks to a rather extreme length of time taken to get through our set of activities – an anomaly noted previously and something that will require deeper investigation with the developers to get to the bottom of.
Detection was excellent though, with very good scores everywhere, and a clear run through the certification sets earns Bitdefender another VB100 award, continuing its splendid run of success.
Main version: 14.1.281.3
Update versions: 7.55555, 7.55905, 14.1.281.8/7.56237, 14.1.283.1/7.56471
Last 6 tests: 5 passed, 0 failed, 1 no entry
Last 12 tests: 10 passed, 0 failed, 2 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Solid
BullGuard’s record is also exemplary, with passes in all of our Windows comparatives for the last two years and more. Installation takes a few minutes, with initial downloads bundled into the setup process. The recently redesigned interface is bright and cheerful with a Windows 8-style tile effect for its various components. Stability was impeccable throughout testing – not even the slightest wobble was noted, even under heavy pressure.
Scanning speeds were OK initially and blazing fast in the warm runs; lag times were very light from the off and improved a little later on. Resource use was low, and our set of activities ran through very quickly. Detection was superb as usual, and with nothing to worry us in the certification sets, BullGuard earns another VB100 award.
Main version: 13.2.029.000
Update versions: 8.3.4.7, 8.3.4.7, 13.3.052.000/8.3.4.7
Last 6 tests: 3 passed, 0 failed, 3 no entry
Last 12 tests: 3 passed, 2 failed, 7 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Stable
Check Point’s ZoneAlarm product line has a long history but first appeared in our tests in 2008, since when it has only participated sporadically until the last year or so, with appearances in the last three comparatives on desktop platforms. The interface hasn’t changed greatly in most of that time (or so it feels), with a busy and wordy layout providing a basic set of controls. It maintained reasonable stability during testing, but froze up a few times during heavier jobs, and we also had some problems with logs not being created.
Scanning speeds were distinctly on the slow side, and file access lags very heavy on the first visit but not too bad once things had settled down. RAM use was a little high, CPU use low but measured over rather a longer time thanks to some noticeable delay imposed on our set of activities.
Detection was very good in our response sets, with no proactive measures recorded thanks to heavy use of cloud technology rendering that part of the test inapplicable. The WildList set was fully covered both with and without the help of the cloud though, and there were no false alarms in the clean sets, meaning that Check Point earns a VB100 award.
Main version: 5.1.31
Update versions: 5.4.11
Last 6 tests: 2 passed, 2 failed, 2 no entry
Last 12 tests: 2 passed, 7 failed, 3 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 12
Stability: Fair
CYREN’s Command product appeared in the very first VB100 comparative back in 1998, and has undergone regular reincarnations and changes of ownership over the years. The product itself hasn’t changed much in quite a while though, with the usual compact installer getting the job done rapidly and the interface plain and unfussy with a minimum of buttons and tabs.
Stability was reasonable, but a number of scans crashed out, while some of those that did complete failed to produce logs. Scanning was not super-fast, and file access lag times were pretty hefty too, with low RAM use but rather high use of CPU cycles and a fairly significant effect on our set of standard tasks.
Detection was very good in the response sets but rather poor in the reactive sets, highlighting the product’s heavy reliance on cloud lookups which are disabled in the proactive part of the test. The WildList set was well covered, but once again in the clean sets we saw a handful of false alarms, mainly items from major PC hardware manufacturers including ASUS, Foxconn, Lenovo and AMD. These false detections mean there is no VB100 award for CYREN this month.
Main version: 9.0.0.4142
Update versions: 3.0.0.600/11.0.1.12, 9.0.0.4183, 9.0.0.4183, 9.0.0.4324
Last 6 tests: 4 passed, 1 failed, 1 no entry
Last 12 tests: 8 passed, 2 failed, 2 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Stable
First taking part in our tests in 2010, Emsisoft has become one of our most regular participants, with a good run of passes and not a Windows comparative missed in the last few years. The product has evolved somewhat since its early ‘A-Squared’ days, but it retained a similar look and feel until very recently. A major revamp has left the interface with a very clean and attractive new look, with large, clear emblems and text, and nice use of colour to indicate status. The layout is simple to navigate and a decent set of basic configuration options are provided.
Stability was decent too, with just a few minor issues – scans crashing out or, on one occasion, simply ignoring a request to get started.
Scanning speeds were rather slow, overheads not the lowest but not too bad, and our set of tasks was noticeably slowed with low resource consumption.
Detection was excellent though, and with the core sets properly dealt with, Emsisoft’s new-look product earns another VB100 award.
Main version: 14.0.1400.1632 DB
Update versions: 7.55523, 7.55914, 7.56237, 7.56471
Last 6 tests: 5 passed, 1 failed, 0 no entry
Last 12 tests: 11 passed, 1 failed, 0 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Fair
The last VB100 comparative without a product from eScan on the test bench was back 2009, and there have been some good strings of results for the product since then. The installation process isn’t the fastest, and updates take a fair while too. Once up, the interface has a little more colour in it these days, but there is still a lot of grey-on-grey. It’s pretty simple to find one’s way around though, and provides an excellent depth of configuration.
Stability was a little shaky this month, with quite a few scans crashing out or otherwise failing to complete properly, and on a few occasions a number of reboots were required to get the on-access component up and running.
Scanning was speedy though – very quick indeed in the warm runs – with lag times pretty low too. Resource use was very low, and our set of activities ran through in almost identical time to the baseline measures.
Detection was excellent, and the certification sets were well managed, earning eScan another VB100 award.
Main version: 7.0.317.4
Update versions: 9997, 10092, 10187, 10284
Last 6 tests: 6 passed, 0 failed, 0 no entry
Last 12 tests: 12 passed, 0 failed, 0 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Solid
Matched only by Bitdefender in the number of tests entered and passed in the last two years, ESET’s unbroken record goes back well into the distant past, with few people old enough to remember a VB100 comparative without a pass from NOD32. After a quick and clear install and update process, the interface is sparse and simple with easy access to a wealth of options. Stability was perfect throughout, with not the slightest sign of weakness.
Scanning speeds were OK to start with and very fast indeed later on, with overheads pretty light but impact on our set of tasks fairly noticeable, and resource use below average.
Detection was excellent, with just a bit of a dip into the proactive sets, and as usual there were no unwanted surprises in the certification sets, thus extending ESET’s unbroken run of VB100 awards still further.
Main version: 25.0.1.3
Update versions: AVA 24.2831/GD 25.3473, AVA 24.3200/GD 25.3597, AVA 24.3510/GD 25.3702, AVA 24.3719/GD 25.3781
Last 6 tests: 4 passed, 0 failed, 2 no entry
Last 12 tests: 8 passed, 0 failed, 4 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Solid
G Data takes part in most of our tests, and the number of times it has failed to achieve VB100 certification in the past decade can be counted on the fingers of one hand. The current product version looks great, with a very clear layout providing instant access to the most important information and controls, and a detailed level of configuration options available. Stability was excellent too, with no issues throughout the test.
Scanning speeds were not bad and fairly consistent, while overheads were similarly reliable but a little on the heavy side. Resource use was low, but our set of tasks took a little while to complete.
Detection was superb as usual, with very little not picked up, and the WildList and clean sets were handled admirably, easily earning G Data another VB100 award.
Main version: 14.2.02.0240
Update versions: 9.180.12501, 9.181.12894, 14.2.0242/ 9.182.12929, 14.2.0244/ 9.183.13110
Last 6 tests: 2 passed, 1 failed, 3 no entry
Last 12 tests: 4 passed, 2 failed, 6 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Solid
K7 products have appeared in most of our desktop tests over the last few years, with a decent number of passes. The current version installs in decent time with a reboot required, and presents a tough-looking GUI with a military theme. The layout is simple and clear with good status information and decent controls. Stability was flawless throughout, with no issues noted.
Scanning speeds were not bad, and overheads a little high initially but soon became very light. RAM use was around average, CPU use rather low, and our set of activities ran through very quickly.
Detection was pretty good in the reactive sets, dropping away a little into the proactive weeks, but there were no issues in the WildList or clean sets and a VB100 award is earned by K7.
Main version: 15.0.0463
Update versions: 15.0.0463(a)
Last 6 tests: 3 passed, 0 failed, 3 no entry
Last 12 tests: 3 passed, 0 failed, 9 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Stable
The third product from Kaspersky Lab this month, the consumer-focused Internet Security suite, has its own entry in our test records, but since it is submitted less frequently than its corporate stable mates, its history is rather sparser. The setup process is simple and reveals a redesigned GUI with a pale, Spartan look and feel with just a few touches of the company’s trademark green. The layout is clear and usable though, with the usual wealth of fine-tuning options.
Stability was reasonable, but on a few occasions we did observe the interface disappearing mid-task – although the jobs seemed to keep running fine in the background. We also noted that some of the configuration options seemed to be ignored by the product.
Scanning speeds were OK to start with, a little slow over our sets of media and document files, but very fast indeed in the warm runs. File access lags were just a touch high first time around, but barely perceptible later, and with low resource use our set of activities ran through extremely quickly.
Detection was good, dropping away somewhat into the proactive sets with no access to cloud lookups, and the core sets were properly handled, earning Kaspersky Lab its third VB100 award this month.
Main version: 2013.SP7.5.040610
Update versions: 2013.SP7.5.071116, 2013.SP7.5.080116, 2013.SP7.5.082018
Last 6 tests: 4 passed, 1 failed, 1 no entry
Last 12 tests: 8 passed, 1 failed, 3 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Stable
Kingsoft’s product is one of several regulars that are available only in Chinese, making for an interesting experience for the test team. For the last couple of years, we’ve seen third-party engines in use, which have helped Kingsoft pick up a decent string of passes.
Initial installation is very fast, but updates can take some time, and on a couple of occasions they caused the whole app to crash out. The interface is glossy and slick, with cartoony icons along the top, and a lot of buttons and options – which suggests that there is a decent level of control, but we didn’t verify the function of most of them.
Scanning speeds were slow to start with, but very fast later on; overheads were fairly light, resource use was a little on the high side, but impact on our set of tasks was negligible.
Detection was almost flawless in our response sets, with excellent scores across the board. There were no problems in the certification sets, and Kingsoft earns another VB100 award.
Main version: 8.3.20.16
Update versions: 7.11.157.6, 8.3.20.34/7.11.160.212, 8.3.24.2/7.11.165.68, 8.3.24.16/7.11.168.124
Last 6 tests: 2 passed, 0 failed, 4 no entry
Last 12 tests: 3 passed, 1 failed, 8 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Stable
The PCKeeper brand was operated by ZeoBit before being taken over by the Kromtech Alliance Corporation, and has appeared in our tests only a handful of times over the last few years. After a simple and speedy setup, the interface is clear and simple too, with nice clean lines and some good status information – including rather flatteringly informing us that our computer was ‘excellent’.
Stability was mostly fine, although we did observe a rather odd situation whereby, after installation of the product, it seemed to be impossible to restart the machine without resorting to a hard reset. After this first reboot, things seemed fine though.
Scanning speeds were decent and very consistent, while overheads were a little high initially, but barely noticeable later on. Resource use was decidedly high though, and our set of activities was slowed down significantly.
Detection was excellent, with high scores even into the later weeks of the RAP sets, and there were no issues in the certification sets, thus earning Kromtech a VB100 award.
Main version: 9.0.6.9
Update versions: 1.1.107.0/87577, 88236, 88469, 88675
Last 6 tests: 0 passed, 1 failed, 5 no entry
Last 12 tests: 0 passed, 1 failed, 11 no entry
ItW on demand: 100.00%
ItW on access: N/T
False positives: 0
Stability: Fair
Another new name, Kyrol comes to us courtesy of MSecure, on whose product it is based. Setup is fast and easy, and the product GUI is arrestingly bright with an orange-on-black colour scheme. Large icons mark the various sections, making navigation fairly simple, and settings are pretty detailed.
Stability was fine for the most part, although we did note one fairly serious issue, which we have previously reported on regarding MSecure’s own version of the product: the default settings rely on an extension list to decide which files to monitor on access. The list includes all the standard file types one might expect, however, at least two of the most commonly used extensions appeared to be ignored, making our on-access detection measure fairly meaningless. There is an option to monitor all file types, which users would be well advised to enable if possible, to ensure better protection.
Scanning speeds were mostly reasonable, but very slow over executable files. Overheads were fairly light with those suspect default settings, and decidedly heavy with ‘all files’ enabled (perhaps explaining why the option is turned off by default). Resource use and impact on our set of tasks were very low, but again this will be affected by the default settings.
On demand at least, detection was pretty good, dipping rather sharply in the proactive sets, and the WildList was covered just fine with no false alarms in the clean sets either. With the on-access component not providing adequate protection by default though, no VB100 award can be granted this month, but with a simple fix there should be no such problems for Kyrol going forward.
Main version: 11.2.5952.0
Update versions: N/A
Last 6 tests: 3 passed, 0 failed, 3 no entry
Last 12 tests: 5 passed, 0 failed, 7 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Solid
Lavasoft’s switch to the Bitdefender engine (on top of its own technologies), along with a major redesign of the interface, recently brought general all-round approval from the testing team. The latest version has a clean and attractive GUI, providing easy access to a good set of basic controls, and underneath it all a high-performing set of protections which have notched up some good performances lately. This month we saw no problems with stability at all, with the product remaining unfazed by even our most high-stress tests.
Scanning speeds were not too bad to start with, and splendid later on thanks to some good optimization. File access lag times likewise shrank from a fairly light start to barely noticeable after settling in. RAM use was a little above average, and once again our set of activities took a bizarrely long time to complete. Some investigations into this showed that almost all of the slowdown was at the download stage, where sample files are fetched over HTTP from a local intranet server. The product’s developers are looking into what may have caused this anomaly.
Detection was excellent, remaining strong into the later weeks of the RAP sets, and the WildList was dealt with without any problems. With no issues in the clean sets either, Lavasoft earns another VB100 award to add to its growing collection.
Main version: 1.1.65.122
Update versions: 4.8.51/4.0.0.4, 1.1.66.123, 1.1.66.123, 1.1.67.124
Last 6 tests: 0 passed, 3 failed, 3 no entry
Last 12 tests: 0 passed, 3 failed, 9 no entry
ItW on demand: 41.11%
ItW on access: 14.57%
False positives: 4
Stability: Stable
Maya is back again after some rather disappointing performances in previous tests, as yet still some way from making the VB100 grade. Installation is pretty speedy, with updates fast too. Once installed, the interface takes rather a long time to open, but when it appears it looks reasonably good, with nice large fonts and big icons leading to a reasonable set of controls. Stability wasn’t too bad – a couple of updates failed to complete first time, and there were some oddities with the controls for the on-access component, but nothing too serious.
Scanning speeds were pretty good, overheads not too bad either, and with high RAM use but very low use of CPU cycles, our set of activities showed a definite slowdown, but not by too much.
Detection was not great, particularly into the later parts of the sets, and the WildList was only covered partially, with the on-access component faring rather worse than the on-demand one. With a couple of false positives too, Maya still has some work to do to reach the required standard for VB100 certification.
Main version: 1.1.107.0
Update versions: 88013, 88236, 88477, 88674
Last 6 tests: 0 passed, 5 failed, 1 no entry
Last 12 tests: 1 passed, 6 failed, 5 no entry
ItW on demand: 100.00%
ItW on access: N/T
False positives: 0
Stability: Fair
As mentioned in passing above, MSecure’s product has been having some trouble with its settings of late, meaning there have been no passes in the last couple of tests. Installation is fairly quick and easy, the interface pretty basic but responsive and navigable with decent options, and stability pretty reasonable for the most part, apart from the ongoing issue with ignoring important file extensions on access.
Scanning speeds were a little on the slow side, overheads not too high, and resource use very low with not much impact on our set of activities – which is not too surprising, given that not much was being monitored.
Detection was good on demand, dipping a little into the proactive sets, but with no problems in the WildList or clean sets. On-access detection is, of course, another story, with the extension problem rendering our efforts at testing rather fruitless – although we did prove that with the ‘all files’ setting enabled, detection was fine. With the default settings though, no VB100 award can be granted this time.
Main version: 10.1
Update versions: 7.04.04
Last 6 tests: 3 passed, 2 failed, 1 no entry
Last 12 tests: 6 passed, 5 failed, 1 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 1
Stability: Solid
Norman’s history in our tests goes all the way back to the start and features a good number of passes, sprinkled with occasional rougher patches. Setup of the current version is not too taxing, and the interface has a simple layout with an appealing grey/blue colour scheme. It remained responsive and stable throughout testing, with no problems observed even under heavy stress.
Scanning speeds were decent with no spikes or troughs; overheads were a little on the high side – especially over binaries – but improved strongly after initial settling in. Resource use was low, but our set of activities did take a while to complete.
Detection was very strong, dipping rather in the later parts of the RAP test. The WildList was well covered, but in the clean sets a single item – a somewhat obscure piece of scientific software – was alerted on by the Sandbox component, which was just enough to deny Norman a VB100 this month, despite a generally strong showing.
Main version: 3.3.27.192
Update versions: 3.3.28.203, 3.3.29.209
Last 6 tests: 1 passed, 0 failed, 5 no entry
Last 12 tests: 1 passed, 0 failed, 11 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Buggy
Another first-timer on the VB100 test bench, WiFi Protector is based around providing a VPN for secure connections from insecure locations, alongside a backup scheme and standard anti-malware protection from the Bitdefender engine. The setup process proved a little complex, requiring Silverlight among other stages, but once up and running, the product interface is pretty clear and detailed with lots of information on the various protective layers.
Stability was a little problematic at times, with a number of crashes during the installation process and throughout testing, including a single blue screen incident.
Scanning speeds were slow initially, but very fast later on, while overheads were distinctly high. RAM use was very high and our set of activities took rather a long time to get through.
Detection was very good though, as one would expect, with high scores across the board. The certification sets were well handled, and Optimal Software earns a VB100 award on its first attempt.
Main version: 3.0.1
Update versions: N/A
Last 6 tests: 4 passed, 1 failed, 1 no entry
Last 12 tests: 7 passed, 2 failed, 3 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Fair
Panda’s free cloud-based product has been putting in some decent performances lately, having returned to the VB100 tests after a lengthy absence a few years ago. The version being tested this month, which installed very rapidly as per usual, has had something of a redesign with a much larger interface laid out in a Windows 8-style tiling arrangement, and it looks bright and cheerful. As in the previous iteration, controls are fairly minimal, but easy to locate.
Stability was mostly OK, although we did find that just about any scan requiring much by way of heavy lifting had a good chance of crashing out, and each stage of the test took rather longer than usual as jobs were split into ever smaller sections to try to get through them before the almost inevitable falling over. More normal workloads (of the type most users are likely to stick to) were less affected, but even some scans of clean files failed to get to the end intact.
Speeds were a little on the slow side on demand, overheads mostly pretty light, mainly thanks to only scanning certain file types on-read. RAM use wasn’t too high either, but our set of activities took quite some time to get through – so long, in fact, that we thought there must have been some sort of problem and re-ran the job, only to get similar figures once again.
Detection was very strong though, at least in the response sets, with no data recorded for the proactive part of the RAP test as the product cannot function without Internet access. The WildList and clean sets were dealt with well, and Panda earns a VB100 award.
Main version: 15.0.0
Update versions: N/A
Last 6 tests: 1 passed, 0 failed, 5 no entry
Last 12 tests: 1 passed, 0 failed, 11 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Fair
The second product from Panda this month is the 2015 edition of the vendor’s suite solution. Once again, setup is very fast, with much of the hard work being done back at Panda HQ rather than on the local machine, and the interface has the same tile layout as the free version, again very slick and professional with some friendly touches.
As with the free product, stability was shaken by a number of problems getting scans to complete happily, either freezing up, crashing out with error messages, or occasionally simply stopping short with no indication that anything had gone wrong. We also saw some errors during aborted attempts to install.
Speeds weren’t too zippy on demand, with overheads a little above average on access too. Resource use was low, and once again our set of activities took a while to get through, although this time only fairly long.
Detection was very strong where it was measured (once again, no proactive measure could be taken thanks to the product’s reliance on the cloud). With nothing missed in the WildList set and no false alarms in the clean set, Panda’s IS product also earns itself a VB100 award.
Main version: 1.0.046
Update versions: N/A
Last 6 tests: 1 passed, 2 failed, 3 no entry
Last 12 tests: 3 passed, 2 failed, 7 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 345
Stability: Buggy
PC Pitstop made some waves last time by opting to use its whitelisting component as the main detection capability, falling back on the ThreatTrack engine only for some parts of the test. Detection was very high indeed, but as one might expect, false positives were fairly high too. Hoping for a better showing this time, we started off with the installation, which required a little work but didn’t take too long, and a look around the interface, which is much the same as in previous tests, with only the bare minimum of controls and occasional moments of unresponsiveness.
Stability was a bit of a problem throughout, with numerous scan jobs falling over, a couple of unexpected reboots, and several other errors.
Scanning speeds were decidedly slow, lag times not too bad, and our set of tasks took a fair amount of time to get through, with fairly high use of memory.
Detection was very good, even in the proactive sets, with that whitelisting component coming into its own there, and this time the WildList was well covered too. However, once again in the clean sets there were a considerable number of false alarms and no VB100 award can be granted despite another interesting effort from PC Pitstop.
Main version: 5.0.0.5044 (x64)
Update versions: 5.0.0.5045 (x64)
Last 6 tests: 4 passed, 1 failed, 1 no entry
Last 12 tests: 9 passed, 1 failed, 2 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Solid
Qihoo has become another very regular participant in our tests of late, with only the Linux comparatives not entered in the last two years and a good pass rate over that time. The current product installs very quickly, and the interface is crisp and clear with a pleasantly pared-down layout and generally usable navigation – although there were a few oddities of language, typos and some bits of text displayed in Chinese.
Stability was good though, with no errors or other problems noted, and with the typographical issues adjudged too trivial to merit marking down, a ‘Solid’ rating is achieved.
Scanning speeds were on the slow side, with no sign of optimization in repeat runs. Overheads were very light thanks to the product’s approach of not intercepting file access in real time, instead scanning and alerting after the fact. Despite this, RAM use was fairly high and our set of activities took quite some time to complete.
Detection was very good indeed though, with just a slight downturn into the later weeks of the RAP sets, and with no problems in the certification sets a VB100 award is comfortably earned.
Main version: 15.00(8.0.8.0)
Update versions: N/A
Last 6 tests: 5 passed, 1 failed, 0 no entry
Last 12 tests: 10 passed, 1 failed, 1 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Fair
Quick Heal’s history in our tests goes back over a decade, with a good number of passes along the way. The latest version installs fairly quickly, with a memory scan included as part of the process, and presents a nicely designed interface with a clear layout and a good set of configuration options.
Stability was reasonable, but we did see a number of scans crashing out or freezing up – the problem recurring often enough to nudge our rating down into the ‘Fair’ category.
Scanning speeds were a little slow over some file types but reasonable elsewhere, while overheads became very light indeed after initial familiarization with the sample files. RAM use was high, CPU use quite low, and time taken to complete our set of tasks a little slow, but not too bad.
Detection wasn’t too bad initially, dropping away somewhat into the proactive sets, and there were no problems covering the WildList. The clean sets threw up no surprises either, and Quick Heal earns another VB100 award without too much trouble.
Main version: 9.0.6.9
Update versions: 1.1.1070/87826, 88236, 88477, 88675
Last 6 tests: 0 passed, 1 failed, 5 no entry
Last 12 tests: 0 passed, 1 failed, 11 no entry
ItW on demand: 100.00%
ItW on access: N/T
False positives: 0
Stability: Fair
Securealive is another product from the MSecure stable, based on the same underlying setup, so things did not look good for it from the outset. The installation process was pretty quick and easy, the interface nice and clean with big clear icons, providing a decent set of controls.
Stability was a little shaky in places, with a couple of scans freezing up, and again we saw the same issue with the on access component ignoring several important file types with the default settings.
Scanning speeds were rather slow, overheads look very light, but that’s likely mainly due to the small number of files being inspected, which will also have led to low resource use measures and a rapid pace through our set of activities.
Detection was decent on demand, with very good scores in the reactive sets and not too steep a drop into the proactive weeks. The WildList was fully detected in a scan but, as expected, the on-access component paid little attention to its contents, meaning there is no VB100 award for Securealive this time despite there being no problems in the clean sets. Things should be considerably better with the main issue resolved though, and we expect to see the product make the grade sometime soon.
Main version: 8.10.25261.501
Update versions: 8.10.25263.501
Last 6 tests: 5 passed, 0 failed, 1 no entry
Last 12 tests: 9 passed, 0 failed, 3 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Stable
Tencent’s first appearance in the VB100 tests was just over two years ago, and since then we’ve seen a string of passes, no fails and very few tests missed. The product is another which the lab team have the task of wrestling with without the assistance of written information, the GUI being available only in Chinese, but it has generally proven simple to navigate, with a rapid install and a clean and clear interface which can be figured out based mainly on icons.
Stability was reasonable, with just a couple of scans failing to complete smoothly, but on one install we couldn’t get the product to show any sign of working and eventually had to wipe the machine and start again.
Scanning speeds were average over executables, fairly zippy elsewhere, with overheads a little on the high side given that on-read protection is not available. RAM use was a little higher than many too, as was CPU use, and there was a medium-sized impact on the runtime of our set of tasks.
Detection was splendid though, with good scores across the board, and with no problems in the core sets, Tencent is well deserving of another VB100 award to add to its string of passes.
Main version: 7.0.6.2
Update versions: 3.9.2592.2/30672, 31750, 32036, 32556
Last 6 tests: 3 passed, 0 failed, 3 no entry
Last 12 tests: 6 passed, 0 failed, 6 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Stable
ThreatTrack’s VIPRE engine has already cropped up in a couple of other solutions this month, and the product’s test history in the last year or so is all to the good. The setup process is fast, with updating also rapid, and the product GUI is pale and restrained with good clear status indicators and a decent set of configuration options under the covers.
Stability was pretty good, with just a single crash of the program UI, which, rather surprisingly, occurred when clicking the ‘help’ button (admittedly, after running a rather large detection job).
Scanning speeds were a little slow, but did speed up somewhat on repeat runs, and file access lag times were barely detectable, possibly in part due to the backgrounding of more intense work on-read. RAM use was a little above average, CPU use low, and time taken to complete our set of activities a little high.
Detection was pretty good, with high scores in the response sets and not too sharp a drop into the proactive sets. With the WildList fully covered and no false alarms in the clean sets, ThreatTrack’s VIPRE merits another VB100 award.
Main version: 9.0.0.141
Update versions: 6973.0.0.0, 7007.0.0.0, 7038.0.0.0
Last 6 tests: 1 passed, 1 failed, 4 no entry
Last 12 tests: 1 passed, 4 failed, 7 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Fair
The last time we saw a consumer product from Total Defense was late last year, since when the business arm of the company has branched off to form its own firm in the name of iSheriff. The home-user product soldiers on though, hoping to improve on a rather rough run of results in 2013.
Installation didn’t take too long, with a scan included in the process, and updates were very fast indeed. The interface is familiar from many previous tests – heavy on the styling with a little room for confusion in places, but a decent set of controls are available for those willing to figure them out.
Scanning speeds started off reasonable and became lightning fast, with overheads a little high over binaries but not too bad elsewhere. RAM use was around average, CPU use low, and time taken to complete our set of activities very slow.
Detection was a little mediocre in the response sets, with no proactive score available thanks to heavy reliance on cloud lookups. The WildList was well handled though, and with no false alarms either, Total Defense once again makes the grade for a VB100 award.
Main version: 1.0.50.1264
Update versions: N/A
Last 6 tests: 1 passed, 0 failed, 5 no entry
Last 12 tests: 1 passed, 0 failed, 11 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Stable
Another name that is new to our tests, Adept comes from Ukraine and arrived just in time for inclusion in this test, although too late to prepare an install for the proactive detection tests.
The setup process requires minimal interaction but takes a fair while to complete. Once up, the interface is strongly reminiscent of another product that is relatively new to our tests: Wontok.
The interface is smart and stylish with a professional feel to it, and responded well for the most part, although a couple of scans failed to complete cleanly.
Scanning speeds were pretty good, especially over non-executable file types, and while simple file access overheads were fairly high, resource use was reasonable and our set of activities completed in good time.
Detection was superb in those parts of the tests we were able to complete, with very little missed, and the WildList was covered flawlessly too. There were no unpleasant surprises in the clean sets, and ULIS earns a VB100 award on its first attempt.
Main version: 1.0.40.1171
Update versions: N/A
Last 6 tests: 2 passed, 0 failed, 4 no entry
Last 12 tests: 2 passed, 0 failed, 10 no entry
ItW on demand: 100.00%
ItW on access: 100.00%
False positives: 0
Stability: Stable
Wontok first entered our tests just under a year ago, passing first time and showing off some impressive detection capabilities thanks to a combination of the Avira and Bitdefender engines. After a speedy install, we got to have another look at the clean and slick-looking interface, with clear navigation and a reasonable set of configuration options. Stability was decent, with just a couple of very minor issues.
Scanning speeds were not too bad, and overheads a little high initially, but calming down quickly. Resource use was a little above average, but impact on our set of activities was not excessive.
Detection was awesome, as expected, with near-perfect scores in the response sets and still extremely high figures in the proactive sets. The WildList was brushed effortlessly aside, and with no issues in the clean sets either, Wontok comfortably earns its second VB100 award.
(Click for a larger version of the table)
(Click for a larger version of the table)
(Click for a larger version of the table)
(Click for a larger version of the table)
(Click for a larger version of the table)
(Click for a larger version of the table)
(Click for a larger version of the table)
(Click for a larger version of the table)
(Click for a larger version of the table)
(Click for a larger version of the chart)
(Click for a larger version of the chart)
In addition to the products listed above, some others were submitted for testing but proved to be untestable for one reason or another. These included Perisai Internet Security.
Another epic test ended with some considerable relief for the testing team. It has been a rather tough one with a large number of products proving intractable or unreliable, and many needing jobs repeating or breaking into smaller and more time-consuming parts thanks to instability. We saw a record number of blue screens this month, although it’s hard to say exactly how much of that can be blamed on the products under test and how much may be due to the new version of Windows, as new OS versions always seem prone to a few wobbles during their first few months.
Apart from the wobbles, things were mostly pretty good, with another high pass rate. Our typical ratio in the past has been around two-thirds of submitted products achieving certification, but in recent tests this has risen considerably. Much of this is doubtless due to the continued growth in use of a handful of popular engines, which are being deployed by an ever wider range of products. Just two engines are included in more than 20 different solutions this month, and the top five are deployed in two-thirds of the products taking part in the test.
We also saw some very high scores in the proactive part of our RAP tests for a number of products – rather higher than has been standard of late. This is in part thanks to some further adjustments to the process for building our test sets as we try to focus on the most relevant and common threats. We will continue to tweak this process as better telemetry information on incoming samples becomes available.
In a few cases, some rather bad luck with fairly minor false positives denied high-performing products a pass, but several of the products failing this month did so thanks to serious problems that dramatically impaired the protection they provide. As usual, we will be chasing up the developers to ensure these bugs are fixed as soon as possible.
As the final touches are being put to this report, the next comparative – on Windows 2008 Server – is already nearing completion, and we hope to get the details of that one published within a few weeks of this test going live.
Test environment. All tests were run on identical systems with AMD A6-3670K Quad Core 2.7GHz processors, 4GB DUAL DDR3 1600MHz RAM, dual 500GB and 1TB SATA hard drives and gigabit networking, running Microsoft Windows 8.1 with update, x64 Pro edition.
Any developers interested in submitting products for VB's comparative reviews, or anyone with any comments or suggestions on the test methodology, should contact [email protected]. The current schedule for the publication of VB comparative reviews can be found at http://www.virusbtn.com/vb100/about/schedule.xml.
