VB100 comparative review on Windows XP Professional SP3

Main version: 8.0.7.5 (build 1398)

Update versions: 2013.03.08.05, 2013.03.14.00, 2013.03.20.00

First up this month, AhnLab is not the most regular of participants in our tests but usually puts in a reasonable performance. The 118MB product installer worried us slightly by appearing to do nothing for quite some time before suddenly waking up and getting going, running through a fairly standard process to complete in a minute or so. Updates added a couple of minutes on average, but reboots were not needed and things were soon ready to go. The GUI is crisp and clear with a fair range of controls; logging is provided in a dedicated log viewer tool, which gives good access to the data and allows it to be exported easily and reliably for external analysis.

Operation seemed fairly stable, although we did note some odd recurrence of on-access alert pop ups at seemingly random times, and also saw no effect from adjusting the on access scanning settings to include self-extracting archives, which appeared to remain unexamined. Speeds were not bad on demand, overheads perhaps a little on the high side on access, but resource use was well below average and our set of activities was completed in decent time too.

Detection was reasonable – a little behind the leaders, but not embarrassingly so, and the certification sets were handled well, with no issues to report in either the WildList or clean sets. A VB100 award is thus earned by AhnLab, getting things off to a good start. The vendor’s test history shows three passes from three entries in the last six tests; four passes and two fails in the last two years. With only a couple of pretty minor issues noted, the product gets a ‘Stable’ rating.

ItW Std: 100.00%

ItW Std (o/a): 100.00%

ItW Extd: 100.00%

ItW Extd (o/a): 99.91%

False positives: 0

Stability: Stable

Main version: 7.0.1474

Update versions: 130213-2, 8.0.1482/130305-0, 8.0.1483/130311-0, 130319-0

Avast released a much-heralded new major version shortly after the test deadline. We had hoped to review it in more depth in these pages, but time and manpower sadly ruled that out – nevertheless, thanks to our policy of applying all product updates offered, we were able to see the new version in action throughout much of the test, with only the RAP test using the version originally submitted. Each install used the same 101MB for the old version 7 product, which ran through pretty speedily, completing in under a minute even when the option to install the Google Chrome browser was left in its default checked state. Updates were fast too, taking not much more than a minute even when the new version was added, although in these instances a reboot was needed to complete the set up.

The design is another Windows 8-aping tiled affair, with a very impressive range of tools and gadgets on offer, many of which are new and some quite unheard of in security suites in our experience. When you burrow down to the anti-malware area though, things are pretty familiar, with a clear and sensible layout providing an irreproachable range of controls. Logging is clear, reliable and configurable.

The new interface seemed robust too, with no sign of wobbles even under extreme pressure. Scanning speeds were very good on demand, with overheads very light on access, and although this is helped by only some file types being covered on-read by default, the ‘full’ measures were still very good in most sets. Resource use was very low for RAM and below average for CPU too, with our activities taking a little while to get through, but not too much longer than the average for the month.

Detection was very good, with some impressive scores in all sets, RAP numbers dropping off just a little into the proactive week. With no problems in the WildList or clean sets, a VB100 award is comfortably earned. A blip in the recent Linux test means Avast has only five passes from the last six tests; ten passes and two fails in the last two years. This month’s performance was impressively stable, earning the product a ‘Solid’ rating.

ItW Std: 100.00%

ItW Std (o/a): 100.00%

ItW Extd: 100.00%

ItW Extd (o/a): 100.00%

False positives: 0

Stability: Solid

Main version: 2013.0.2897

Update versions: 2639/6085, 2013.0.2899/2641/6152, 2013.0.2904/2641/6164, 2641/6198

We have already seen AVG’s flashy new version in our recent Windows 8 test, with the new interface designed to fit in with Windows 8 stylings. Installation from the 118MB package provided was fairly straightforward. An ‘express install’ option is offered, but only after several dialogs have been clicked through, and with another shortly afterwards, in this case offering a toolbar. Unusually, but commendably, all options default to unchecked. The process took a few minutes, with initial updates adding a few more minutes, but with no reboots required.

The interface is crisp and glossy but seemed a little wobbly in places. Configuration is ample and reasonably simple to navigate, but we did see a few instances of wonky rendering, and logging was very problematic, with logs of any significant size causing all kinds of headaches. Scans would complete happily, but trying to access or export log data brought a number of GUI freezes and crashes, or simply dumped out incomplete (and in some cases completely empty) logs. On a couple of occasions we had scans which initially reported large numbers of detections, but when revisited after recovering from a crash reported the same job had completed and found nothing. Although these issues mainly occurred when scanning large infected sets, which is unlikely to happen often in the real world, it is enough to dent our stability rating for the product fairly noticeably.

Scanning speeds were good though, zooming through the warm runs, and overheads were low, again with great improvements after initial familiarization. CPU and RAM use were around average, with impact on our set of tasks pretty light. Detection was very good too, with a solid position in both of our scatter graphs and a noticeable but slow decline through the weeks of the RAP sets. With no issues in the WildList sets, and just a few warnings about corrupt, unsigned or potentially over-packed files in the clean sets, a VB100 award is easily earned. AVG now has five passes and one fail in the last six tests; ten passes and two fails in the last two years. This product’s stability was a bit of a worry for XP users – perhaps suggesting that too much focus has been given to early adopters of Windows 8 – and issues encountered with both the interface and logging mean a stability rating of no more than ‘Fair’.

ItW Std: 100.00%

ItW Std (o/a): 100.00%

ItW Extd: 100.00%

ItW Extd (o/a): 100.00%

False positives: 0

Stability: Fair

Main version: 13.0.0.3185

Update versions: 7.11.60.212, 7.11.63.170, 7.11.64.152, 7.11.65.136

As usual in our desktop tests, we have a pair of products from Avira, starting with the free edition. The 108MB installer took a few seconds to unpack but then needed only a single click to kick off an express install, which completed in under a minute including a ‘quick’ scan. Updates mostly took another minute or so, although on one occasion the product announced that a new version needed to be downloaded which would take around 23 minutes – in the end this took less than two minutes, but did require a reboot.

The interface is simple and unflashy, providing a decent if not quite complete level of control; logging is likewise workmanlike and reliable. Scanning speeds were reasonable considering the very thorough settings, very fast indeed over some of our sets, and overheads were not too high either, with RAM and CPU use pretty low. Our set of tasks did take quite some time to get through though.

Detection was superb as usual, with very little missed in any of our sets, and the product takes up a strong position in the leading cluster on our RAP chart. The core sets were properly dealt with – nothing was missed in the WildList set or flagged up in the clean sets, thus a VB100 award is comfortably earned. Participating only in our desktop tests, Avira’s free version has a strong record with three passes from three entries in the last six tests; six from six in the last two years. There were no stability problems, earning a ‘Solid’ rating.

ItW Std: 100.00%

ItW Std (o/a): 100.00%

ItW Extd: 100.00%

ItW Extd (o/a): 100.00%

False positives: 0

Stability: Solid

Main version: 13.0.0.3185

Update versions: 7.11.60.212, 7.11.63.170, 7.11.64.152, 7.11.65.136

The Pro version of Avira’s product is not very different from its free sibling – the installer is a little larger at 121MB, but the general look and feel is much the same. Once again, the installer offers an express path, which completes in a minute or so with no more than another minute needed for updates; again, in later instances a reboot was needed to complete a more thorough update.

The GUI is sober and sensible, with a comprehensive set of controls this time, and once again logging is solid and dependable. Scanning speeds were similar, overheads a tiny bit higher in some areas, but a fraction lower in others, with RAM use a little higher and CPU use notably increased but both still well below average. Our set of activities did take rather a long time to complete, with a time very close to that of the free version.

Detection scores were pretty much identical to those of the free version across the board, meaning another excellent set of figures. The certification sets were properly dealt with, earning Avira another VB100 award. The mainline product’s test history now shows an impeccable 12 passes in the last two years. This month’s performance also earned a stability rating of ‘Solid’.

ItW Std: 100.00%

ItW Std (o/a): 100.00%

ItW Extd: 100.00%

ItW Extd (o/a): 100.00%

False positives: 0

Stability: Solid

Main version: 7.0.1

Update versions: 1.2.2534, 1.2.2570, 1.2.2579, 1.2.2591

The company formerly known as eEye Digital Security has renamed the product formerly known as Blink to the rather less concise PowerBroker EPP for Desktops. There has been a fairly significant overhaul of the interface with apparently some serious improvements under the covers too, but the product is still recognizable. The fairly large 262MB installer runs through a few stages and requests a swathe of personal details towards the end, but does not need a reboot to complete. Updates pulled down well over 200MB of data, taking quite some time, and in some cases seemed to perform the same download several times, failing at an unspecified point with minimal information. On some occasions the updates refused even to start, although we found that this issue could be skirted around by tweaking the configuration of the updater module, or by installing a more recent IE version, which seemed to change some useful security settings. These annoyances added considerably to the overall install time.

The interface is rather different, browser-based and laden with information, with plenty of options down one side, but the configuration dialog is pretty much as was, providing a range of controls for the various components, a good basic set for the anti-malware side of things. Logging is fairly clear, exportable and usable, but not very efficient – large swathes of text are needlessly repeated on line after line. Stability was mostly good, although a few large scans did appear to give up halfway through and, rather worryingly, there was no indication that they had not covered the entire area requested.

Scanning speeds were decent if far from blinding, and on access lag times were a little high in some areas. Use of RAM and CPU cycles were both high, but our set of tasks didn’t take too long to complete. Detection was pretty good, with good scores in both the RAP and Response tests, and the WildList set was well handled too. In the clean sets, however, a single item was labelled as a threat – a fairly generic identification, but enough to deny BeyondTrust a VB100 award this month. The product’s test history is a little rocky, with two passes and three fails in the last six tests; six passes and four fails in the last two years. A number of issues were noted this month, including some tricky updating problems, meaning a stability rating of only ‘Fair’.

ItW Std: 100.00%

ItW Std (o/a): 100.00%

ItW Extd: 100.00%

ItW Extd (o/a): 100.00%

False positives: 1

Stability: Fair

Main version: 16.26.0.1739

Update versions: 7.45417/8751169, 7.45851/9221234, 7.45990/9239152, 7.46152/9305085

Bitdefender’s product range has also had a pretty snazzy overhaul recently, and the vendor’s engine seems to get ever more popular among OEM vendors.

The installer submitted this month was a fairly large 268MB executable, with all latest updates included, and one of the first things it asks is whether a check should be made for a newer version. This seemed to find something to download every time we ran it, but it rarely took more than a few seconds to fetch what it needed. The install itself is another one-click affair, which runs through in a few minutes. Updates are then required despite the initial download, taking a few more minutes and usually fetching a little over 20MB of data. The whole process completes with no need to restart.

The interface is dark and brooding, but reasonably easy to navigate, offering a wide range of controls. Logging is in XML format and reasonably usable, and stability seems good throughout. Scanning speeds started off decent and ramped up hugely in the warm runs, while overheads were very light indeed. RAM and CPU use were both on the low side, and our set of tasks ran through in reasonable time. Detection was excellent, as we have come to expect, with very high scores in the RAP sets, even in the proactive week, and not far short of perfect through the Response sets. The certification sets were handled impeccably and a VB100 award is easily earned. Bitdefender is another member of the elite club of products achieving 12 VB100 passes in the last two years. Stability was flawless, earning a ‘Solid’ rating.

ItW Std: 100.00%

ItW Std (o/a): 100.00%

ItW Extd: 100.00%

ItW Extd (o/a): 100.00%

False positives: 0

Stability: Solid

Main version: 13.0.256

Update versions: 7.45415, 7.45851, 7.45990, 7.46152

The first of the army of products including the Bitdefender engine amongst their armoury, BullGuard has a very solid record in our tests of late. The installer weighed in at 171MB, and ran through with minimal input required, completing in under a minute, with a couple more minutes taken for initial updates. The interface is slick, stylish and simple, with a few little quirks of design, but with a little exploration it soon becomes clear as to where things are, and a decent level of configuration is provided. Logging is once again in XML format, with a viewer system in place to keep track of things, but exporting and external analysis is fairly simple.

Speeds were impressive on demand, with strong improvements in the warm runs, and overheads were fairly light too. Resource use was a little high though, with CPU use particularly rocketing up at busy times, but our set of tasks got through in good time, and detection was splendid, with highly impressive numbers across the board. The core sets proved no difficulty, and a VB100 is duly earned, putting BullGuard on ten passes in the last two years, only our annual Linux tests not entered. Stability was again impeccable, earning a ‘Solid’ rating.

ItW Std: 100.00%

ItW Std (o/a): 100.00%

ItW Extd: 100.00%

ItW Extd (o/a): 100.00%

False positives: 0

Stability: Solid

Main version: 5.1.20

Update versions: 5.3.20/201302130928, 201303072220, 201303140338, 201303212222

Commtouch’s performances have left a little to be desired of late, with false positive issues rife in recent tests. The latest submission was a tiny 12MB installer, with updates similarly compact at 29MB. The install process runs along the usual lines, completing in under half a minute, and after a quick licensing stage updates are speedy too, the whole job done within under a minute (assuming speedy typing or pasting in of licence codes).

The interface is simple and a little lacking in modern sparkle, but does a reasonable job of providing a basic set of configuration options in an easily accessible manner. The GUI crashed a few times under heavy stress – mostly when running scans of large sets of infected samples or trying to export large logs – but protection never seemed to be impaired. Scanning speeds were a little on the slow side, and lag time accessing files considerably heavier than most. RAM use was low but CPU use high, and unsurprisingly our set of tasks took some time to get through – around double what they took on an unprotected system. Detection was a little low in the RAP sets, but much better in the Response sets thanks to heavy and clearly quite effective use of cloud look ups.

The WildList sets were handled well, but in the clean sets a number of items were alerted on with vague heuristic warnings, including some chipset drivers from a CD bundled with ASUS motherboards. This was enough to deny Commtouch a VB100 award, leaving the vendor with just one pass and four fails in the last six tests; three passes and seven fails in the last two years. A few minor issues with GUI stability were noted, but only under unusual pressure, so the product just qualifies for a ‘Stable’ rating.

ItW Std: 100.00%

ItW Std (o/a): 100.00%

ItW Extd: 100.00%

ItW Extd (o/a): 100.00%

False positives: 7

Stability: Stable

Main version: 6.0.264710.2708

Update versions: 15495, 15558, 15649

Comodo has had little luck so far in our comparatives, but we’ve seen steady improvement in the vendor’s products in terms of design, comprehensiveness of components and quality, all of which are highly encouraging. The latest version was provided as a 128MB installer, but seemed to pull down a lot of extras during the set up process, which was nevertheless quick and required minimal interaction. A reboot was required at the end, and an impressive six desktop icons were put in place for the various components, including the company’s ‘Dragon’ web browser. Updates weighed in at over 100MB in some cases, but rarely took more than a few minutes to complete.

The interface is very jazzy and glitzy, but on this platform seemed a touch laggy and prone to rendering errors. On several occasions it froze up entirely, and we had problems completing some scans thanks to further freezes. When working, though, the layout is clear and sensible and provides a good set of controls, with clear and usable logging.

Scanning speeds were no more than OK initially, but super fast in the warm run, while overheads were a little on the high side throughout. Resource use was around average, as was impact on our set of tasks. Thanks to a problem with images taken on the deadline date, and no offline update package being provided by the vendor, detection scores were not available for the RAP sets, but the Response sets start pretty high, dropping steadily into the most recent few days (this may have been impacted by some parts of scan jobs failing to complete properly).

The clean sets were well dealt with, and the traditional WildList was covered without problems on demand, but on access several items were missed, with more issues on demand in the Extended set, meaning Comodo cannot be granted a VB100 award this month. The IS product is not a regular in our tests, with just this single entry in the last six tests; four fails from four attempts in the last two years. A number of issues with both the interface and scanning processes were seen this month, generally under heavy pressure only, meaning the product is given a ‘Fair’ rating for stability.

ItW Std: 100.00%

ItW Std (o/a): 99.80%

ItW Extd: 99.27%

ItW Extd (o/a): 98.02%

False positives: 0

Stability: Fair

Main version: 7.0.0.18

Update versions: NA

Having recently switched the underlying engine from Ikarus to Bitdefender, Emsisoft’s prospects looked good from the off this month, given the performances already detailed here. The 230MB installer was downloaded from the company’s public website on the deadline day, and installed using a familiar process that has changed little despite the radical changes under the hood. The initial process is fairly speedy, but updates took a while, fetching over 160MB of data in some cases and taking up to ten minutes to complete.

The GUI is a little quirky but clear and pleasant, with a good basic set of controls which were mostly responsive. Logging is clear and reliable, but on a number of occasions the GUI locked up and required a reboot to get itself back in order. This was particularly difficult in the RAP tests where several parts of the test could not be completed; we broke things down into the smallest parts possible to get the best cover we could, but scores will doubtless reflect the issues encountered.

Scanning speeds were not bad, though, when only covering clean samples, and overheads were very light, thanks in large part to the lack of full on-read scanning by default – they slowed down considerably in the ‘full’ measures, which approached more closely the standard settings for most other products. Resource use was fairly low, and impact on our set of tasks around average.

Detection was generally pretty decent despite the missing data – scores were not quite as good as they should have been, but still highly respectable, and most users would find their experience rather better. There were no false alarms and no issues in the WildList sets, and a VB100 award is earned. Emsisoft’s recent test history is good, with three passes and two fails from five entries in the last six tests; longer term things are less impressive, with seven fails and three passes in the last two years, but given the change of engine it looks likely that things will continue to improve. There were some problems this month, mostly during unusual circumstances like scanning large sets, but these mean the product earns only a ‘Fair’ rating for stability.

ItW Std: 100.00%

ItW Std (o/a): 100.00%

ItW Extd: 100.00%

ItW Extd (o/a): 100.00%

False positives: 0

Stability: Fair

Main version: 14.0.1400.1351

Update versions: NA

Another product using the Bitdefender engine, and another that introduced a heavily redesigned GUI in our Windows 8 test late last year, eScan is one of our most regular participants, and given the issues observed so far with glitzy new interfaces, we were intrigued to see how this one would fare on creaky old Windows XP. The installer was a monster 416MB, and got going after only a few clicks. The set up process takes a few minutes, but keeps the operator entertained with a slideshow featuring a wide range of people smiling happily into their laptops. After five minutes or so a message appears announcing that a reboot is required to complete things, and offering to initiate one. Accepting the offer to reboot seemed to have no effect though, and trying to restart using all normal Windows methods also proved fruitless. It appeared that the product had disabled rebooting – one of the core functions of Windows.

Further probing found that by killing some of the eScan processes rebooting could be reinstated, but as the same issue emerged on each install (and also since one attempt to force a reboot caused a dreaded Blue Screen), we found it easier simply to hit the reset button each time we needed to restart. Once fully installed, the interface was much the same as we remembered: dark and glowering and fairly hard to see at times, but reasonably well laid out and providing ample controls. For the most part it remained fairly stable, although the disabling of reboots seemed permanent, and on a couple of occasions scans of our clean sets froze, refusing to complete or let themselves be cancelled. This seemed particularly prone to occurring when the on-access thoroughness settings were turned up.

Scanning speeds were pretty decent, and overheads mostly not bad, although with the settings fully turned up our set of binaries took an age to complete. RAM use was a little high, but CPU use not too bad, and our set of tasks was slow to complete, but not excessively so. Detection was as excellent as we would expect, with all sets very well handled – even the proactive week of the RAP sets – and only a slight drop in the very latest day of the Response sets. The WildList set was dealt with perfectly, and the clean sets too, earning eScan VB100 certification and maintaining its flawless record of 12 passes in the last two years. However, there were some serious stability issues here: a Windows system that cannot reboot is like a house with no doors – usable, but rather awkward – and disabling this pivotal feature is a fairly major error for any product. Along with a few other less significant problems, this pushes eScan’s stability rating deep into ‘Buggy’ territory.

ItW Std: 100.00%

ItW Std (o/a): 100.00%

ItW Extd: 100.00%

ItW Extd (o/a): 100.00%

False positives: 0

Stability: Buggy

Main version: 6.0.308.0

Update versions: 8006/1381, 8085/1382, 8104/1382, 8135/1383

ESET’s submission this month came as a standard installer, with instructions to set up, update and freeze on the deadline day. Installation was straightforward, the only thing of note being the unusual forced choice of whether or not to detect PUAs, and completed very rapidly. Updates were very speedy too. The interface is clear and unfussy, with comprehensive configuration, and logging is detailed, reliable and usable.

Stability was generally fine, although we did have a single instance of the GUI crashing – which was easily recovered from, did not affect protection and could not be reproduced despite our best efforts. Scanning speeds were zippy, overheads light, RAM use low, CPU use very low and impact on our set of tasks very reasonable too.

Detection was very strong, well up with the leading group in the RAP chart and hard to criticize in the Response sets; the WildList was covered without issue, and with just a handful of (quite accurate) alerts on potentially dodgy items in the clean sets, a VB100 award is well deserved, further extending ESET’s epic unbroken run of passes – 12 from 12 in the past two years, and stretching back much further than that. With just one odd incident, a ‘Stable’ rating is well deserved too.

ItW Std: 100.00%

ItW Std (o/a): 100.00%

ItW Extd: 100.00%

ItW Extd (o/a): 100.00%

False positives: 0

Stability: Stable

Main version: NA

Update versions: NA

Filseclab continues its quest for that elusive first VB100 pass, showing steady improvement with each appearance, particularly in the RAP tests. The latest build came as a 165MB installer, with offline updates in a 29MB package. The set up process was very speedy, completing in under a minute with a reboot required after some updates. After install a brief set up wizard asks some probing questions to figure out the user’s level of competence before suggesting a more or less advanced version of the interface. The GUI itself is rather attractive, slick, and clear and easy to navigate, with a detailed configuration selection. Logging seemed lucid and reliable.

Scanning speeds were a little slow initially, but blazing fast through the warm runs, while on access things were also a little sluggish, sadly with no signs of improvement later on. RAM use was low but CPU use was decidedly high, and our set of tasks ran through rather slowly.

Detection was quite competitive in the RAP and Response sets, and not bad in the WildList sets either, although not quite reaching the 100% coverage required for certification. There were a handful of false alarms in the clean sets too, including components of Audacity, Gimp, iCloud, Thunderbird and uTorrent. Nevertheless, Filseclab continues to edge closer to the goal. The vendor has no passes from three entries in the last six tests; none from five entries in the last two years. Stability was good, with just a slight wobble during one of the updates, earning a ‘Stable’ rating.

ItW Std: 97.98%

ItW Std (o/a): 97.98%

ItW Extd: 95.72%

ItW Extd (o/a): 95.72%

False positives: 8

Stability: Stable

Main version: 5.0.1.199

Update versions: 5.0.53/17.157/17.156, 17.248/17.218, 17.248/17.256, 17.335/17.312

After a couple of tests with a ‘Lite’ version of FortiClient, this month saw another new version, although it was clear from the submission details that, like the ‘Lite’ version, it did not offer the full range of controls featured in the old favourite from the last few years. The installer remains very compact at just 12MB, with an offline update package of 158MB also provided, and initial set up is fast and simple, with just a couple of clicks required and taking little more than a minute.

Our first look at the interface was rather upsetting, with much of it clearly not functioning properly, and it took us a few minutes to realize that this was due to the browser installed on our systems – as the developers had informed us, the product requires a more recent version, but there is nothing in the product itself to warn of this. With the browser updated things worked a lot better, although it is clear that there is still work to do on the interface. It provides a reasonable range of basic controls, although they can be a little hard to find and operate, partly thanks to the lagginess and shakiness that seems an inevitable part a of browser based interface (making one wonder why anyone who didn’t specifically aim to irritate their users would opt for such an approach). There were a number of error messages, a few scans freezing up, and a few stranger oddities too, such as changing some Windows desktop and user access options without being asked to.

Once we had figured out how to operate things, though, the tests moved along reasonably well. Scanning speeds weren’t too bad, on access lag times were a little high initially but much lighter in the warm runs, while resource use was high and our set of tests took a fair while to complete. Detection was excellent though, continuing an upward trend observed over recent tests as heuristics are gradually tweaked upwards. The core sets presented no issues, and Fortinet earns a VB100 award, putting it on five passes from five entries in the last six tests; ten from ten in the last two years with only our annual Linux tests missed. There’s clearly a little more work to do on this new GUI before it’s ready for real-world use though, and as it stands it only just scraped into the ‘Fair’ category.

ItW Std: 100.00%

ItW Std (o/a): 100.00%

ItW Extd: 100.00%

ItW Extd (o/a): 100.00%

False positives: 0

Stability: Fair

Main version: 10.00 build 413

Update versions: 9.50 build 19031

A pair of products were submitted by F-Secure, as usual, with this first one presumably the more corporate-focused of the two. In this case a 72MB installer was provided, but updates had to be performed online. Set up was speedy, although a reboot was required, but the speed of updates was hard to measure, as they dribbled in slowly and only displayed progress deep in the settings area of the interface. Despite clear messages in the interface stating that we were safe and secure from the off, trying to run tests before this process had completed showed some rather gaping holes in protection, so we had to wait it out each time.

The GUI itself is clean and sparse, providing a basic range of controls; it had a few wobbles here and there, including the apparent inability to run any kind of on-demand scan if an on-access detection had previously been made. A reboot was required to rectify this. Logging remains a little sketchy, with on demand logs at least mostly reliable these days – the HTML file generated after each scan farms out the list of detections to a text file if it gets too large. On access logging seems to be entirely absent, however. Scanning speeds were quite fast, speeding up even further in the warm runs, with lag times quite low on simple file access. RAM use was around average too, but our CPU use measure was well into negative figures. The reason for this is the extreme length of time taken to complete our set of activities – each run taking close to half an hour where the baseline measures were little over two minutes. Worrying that this might be some kind of freak anomaly, we re ran the job later but got similar figures, and this was confirmed by seeing similar times for the consumer product. We could only assume that it was trying to perform some kind of cloud look ups, which were taking some time to complete. Either way, there was clearly a lot of idle time during the process, as our CPU measures taken every five seconds recorded very little activity.

Detection was solid though, keeping well up with, if not overtaking other products including the Bitdefender engine in the RAP and Response sets. The clean sets saw just a warning on a screensaver from nVidia flagged as suspicious by the cloud, and the WildList sets looked to be well handled until we noted a single item in the Extended WildList not being picked up, either on access or on demand, in any of the three runs. Closer investigation into this issue had both ourselves and the developers stumped for some time, as detection seemed fine on repeating the tests later on. It was eventually discovered that the item in question had been erroneously whitelisted in the cloud for a brief spell, which by shocking bad luck had coincided exactly with our testing period. This misfortune is enough to deny F-Secure a VB100 award this month, the history for this corporate solution showing three passes and a single fail in the last six tests; seven passes and two fails in the last two years. There were a few little issues noted, assuming the extreme slowdown in our activities set was normal, and the product earns a ‘Stable’ rating.

ItW Std: 100.00%

ItW Std (o/a): 100.00%

ItW Extd: 99.95%

ItW Extd (o/a): 99.95%

False positives: 0

Stability: Stable

Main version: 12.71 build 102

Update versions: 10.00 build 18410

The consumer end of F-Secure’s product line was provided in a rather different format, with a tiny 600KB executable all that was provided initially. This pulled down the rest of the product, and seemed to do so in commendably good time, the whole install process taking less than two minutes and completing with a reboot. Once again, updating is a little hard to keep track of, and again while this was going on the interface suggested that a greater level of protection was in place than was actually the case. The GUI itself is very similar to that of the product’s corporate sister, providing a reasonable level of control and mostly seeming responsive and simple to navigate. Logging is just about usable and reasonably dependable these days, at least as far as on demand data goes.

Performance measures were fairly similar, with scanning speeds a fraction slower, overheads a trifle higher, and RAM use much the same. Once again our set of tasks took an enormous amount of time to get through, and long idle periods meant extremely low CPU use during this period. Detection was splendid for the most part, but again that single item in the Extended WildList was ignored, and no VB100 award can be granted. This product line appears a little less regularly in our tests, now showing two passes and one fail from the last six tests, with no entries for a while before that. Stability was reasonable, falling into the ‘Stable’ category.

ItW Std: 100.00%

ItW Std (o/a): 100.00%

ItW Extd: 99.95%

ItW Extd (o/a): 99.95%

False positives: 0

Stability: Stable

Main version: 23.1.0.2

Update versions: NA

G Data’s desktop product came as a fairly large 298MB installer, which took half a minute or so to find its feet before things got moving. It ran through a few standard questions then took another minute to complete, needing a reboot at the end. Updates were a little on the slow side, taking almost ten minutes on one occasion. The interface is businesslike, providing an exemplary range of controls in a simple, clear manner, and proved rock-solid under everything we could throw at it.

Scanning speeds were not bad to start with and a lot better in the warm runs, with overheads a little high initially and again improving a lot after an initial settling-in period. Resource use was a little above average, but our set of tasks completed in reasonable time. Detection rates were stellar, with all our sets brushed aside and very little indeed missed in our Response sets. With no issues in the certification sets a VB100 award is comfortably earned. G Data now has four passes from four entries in the last six tests; eight passes and one fail in the last two years. This month’s performance earned a ‘Solid’ stability rating.

ItW Std: 100.00%

ItW Std (o/a): 100.00%

ItW Extd: 100.00%

ItW Extd (o/a): 100.00%

False positives: 0

Stability: Solid

Main version: 6.0.0.0

Update versions: NA

Hauri’s products have given us some problems in recent tests, routinely underperforming compared to others based on the same Bitdefender engine. We hoped that some of the biggest issues would be fixed for this month’s test. The installer was a fair size at 214MB, and ran through quite a few steps but took minimal working time, completing in little over a minute. Somewhat oddly, a warning appeared halfway through, stating that there was no web access, but initial updates (an area where we have had problems in the past) proceeded apace; this time all went fine, and we managed to get the definitions updated to current levels, somewhat to our surprise.

The interface is a little short on charm, but works reasonably well, providing at least some configuration, although much of it is less than clear. Logging is decent for the most part, but at least one scan clearly gave up well before it had finished, with no warning. Scanning speeds were sluggish, and overheads look light but cannot be compared with most products as there seems to be no on-read protection available. Use of resources and impact on our set of tasks also look light, and again the unusual approach to protection will doubtless have played a major part in this.

Detection was excellent in the RAP sets, well up with the leaders and closely clustered with other users of the Bitdefender engine, but in the Response sets numbers were well down, suggesting that perhaps updates were not as successful as they appeared. The WildList sets were well handled, but in the clean sets a single item, part of the RealPlayer package, was flagged as malware, denying Hauri a VB100 award this month. The vendor’s test record shows a good run of late, with four passes and now a single fail in the last six tests; two fails and four passes over the last two years. A few wobbles were noted, earning a ‘Stable’ rating.

ItW Std: 100.00%

ItW Std (o/a): 100.00%

ItW Extd: 100.00%

ItW Extd (o/a): 98.65%

False positives: 1

Stability: Stable

Main version: 2.2.14

Update versions: 1.4.0/83441, 83615, 83668, 83725

Quite a regular these days with a number of good showings of late, Ikarus provided its product in the form of a CD ISO image, along with a separate 150MB offline update bundle. The set up process includes the requirement to install the .NET 2.0 framework, which is thoughtfully included in the ISO, and this adds a fair amount to the initial set up time. Otherwise things moved along nicely, with a full set of stages to click through (including the option to provide automated feedback rather sneakily hidden on a greyed-out update settings page). The interface is plain and minimalist, with only a basic set of controls, but is reasonably simple to operate and provides ample, reliable logs of its activities. On a few occasions under heavy stress it became a little shaky, but quickly recovered itself without intervention.

Scanning speeds were impressive, and overheads very light indeed, with RAM use a fraction above average, CPU use a little below, but impact on our set of tasks barely perceptible. Detection was pretty good in the RAP sets, tailing off very slightly into the proactive week, and excellent in the Response sets, with just the slightest decline in the most recent few days. The WildList sets were well handled, but in the clean sets a single item, part of the Gimp package, was mislabelled as malicious and Ikarus just misses out on a VB100 award this month. The vendor’s test history now shows two passes and three fails in the last six tests; three passes and six fails in the last two years. This month’s performance merited a ‘Stable’ rating.

ItW Std: 100.00%

ItW Std (o/a): 100.00%

ItW Extd: 100.00%

ItW Extd (o/a): 100.00%

False positives: 1

Stability: Stable

Main version: 4.5.1.7

Update versions: NA

Having been absent from our tests for a while, Iolo returns with its product looking much the same. The install package provided was a tiny 500KB downloader which fetched the main 57MB installer from the web – an option is provided to keep this installer in case it is needed again. The initial set up zips through, then pauses for a minute or so at a stage labelled ‘finalize’. Eventually a reboot is needed, after which an update runs extremely quickly, completing in less than half a minute. The product interface has a brisk, professional but not unfriendly look, and provides a surprisingly detailed level of fine-tuning, although our preferred option, to only log malware detections, is absent. In this case it would be of little use anyway, as despite repeated requests to the developers we have been unable to find any information on how to decrypt the log data, which is stored in some unfriendly database format and is not exportable from the product GUI.

Stability was mostly good, but after running our performance tests it seemed as if something odd was going on: trying to disable the on-access element to replace some test samples appeared to have worked, but access was still being blocked, so the system was rebooted. On restart, however, a message appeared saying that the product could not access a required RPC service, and was therefore unusable. Despite much effort we could find no way of repairing things, and in the end had to wipe the system and reinstall to let us move along. Otherwise, speed tests went well and seemed reasonably zippy, but overheads were very high on access; RAM use was low but CPU use very high, and our set of tasks was very heavily impacted, much as we saw with Commtouch whose engine is included in this product.

Detection rates in the RAP tests were disappointing, a little behind those of Commtouch, while Response scores lagged well behind, implying that only local detection is available here, without the additional cloud look ups that so boosted Commtouch’s own scores. Thanks to the heavy slowdown in our activities measure, the product falls well outside the scope of our new detection-performance scatter chart. Note that these scores are based on our best interpretation of extremely unwieldy logs, and as in the past it seems likely that some data may have been rendered unreadable despite our efforts.

The WildList sets were handled well though, with no problems on access and, after much careful scrutiny, a full complement of detections were unscrambled from the on demand logs. With no FPs either, a VB100 award is earned – Iolo’s first from two attempts in the last six tests. The vendor now has two passes and four fails from six entries in the last two years. Stability was generally OK, but one incident was pretty severe, albeit not reproducible, putting the product well down into the ‘Fair’ category.

ItW Std: 100.00%

ItW Std (o/a): 100.00%

ItW Extd: 100.00%

ItW Extd (o/a): 100.00%

False positives: 0

Stability: Fair

Main version: 12.2.0.174

Update versions: 9.160.8215, 13.1.0187/9.162.8300, 13.1.0188/9.163.8351, 9.164.8404

Having undergone a fairly radical redesign not too long ago, K7’s current product continues to draw appreciation from the lab team for its clear and pleasant styling. The 93MB installer runs with a single click and completes very rapidly indeed, with updates taking between one and three minutes – only the later, slower one requiring a reboot. The GUI is crisp with a strong theme, and provides a solid level of control in easily accessible fashion. It operated cleanly and smoothly throughout testing.

Scanning speeds were pretty decent, and overheads not bad either with some good speed up in the warm runs. RAM use was a fraction above average but CPU use was low; our set of tasks took a little longer than average to complete. Detection was decent – not quite up with the leaders in the RAP test but not far behind, with similar figures in the Response test too. The core sets were well handled, and a VB100 award is earned without fuss. K7 now stands on two passes and one fail from three entries in the last six tests; four passes and one fail in the last two years. This month’s assured performance earns a ‘Solid’ stability rating.

ItW Std: 100.00%

ItW Std (o/a): 100.00%

ItW Extd: 100.00%

ItW Extd (o/a): 100.00%

False positives: 0

Stability: Solid

Main version: 10.1.0.867

Update versions: NA

Kaspersky’s latest corporate product came as a jumbo 410MB installer, with an update bundle weighing in at 315MB but supporting many other products. It ran through a number of steps to complete in around a minute and a half. Initial updates were reasonably speedy, adding a couple more minutes to the total install time. The product interface is glossy and slick, with a few quirks but little difficulty accessing the comprehensive set of controls for this multi featured product. Logging is also comprehensive and easily manipulated, as befits a business-oriented product. Stability was mostly decent, with some significant slowness in exporting large logs and a single instance of the scanner crashing during a high-stress scan.

Scanning speeds were pretty slow to start with, especially in our set of executables and other binaries, but they sped up to almost instant in the warm runs. Overheads on access were light, with fairly high CPU use at busy times but use of memory and impact on our set of tasks were both around average. Detection was very strong – very steady through the RAP sets with just a small drop into the proactive week, and similarly stable in the Response sets with only a tiny dip into the latest few days. The WildList and clean sets brought no issues, and a VB100 award is easily earned by Kaspersky, which now stands on six passes out of six in the last year; nine passes and two fails from 11 entries in the last two years. The product is given a ‘Stable’ rating this month.

ItW Std: 100.00%

ItW Std (o/a): 100.00%

ItW Extd: 100.00%

ItW Extd (o/a): 100.00%

False positives: 0

Stability: Stable

Main version: 2013.SP2.0.020411

Update versions: NA

After a fairly lengthy absence from our tests, Kingsoft returned late last year with a glitzy new look and a new third party engine under the hood, courtesy of Avira. Back for more this month, the 91MB installer has a very zippy and funky progress meter which blasts through in seconds. Updates are speedy too for the most part, although on one run a couple of attempts failed before finally doing the business. The GUI is glossy and stylish but not the easiest to navigate, with some rather confusing language, but a reasonable degree of configuration is provided once you’ve found your way around.

Scanning speeds were slow over archives but good elsewhere, while overheads were perhaps just a trifle high. Resource use was distinctly on the high side, particularly use of CPU cycles, but our set of activities completed in reasonable time. Detection was very strong, as we would expect from the Avira engine, with only the very slightest of drops in the most recent RAP and Response sets. The core certification sets were handled impeccably, thus the product easily earns a VB100 award. The vendor now has two passes from two entries in the last six tests, plus an extra one just under two years ago from its previous incarnation. Kingsoft earns a ‘Stable’ rating this month, let down only by some shakiness in the updater – next time, we are promised a Chinese language only version.

ItW Std: 100.00%

ItW Std (o/a): 100.00%

ItW Extd: 100.00%

ItW Extd (o/a): 100.00%

False positives: 0

Stability: Stable

Main version: 10.4.49.4168

Update versions: 15522, 10.5.1.4369/15898, 16018, 16322

Lavasoft’s Ad-Aware is the first of a clutch of products on this month’s test bench that use the VIPRE engine (the company behind which has recently been redesignated as ThreatTrack after splitting off from mother company GFI).

We have encountered some stability issues with the Lavasoft product in the past, but things seem to be improving gradually. The installer submitted was a tiny 5.8MB with no built in detection data, and took us through a number of steps, including adding a browser plug in and adjusting the default search engine to Blekko. It also offered the option to perform a ‘compatible’ install, allowing it to operate alongside another solution. The main install took little more than a minute, with a reboot at the end, but updates were slightly slower, averaging four minutes for the first run.

The product interface is a little different from the norm but is colourful and looks professional; very minimal configuration is provided, at least as far as we could tell. Logging is in XML format and is fairly usable, but rather frustratingly it seems to wipe most of the data as soon as a task has ended – we lost a good chunk of time before we realized that this was the case. On one occasion, during a scan of part of the clean sets, the interface froze up completely and a reboot was needed to access the controls again – however, this seemed to be only a surface issue, with the scan completing successfully in the background.

Scanning speeds were very slow over archives, despite only a limited set of archive types being looked at, but not bad elsewhere, speeding up a lot in the warm runs in the other sets. On-access lag times were pretty light, although the fact that some scanning for time-consuming files has been moved to the background will have had an impact here. Resource use was low, particularly the CPU measure, and our set of tasks completed in reasonable time.

Detection rates were consistently high, with not much missed anywhere, and with nothing missed from the WildList set and no false positives either, a VB100 award is easily earned. This puts Ad-Aware on one pass and one fail from two entries in the last few months, with the vendor’s last appearance way back in early 2011. There were a few little wobbles in the GUI, but nothing too serious and a ‘Stable’ rating is merited.

ItW Std: 100.00%

ItW Std (o/a): 100.00%

ItW Extd: 100.00%

ItW Extd (o/a): 100.00%

False positives: 0

Stability: Stable

Main version: 4.2.223.0

Update versions: 1.1.9103.0/1.143.2136.0, 1.1.9203.0/1.145.1212.0, 1.1.9302.0/1.147.81.0, 1.1.9302.0/1.147.556.0

Microsoft’s free-for-home-use desktop solution has become a fixture in our desktop tests and generally proves reliable. The package provided this month was a slimline 10MB installer with offline updates also fairly small at 75MB. Set up is very fast, with initial updates taking an extra couple of minutes, but no reboots were needed. The interface is neat and clean, although occasionally there were some rendering issues, and a good basic set of controls is provided. Logging is mostly detailed and clear, although on a couple of occasions scans seemed to complete without leaving any evidence of their passing. We also saw a couple of jobs fail to run at all, with rather stark error messages, and at one point there was a full crash of the interface during an overnight scan. Most jobs ended up including an overnight period, as it seemed that all our core malware tasks – which many products manage to complete in a couple of hours – would last longer than a working day.

In the end, though, we got everything done successfully, with scanning speeds not too bad over clean files, a little low in the archive sets but still quite acceptable given the thorough unpacking defaults. On access lag times were pretty light and showed signs of good optimization in the warm runs, while resource use was very low and our set of tasks got through in good time. Detection was reliably decent in the Response sets, showing a gradual downward curve in the RAPs, but there were no issues in the WildList or clean sets and a VB100 award is well deserved. This puts Microsoft Security Essentials, which usually enters all desktop tests but missed out in the Windows 8 test thanks to the built-in Windows Defender, on two passes from two entries in the last six tests; five from five in the last year. A few stability issues were noted, including failure to log data and a GUI crash, which nudges our rating just over the edge into ‘Fair’.

ItW Std: 100.00%

ItW Std (o/a): 100.00%

ItW Extd: 100.00%

ItW Extd (o/a): 99.72%

False positives: 0

Stability: Fair

Main version: 1.1.107.0

Update versions: 83441, 836155, 83668, 83725

A relative newcomer to our tests, MSecure’s solution includes the Ikarus engine. Initial set up was from a small 16MB installer, and followed a standard path, completing in good time. Updates were provided offline, measuring 97MB, and when run online proved fairly speedy. The interface is angular and a little cluttered in places, providing a basic but reasonable set of controls, and was mostly responsive with just a few moments of jerkiness.

Scanning speeds were on the slow side, and overheads started out fairly heavy too, but improved a lot in the warm runs. Resource use was low though, and our set of tasks completed in very good time indeed. Detection was very strong, with only the proactive week of the RAP sets showing any real decline. The WildList sets were well covered on demand, but on access a few items seemed to go unnoticed and this, coupled with the expected false alarm on part of the Gimp package in the clean sets, was enough to deny MSecure a VB100 award this month despite a reasonable showing. With only two entries under its belt, the vendor now has one pass and one fail; this month a ‘Stable’ rating is comfortably earned.

ItW Std: 100.00%

ItW Std (o/a): 100.00%

ItW Extd: 99.54%

ItW Extd (o/a): 94.76%

False positives: 1

Stability: Stable

Main version: 7.00.22

Update versions: NA

Norman’s product is another that has been given a shiny new look to coincide with the arrival of Windows 8, and we were interested to see how well it worked on a rather older, plainer platform. The installer submitted was a fairly large 226MB, and after a few clicks the set up seemed to complete fairly quickly, but there was clearly much still going on in the background, with a message appearing several minutes later indicating it was finally done and requesting a reboot. Updates took quite a long time, and required another reboot, and on some occasions further updating went on after this. On a couple of occasions the installer crashed out with a less than helpful error message, but each time it ran without problems on the second attempt.

Once everything was set up, we found the new interface working rather well, looking bright and cheery, generally responding at reasonable speeds but occasionally lagging a little. Settings are basic but fairly simple to operate, and logging was good on demand, but a little baffling on access, with data often seeming to be entirely absent.

Scanning speeds were a little slow in some areas, but not bad at all in others, with lag times fairly high initially but dropping considerably in the warm runs. Resource use was a little above average, but thanks to the optimization our set of tasks completed very quickly. Detection was impressive, dropping off a fair bit in the proactive part of the RAP sets but solid elsewhere. There were no problems in the clean sets, but in the extended WildList set a couple of items were missed, rather surprisingly. On consultation with the vendor it was eventually spotted that a version of the scan service that was in use for only a brief period – which just coincided with the test – contained a small bug affecting a small number of detection identities, and it was this that caused the problems we observed. This bad luck denies Norman a VB100 award this month, putting the vendor on four passes and two fails in the last six tests; nine passes and three fails in the last two years. With a few little wobbles noted, a ‘Stable’ rating is earned.

ItW Std: 100.00%

ItW Std (o/a): 100.00%

ItW Extd: 99.91%

ItW Extd (o/a): 99.91%

False positives: 0

Stability: Stable

Main version: 10.9.82 (build 3350)

Update versions: NA

Optenet is not the most regular of participants in our tests, but generally performs decently, with its current product line based on the Bitdefender engine. The submission for this month’s test was a small 26MB installer without built in detection data, relying instead on initial updates. The set up was a little confusing, with one dialog box appearing behind another, making us think at first that nothing was happening. After that it zipped through quite rapidly. Updates took a little while to complete though, and on one occasion seemed to get nowhere at all; after a full hour, the system was rebooted and the update process restarted, but after another two hours there was still no progress. The system was then reimaged and the product reinstalled, and left overnight, after which all seemed in order at last.

The interface has been redesigned fairly recently, with a browser-based affair prone to all the usual lags and wobbles. A preponderance of pop ups caused some nasty issues in the on access tests, where a fair number of samples are accessed in rapid succession; on several runs the flood of windows caused the system to grind to a crawl, with multiple error messages complaining of lack of memory, run-time errors and other ‘unexpected’ issues. Memory issues cropped up in on-demand scans too, with our RAP scan bringing the machine almost to a halt, an epic total 2GB of RAM being used by the product’s two main threads, 1.75GB for just one of them – little surprise, then, that the machine was having trouble.

These issues mostly occurred in fairly unusual circumstances (apart from the update problems), and our speed and performance measures, dealing mainly with clean samples, ran through OK. On demand speeds were reasonable, with a small improvement in the warm runs, while on access lag times were fairly decent too. RAM use was a little high, CPU use not bad, and our set of activities completed in pretty good time.

Detection was very strong, well up with other products based on the same engine, and with no issues in the core sets a VB100 award is duly earned. Optenet now has one pass and one fail in the last six tests; two passes and one fail in the last two years. There were a lot of issues here, mainly but not all occurring in stressful situations, and stability is rated as ‘Buggy’.

ItW Std: 100.00%

ItW Std (o/a): 100.00%

ItW Extd: 100.00%

ItW Extd (o/a): 99.26%

False positives: 0

Stability: Buggy

Main version: 2.1.1

Update versions: NA

Panda’s free cloud-based solution has had a run of good performances since rejoining our tests in the middle of last year. The set up is performed from a tiny 800KB executable, with a few extra bits pulled down from the web in a few seconds, and the whole process is complete in under a minute. The interface is clean and simple, with little more than some basic stats and a scan button, but there is some configuration available. Stability was mostly good, but during large scans there seemed to be some occasional loss of connectivity to the cloud; on some installs, Windows Update appeared to have been enabled without being asked for, and on some of the jobs the system was very slow. After a reboot a message hovered over the system tray reading ‘detener antivirus’.

Things went smoothly almost all the time though, with scanning speeds pretty good – a little slow at first over binaries and archives, but showing improvement in the warm runs. Overheads were fairly light, although not as light as one would expect given the lack of full on-read protection (only some file types seemed to be blocked on-read), while resource use was low and our set of tasks completed in pretty good time.

Detection was a little hard to measure, as standard logging is capped at a fairly low size, while the ‘advanced’ logging records a huge amount of data, but cuts out when it reaches 500MB (easily done with some of our bigger scans). Despite great care, splitting scan jobs into multiple small parts and frequently backing up logs, when the time came to process the data it still seemed to be incomplete, so averages of the available data had to be used. Scores were pretty decent in our Response sets though (bearing in mind some of the data was likely to be missing), and there was no RAP score as the product cannot operate without access to the Internet. The WildList set caused no problems, and the clean sets were well handled too, earning Panda another VB100 award, its third from three appearances since rejoining our tests last year. A few fairly minor stability issues were noted this time, but a rating of ‘Stable’ is still just about earned.

ItW Std: 100.00%

ItW Std (o/a): 100.00%

ItW Extd: 100.00%

ItW Extd (o/a): 99.83%

False positives: 0

Stability: Stable

Main version: 1.0.0.34

Update versions: NA

The PC Pitstop products have been sent in for us to look at on a few occasions over the last year or so, but to date we have not been able to include one in a full report – thanks in part to the way the on-access mode was implemented and in part to some trouble handling large infected sample sets. The current version, which is based on the VIPRE engine, promised some improvements which were expected to set things on the right lines.

The installation process is a little unusual, with a small 1.5MB package performing the initial set-up, most of which seems focused on system optimization and other rather dubious features. Once a user has logged into an account, further features become available, including the ‘Nitro’ download optimizer and ‘SuperShield’, the main anti-malware component. Installation of this portion runs in the background, with the stipulation that it could take quite some time to complete, but in most cases it seemed to be done within ten minutes or so. The real time component seemed similar to that offered by VIPRE itself and other products using the same engine, with some scanning going on in the background, so that some files are rather disconcertingly granted access to at first, only to be condemned retrospectively. Oddly, though, when running the on access component over the WildList set it took several hours to perform a job that most products handled in minutes.

There are few controls for this beyond on and off, while the on-demand component consists of a single catch-all scan function which promises all manner of checks, for a variety of things which might be suspected of slowing a system down or rendering it vulnerable. The only way of scanning a folder for malware is to disable most of these checks and then set it to scan a specific area only – rather a fiddly task. Even after all this effort quite some time seems to be spent on other tasks, but in the end we managed to perform all the jobs we wanted to. Very few of them completed satisfactorily, with almost every scan which included a detection fading away with a variety of messages about Ajax errors and other GUI nasties. Logging was mostly OK in the background though (in the usual fussy XML format).

Speed measures were about the only tasks that did complete without errors, and showed some very slow speeds indeed, although on access lag times were not too bad, showing some improvements in the warm runs (this will have been helped by the background scanning feature). RAM use was a little high, CPU use very low, and our set of activities completed at a good pace. Detection was as solid as we would expect, dipping very slightly in the latest day of the Response test and a little more deeply in the proactive week of the RAP test. With the WildList properly handled and the clean sets producing no surprises, PC Pitstop earns a VB100 award on its first full appearance. Stability was a little suspect though, with a number of GUI errors not all of which seemed to be caused by handling unusual amounts of malware, thus meriting a stability rating well into the ‘Fair’ category.

ItW Std: 100.00%

ItW Std (o/a): 100.00%

ItW Extd: 100.00%

ItW Extd (o/a): 99.95%

False positives: 0

Stability: Fair

Main version: 4.0.0.4030

Update versions: NA

Qihoo’s 360 is another product with an unusual approach to ‘real-time’ protection – in past tests allowing malicious samples to be copied and pasted, but somewhat later announcing that they had been found to be suspect and access blocked, although the files had been written. This time an installer of 157MB needed minimal interaction and finished rapidly; updates added a little less than a minute on average. The product interface is another to have had a Windows 8-style tiled facelift, but the controls underneath are little changed, providing a reasonable level of fine-tuning. Logging is thorough and reliable.

Scanning speeds were slow in the archive sets but pretty zippy elsewhere, with very light overheads on access (thanks to not actually checking anything in real time), with correspondingly low resource use and impact on our set of tasks. Detection was splendid, in line with others based on the same Bitdefender engine, with only the slightest decline in the most recent parts of the sets; nothing was missed in the WildList, and with no issues in the clean sets a VB100 award is well deserved. Qihoo now has four passes from four entries in the last six tests; five passes and two fails in the last two years. Only a couple of very slight wobbles were noted, earning a ‘Stable’ rating.

ItW Std: 100.00%

ItW Std (o/a): 100.00%

ItW Extd: 100.00%

ItW Extd (o/a): 100.00%

False positives: 0

Stability: Stable

Main version: 14.00 (7.0.0.4)

Update versions: NA

Another of our most regular participants, Quick Heal’s current solution was provided as a 285MB installer, which required little work and completed in under half a minute. Updating added a couple of minutes to the total set up time. The interface is smooth and attractive, with a few odd quirks but a thorough level of configuration is available, once the layout has been mastered. Stability was good, with only a single small stumble in the RAP sets.

Speeds were zippy in most cases, only slowing down in the archive sets, although default settings are rather light. On access overheads were pretty light, as was CPU use, although RAM use was a little high; the time taken to complete our set of tasks was pretty much spot on the average for this month.

Detection was rather mediocre, declining slowly through the Response sets, remaining fairly steady through the RAP sets but lagging behind the top performers throughout. The core sets were well managed though, and a VB100 award is earned, putting Quick Heal on four passes and one fail from five attempts in the last six tests; seven passes and two fails from nine entries in the last two years. There were very few issues with stability, and the product earns a ‘Stable’ rating.

ItW Std: 100.00%

ItW Std (o/a): 100.00%

ItW Extd: 100.00%

ItW Extd (o/a): 100.00%

False positives: 0

Stability: Stable

Main version: 10.2.4

Update versions: 3.40.1/4.86G

Another of our regular entrants, Sophos’s product has remained little changed over the last few years. This month’s installer was a 100MB executable, with updates pretty small at 8MB. The set up process involved unpacking to one location, then installing to another, with the whole job taking little more than a minute, initial updates adding about the same amount of time. The interface is clear and efficient, providing a wide range of controls in the main configuration areas and much more detail in an advanced dialog. Logging is comprehensive and reliable, and stability was impeccable throughout.

Scanning speeds were good to start with and blindingly fast in the warm runs. RAM use was a little above average, CPU use quite well below, and our set of activities completed in impressively short time. The certification sets were well handled and a VB100 award is easily earned, putting Sophos on a perfect six passes in the last six tests; ten passes and two fails in the last two years. There were no issues with stability, earning the product a ‘Solid’ rating.

ItW Std: 100.00%

ItW Std (o/a): 100.00%

ItW Extd: 100.00%

ItW Extd (o/a): 100.00%

False positives: 0

Stability: Solid

Main version: 7.4.24969.501

Update versions: NA

Testing Tencent’s product is always a bit of a wild ride, the lack of any language options beyond the company’s native Chinese making operation a bit of a lucky dip, even with the aid of the detailed guide provided by the developers. The installer is mid-sized at 119MB, set up is fast and simple with only a handful of clicks required (although exactly what we are agreeing to remains something of a mystery); it completes within half a minute, with updates adding another minute or two.

The GUI looks fairly complicated, suggesting a wide range of components, with only one small area required for our needs. The limited controls we were party to were clear and simple enough. Logging seemed reliable, and scanning speeds were fast in some areas, average in others, and slow in the archive sets. Overheads were low, and once again (like others from the same region) on-read protection was absent, at least by default. Resource use seemed on the low side, and our set of tasks zipped through very rapidly, but this will have been helped by the less thorough level of analysis than is provided by most.

Detection was excellent, thanks to the underlying Avira engine, with a minimal number of misses, and the core sets were handled perfectly, easily earning the vendor another VB100 award. Tencent has an impeccable record of four passes from its first four appearances in our tests, and this month we once again saw no stability issues, earning it a ‘Solid’ rating.

ItW Std: 100.00%

ItW Std (o/a): 100.00%

ItW Extd: 100.00%

ItW Extd (o/a): 100.00%

False positives: 0

Stability: Solid

Main version: 6.1.5493

Update versions: 15502, 6.2.1.10/15900, 16018, 16194

The team behind VIPRE have recently been farmed out from mother company GFI to form a new company called ThreatTrack – but the changes behind the scenes don’t seem to have had any great effect on the product itself so far. The installer is a compact 8MB, with offline updates measuring 93MB. Installation requires an Internet connection and features a slideshow to keep the user from boredom – however, there is little chance of that as it is over in no time, with initial updates bringing the whole process to around two minutes, with a reboot to complete. The product interface is fairly clear, sparse and simple but provides some basic controls, and seemed reasonably free from wobbles – although as usual handling large infected sets proved something of a lottery, with detection data stored in memory and scans becoming ever slower as space filled up.

Things mostly went well though, and scanning the clean sets was pretty speedy, becoming very fast indeed in the warm runs. Overheads were mostly light too, although somewhat oddly they were higher in the sets of media and documents than elsewhere, but again the warm runs showed great improvement. Resource use was below average and our set of tasks completed in excellent time. As noted elsewhere, moving some intensive tasks to the background may have contributed to this.

Detection was very good too, declining gently through the RAP sets but with a fair drop into the proactive week. The Response sets were well covered with only a slight decline in the last few days. The core sets were properly handled and a VB100 award is easily earned, putting ThreatTrack (formerly listed in our test history as GFI) on two passes and one fail from three entries in the last six tests; six passes and two fails in the last two years. Stability was pretty good this month, earning a ‘Stable’ rating.

ItW Std: 100.00%

ItW Std (o/a): 100.00%

ItW Extd: 100.00%

ItW Extd (o/a): 100.00%

False positives: 0

Stability: Stable

Main version: 5.0.0.313/12.163

Update versions: 5.0.1.0324/12.163, 5.0.0.0313/12.163, 5.0.0.0318/12.163

There was something of a surprise this month, as not long after setting up the usual business solution from Total Defense, we started receiving urgent messages from the developers, requesting that we replace it with a cloud oriented solution which has apparently been on the market for several months but has not as yet appeared in our tests. Thus users of the traditional Total Defense business solution (referred to as ‘r12’ or more recently ‘r14’) are urged either to replace it with this product, or to disregard the results reported here as they will not apply – this product features a third-party detection engine and has many other differences from the product we have looked at before. Set up was rather baffling, perhaps in part thanks to rather limited initial communication from the developers, and for the first time ever, after putting well over 1,000 products through the VB100 process, we had to be led through the installation in a WebEx. Monitoring its progress, we were able to memorize the core points and repeat it later with reasonable success, although much searching through of options was required.

Both installation and configuration are run from a web console, with a local web console also available once things are installed, providing some basic controls. Navigation was clear in some areas, but much less so in others. Logging seemed very thorough, to the point of overwhelming with detail. Stability was a bit of a mixed bag, with initial attempts to run the on-access tests seeming unsuccessful, and only around half of our WildList samples were blocked, but running updates several times and rebooting the system to ensure they were fully in place seemed to make all the difference, with detection improving dramatically on the third and final attempt. The same effect seemed to repeat on each run, implying that full protection takes quite some time to come into play after updates. On one occasion the product seemed unable to load up at all, with a rather unhelpful message simply reading ‘module error’ in the product’s status box; this was easily fixed by using the ‘reinstall product’ feature – the very fact that this button is provided, as part of a GUI with very few controls indeed, suggests that this might be something that is needed on a regular basis.

The shortage of controls extended to basic scanning options, with the only choice being between a quick scan and a full scan, the latter covering all locally attached drives. Thus, our on-demand speed tests could not be performed, and all other on-demand tests had to be done by copying sets to the C: partition with the product disabled, unmounting all other partitions and then running the full scan – a rather unwieldy and time-consuming approach but one that seemed to work out OK. On access overheads were measurable at least, and showed some reasonable times – a little heavy in the sets of binaries and media files, but pretty light elsewhere. RAM use was pretty high, probably mostly thanks to the need for a browser to be running in order to access the product GUI, but CPU use was barely perceptible; our set of activities took a little while to get through, but not excessively long.

Detection rates were stellar though, thanks to the wise choice of third-party engine, putting the product well up amongst the leading cluster in our RAP chart and showing some pretty impressive scores in the Response tests too. The WildList and clean sets were properly dealt with, and Total Defense earns a VB100 award for its cloud-centric solution. This is the first VB100 for this particular product but in our history it will follow on from previous business offerings from the vendor, which now show four passes from four entries in the last year; eight passes and a single fail in the last two years. A few oddities were noted, some of which were fairly serious as they affected protection, resulting in only a ‘Fair’ rating for stability.

ItW Std: 100.00%

ItW Std (o/a): 100.00%

ItW Extd: 100.00%

ItW Extd (o/a): 100.00%

False positives: 0

Stability: Fair

Main version: 8.0.0.215

Update versions: 6196.0.0.0, 6227.0.0.0, 6233.0.0.0, 6246.0.0.0

The second product from Total Defense this month is the company’s rather more familiar consumer solution, based on its own detection technology. The set up is run from a 172MB package, which runs through quickly and requires a reboot. Online updates ensue, lasting a couple more minutes, and then a second reboot is requested, followed in some cases by further update activity. On one install, applying the licence key supplied by the developers brought up a message insisting that the licence had expired, and that we should urgently renew it as the product (as yet unused) had already ‘purged’ three threats from the system – these appeared to be cookies dropped by the MSN website visited when we fired up Internet Explorer to check connectivity when first setting up the test machines. Both before and after this, however, the same licence key worked fine and seemed to have ample time remaining, so we were able to avoid spending a further $69.99.

The product GUI suffers somewhat from style over substance, looking very glitzy and funky but often being a little tricky to operate and navigate, with some very confusing labels in places. A basic level of control is available though, after some exploration. Logging was mostly decent, but on a couple of runs it failed to produce any logging at all; fortunately it was a fairly zippy process to re run the scans and details were recorded. Some other scans in the Response sets froze up entirely and could not be completed, meaning some data had to be extrapolated by averaging results from other runs.

Detection rates in the Response sets were uneven and generally mediocre; as the developers insisted the product relies largely on the cloud, it was omitted from the RAP test, but an unofficial run over part of this showed that detection was indeed worse without web access. The WildList sets were properly handled, but in the clean sets a handful of false alarms were recorded, including parts of packages from Adobe, Roxio and Vuze. This was enough to deny Total Defense a VB100 award for its consumer suite this month, putting it on one pass and two fails from three entries in the last six tests; three passes and three fails in the last two years. A number of stability issues were observed, most of which were fairly minor, but there were enough of them for stability to be rated only as ‘Fair’.

ItW Std: 100.00%

ItW Std (o/a): 100.00%

ItW Extd: 100.00%

ItW Extd (o/a): 100.00%

False positives: 7

Stability: Fair

Main version: 13.0.9.5102

Update versions: NA

Yet another product including the Bitdefender engine, this time in parallel with that of AVG, TrustPort’s dual-engine approach makes it routinely a high performer in our detection tests. The product is provided as a 240MB installer, which runs through the usual steps to complete within a minute or so; initial updates are not the fastest, adding a few more minutes to this process. The product interface is a little unusual, with some fairly basic buttons covering the main functions and everything else accessed via a couple of drop-down menus. Once it has been ferreted out, the ‘advanced’ configuration dialog provides ample fine tuning options. Logging is a little unreliable, with a fixed size cap on the main records which seems not to change when the option for setting the cap is adjusted – dumping data which the user had every right to expect to see retained. Scan logs are also recorded in separate XML files though.

Scanning speeds were a little on the slow side, and overheads were a little high, especially with the scan depth options turned to high, but speeds increased considerably in the warm runs. Resource use was a little high too, but our set of tasks got through in reasonable time. Detection was excellent, as one would expect, with superb scores in both the RAP and Response tests, and there were no problems in the core certification sets, thus earning TrustPort a VB100 award. The vendor now has four passes and a single fail in the last six tests, skipping only our annual Linux comparative; seven passes and two fails in the last two years. There were a few fairly minor issues noted, and the product earns a ‘Stable’ rating.

ItW Std: 100.00%

ItW Std (o/a): 100.00%

ItW Extd: 100.00%

ItW Extd (o/a): 100.00%

False positives: 0

Stability: Stable

Main version: 6.1.36.15526/15526

Update versions: 6.1.36.15898/15898, 6.1.36.16018/16018, 6.1.36.16192/16192

UnThreat is another product that uses the VIPRE detection engine, and has given us a few headaches in the past. The current version came as a small 11MB installer, which runs simply and rapidly, with updates a little slower, fetching down close to 100MB of data in the initial run, with a reboot needed afterwards. The product GUI is clear and simple, following usual standards and not trying to be too clever, and provides a decent level of control. Logging was mostly in a rather inefficient and repetitive XML format, but it was fairly simple to process thanks to a generic tool provided long ago by the engine developers. As in others using the same engine, logging is not written to disk until a scan completes, and in this case only the last log is kept.

Completing scanning proved far from simple, once again. Many, many scans died, froze, locked up the entire machine, or otherwise came to an end unsatisfactorily. Although we had several issues in the RAP sets, the scanning of malware samples seemed largely unaffected, with the Response tests mostly running through smoothly. The main problem seemed to be with some archive files, mostly JAR files, which could easily send the product into a bewildered and unusable state. Several attempts to get through sections of our clean sets were left running over weekends only to sit stuck near the start for days on end, and generally a reboot was needed to clear things up. On one occasion (while scanning a batch of simple bitmap images inside Zip archives) we managed to kill a frozen task using the Task Manager, but found the system entirely unusable, with both the start button and Windows Explorer refusing to respond. Most of the clean sets and several parts of the speed sets had to be picked apart carefully to remove the files which seemed to cause upset, but in many cases we observed that there only seemed to be an issue with the files in specific locations: samples which were definitely causing crashes when scanned as part of the main sets were handled fine when moved to a different spot and scanned separately there. In the end we handed as much information as possible to the developers, gathered as much data as we could for ourselves, and made the best of a bad job.

With sets perhaps slightly depleted to allow us to get through them, scanning speeds proved pretty reasonable: slow to start with but speeding up a little in the warm runs. Overheads on access were a little heavy, again much better in the warm runs, and resource use was low, with a good time taken to complete our set of tasks. Detection was solid, with good scores throughout, and the WildList sets presented no difficulties. What was tricky was nursing the product through the clean sets, which took several times longer than most other products and required constant hands-on attention, but in the end we satisfied ourselves that all files had been examined with no false alarms reported, so a VB100 award is just about earned. With a fairly limited test history, UnThreat has one pass from a single entry in the last six tests; two from two in the last two years. Stability was a major worry this month though, with a number of issues mostly related to scans failing, some of which caused some nasty side effects on the test machines; a ‘Flaky’ rating is deserved.

ItW Std: 100.00%

ItW Std (o/a): 100.00%

ItW Extd: 100.00%

ItW Extd (o/a): 99.95%

False positives: 0

Stability: Flaky