VB100 Comparative review on Windows Server 2003 R2

Main version: 7.0.1644

Update versions: 120815-3; 7.0.1646/120910-0

Avast! comes first in our list this month, thanks to the absence of a handful of regulars that usually appear ahead of it alphabetically. However, it is often one of the first products we try out in a new test, its reliability, rapidity and ease of use making it ideal for sanity-checking our set-up.

The package provided for this dedicated server edition was a single executable measuring 105MB, and the set-up process is not too different from that of the home-user and free solutions we generally see on our test bench, with plenty of colour and clear, friendly language. It requires only a handful of clicks – rather surprisingly it presents some advertising for a mobile solution, but completes the bulk of the process in less than half a minute. Updates are mostly swift and reliable, although on some occasions an additional ‘program update’ is required alongside the standard data enhancement, and this can take a little longer, with reboots required on some occasions. Nevertheless, few installs took more than two minutes to complete.

The interface is also very similar to other products in the company’s range, with a very clear design and layout, attractive shapes and colours and an all-round pleasant user experience. Configuration is excellent and mostly reliable, although we did have an issue with some of the logging settings, having adjusted the location of the real-time log but finding our changes ignored. Fortunately, logging was gathered properly elsewhere and no data was lost.

Tests blasted through at a zippy pace, with no further issues to report. Scanning speeds were good, lag times fairly low (although this figure is hard to compare with the bulk of the field thanks to there being only minimal on-read protection by default). Our set of activities got through in pretty good time, with reasonable RAM use and CPU use a fraction below the average for the month.

Detection was excellent in the RAP tests, but a little unpredictable in the Response sets, with some days covered notably less well than expected. This dented the averages somewhat, but final scores were still more than respectable. The core sets were handled excellently, and a VB100 award is easily earned. That gives Avast! five passes and a single fail in the last six tests; 11 passes in the last two years – a very solid record. Stability was good, with only a minor issue with logging noted, thus earning a ‘stable’ rating.

ItW Std: 100.00%

ItW Std (o/a): 100.00%

ItW Extd: 100.00%

ItW Extd (o/a): 100.00%

False positives: 0

Stability: Stable

Main version: 2012.0.2197

Update versions: 2437/5200, 2012.0.2221/2437/5272, 2441/5277

AVG submitted its standard corporate Internet Security edition, which was provided as a 171MB executable. Set-up takes a little while, with a few clicks required before the option of an ‘Express’ install path is offered. From there on it takes a little over three minutes for the initial install, and then on completion it runs a quick update, which claims to be complete after only ten seconds or so. If the user then clicks the ‘update’ button, a window opens with a list of updates which are required, and this process takes a little longer (several minutes in some cases, with occasional freezes of the progress screen and other general wobbliness in the interface), and often requests a reboot to complete.

When everything was finally in place, testing proceeded fairly smoothly, with scanning speeds not bad, lag times decent too, and resource use very low indeed; our set of tasks also completed in good time. Detection was decent in the RAP sets, dropping off fairly sharply in the proactive week, and very good in the Response sets, with only a slight decline on the last day. The core sets were again well handled, with no issues in the WildList or clean sets, and AVG comfortably earns a VB100 award.

The vendor’s record shows just one fail and five passes in the last six tests; two fails and ten passes in the last two years. With some slight wobbliness during the rather confusing update process, a ‘stable’ rating is earned.

ItW Std: 100.00%

ItW Std (o/a): 100.00%

ItW Extd: 100.00%

ItW Extd (o/a): 100.00%

False positives: 0

Stability: Stable

Main version: 12.0.0.2309

Update versions: 8.02.10.132/7.11.39.182, 8.02.10.158/7.11.42.206, 8.02.10.162/7.11.43.62, 8.02.10.164/7.11.43.130

Another full server solution, Avira provided its product as a fairly compact 91MB installer, including all required definition data for the RAP tests. Our first attempt at installing it was aborted fairly speedily, with several messages saying parts of the package could not be unpacked, and there were some rather odd effects on the operation of the desktop. After a reboot however, everything proved perfectly normal, and we were unable to reproduce the oddity over multiple repeat attempts. Under normal circumstances, the install process requires only a couple of clicks and zips through very speedily; updates are so fast it’s hard to spot them happening at all, and even with a ‘quick scan’ at the end the whole process never took more than a minute, with no reboots required.

The interface uses the MMC subsystem, which has caused much displeasure among the lab team in the past, but in this instance it seems fairly well implemented, with navigation and operation fairly simple and user-friendly. Configuration is available in great depth, and responsiveness under pressure seemed good.

Detection was very good indeed, with RAP scores excellent in the reactive weeks and still solid in the proactive part. Response tests maintained a very high level throughout. No problems were noted in the clean sets, and a VB100 award is thus earned without difficulty. Avira maintains an impeccable run in our tests with 12 passes in the last two years; this month’s performance earns a ‘stable’ rating, with just the single odd and non-reproducible freak-out at the install stage.

ItW Std: 100.00%

ItW Std (o/a): 100.00%

ItW Extd: 100.00%

ItW Extd (o/a): 100.00%

False positives: 0

Stability: Stable

Main version: 6.00

Update versions: 1.1.2248, 1.1.2303, 1.1.2312, 1.1.2343

A few months ago, as part of a swathe of changes across the industry, vulnerability specialist and developer of the Blink product, eEye Digital Security, was acquired by BeyondTrust, a major player in the access control field. We have updated our naming, although there has so far been little change either in the product or in the company websites, other than posts announcing the acquisition. The product has been a regular on our test bench for some time, with this month’s submission arriving as a fairly hefty combo of a 264MB main installer and a 162MB update bundle for offline use.

The set-up process is fairly simple and speedy, taking little more than a minute, but initial online updates were rather slow, taking at least ten minutes and sometimes longer. This pushed the overall install time close to a quarter of an hour. On one occasion an update failed to complete successfully, with little information provided, and on repeat attempts it seemed to freeze for some time, finally taking close to 20 minutes to complete.

The product interface is crisp and clear, covering a wide range of security areas of which anti-malware, based on the Norman engine, is but a part. Configuration is thus limited by the available space, but a reasonable amount of fine-tuning is provided, and it seems clear and responsive. As noted in past tests, scanning speeds were slow over archives and binaries – where the sandbox solution incorporated in the scanner adds some time to the analysis of unknown items – but reasonable over documents and other items. Lag times were rather heavy but not outrageously so, and while RAM use was fairly low, CPU use was very high. Our set of tasks completed in reasonable time though.

Detection was very good in the earlier RAP weeks, declining steeply into the most recent reactive week and the proactive part of the sets. This impression was confirmed in the Response sets, with fairly unimpressive and rather uneven scores. The core sets were dealt with well though, with no problems in the WildList sets and in the clean sets just a few alerts on suspicious and ‘potentially unwanted’ items. A VB100 award is thus earned, putting Blink on four passes and one fail in the last six tests; eight passes and two fails in the last two years. With just one non-recurring incident noted during an update, a ‘stable’ rating is earned.

ItW Std: 100.00%

ItW Std (o/a): 100.00%

ItW Extd: 100.00%

ItW Extd (o/a): 100.00%

False positives: 0

Stability: Stable

Main version: 3.5

Update versions: 3.5.20.2/7512929, 7547254, 7572488

Bitdefender’s submission came as a 191MB executable, which threw us a little right at the start by informing us that our test system was not good enough to host the solution. On closer inspection we quickly realized that it was merely complaining about the absence of the .NET 2.0 framework, and that this was in fact only required to support Exchange components. Moving quickly on, the rest of the install process was uneventful, with most defaults set to ‘let me decide’. This is another full-blown server solution using the MMC subsystem for its interface, and again despite the lab team’s ingrained distaste it was generally found to be pleasantly designed and usable, providing an excellent range of configuration options. At one point we did observe a message complaining about a script error, but for the most part it ran smoothly. Updating was very speedy, taking little more than half a minute even for the initial download.

Scan rates were slow over archives, which are analysed very thoroughly by default, but nice and fast elsewhere, with fairly light overheads, particularly in the warm runs. Resource use was low and our set of tasks got through in good time. Detection was once again superb, with excellent scores in both the RAP and Response tests, both showing just a slight downward trend into the more recent sets.

The certification sets were dealt with efficiently and a VB100 award is well deserved, keeping Bitdefender in the elite group of products that can boast 12 consecutive passes in the last two years. With just some minor interface wobbles noted, a ‘stable’ rating is earned.

ItW Std: 100.00%

ItW Std (o/a): 100.00%

ItW Extd: 100.00%

ItW Extd (o/a): 100.00%

False positives: 0

Stability: Stable

Main version: 12.0.230

Update versions: 12.0.0.29/12.0.0.27/12.0.0.58/12.0.0.52

A sibling of Bitdefender with a similarly strong performance record, BullGuard’s product arrived as a 160MB executable including all necessary updates. Set-up requires only a couple of clicks – the standard steps of welcome, choosing options and accepting a EULA all compressed into a single stage – and after ten seconds warned us that a driver for the behavioural component could not be installed (presumably due to a slight incompatibility with the platform, the product’s main focus being home users).

The interface is bright and cheery, with a rather unusual approach to design and operation but it provides a good basic set of controls. The real-time section claimed that the on-access component was ‘starting’ for rather a long time after installation, and accompanying buttons were greyed out, but there seemed to be no interruption in protection.

Scanning speeds were very fast, even in initial runs, with warm runs barely measurable, and overheads were pretty light too, again speeding up considerably after a settling-in period. RAM use was around average, but CPU use notably on the high side, while our set of activities got through in decent time. There were no issues in the certification sets, and BullGuard comfortably earns another VB100 award. Having skipped a few tests (mainly the annual Linux comparative), the vendor remains on nine passes but no fails in the last two years. With little to report other than the unsettling issue with the ‘starting’ message, a ‘stable’ rating is earned.

ItW Std: 100.00%

ItW Std (o/a): 100.00%

ItW Extd: 100.00%

ItW Extd (o/a): 100.00%

False positives: 0

Stability: Stable

Main version: 2.1.91

Update versions: 15.0.147, 15.0.188, 15.0.192, 15.0.194

Given the recent upheavals behind the scenes, with engine developer VirusBuster handing over control of development and support to Agnitum, we were surprised to see entries from the regular cluster of products based on the Preventon SDK, but they bravely arrived at the close of the deadline day. First up in this crew is Clearsight, which provided a compact 85MB installer for its product. Installation was speedy and simple, and updating seemed fast too, but appeared not to be working properly: after the download stage, a message suggested that the update was complete, but also stated that definition data was well out of date. Repeat attempts didn’t help, and only rebooting nudged the display system into updating its message, showing the correct information on the data installed.

With that hurdle out of the way things moved along nicely. The product interface is unchanged and provides a decent basic set of controls in a simple and generally reliable fashion. Speed tests plodded through with scan times a little sluggish and overheads surprisingly high. Resource use was also a little above average, but our set of tasks didn’t take too long to complete.

Detection rates were not bad in the earlier part of the Response sets but dropped away fairly sharply; RAP scores were a little unpredictable but also showed a steep dip into the later weeks. There were no problems in the clean sets but the WildList sets showed quite a few misses, improving slightly in later runs but still leaving much to be desired. This might have been expected given the transfer period in the development of the underlying engine. No VB100 award can be given to Clearsight this month, but the vendor’s record looks quite decent of late, with a single fail and three passes from four entries in the last six tests; seven passes and one fail in the last two years. With some rather odd behaviour during the initial installation, which was repeated on every run, a ‘stable’ rating is only just earned.

ItW Std: 98.25%

ItW Std (o/a): 98.07%

ItW Extd: 95.89%

ItW Extd (o/a): 95.84%

False positives: 0

Stability: Stable

Main version: 5.1.16

Update versions: 5.3.14/201208150952, 201209130125, 201209181239, 201209190937

It’s all change at Commtouch too, with the acquisition of Frisk meaning that it is now the owner of the engine it formerly made use of as a third-party offering. Ties between the two firms have long been close though, and the merging of teams appears to be going smoothly. The product is generally among the smallest at submission time, this month measuring just 14MB for the main installer and offline updates weighing in at 28MB.

Set-up is simple and speedy with just a handful of clicks required, and updates were very fast too, taking under 30 seconds in most instances. The product interface is basic and starting to look a little in need of a refresh, but it provides decent controls and is fairly easy to use, as well as mostly very reliable. Scanning speeds were pretty slow, and overheads very high with most of our lag tests dragging on for quite some time. RAM use was low but CPU use was off the scale, as was impact on our set of activities which took an age to complete.

Detection rates were pretty disappointing, as ever, in the RAP sets, but pretty decent in the Response test, illustrating the impact of cloud detection systems. In both cases a slight downward trend was discernible going into the more recent sets. The WildList was well handled, but a small number of items in the clean set were alerted on, all under some vague heuristic or generic flag. These included items from prominent business developers such as HP and Sage, and were enough to deny Commtouch a VB100 award. Luck has been against the vendor of late, with two passes and now three fails in the last six tests; five of each in the last two years. Stability, on the other hand, was excellent, with no issues at all earning the product this month’s first ‘solid’ rating.

ItW Std: 100.00%

ItW Std (o/a): 100.00%

ItW Extd: 100.00%

ItW Extd (o/a): 100.00%

False positives: 5

Stability: Solid

Main version: 1.005.00006

Update versions: 22852, 23166, 23220, 23251

Trouble loomed immediately for Coranti, with its multi-engine approach inevitably imposing a higher risk of false alarms and the ‘Frisk’ engine one of those included here, alongside Bitdefender and Lavasoft/GFI. The main installer is fairly small, at only 53MB, with a simple and speedy installation process, online updates pulling down just under 240MB of data on each install but not taking too long over it – rarely more than five minutes in total.

The product interface is clear and simple, with a good range of options presented in a fairly easy-to-access, if slightly wordy manner. It operated relatively smoothly, with no issues to report, and demonstrated good scanning speeds with impressive improvements in the warm runs, and fairly light overheads too. Resource use was fairly high, but our set of tasks completed in decent time.

Detection was pretty impressive, with high scores everywhere, dropping off only slightly in the very latest sets. The WildList was handled well, but in the clean sets there were issues as feared – not as bad as we expected, with just a single full alert, but still enough to spoil Coranti’s chances of a VB100 award this month. With a rather sporadic pattern of entries Coranti now has two passes and one fail from three appearances in the last six tests; four passes and two fails in the last two years. Stability was not a problem though, with a ‘solid’ rating earned.

ItW Std: 100.00%

ItW Std (o/a): 100.00%

ItW Extd: 100.00%

ItW Extd (o/a): 100.00%

False positives: 1

Stability: Solid

Main version: 2.1.91

Update versions: 15.0.147, 15.0.188, 15.0.192, 15.0.194

Another from the Preventon stable, Digital Defender’s server solution has been seen several times before in these pages. The slim 85MB installer ran well, but as predicted following our earlier encounter with a product from the Preventon clan, there were some oddities with the update process, data not righting itself and showing some distinctly worrying information at least until the next reboot. Otherwise operation was decent – scanning speeds were not great but not too bad either, overheads were a little heavy, RAM use was OK but CPU use a bit high, and impact on our set of tasks was not bad.

Detection was reasonable to start with but showed a steady downward slope. There were no false positives, but the WildList sets were handled far less reliably than we require and no VB100 award can be granted to Digital Defender this month. That puts it on three passes and a single fail in the last six tests; seven passes and two fails in the last two years. With some issues encountered during the update process, the product just about scrapes a ‘stable’ rating.

ItW Std: 98.25%

ItW Std (o/a): 98.07%

ItW Extd: 95.89%

ItW Extd (o/a): 95.84%

False positives: 0

Stability: Stable

Main version: 6.6.0.4

Update versions: 6631272, 7734128, 7827397, 7854621, 10863274

With little changed for now but some major enhancements on the way, Emsisoft's solution will soon boast the Bitdefender engine in place of that of its current partner, Ikarus. The product installer is a fair size at 151MB, including all the latest data, but initial installation is fairly speedy. Updates are on the slow side though, taking up to five minutes in some cases, and in one instance the update process failed, claiming a lack of Internet connection (although a connection was in place). Re-running the task once the install was complete took some 10 minutes and ended with an error message warning of a ‘major problem’. This led to the product interface refusing to open even after a reboot. In the end the decision was taken to retry the install from scratch on a fresh machine, and this time all went perfectly smoothly.

The interface is a little slicker than in past tests but much the same in layout, providing a decent range of controls which are mostly fairly accessible. Scanning speeds were not bad and overheads were reasonable too, with very light RAM use. CPU use was also very low, and our set of tasks took just a little longer than the average for this month.

In the detection tests things were much less smooth however, with many jobs ending with a catastrophic ‘a major problem has occurred’ message. In some cases this left the product inoperable and protection disabled; in others it was possible to continue working. When things were snarled up a reboot generally cured things, but on at least two occasions even after multiple reboots and attempts to restart various services we were unable to access the product interface, run any scan tasks or find any evidence of active protection – the test system had to be wiped and testing restarted on clean hardware. This made for quite some work. In most cases the problem seemed to be unrelated to the files being scanned, as the same scan could successfully be run to completion on later attempts. However, in some instances, particularly in the RAP sets, it was clear that particular samples were causing issues, with jobs repeatedly stopping in the same places. The developers suggested the issue might be down to the product’s quarantine filling up too quickly, but as we routinely disable quarantining and cleaning before running large scans this seemed unlikely to be a factor here.

Efforts were made to cover as much of the test sets as possible, but inevitably some items went unscanned where stability was too shaky to cover much of a folder. This will have impacted detection rates, which were excellent in some sets but a little less solid elsewhere, mainly as a result of the problems encountered. The WildList was not hit by the issues and was covered well, but in the clean sets a single file, part of a business package from HP, was alerted on as a dropper trojan, and Emsisoft does not make the grade for a VB100 award this month. Its ongoing streak of bad luck with false positives puts Emsisoft on one pass and four fails in the last six tests; two passes and eight fails in the last two years. With a large number of problems encountered – most of them described by the product itself as ‘major’ – stability is rated at the lowest level, ‘flaky’.

ItW Std: 100.00%

ItW Std (o/a): 100.00%

ItW Extd: 100.00%

ItW Extd (o/a): 100.00%

False positives: 1

Stability: Flaky

Main version: 11.0.1139.1250

Update versions: NA

Another client of the ever-popular Bitdefender, eScan has been using the engine rather longer and has done pretty well out of the deal. The current version came as a hefty 192MB installer, which took some time and quite a few clicks to get through. The install process is lively, with lots of progress bars, message windows and so on to keep the operator amused. After a couple of minutes of activity it launches straight into a scan, and then is all done. Updates appear not to start automatically (or at least not soon enough for impatient types), so were kicked off manually, taking a further three to four minutes to complete the set-up process.

The interface is glossy and colourful, with some funky buttons which swell up when hovered over like Mac menus. The developers have obviously spent a lot of time on this, but their time would perhaps have been better spent looking more closely at the product’s operation and messaging: initial tests showed no sign of active protection, and this was only remedied by a reboot, which was not requested at any time by the product itself, yet appears to be essential. On restart, on several occasions we encountered a message warning that the monitoring tray component had experienced a problem – but this didn’t seem to affect the protection, which was now fully operational.

Testing advanced more rapidly from here on, with speed and performance measures hitting few snags. Scanning speeds were sluggish and overheads a little high, but resource use was low and there was a low impact on our set of tasks. Detection tests were more difficult, with the interface locking up several times towards the end of larger jobs, and on one occasion, when the product was installed with no Internet connection for the RAP tests, it appeared to detect nothing at all – scans were run over several sets including the EICAR test file, and claimed to have completed successfully with nothing to report. Rebooting and tweaking the settings, and even running online updates appeared to produce no improvement, and in the end we were forced to start again from scratch with a fresh install – fortunately this time everything went smoothly and the previous oddities could not be reproduced.

Given the underlying engine it was little surprise that detection rates were excellent, with just a slight drop into the later sets, and the core sets were handled well, comfortably making the grade for VB100 certification. That puts eScan on six passes in the last six tests; 11 passes and a single fail in the last two years. However, multiple issues with the product earn it a less than pleasing stability rating of ‘buggy’ this month.

ItW Std: 100.00%

ItW Std (o/a): 100.00%

ItW Extd: 100.00%

ItW Extd (o/a): 100.00%

False positives: 0

Stability: Buggy

Main version: 5.0.2126.0

Update versions: 7389, 7477, 7389, 7497

Another long-term high performer, ESET’s latest corporate solution picks up the ‘Endpoint’ tag which seems to be all the rage these days. It seems fairly similar to the consumer products we are more used to seeing in our tests though, with the svelte 64MB installer starting off by warning that we were installing an endpoint solution on a server, which might be better protected by a dedicated server product. The rest of the set-up was smooth and quick, with updates completed in a flash and the whole process taking under a minute on each install.

The interface is crisp, clean and pleasant, with a splendidly comprehensive set of configuration options, and ran smoothly with no issues throughout testing. Speeds were excellent on demand, particularly in the warm runs, and overheads extremely light; resource use was well below average and our set of activities ran through very quickly indeed.

Detection rates were solid throughout, with no issues in the core sets easily earning ESET yet another VB100 award – the vendor has entered and passed every VB comparative since June 2003. With a very impressive performance this month, a ‘solid’ stability rating is well deserved.

ItW Std: 100.00%

ItW Std (o/a): 100.00%

ItW Extd: 100.00%

ItW Extd (o/a): 100.00%

False positives: 0

Stability: Solid

Main version: 4.1.3.149

Update versions: 5.0.26/16.37, 16.386, 16.403, 16.421

Fortinet’s scores have been climbing steadily of late, from a fairly low spot a year ago to some very impressive heights. The current version was provided as usual as a tiny 10MB installer, with an offline update bundle of 137MB. The set-up process offers the choice of free or premium products, with the latter option selected for testing this month. The process completes after the usual steps in under half a minute. Updates are fast and after some runs request a reboot, the entire install process never taking more than a minute. The design is simple and clear, and the interface proved reliable and responsive throughout testing.

Scanning speeds were pretty good, with overheads a little high to start with but dropping sharply after initial settling in. RAM use was low, CPU use perhaps a little higher than average, but our set of tasks ran through in very good time.

Detection scores were once again excellent across the board, taking a commanding position on the RAP chart, and with no issues in the certification sets a VB100 award is easily earned. Having missed only our annual Linux tests, Fortinet now has five passes from five entries in the last six tests; nine passes and a single fail in the last two years. This month’s splendid performance is capped off with a ‘solid’ stability rating as we encountered no issues whatsoever.

ItW Std: 100.00%

ItW Std (o/a): 100.00%

ItW Extd: 100.00%

ItW Extd (o/a): 100.00%

False positives: 0

Stability: Solid

Main version: 11.5.2.133

Update versions: AVA 22.5792/AVL 22.1117, AVL 22.1181, AVL 22.1190, AVA 22.6167/AVL 22.1197

As in several previous server tests, the G Data product was provided as a combination of administration system and protection client agent. This made the submission rather large, with the admin kit weighing in at a hefty 659MB, and the client was provided as a separate 215MB package. Installation of the management side of things was fairly simple but did take some time, much of which was devoted to the .NET framework. Once the control system was in place deployment to clients was generally smooth and simple, although the discovery of connected systems seemed more complete on some runs than others. The whole process took between six and eight minutes, with a reboot partway through. Updating was a little harder to measure, as it ran as a background task with little indication of exactly when everything was safely installed, but it seemed to add another seven or eight minutes at least to the total set-up time.

The interface is reasonably easy to navigate but has a tendency to dawdle when refreshing screens and seemed a little shaky at times, as well being rather short on feedback and information. On one install, an on-access run showed some odd results, with protection clearly shutting off for a spell in the middle of the run, possibly while an update was applied. In the on-demand work, a couple of scans stopped short with large chunks of the area we had asked to be checked ignored; in both these cases, re-running the same job worked fine, so the issue was clearly simply one of wobbliness rather than a recurrent bug. Nevertheless, unreliability is not to be expected in a server-grade solution.

Scanning speeds were more impressive, especially in the warm runs, and on-access lag times were also good after more thorough initial checks, which did take some time. Resource use measures appear a little high, but this figure is difficult to compare with other products as the administration suite was installed on the same system as the client and doubtless hogged the bulk of the memory and CPU time taken up. Our set of tasks ran fairly slowly too.

Detection rates were excellent as ever, with RAP scores hard to fault even in the proactive week and Response scores splendid across the board. The core sets were well handled and G Data earns a VB100 award, putting it on five passes from five attempts in the last six tests; nine passes and one fail in the last two years. Stability was a little questionable this month, with a selection of mostly minor issues mounting up to a score just nudging into ‘fair’ territory.

ItW Std: 100.00%

ItW Std (o/a): 100.00%

ItW Extd: 100.00%

ItW Extd (o/a): 100.00%

False positives: 0

Stability: Fair

Main version: 6.0.0.0

Update versions: 2012-08-15.00(7512929)

Yet another product based on the Bitdefender engine, Hauri has underperformed in the past thanks to some issues with updating. Hoping for better things this time around, the 182MB install package ran through the usual stages, taking minimal time despite the pre-install scan. On completion it presented the option to run an update. However, this and repeated subsequent attempts to launch updates proved fairly profitless – while a few did at least open a progress dialog with some sign of action, on most occasions very little seemed to happen at all. On all attempts the version information displayed in the main interface failed to change, although on some occasions it was at least marked with a red ‘x’ to indicate that it was in need of updating; exactly how this is supposed to be achieved remains something of a mystery.

The interface itself is fairly straightforward, providing a decent if not exhaustive set of controls, and it generally seemed to respond well, at least when not being asked to perform updates. At one point a scan of clean items in one of our false positive tests did completely freeze up, but we were able to kill the job and restart it fairly easily once we realized there was a problem. Eventually completing our work, we saw some rather sluggish scanning speeds, reasonable overheads, RAM use slightly above average but CPU use well below, and a fairly heavy hit on our set of activities.

Detection was excellent in the RAP sets but fairly mediocre in the Response sets, confirming our suspicion that updating was once again completely ineffective. The core sets were not affected by this however, and a VB100 award is just about earned, putting Hauri on three passes from three attempts in the last six tests; three passes and three fails in the last two years. With a number of issues this month, some of them fairly serious, stability could be rated no better than ‘buggy’.

ItW Std: 100.00%

ItW Std (o/a): 100.00%

ItW Extd: 100.00%

ItW Extd (o/a): 100.00%

False positives: 0

Stability: Buggy

Main version: 2.2.12

Update versions: 1.1.122/82051, 82267, 82295, 82312

Ikarus recently tweaked the name of its product, but little else appears to have changed. As usual, the submission was provided as a full ISO image for an installation CD, doubtless including much else besides the basic product but still fairly compact at just over 200MB. Updates are provided separately for offline use, weighing in at 89MB. Set-up has a fair number of steps and much of the fairly lengthy time it takes is down to the installation of the .NET framework – those who already have this in place will of course enjoy a faster set-up time. On completion of the job a message reported errors in the network which prevented updating, but Internet access seemed to be fine and re-running the update task once the install was complete proved problem-free – it did take some time however, with the whole install process averaging more than ten minutes.

The product interface, in the .NET style, is sparse and simple. It is a little slow to respond at times but reasonably stable; it does become very laggy when under heavy pressure though. Configuration is a little more than minimal, but fairly easy to navigate, and logging is thorough and readable. Scanning speeds were not great – better in the sets of media and miscellaneous files than elsewhere – while lag times were distinctly heavy. Resource use was also high – particularly CPU use, which was off the chart – but our set of tasks didn’t take too long to get through.

Detection rates were superb, dropping off only very slightly in the last few days of the response tests and remaining above 90% even in the proactive part of the RAP sets – a remarkable achievement this month. As has so often been the case in the past though, this stellar detection is counterbalanced by a tendency to false alarm, and a single item in the clean sets, again from HP, was flagged as a trojan. This was enough to deny Ikarus a VB100 award despite good coverage of the WildList sets. Ikarus’s luck remains highly varied, with two passes and three fails in the last six tests; three passes and six fails in the last two years. Stability this month was decent, with just some minor wobbliness in the GUI under heavy pressure, earning the product a ‘stable’ rating.

ItW Std: 100.00%

ItW Std (o/a): 100.00%

ItW Extd: 100.00%

ItW Extd (o/a): 100.00%

False positives: 1

Stability: Stable

Main version: 8.1.0.831

Update versions: 8.1.0.831 (a)

Kaspersky’s business solution was provided this month as a sizeable 319MB install package, with updates, in the form of a mirror of the company’s update servers, measuring close to 300MB but including much more than required for this product alone. Set-up followed standard lines and seemed to run fairly speedily, although at one point it did worry us slightly by vanishing for some time, with nothing to indicate any kind of activity for close to 30 seconds. After this the final stages zipped through very quickly, but the install was on the slow side. Updates were also slow, taking more than six minutes in every run, and in each case a reboot was requested rather quietly, so that only the more observant of users would be likely to notice it.

The interface is glossy and modern, with a quirky take on buttons and links in places, but it is reasonably simple to find one’s way around and provides a superb degree of fine-tuning. It also seemed very stable under pressure. On one occasion an on-access job produced odd results with a number of misses, but re-running the same task moments later showed perfect coverage and the issue could not be reproduced. Scanning speeds were a little slow initially and blindingly fast in the warm runs, with very light overheads. Resource use was a little high, but our set of tasks got through in good time.

Detection was solid, with good scores across the board, and with no problems to report in the certification sets a VB100 award is comfortably earned. That leaves Kaspersky on five passes and a single fail in the last six tests; eight passes and three fails in the last two years. With only a single, non-reproducible issue observed, a ‘stable’ rating is earned.

ItW Std: 100.00%

ItW Std (o/a): 100.00%

ItW Extd: 100.00%

ItW Extd (o/a): 100.00%

False positives: 0

Stability: Stable

Main version: 2.2.903.0

Update versions: 1.1.8601.0/1.131.1805.0, 1.1.8704.0/1.135.1150.0, 1.135.1404.0, 1.135.1568.0

Microsoft’s business product is another that has undergone a rebranding of late – once again with little difference in the overall user experience. The package provided was an 83MB zip archive, containing the compact 17MB main installer and an offline update bundle. Set-up was fast and simple, with minimal interaction required; updates averaged around a minute, taking the total install time to not much more than a minute and a half.

The interface is simple and mostly straightforward, although a little wordy in places. It was generally responsive, although during one of the RAP jobs we did fear it had frozen up entirely – we later found it was simply taking its time. Later on, though, we hit a more serious problem when a scan of one of the Response sets which had been set to run overnight was found in the morning to have got nowhere, a message simply stating that the scan could not be run. On rebooting the system prior to another attempt, we were shown a message indicating that the main service had stopped running unexpectedly. The reboot seemed to clear things up adequately though.

Speeds were no more than OK, but overheads were pleasingly light, with low RAM use, CPU use a little below average for the month, and our set of tasks barely affected by the protection. Detection scores were not bad, dropping rather sharply into the proactive part of the RAP sets, but remaining impressively stable through the Response sets. The core sets presented no difficulties, and a VB100 award is well deserved; our test history for Microsoft’s business line shows rather sporadic entries but solid performances, with two passes from two attempts in the last six tests; five from five in the last two years. A few oddities were noted this month, but a ‘stable’ rating is still merited.

ItW Std: 100.00%

ItW Std (o/a): 100.00%

ItW Extd: 100.00%

ItW Extd (o/a): 100.00%

False positives: 0

Stability: Stable

Main version: 9.00

Update versions: 6.08.06

Norman’s scores have improved significantly over the last six months, and we came into this month’s test hoping to see more of the same. The package submitted for testing was a 253MB executable, and as usual it seemed to run through its business impressively quickly. Once all appeared to be done with however, the product interface and system tray icon were unavailable or incomplete for some time – there was clearly much more going on in the background. Judging the actual time taken for installation was thus rather tricky, but on most runs it seemed to be at least three minutes before everything was fully functional. Several more minutes would then be required for initial updates.

The interface is displayed by a browser, and under the default security settings in Windows Server 2003, a number of warnings have to be dealt with before local content can be displayed, including acceptance of exceptions to security rules. With this done, a number of scripting errors were alerted on – a total of seven each time we opened the interface – and once up it took some time to fully display, leaving several areas worryingly blank to start with. The layout is reasonable but can be tricky to handle as it lacks the flexibility of a proper interface. The lack of warnings when navigating away from a page without having saved any changes made to the configuration have confused things for us in the past. This month we noted an oddity in the logging system, with on-access detections appearing fleetingly when the real-time log is opened, but vanishing before anything can be read properly.

Otherwise things moved along nicely, with long, slow scans of our speed sets, particularly the archive set, but lag times not looking bad at all thanks to some smart improvements in the warm runs. RAM use was low, CPU use a little high, and our set of tasks ran through quickly. Detection rates were solid in the earlier part of the RAP sets, dropping off fairly sharply into the proactive week, and also started well in the Response sets, again tailing away quite severely into the more recent sets. The core sets were dealt with well though, and a VB100 award is earned, keeping Norman on a very respectable six passes in the last six tests; 11 passes and a single fail in the last two years. Stability was rated ‘fair’ thanks to a number of mostly minor problems with the interface and logging.

ItW Std: 100.00%

ItW Std (o/a): 100.00%

ItW Extd: 100.00%

ItW Extd (o/a): 100.00%

False positives: 0

Stability: Fair

Main version: 4.3.91

Update versions: 15.0.147, 15.0.188, 15.0.192, 15.0.194

Progenitor of the usual cluster of participants, a few of Preventon’s offspring have already been discussed this month, and attentive readers will be able to predict how Preventon itself fared. Installation was highlighted by issues with the updating display, and also on one occasion by the product switching from English into German after the initial update. The interface was mostly reliable after install, and speeds were not too bad. Overheads were rather high, resource use a little high, particularly for CPU cycle use, and impact on our set of tasks a tad high too.

Detection was rather mediocre, with fairly steep downward slopes through both the RAP and Response sets, and although a clean sheet was managed in the false positive tests the WildList was not well handled, meaning no VB100 award for Preventon this month. The vendor’s test history is mostly decent, with three passes and now a single fail in the last six tests; seven passes and two fails from nine entries in the last two years. How things go in the future will depend greatly on how well the new engine developers can handle the transition. Stability was also a little shaken this month by some oddities that occurred during install and update, with a score only just within the ‘stable’ range.

ItW Std: 98.25%

ItW Std (o/a): 98.07%

ItW Extd: 95.89%

ItW Extd (o/a): 95.84%

False positives: 0

Stability: Stable

Main version: 3.0.0.3051

Update versions: NA

Qihoo’s 360 is another product that uses the Bitdefender engine. We noted recently that it also offers access to the Avira engine, but as this is not enabled by default we have never looked at its impact in our tests. In the past, the product’s performance has generally been decent, although we have encountered issues with updating and with the distinctly unusual approach to what can only very loosely be described as real-time protection. The current version came as a 134MB executable, which ran through rapidly with minimal interaction. Installation seemed to be complete in well under a minute, but once again updates were less than reliable, with several attempts returning a rather unhelpful ‘failed’ message, while others claimed success but made no changes to the reported version information. On most runs we did eventually get things into what appeared to be an updated state, with more than a little effort in most cases.

The interface is fairly simple and easy to operate, with a reasonable level of control, and in general it seemed to run reliably. As noted repeatedly in the past, on-read protection is more than a little quirky, apparently observing that files have been accessed and adding them to a queue to be checked; some time later (several hours if a heavy barrage of detections is forced), a message appears informing the user that a threat has been detected and access to it has been blocked – but this often seems to be rather a case of shutting the stable door long after the horse has made off with your sensitive data. Fortunately, as far as we can tell, on-execution checks are a little more rigorous.

Scanning speeds were around average, with overheads heavier than might be expected given the minimal intrusion. Resource use was barely noticeable though, and our set of tasks apparently bypassed the notice of the product entirely. Detection rates were excellent throughout, confirming that our hard work with the updates had paid off, and the core sets were well dealt with, earning Qihoo a VB100 award. That puts the vendor on two passes and two fails from four entries in the last six tests; five passes and two fails in the last two years. Ignoring the real-time oddness as a feature rather than a bug, there were a few little wobbles mainly in the updater component, and a ‘stable’ rating is earned.

ItW Std: 100.00%

ItW Std (o/a): 100.00%

ItW Extd: 100.00%

ItW Extd (o/a): 100.00%

False positives: 0

Stability: Stable

Main version: 13.00 (6.0.0.4)

Update versions: NA

One of our long-time regulars, Quick Heal has hit a bit of an unlucky patch of late. The vendor’s latest version for servers was provided as a 257MB installer, which started with some initial preparation and a scan of system memory, before following a more standard path to complete the set-up in under a minute. Updates downloaded rapidly but took some time to complete installation, making for a total install time of around four minutes.

The interface is glossy on the surface but crisp and business-like underneath, with a good level of control offered. One thing lacking which we would like to see is an option to export on-access logs to plain text. Things were mostly easy to find and seemed to run solidly under pressure.

Scanning speeds were good and overheads light, with resource use well below average and an impressive time taken to complete our set of tasks. Detection rates were less than stellar, dropping steadily in the RAP sets and rather unpredictable in the Response sets. The core sets were handled properly though, with no issues to report, and a VB100 award is earned. No stability issues were observed, and Quick Heal joins the elite ranks of those rated ‘solid’ this month.

ItW Std: 100.00%

ItW Std (o/a): 100.00%

ItW Extd: 100.00%

ItW Extd (o/a): 100.00%

False positives: 0

Stability: Solid

Main version: 10.0

Update versions: 10.0.7/3.34.0/4.80G, 10.0.8/3.35.1/4.81G

Sophos recently suffered one of those periodic major false positive disasters which make news headlines even outside of the technical press – and to which all big security vendors seem prone from time to time. Luckily for Sophos, the incident occurred just outside of this month’s VB100 testing cycle. The company provided its submission as a 98MB installer with updates of only 7MB; set-up dished out no surprises and completed in around two minutes, with updates adding another minute or so on average.

The product GUI is clear and simple to operate, with decent main controls and a huge amount of fine-tuning provided behind a warning not to meddle if you don’t know what you’re doing. Operation was steady and reliable, with no issues noted.

Scanning speeds were pretty good in most areas, apart from archives which are analysed in some depth by default. On-access lag times were fairly light, and in this case extra light on archives, which are ignored by default in this mode, as are files with non-threatening extensions. With settings turned up things do get a little slower, but not significantly so. Resource use was around average for the month, and the time taken to get through our set of tasks was quite decent.

Detection was solid in the Response sets, although it tailed off noticeably in the most recent few days, and the RAP scores showed a similar pattern of starting high but dropping quite a bit into the proactive week. With the vendor’s false positive incident hitting just days after testing had finished, the core sets presented no problems, and a VB100 award is earned. That puts Sophos on four passes and two fails in the last six tests; its longer-term test history is much better with ten passes and two fails. With no stability issues observed, Sophos earns a ‘solid’ rating this month.

ItW Std: 100.00%

ItW Std (o/a): 100.00%

ItW Extd: 100.00%

ItW Extd (o/a): 100.00%

False positives: 0

Stability: Solid

Main version: 7.1.258

Update versions: NA

Using the Preventon SDK to the engine now referred to as Agnitum’s (formerly VirusBuster), SPAMfighter has made a few more changes to the look and feel of its solution than most related offerings. The 85MB installer runs quickly and simply, although somewhat unusually it asks for a user email address. The GUI is fairly clear with a decent basic level of controls.

Scanning speeds were on the slow side, with file access lags fairly heavy. RAM use was low but CPU use high, while our set of activities didn’t take too long to get through. Detection was not great, with the expected fairly sizeable batch of misses in the WildList denying SPAMfighter certification this month. As in most tests, we observed some serious instability in the interface under any kind of pressure – detecting more than a few dozen samples in a row sends it into something of a tizzy, flickering, juddering and eventually freezing completely. As this prevents access to some other windows, including the one requesting reasons for rebooting presented in this server platform, rebooting to cure the issues was often fairly tricky – but always effective. Hopefully such issues would not affect too many real-world users, but on server platforms this kind of scenario is quite possible.

The product’s test history is decent, with three passes and now a single fail in the last six tests; seven passes and two fails in the last two years. The issues encountered were fairly inconvenient, but only occurred under heavy pressure, and the stability score falls into the ‘fair’ range.

ItW Std: 98.25%

ItW Std (o/a): 98.07%

ItW Extd: 95.89%

ItW Extd (o/a): 95.84%

False positives: 0

Stability: Fair

Main version: 6.6.2284.201

Update versions: NA

Tencent returns for its third consecutive test, giving the lab team a change from the everyday with its Chinese-only interface. Installation from the 117MB package submitted was something of a mystery tour, with several dialogs to click through without much idea of what they were asking. The whole business was completed in under a minute though, with no need to reboot (as far as we could tell). Updates mostly ran to around two minutes for the initial run, although on one occasion they froze at around 2% complete, sitting there for over an hour before we noticed nothing was happening. Rebooting and re-running the tasks was more successful however, with no repeat of the issue.

Scanning speeds were rather slow, and the overhead measures are not comparable with others as on-read scanning appears to be unavailable. Resource use and impact on our set of tasks was low, but again the lack of the full real-time protection provided by most other products affects this measure greatly. Detection rates from the Avira engine underlying the anti-malware component of what appears to be a multi-part suite were good, but perhaps not quite as excellent as we would expect, with solid RAP scores but Response scores some way below those scored by Avira’s own solution – hinting perhaps that updates either lag a little behind the times or were not fully successful in every run.

Nevertheless, the core sets were well covered with no misses in the WildList and no false alarms in the clean sets, and Tencent earns its third VB100 award on its third attempt. Stability was decent, with just a single issue occurring during one of the updates, thus earning a ‘stable’ rating.

ItW Std: 100.00%

ItW Std (o/a): 100.00%

ItW Extd: 100.00%

ItW Extd (o/a): 100.00%

False positives: 0

Stability: Stable

Main version: 12.0.0.833

Update versions: 12.0.0.832/1.6.0.1884/5935.0.0.0, 5978.0.0.0, 5981.0.0.0, 5987.0.0.0

The business offering from Total Defense was installed as usual from a full DVD ISO image measuring over 3GB, but this of course includes far more than the client solution tested here. Updating could apparently only be provided online on the deadline day, despite options in the product interface clearly offering the choice to update from a local file. Set-up is slowed considerably by the need to have the .NET framework in place, which takes around four to five minutes to set up if not already present, and also requires a swathe of personal data to be filled in. However, once this was out of the way the set-up process was fairly speedy, with updates also fast, taking under a minute on each run. Our overall install time of close to ten minutes includes the .NET stage – without this it would more or less be halved. On completion of the installation a reboot is required, and another was requested after most updates. On one occasion, after this second update the interface could not be accessed, with no response from the system tray icon either, for well over five minutes. A third reboot was blocked by the ‘catm.exe’ process, which had to be forcibly closed, but after this restart things went smoothly once again.

The product interface is simple and clear, with reasonable if not comprehensive controls, and it operated well under pressure for the most part. Scanning speeds were very zippy even at first attempt, and even faster in the warm runs, with overheads pretty light, at least until settings were turned up high. RAM use was fairly high, but CPU use was below average and our set of tasks tripped through in excellent time.

Detection was pretty disastrous, dropping below 50% in some of the Response sets and even one of the reactive weeks of the RAP sets. Fortunately this poor showing did not extend to the WildList, which was well covered, and with no false alarms either Total Defense earns a VB100 award. With some minor interface problems observed, the product’s stability score falls just inside the ‘stable’ range.

ItW Std: 100.00%

ItW Std (o/a): 100.00%

ItW Extd: 100.00%

ItW Extd (o/a): 100.00%

False positives: 0

Stability: Stable

Main version: 13.0.2.5069

Update versions: 13.0.4.5077

In the last test we noted that TrustPort’s offering seemed to have been reduced to a single engine, but things were back to normal this time, with both AVG and Bitdefender on board. The installer submitted weighed in at 216MB, including all required data for both engines, and ran through fairly speedily, including what appeared to be an update in a total runtime of under a minute. A reboot was required at the end, after which the system took some time to wake up, at which point it was clear that further updating was needed. This varied in time from two to ten minutes, with total data of around 80MB to download.

The interface remains a little unusual and can be tricky to find one’s way around, but mostly makes sense after a little practice and exploration; configuration is limited in the main areas, but fairly in-depth once the advanced control panel is discovered.

Scanning speeds were not very exciting, with overheads a little high, but use of resources was impressively low for a dual-engine solution, and our set of tasks got through in good time. Detection was uniformly excellent, with only the very latest day of the Response sets and the proactive part of the RAP test falling below 99%. The core sets were effortlessly brushed aside, and a VB100 award is easily achieved by TrustPort. Recovering from a recent rough patch, its test history now shows three passes and two fails in the last six tests; seven passes and two fails in the last two years. With the usual odd window behaviour observed several times during the test but no more serious issues, a ‘stable’ rating is earned.

ItW Std: 100.00%

ItW Std (o/a): 100.00%

ItW Extd: 100.00%

ItW Extd (o/a): 100.00%

False positives: 0

Stability: Stable

Main version: 3.1.46

Update versions: 15.0.147, 15.0.188, 15.0.192, 15.0.194

Another member of the Preventon family that has been plagued by problems this month, UtilTool’s chances of achieving a VB100 seemed slim. The 83MB installer ran through in good time and updates were fairly speedy. Once again, on completion the updates seemed to have made no difference – a reboot was not requested but seemed to be required to get the version details to catch up with reality. The interface is a newer version than the others tested this month, and has a few bugs of its own. For example, reboots did not work on first attempt, doing no more than shutting down the product interface – a second reboot was needed to actually restart the machine.

The interface remains fairly accessible though, providing a reasonable degree of control. Scanning speeds were not great but not too bad either, and overheads pretty high. RAM use was low, CPU use a little above average, but our set of activities was not badly affected at all. Detection was mediocre, with fairly dismal scores in the RAP sets, earlier parts of the Response tests not too bad but soon falling away, and as expected, the WildList was not well dealt with, meaning there is no VB100 award for UtilTool this month. Our test history for the vendor stretches back just over a year, with two passes and two fails from four entries in the last six tests, one more pass to add to that from just over a year ago. Stability this month was hit by a few fairly minor problems, putting it in the ‘fair’ range.

ItW Std: 98.25%

ItW Std (o/a): 98.07%

ItW Extd: 95.89%

ItW Extd (o/a): 95.84%

False positives: 0

Stability: Fair

Main version: 3.1.46

Update versions: 15.0.147, 15.0.188, 15.0.192, 15.0.194

The only new name on this month’s list, Vexx Guard promised to liven things up but it quickly emerged as being yet another part of the ever-expanding Preventon clan, choosing a rather unlucky month to make its first appearance. The installer was larger than expected at 87MB, but that’s where the surprises stopped – set-up was fast and simple, updates hit by the issue with the version information not updating properly, while reboots were again odd thanks to the implementation of the newer-style interface. Speeds were slowish, overheads heavy, RAM use OK but CPU use high, and the hit on our set of tasks not too bad.

Detection scores were disappointing, with low RAP scores, Response scores starting off passable but ending up poor, and the WildList showing a number of misses. There was thus no VB100 award for Vexx Guard on its first attempt. Stability was rated as ‘fair’ due to a number of small irritations.

ItW Std: 98.25%

ItW Std (o/a): 98.07%

ItW Extd: 95.89%

ItW Extd (o/a): 95.84%

False positives: 0

Stability: Fair