VB100 on Windows 2003 Server x64

Agnitum’s Outpost suite is essentially a desktop product, but should provide ample protection for a server platform. The installation process includes options for the numerous components, including web and spam filtering and behavioural monitoring as well as the famed client firewall. Installation took quite some time thanks to various network scans, attempts to update (foiled of course by the isolated nature of our lab), and finished with a recommendation to reboot to ensure full efficacy.

Once ready to use, the interface impressed the lab team with its simple, uncluttered layout, but it seemed somewhat lacking in the fine-tuning options likely to be required by most server admins to ensure best fit with their specific requirements. Scanning speeds were no more than fairly good, and on-access lags were somewhat above average, but a caching system should provide better speeds once the product has familiarized itself fully with its environment (something which we hope to be able to test more accurately in the near future). Our tests didn’t cover the behavioural and other aspects of protection provided, but the detection rates recorded represent a fair measure of the product’s ability to protect fileshares from infiltration. These rates proved fairly decent in general, with a steady decline in detection of the RAP sets as time to product freezing drew closer, as expected. In the polymorphic set, a fair number of samples of the latest Virut variant were missed, suggesting that some more work may be needed to make the grade next time around, but with no issues in the current WildList set, no false positives and no other problems, Agnitum starts this month’s comparative off well by winning a VB100 award.

AhnLab’s dedicated server product proved much simpler to install, with the option of a pre-install scan to ensure the system is clean before getting under way. The install offers an optional ‘anti-hacking’ feature alongside the standard choices, and is up and running with no reboot required. The interface, closely mirroring the desktop product, is clean and simple with most of the basics easy to find, but once again, the more in-depth configuration which seems appropriate for server products was absent.

Speeds were somewhat slow on demand, but not bad at all on access. Detection rates also seemed fairly impressive, with again a pretty steady decline across the RAP sets as expected, and mediocre coverage of the latest Virut variant.

In the WildList set, things seemed fine on demand but less so on access, where a small selection of samples were not blocked immediately. Probing this issue, it seemed that the product continues the somewhat outmoded path of separating ‘virus’ and ‘spyware’ detection, to the extent of requiring separate filesystem scans to check for each type of malware. Both types of detection are active on access, and some WildList samples were being detected by the anti-spyware portion of the product. Despite appearing to be configured to deny access on detection, the anti-spyware module seemed not to do this as well as the anti-virus module, which was blocked from scanning the files as they had already been alerted on by the anti-spyware component. Although this seems like a rather nasty situation, logging of detection is all that the VB100 rules demand and thus the product is credited with full coverage of the WildList. In the clean sets, a false positive emerged on a fairly obscure browser product, relieving us of the pressure of making a tricky call on the WildList behaviour, and AhnLab does not quite make the grade for a VB100 award.

Alwil’s server version provides a speedy and straightforward installation process, at the end of which a reboot is not forced, but those choosing not to do so are warned that ‘system failure’ may result. The interface closely resembles the desktop edition, with the advanced version providing a wide range of controls and options but proving rather cluttered and difficult to navigate. While a simple version is also available, the default settings provided are fairly basic and likely to be inadequate for most admins. Scanning speeds were fairly mid-range, and detection rates a fraction below the outstanding levels expected – but were nevertheless impressive, with no problems having been encountered in detecting the large numbers of new W32/Virut samples in the polymorphic set. The WildList was likewise covered cleanly, and with the clean set presenting no serious problems either, a VB100 is awarded to Alwil.

AVG’s product again provides a slick and fast install, with no reboot necessary, and a ‘first run wizard’ provides configuration for things like updating, scheduled scans, trusted networks and so on. The interface seems identical to the standard desktop version – rather busy, with icons for numerous components and modules leading to more advanced configuration in tree format, which can also become a little tricky to navigate in its rather small default window. Some options that would be of relevance to server admins, such as processing of archive files on access, seemed to be absent, but could merely have been overlooked in the confusion.

Speeds were in the medium range, and detection rates continued their recent upward climb, with once again no problems with either the WildList or the new Virut strain expected to join it next time around. The clean sets were ably handled too, and AVG’s superb performance earns a VB100 award.

Avira also impressed with the speed of its installation process, despite the need to set up some Visual C++ components on the system, and again no reboot was required. This is a proper server edition, with an MMC-based console to control configuration – which appeared to be provided in considerable depth. The neatly laid out tree structure proved simple to navigate and easy to use, and overall the design was declared excellent by the lab team, although the default settings on access were once again fairly basic. Running through the test quickly and easily, we noted that on-demand speeds, normally extremely fast, were not as far ahead of the pack as usual, although on-access overheads were as excellent as ever.

Detection rates were similarly superb across the board, with some truly remarkable figures in the RAP sets and no problems handling the expanded polymorphic sets. Sadly, however, a single false alert on a fairly minor item in the clean sets scuppered Avira’s hopes of earning a VB100 this month.

Another full server edition, BitDefender offers admins control over the number of scanning processes implemented, and during installation does some probing to estimate an optimal default level. Also included with the otherwise fairly standard install process is a request for permission to send crash information back to the developers to smooth out any wrinkles in the product’s stability, and at the end a reboot is required to finalize the install. The interface again uses the MMC system and a tree of configuration and option controls, which the team found clear and well laid out. It also provides lots of statistical information on its own performance, which many server admins may find useful, and provides a wealth of other server-oriented extras such as importing schedule settings from a file.

An initial run over the test sets found no problems on access, but the on-demand tests were held up while we tried to persuade the scanner to run. A batch of scheduled scans were set up to run over a weekend but failed to activate, and attempts to kick-start the same jobs manually also proved fruitless. Little information seemed available and it was not even clear whether scans were in fact running in the background and simply snagged somewhere, or not running at all. Reinstalling the product on a fresh system fixed all this however, with no repeat of the odd issues, and all tests were completed without further upset.

Scanning speeds were not excellent, with some rather heavy overheads on access, but detection was very good across all sets, with a gentle decline through the RAP sets but little missed elsewhere, including full coverage of the WildList and polymorphic sets. In the clean test sets, logs confused us for a while with their tendency to include password-protected files in the ‘virus’ category, and a single item, a component of the popular open-source graphics tool the Gimp, was mislabelled as a trojan. BitDefender thus also misses out on a VB100 award despite a strong showing.

CA's eTrust is a corporate-focused product and has remained unchanged for several years, although the anti-malware side of the giant company has gone through a major evolution lately and we hope to see a significant overhaul of the product in the near future. Installation was somewhat arduous, with a number of lengthy EULAs which had to be scrolled through to the end to simulate reading them, and a form requiring plenty of personal information. A reboot is required to finalize the process. The interface has never been the most popular with the lab team, but worked better than usual on a server platform, presenting fewer of the slowdowns noted on some desktop tests. Testing ran through at a rapid rate, aided by the product’s remarkable scanning speeds. On-access overheads were similarly feather-light, but completing the testing process was somewhat hampered by the product’s horribly unfriendly logging format, which required some fairly crude hacking into shape before any useful data could be extracted.

Results were much along the lines of recent experience: fairly mediocre in the trojans and RAP test sets and with some work to do in the polymorphic set too – a fair number of samples of the new strain of W32/Virut were missed. In the WildList set there were no problems however, and with no false positives either CA earns another VB100 award.

ESET’s NOD32 has an excellent history in VB100 testing, with excellence in both detection rates and speeds, but in recent years has lost some ground in the speed area. We were interested to see if the release of version 4 would have any impact on this trend, and initial impressions during installation were fairly promising. There was a brief lag during the ‘preparing to install’ stage, but otherwise it was a very fast and highly user-friendly set-up process, not needing a reboot to get full protection up and running.

Running through the speed tests first, on-demand settings were pretty thorough by default and throughput seemed fairly sluggish, although it is perhaps unfair to judge against sky-high expectations and in fact it proved to be among the faster products under test, while on-access overheads were barely noticeable. A detailed and well-designed interface appeared well stocked, but a notable omission was the ability to scan archives by default – an option some admins may find useful and one which would have enabled the full running of our speed comparisons.

Moving on to the infected sets, things went a little less smoothly. Some quirks in the operation of the on-access scanner meant having to run parts of the test by copying the sets to the test system across the network to activate detection, but this seemed reasonable in a test of fileshare protection. There were also a few occasions when the product seemed overwhelmed by the high stress it was put under, with the interface freezing up for long periods and on one occasion a reboot being needed to get things moving along.

Detection rates in the expanded polymorphic sets were impeccable, and fairly reasonable in the trojan and RAP sets, although perhaps a fraction below the excellent standards we have come to expect. This was thanks in part to a quirk which seemed to cause the on-demand scanner to ignore a fairly large number of items alerted on on-access – as these broadly fell into several clusters of near-identical files, counted as single items when calculating percentages, this impacted more heavily on the raw numbers than the percentage scores, but does seem somewhat worrying. With the WildList covered with no difficulties, however, and no false positives or other issues, ESET earns a VB100.

Fortinet’s installation process was interrupted by several rather worrying alerts from Windows warning that some components were not approved by Microsoft and may threaten the stability of the system. Ignoring strong recommendations to abort the install, it continued fairly smoothly, but spent several minutes apparently ‘optimizing performance’ before the set-up process was complete. Once up and running, a revamped interface presented a smooth and colourful outlook, much more cheery than the previous effort which, while thorough and businesslike, lacked a little charm. It also seems somewhat less cluttered than the old version, while still providing a very good level of configuration and range of fine-tuning options. The defaults, set to thorough and secure, provided a stark contrast with many of the other products looked at so far, which seemed to err on the lax rather than cautious side.

With this thoroughness in evidence in some rather slow scanning speeds, one area where the defaults seemed bizarrely lacking was in the detection capabilities. The standard settings, while capable of handling the WildList quite happily and scoring reasonably well in the other older sets, showed fairly limited coverage of the new Virut strain and miserably low scores across the trojans and RAP sets. Having diagnosed this issue in previous tests, we re-ran scans after activating some additional options. With ‘extended databases’ enabled, as well as greyware detection and heuristics, detection rates shot up to impressive levels, with a huge leap to over 80% in the trojans set and similar levels achieved across the RAP sets, dropping fairly sharply in the ‘Week+1’ set.

Admins would be best advised to enable full detection capabilities, but under the VB100 rules defaults must be used (however bizarre they may seem), and the figures reported in our tables thus do not include the additional detections. Activation of the full range seemed to have little impact on the clean sets, with a few additional files labelled as suspicious, and with the default settings not raising any issues at all here a VB100 is duly awarded.

Frisk’s F-PROT product is a pretty pared-down, bare-bones kind of affair, providing straightforward malware protection for the filesystem, with a little extra in the form of web and mail scanning. The installation process is therefore fairly simple, but seemed a little sluggish at times and needed a reboot to complete. With only basic configuration available, we relied on the defaults to see us through and got the test battery over with fairly quickly, with very good scanning speeds and minimal overheads on access.

Detection rates were a little below expectations in the RAP sets, but much better in the slightly older trojans set and splendid elsewhere, handling all the new Virut samples with aplomb. The WildList proved no problem, but in the clean set a handful of files included with some UPS management software from a major vendor were alerted on by heuristics, which was enough to count as a false alarm under our rules, thus disqualifying Frisk from a VB100 award this month.

F-Secure’s server edition seems fairly similar to its standard desktop range, although the normal installation process also includes options for centralized or local management policies. Configuration in the simple, sensible interface is available in great depth, although the scheduler seemed to lack sophistication, allowing only a single job with a single target to be specified.

Scanning speeds were, as usual, on the slow side, and on-access overheads pretty hefty, but detection was generally solid, if not quite up to the expected high standards in the RAP and trojan sets. In the polymorphic set, a single instance of the latest Virut variant was not detected, but the WildList set was handled thoroughly. In the clean set, a couple of suspicious alerts were no barrier to F-Secure achieving a VB100 award this month.

K7’s main market is in Japan, but the English version of the product seems pretty smooth and solid. The installation process had a slightly boxy feel, but ran through quickly, with a pause to gather some user information and a reboot at the end. It seemed to make startup slightly slower than expected, but once up and running provided a straightforward and responsive interface with a reasonable level of configuration available. Something that may prove problematic for server admins is the apparent inability to scan more than one level deep into archives, even in thorough on-demand scans.

Perhaps thanks in part to these fairly minimal settings, scanning speeds were through the roof, and on-access overheads very low indeed, but detection rates were medium at best, with a fairly steep week-on-week decline in the RAP sets and large swathes of the new Virut samples not covered. The WildList was handled without any difficulty, and the clean sets likewise, so K7 also meets the requirements for a VB100 award.

Kaspersky’s server edition is a quite separate beast from the company’s desktop range, with a long and complex installation process tripping through a large number of options screens. Once the process is complete many admins will also require the administration component, which rather than being an option to the main installer is in fact its own standalone module with a separate set-up process. Once everything is ready, an MMC interface provides a long and complex tree of configuration, monitoring and reporting options. This proved generally fairly simple to navigate, although there were a few moments of confusion thanks to unexpected behaviours and surprising placement of controls. A few times setting changes were rejected, and for a time some error messages appeared to say that the product had lost connection to itself. Most disturbingly, the on-demand scan settings seemed to constantly revert to defaults when changing views from one tab to another, leading to several frustrating runs through the tests as samples were trashed against our instructions. This could be a fairly serious issue in enterprise environments, where experienced admins will want to know exactly what has been found on their networks – with physical copies of files so they can be analysed and any potential breach of data privacy recorded.

Finally gathering the required data for the infected sets, detection rates proved good, but not as excellent as usual, with a sharp drop in the ‘Week+1’ RAP set contrasting sharply with the desktop product’s performance in the last comparative (see VB, April 2009, p.15). Nevertheless, scores were still commendable, the WildList was covered without difficulty, and with no false positives, a VB100 award is duly granted.

While most competitors have evolved their installers and interfaces into more shiny, colourful and cuddly versions, McAfee’s set-up remains sober, sensible and grey. The GUI is simplicity itself, but all the options an admin could desire are neatly tucked away in its easy-access corners. Not everything is same-old, same-old though: the new ‘Artemis’ in-the-cloud detection layer which has been attracting much attention in recent months, is apparently rolled into this version, as shown by a button offering additional online heuristic data. As this was disabled by default, its input did not count towards detection scores under the VB100 rules.

On-demand speeds were reasonable, on-access overheads a little heavy, with executable files particularly slow to process, and detection rates proved pretty solid, with a gradual decline across the RAP weeks to a fairly steep drop in the ‘Week+1’ set. The new Virut strain was not quite fully covered with a single item missed, but the WildList presented no problems and with the clean sets free from upset too, McAfee takes away another VB100 award.

MicroWorld’s eScan has a rather cuddly, cartoony feel to it in places but retains an air of solidity and thoroughness nevertheless. Set-up is a breeze, but once finalized the main interface did seem rather reluctant to show itself, on occasion taking as long as 20 seconds from click to full display. There were a few similarly long lags accessing logs at times too, mostly thanks to their large size after scanning large infected sets, but otherwise things were smooth and reliable. On-demand scanning speeds were very slow, but on access speeds were around the middle of the field. Detection rates were pretty good, with a very slow decline in the RAP sets and an excellent showing in the ‘Week+1’ set, as well as flawless coverage of the polymorphic sets. With no untoward issues in the WildList or clean sets, eScan comfortably wins a VB100 award.

A newcomer to the VB100 this month, Netgate’s Spy Emergency suffers from a rather improbable name with more than a hint of the rogue product about it. The product itself provides a very slick and professional installation and set-up process however, dented in seriousness only by the option to select the GUI skin colour at the end. The interface itself is also attractive and well designed, with only a minimum level of configuration, but what controls there are proved responsive. Logging proved a little less reliable, possibly thanks to inept user interaction, but nevertheless numerous pop-up alerts failed to be recorded in initial attempts. When full detection data was finally gleaned, coverage of the sets was fairly poor, with large numbers missed in the WildList set. False positives were also an issue, with handfuls of false alarms in several of the sets, most notably a selection of samples taken from clean Windows 98 installs, including notepad.exe, calc.exe and explorer.exe. Polymorphic detection was also fairly poor, with very few samples detected at all and no single variant fully covered. There is clearly a good deal of work to be done here before the product is ready for VB100 certification, but it seems like a decent start has been made and those hints of roguishness implied by the unfortunate title should soon be dispelled.

Norman’s installation is simpler than most: a bare InstallShield-style process trips through the standard steps and ends, after suggesting that a reboot may be required, with no call for one. The VC control system is a rather fiddly, multi-interface system which requires several different windows to design and initiate a scan – however, with the benefit of some familiarity, it presented no serious problems. A few irritations included the absence of some options that would have been useful, some options not seeming to work, and despite explicitly setting all actions to log only, numerous samples were removed or disinfected in the various scans run. Scanning speeds and overheads were mostly fairly good, although the executable speed test set took quite some time on demand, and results were fair to middling across the various sets, with some issues apparent over the new Virut samples. The Wildlist presented no such problems however, and with no false positives either Norman earns a VB100 award.

Quick Heal continues to live up to its name, providing a rapid and simple installation to go with its fast, uncomplicated product. The latest version of the interface has a crisp, clean glow about it that is very easy on the eye, and the layout remains basic but highly usable. An absence of in-depth options may put off more demanding admins, and some other issues emerged, including an apparent inability to save on-access logs and a tendency to ignore instructions not to interfere with any infections discovered. Also rather frustrating was a lengthy delay accessing browse windows when selecting targets for on-demand scans, sometimes taking over half a minute to display the filesystem. With this hurdle overcome, scanning speeds and overheads were most impressive.

Detection was also very good, with an excellent showing in the trojans set, full coverage of all our Virut samples, and a decent performance in the RAP set-up too. With the WildList presenting no problems, only the clean sets remained an obstacle to VB100 certification, and here sadly the same browser product which tripped up another product earlier was alerted on, using the same identification – suggesting some contamination of shared sample sets somewhere – and Quick Heal also misses out on a VB100 award by a whisker.

Sophos’s set-up procedure starts with a simple unzipping and leads through the standard stages, via an offer to remove competitors’ software and a couple of command prompt windows which flicker up briefly, to full activation in short order, with no reboot required. The interface looks much as it has done for some time: a fairly plain and bare look with a splendidly complete range of configuration options available beneath the surface, including a highly advanced area where interference without expert guidance is strongly discouraged.

Initial attempts at the speed tests found that on-demand scans invariably included additional scanning for rootkits and suspicious files in standard areas. This added several minutes to each scan, even over a small handful of files, so tests were redone using the right-click option to more closely approximate the standards set elsewhere. The progress bar remains worse than useless, invariably shooting to 80% in the first few minutes of a scan and lingering there for most of the remainder, be that five minutes or 90, but several other products also had some issues in this area. In the final reckoning, scanning speeds were very good, on-access overheads a little heavy, but detection rates were really quite excellent across the board, with a commendably stable set of figures across the trojans and first three weeks of the RAP sets. The WildList was handled easily, and while a sprinkling of items in the clean sets were labelled vaguely suspicious, this is permissible within the VB100 rules and Sophos wins another VB100 award.

Symantec’s corporate product provides options for central or local management to kick off its installation. We opted for local controls, and the rest of the set-up followed the usual path, although when it reached the end and suggested it would require ‘several minutes’ to tidy up after itself, it was something of a surprise to find that it actually meant it. A reboot was then required, after an attempt to update. The latest interface is a bright and shiny thing, not unpleasant to look at and providing a fair degree of configuration options in its more advanced regions (although some items we looked for were not available). The product includes some additional ‘proactive’ protection mechanisms, but these were disabled by default.

The system for designing and running on-demand scans proved pleasingly simple and quick to respond, and the bulk of the tests were handled with ease, producing somewhat below-par speeds in both modes but decent detection rates in most sets. These last figures were obtained only with great patience, as scanning large numbers of infected files takes some time – fortunately not a situation most admins would expect to encounter. Logging also proved rather fiddly, with the product taking an enormous amount of time to display and export logs, which in some cases seemed incomplete. Once data was finally accessed, a single sample of the latest Virut strain proved not to have been detected but the WildList was covered with ease. With no false positives either, Symantec earns another VB100 award.

TrustPort is another product to have had something of a facelift of late, with a curvaceous new company logo, some new fonts and a new colour scheme enlivening what is essentially a very similar layout to earlier versions. The installation process includes a strongly worded warning about installing on machines running other security products, and has a post-install configuration scheme including options to control the order in which the two engines included are applied. The interface is available as a highly simplified version, or as a more advanced one. This does indeed provide an advanced level of configuration, although once again some options were clearly absent, and indeed one – the choice to scan compressed files on access – seemed to have little effect when activated. A few other small worries were encountered, most notably some slow startup times for the on-access protection, which seemed still not to be working long after the newly booted machine was responding to commands. On one occasion a scan came to a halt with the stark message that an API error had occurred. Scanning speeds were rather slow, as one would expect from a multi-engine product, but detection rates were generally very good, although a worryingly large number of new Virut samples were not flagged. There were no problems in the WildList or elsewhere, and TrustPort thus also earns a VB100 award.

Bringing up the rear of the alphabetical product list as usual, VirusBuster’s server edition presents a rather confusing mix of the desktop and server approach. The installation process is fairly simple, and when up and running an interface can be accessed from the system tray and looks very similar to the standard desktop GUI. A brief browse through it, however, revealed that several standard options, and indeed sets of options, are not available here. To find them, one must turn to a second, MMC-based console, for which a shortcut is dropped onto the desktop. This made for some slightly odd flipping between the two as different tweaks needed to be made in various places. Occasionally some slow response times also frustrated, particularly when adjusting the targets of a scan, with long pauses after each stage of the set-up process. Finally, an issue which has been noted here several times before: the option to enable on-access scanning of archives is provided but appears entirely ineffectual.

Despite these minor irritants, scanning speeds were excellent and detection rates not bad at all, although as with so many other products this month, some work may need to be done on the latest W32/Virut strain. For now, however, the WildList set presented no issues, and without false positives either VirusBuster earns another VB100 award.