VB comparative review: Windows Server 2003

AEC’s Trustport suite contains a number of items beyond the anti-virus component, but as usual only this module was submitted for testing. This made installation a pretty straightforward process, and left me with no main interface from which to operate – configuration and functions are instead accessed from a system tray menu. The default settings are pretty thorough, detecting everything in our archive and file-extension scanning test set, and combined with the multi-engine layout this led to some rather languid scanning times.

AEC’s entry in the last comparative review (see VB, December 2007, p.16), its first since the BitDefender engine was dropped from the product in favour of those of Dr.Web and VirusBlokAda, suffered from some false positive issues as well as several WildList misses. Detection was greatly improved this time, with nothing at all missed on demand, and only a few older items missed on access (where not all the available engines are used by default). However, despite one of the engines apparently being disabled entirely, and greyed out in configuration dialogs, several false positives were recorded, which once again deny AEC a VB100 award.

Agnitum’s Outpost was subject to an in-depth review last month (see VB, January 2008, p.17), after achieving its first VB100 certification in the previous comparative. With the review still fresh in my mind, the installation process and configuration were fairly straightforward, although the product is sufficiently well designed to present few difficulties for those without any prior knowledge.

The available configuration is somewhat limited, with no option to scan archives in on-access mode, but other files did seem to be inspected regardless of their extension, and speeds were fairly reasonable considering. False positives were entirely absent, and detection in most of the test sets at the pretty high level expected from the VirusBuster engine in use. In the WildList set, however, a single instance of a W32/VB worm was missed, as well as two samples of one of the new W32/Virut variants. This presaged problems for some of the products further down the list using the same technology, and meant Agnitum didn’t quite manage to add to its VB100 tally.

AhnLab has not been a regular participant in VB100 tests recently, but the AVAR conference the company hosted in Seoul a few weeks before the test deadline provided an opportunity to pester the developers into joining in again – an effort which paid off with this entry.

The V3Net product is quick and easy to install and set up, with a clear and pleasant interface adorned with a touch of cartoonishness without seeming too silly. The configuration is a little lacking on access, with no option to delve inside archives in this mode – something which seemed a little odd in a dedicated server product, as one might expect experienced admins to be interested in having a fuller range of options available. Even in on-demand mode, where most archive types were examined quite deeply, self-extracting executables and installer files were omitted. Another oddity which may cause admins frustration is the format of logs, which record only filenames with no information as to where the files in question may be found – this made for considerably more work in processing the test results.

Detection itself was less of an issue. No false positives were recorded and, despite a handful of misses in some of the older test sets, nothing significant was missed in the WildList set, thus AhnLab earns a VB100 award on its return to the test bench.

The server version of avast! seems little different from the standard version, or at least from the ‘enhanced’ interface usually necessary for the VB100 test. All the required configuration was readily available, with the defaults set not to scan archives internally but options available to scan the full range of archive types included in our test sets. Oddly, the renamed version of the EICAR test file was spotted on access but not on demand, implying that the on-access scanner is set up a little more thoroughly than the normally more in-depth manual scans.

Speeds were impressive, and still fairly decent with the more complete scanning options enabled. Detection levels were reasonable across the sets, with nothing at all missed in the WildList set. With no false positives either, Alwil wins another VB100 award.

Despite an installation process which seemed very familiar, after the required reboot AntiVir’s Server edition displayed significant differences from the desktop variant, with an MMC-based console provided for most of the required configuration options. The interface was not as simple to navigate and use as Avira’s desktop range, but seems to provide a pretty thorough range of controls for the administrator. On-access scanning was fairly straightforward, and thorough once fuller scanning was enabled, although a few files compressed with the ACE algorithm were missed despite more deeply nested samples of the same format being detected.

Some very good speeds were recorded in both modes, although the actual setup and running of on-demand scans took much more time, with a rather awkward and fiddly setup process, and no indication of scanning progress at all. Once the complexities of the design were cracked, scan results showed the product’s usual excellent detection rates and no false positives, giving Avira another VB100 award.

BitDefender also provided a special server version for this test, again incorporating a console interface using the MMC framework. This seemed rather more logically laid out and took less effort to decipher, but also seemed to be missing some useful options. The on-access scanner, for example, seemed to offer no option to block access only, making this action available only after attempts at other ‘cleaning’ methods had failed. This resulted in my test collection being trashed and requiring restoration between tests. Another apparent failing was an issue with setting up on-demand scans. Assuming at first that these could again only be run from the scheduler, I set up a scan using the default time offered, which was in fact the current time – ideal for my needs. However, by the time the setup process had finished, the moment had passed and the scan thus failed to initiate, waiting instead for the same time to roll around the following day. My frustration was quickly sidestepped when I found the proper place to run manual scans, with a ‘scan now’ option available.

Having deciphered the interface, testing continued without further stumbles, with fairly good speeds and the default settings covering most file types in depth. Detection was pretty close to flawless across the test sets including the WildList, and in the clean sets a few items were flagged as adware but no false positives were recorded, granting BitDefender a VB100 award.

CA’s eTrust is a corporate-focused product, and has been submitted in much the same form for just about all VB100 tests I have run. This month was no different, and the familiar interface, its frustrations of slow connection times slightly less intrusive than usual, powered through the tests in splendid time. On-access archive scanning appeared to be absent, despite a number of options relating to such scanning being activated – single-level zip and jar archives were penetrated in this mode, but no other types or greater depths. On-demand scanning proved more thorough, although ACE and self-extracting EXEs were only probed one level deep.

Detection levels were very high, with almost complete coverage across the test sets and the WildList covered without difficulty. Without false positives CA easily makes the grade required for a VB100 award.

Doctor Web’s product presented the same slick and solid design which impressed me in the last test, although the rather basic font used in the installer looks slightly out of place in its glossy surroundings. The clear layout of the interface made testing smooth and problem-free, with sensible defaults and deep configuration available. A few times on shutting down the on-access scanner there were error messages that claimed there were issues with disabling the protection, but it certainly seemed to have closed properly and restarted without further problems.

Scanning speeds were excellent, particularly in the default mode, which uses a ‘smart’ setting to determine which files are worth scanning. With thorough scanning of all files enabled things slowed down somewhat, but detection was pretty good across the board, with no more than a few files missed in each set, most of them down to file types not scanned by default. No false positives were in evidence, but unfortunately for Doctor Web a few items added to the latest WildList were not covered, and the VB100 award remains just out of reach.

The latest incarnation of ESET’s product was reviewed on its release a few months ago (see VB, November 2007, p.19), and received some rather effusive praise for its stylish looks and smart design. As NOD32 version 3 appeared on the VB100 test bench for the first time, the stylishness and clever layout continued to impress, allowing the tests to be run through with great simplicity and making the testing experience a joy.

Speeds were as excellent as ever, although probing into archives slowed things down somewhat, and this depth of scanning was not available on access – one of the only options notably absent. Detection could not be faulted in most sets, although a set of samples of an aged DOS polymorphic virus which caused no problems in previous tests were not detected with this version, returning an ‘internal error’ message in logs. This does not affect NOD32’s qualification for the VB100 award, which was achieved easily with full detection of the WildList set and no false positives.

Fortinet’s product provided a similarly problem-free run through the tests.

The installation, updating and configuration processes are familiar, the core interface having changed little for some time. The product is clearly laid out with all the required elements readily to hand, despite a wide range of other functionality (beside the anti-malware protection) being controlled from the same interface.

Little configuration was required, with the default settings including most file types. Somewhat oddly, ZIP files – perhaps the most common archive format – were scanned less deeply than others. This could be a resource-saving measure introduced due to the very popularity of the format. Despite the thoroughness speeds were quite impressive, and coverage of the sets excellent, with no misses and no false positives earning Fortinet a VB100 award.

F-Prot is a far simpler product than many, with a pared-down interface offering basic control of anti-malware protection and scanning, and little else. With minimal configuration available, and functionality such as logging generally excellently implemented, testing zipped through. Minimal configuration options cut the speed test requirements down, with only the product’s seemingly unstoppable urge to remove infected files drawing out the process (an initial run was stopped and replaced with one in which detections were logged only after the first attempt proved to be spending considerable time disinfecting and quarantining).

Default archive settings were among the most sensible so far, with most archive types covered in depth on demand and the basics, self-extractors, ZIPs and the almost identical JAR files delved into a couple of levels deep on access. Speed times were splendid, and detection almost impeccable, earning Frisk a VB100 award too.

F-Secure’s product is a little more complex and in-depth, though the server version tested here seems little different from the desktop editions seen in previous comparatives. The installation is slick and smooth, lending a solid and trustworthy feel to all components. This weightiness is not too evident in the scanning times, which were surprisingly good over most of the sets although, with the default setting to scan most archive types to a depth of five levels, this set took rather longer. Somewhat oddly next to this thorough setting, file types are identified only by extension, but scanning with ‘all files’ enabled did not take too much longer to complete, although an occasional moment of sluggishness was observed during operation of the machine thereafter.

F-Secure has presented me with considerable difficulty recently thanks to its rather flaky logging behaviour, which was in evidence once again here, with the ‘display log’ button bringing up an attractively formatted HTML log in a browser window. As in previous tests, the contents of this log varied wildly, apparently containing a random sampling of items discovered during a scan. Attempting to access the results of scanning the full test collection produced logs varying in size from 50 to 1500 KB. After much frustration trying to achieve the best results with this method, a series of smaller scans set to delete files proved the simplest way of judging the product’s effectiveness.

This effectiveness was considerable, with splendid detection rates and no false positives, just a few alerts on suspect tools with potentially unwanted uses. With no problems at all in the WildList F-Secure also qualifies for a VB100 award.

After an initial problem with an activation code inappropriate for use on a server, AVG proved somewhat simpler to handle, slipping slickly through its install and skipping lightly over the test sets. Although the multiple-window configuration system remains somewhat baffling, the limited configuration options were eventually tracked down and testing produced no major frustrations.

Scanning times were fairly decent, although again by default files with altered extensions are ignored. Detection rates were similarly solid rather than excellent, but the WildList was covered without difficulty, and with no false positives recorded Grisoft also makes the VB100 grade.

Ikarus has bravely battered at the VB100 door for some time now, and has gradually moved closer to the required standard for qualification, with high levels of false positives having been the major stumbling block in recent tests.

The product’s interface uses the .NET framework, and has suffered some flakiness in the past, which this month was considerably lessened. However, on a few occasions the GUI seemed to fail to open, and during the scanning of large infected sets the whole thing seems to flicker and spasm rather worryingly.

An initial run over the clean test set produced some remarkable speed times and an even more eyebrow-raising absence of false alarms. Some quick investigation quickly showed that I had omitted to apply the update, and that in its bare state the product has hardly any detection capabilities at all. Re-running the tests showed that a small number of clean files has been mislabelled, and a handful of WildList items missed, a few odd samples of several of the latest polymorphic additions. Although speed times were impressive and detection in the other sets fairly reasonable, Ikarus still has a few more issues to resolve before attaining a VB100 award.

Kaspersky, meanwhile, is a seasoned competitor with a long history of excellent performance, a few minor technical issues in recent tests notwithstanding. The product, not quite as glossy and glitzy as the home-user offering provided lately, is no less solid or reliable for it, and offers a well-designed, intuitive interface with an excellent level of configuration, although scanning of archives on access seemed to produce a fairly erratic selection of depths for different formats.

After a few brief and easy tweaks the product stomped through the tests, speeds reflecting a more thorough attitude to scanning than many, but results showing splendid coverage and no false positives, thus earning Kaspersky yet another VB100 award.

Kingsoft is another firm which has had some trouble in recent comparative reviews but has nevertheless continued to strive for the excellence required for a VB100 award. The company’s product has grown in stability and responsiveness in the year or so since it first visited the VB test bench, and seems very pleasant to look at and rational to use.

Available configuration is less than complete but adequate for my needs, and testing trotted nicely along with impressive scanning times. False positives were pleasingly absent and detection rates showed further improvement, but alongside a fair number of recent items in the older sets (including some quite significant W32/Sdbot and W32/Mytob variants), several worms in the WildList set were missed, as well as a few samples infected with W32/Virut and W32/Bacalid. As a result, a VB100 award still proves to be a little way out of reach for Kingsoft this month.

McAfee’s enterprise product is a regular on the VB test bench and it took me little time to find my way around it. The layout is somewhat individual, but simple to operate and provides the full range of settings and controls expected in a complex corporate environment.

Adjusting the defaults to cover a wider range of file formats did not add too significantly to the pretty fair scanning times, although of course delving deeply into a broad range of archives was a little slower than leaving them unchecked.

The solidity of design and implementation was reflected in some effortlessly impressive detection rates, with nothing missed or mislabelled anywhere, and McAfee thus wins a VB100 award.

Microsoft’s product seems to take quite the reverse approach, assuming a mother-knows-best attitude and offering almost nothing by way of configuration options.

Rather amusingly, the installation process required an update to the Windows Update Agent before it could complete, and once installed the simple interface offered some basic information and a page of rather random controls.

The client in use here is part of a more complex suite of products, so it is possible that much of the configuration can be controlled from above. Nevertheless, it would seem appropriate to provide the user with a little more information on how their system is being monitored.

After running some scans and on-access tests a small amount of information emerged about how the product was operating, though little of this came from the product itself. After scanning several thousand infected files the GUI displayed the message ‘Items Detected – Severe/High Alert level: 24’, while all detections were logged only to the system event log once the on-screen display was closed. A ‘History’ button reopened the display from each scan, but regularly froze while trying to access the results of large scans and on occasion caused the whole interface to disappear from view.

Despite these annoyances, results were eventually dragged together and showed fairly good speeds. A sensible default selection of files handled all the archive sets without problem on demand and looked briefly into the most common types on access. Detection rates were very good indeed, and without any false positives Forefront is awarded a VB100.

The corporate edition of eScan is a little more sober than the normal home desktop version, although its installation process with automatic scanning of important system areas remains much the same.

Configuration is provided via a console resembling the MMC, but dubbed ‘EMC’, and seems fairly comprehensive. However, little adjustment was needed as the default settings scanned pretty much everything thrown at it.

This resulted in some rather slow scanning speeds but of course excellent detection rates. A couple of items spotted as suspected malware by the Kaspersky engine in its other guises were missed here on access, and a few others that were not identified elsewhere were flagged here as potentially risky. However, with no samples missed in the WildList test set, and no false positives, eScan also qualifies for a VB100 award.

Norman’s product is another which makes use of a variety of windows for various facets of its control and operation, and as usual this led to a certain amount of confusion and frustration. However, once their interoperation had been mastered things proceeded reasonably well, with the only issue found in the actual running of the product being a problem with the redirection of logs. An option to change the folder in which logs are saved seemed ideal for my use, but on checking the selected location at the end of the test it was found to be entirely log-free. Fortunately all the required data was stored within Norman’s own logging folder and results were thus gathered after only a brief moment of worry.

There was not a great deal of flexibility in the types of files scanned, with a handful of the more common archives investigated on demand but none on access. All file extensions were analysed for malicious content by default however, and this resulted in some rather below average speed times, as well as a single file in one of the clean sets being labelled as malware.

Detection rates were also less than perfect, with a handful of polymorphic variants in the WildList set not fully covered, the on-demand scanner faring slightly better than the on-access. Norman thus misses out on a VB100 award this month.

PCTools products have been a little awkward in the past, with an inflexibility of configuration providing some frustration. This time, however, everything I needed seemed to be both available and easily accessible. The installation offers an accompanying install of the Google toolbar, which I turned down for my tests, but few other difficult decisions were required.

Despite the default settings covering no archive types or renamed files on access, scanning speeds were on the slow side, and the system seemed less than usually responsive. On-demand scans had slightly more thorough settings, with most archives probed to a single level, and the resulting speeds were even less impressive.

Scanning infected sets brought up a beautiful cascade of alert popups, scrolling and interweaving with each other down one side of the screen. Detection rates closely mirrored those of Agnitum, as both products use the VirusBuster engine, and thus it was hardly a surprise to see the same handful of misses in the WildList. Thus, despite a lack of false positives, PCTools does not receive a VB100 award for its efforts.

Quick Heal is one of the few products to scan the system prior to installation, but the setup process is nevertheless speedy and efficient, offering a friendly ‘Welcome’ message flashing in the system tray. The interface is visually appealing and seems very stable and solid, but again configuration is kept to a minimum.

On-access settings can barely be adjusted at all, with no way of forcing files such as my renamed EICAR file to be watched for, and archives left unprobed. On-demand scanning is a little more thorough, with a few items delved into lightly by default and slightly more depth available for those who want it.

This lightness of scanning may contribute somewhat to the speed of the product, which was uniformly excellent. Detection rates were a little below average over the older sets but the WildList was covered without difficulty. In the clean set, a few items were incorrectly flagged as malicious, mostly identified as ‘I-Worm.Sohanad.T’, suggesting some overzealousness in the detection of this item. This inaccuracy is enough to deny Quick Heal a VB100 award this time.

Redstone returns for a second attempt at the VB100, having been denied last time by a small technicality in the settings of the Kaspersky engine on which it is based. This is another .NET product, again at a fairly early stage in its development, and some flakiness is evident in the running of the interface, with occasional unexpected shutdowns and the odd error message, particularly when trying to access logs. Configuration is extremely minimal here, with the controls accessible from the system tray icon limited to running a scan and shutting down the on-access scanner. With the ‘default’ settings provided in the form of a series of registry keys it is here that adjustments must be made if needed – changing the default on-access behaviour (which seems to be to prompt users with a message offering not to delete if they respond within 30 seconds) seems not always to respond as expected, interrupting a few scans with its warnings.

After some struggles extracting scan data from a series of XML files and allowing the on-access scanner to delete most of the infected test set, results were obtained. The results proved as excellent as those achieved by other products using the Kaspersky engine.

With detection almost impeccable and false alarms completely absent, Redstone qualifies for its first VB100 award.

The entire Sophos product line has a resolutely corporate focus, and thus the offering for this test seems identical to those that have appeared in previous comparatives. With the usability never too taxing, the installation and configuration of the product slid by without any trouble.

Testing proved just as simple a process, although the progress bar proved as errant as ever (which proved to be a common issue in this test in cases where an attempt was made to estimate the remaining scanning time), and the logging seemed rather strangely organised and confusing.

The deep configuration available did not extend to scanning archives beyond five levels deep, but most types were covered, and scanning speeds – excellent with the default settings – were fairly good.

Detection rates were splendid, and although the switching on of a wider range of suspicious detection flagged up a number of unusually packed files in the clean set, alongside a handful of ‘adware/PUA’ and ‘Hacktool’ alerts, no full false positives arose and Sophos is able to claim another VB100 award after a couple of unlucky months.

Symantec’s corporate desktop product has undergone a considerable change recently, and still seems to be suffering a few teething problems.

Although the installation was impressively speedy, the automatic attempt at online updating took some time and effort to put a stop to (including a warning that it may take a few minutes to ‘clean up’), followed by a reboot.

Logging of scan results also proved problematic, with attempts to open the logs via the interface causing some nasty freezes for the on-access data, and simply a blank page for on-demand data, despite several scans and several tens of thousands of items detected. The freezes were resolved by killing the process with the Task Manager, which brought up an increasing number of alert messages from Symantec’s anti-tamper system, informing me that attempts to shut it down had been ‘blocked’ – in one instance, after several dozen of these messages protection was in fact stopped and the interface restarted.

These minor issues, likely due to the generation of a log exceeding 150 MB, did little to affect the results themselves however. Scan times were fairly good, with on-demand defaults delving three levels deep into most archives and more available. The on-access scanner seemed to offer only limited configuration but did identify disguised file types. Parsing the enormous log showed superb detection rates and a complete absence of false positives, and Symantec also qualifies for a VB100 award.

VirusBuster’s server product again seems much the same as the home-user version, with the addition of an MMC-based console for some extra configuration. This included options which seemed to imply archives would be scanned internally on access, but apparently only cover normal executables renamed as archives to conceal their intentions (which would be ignored in the default modes).

The interface itself is pleasant if a little fiddly when setting up scans, and suffers a tendency to linger a little over saving its logs, even those with minimal content. This did little to dent a good performance in terms of both speed and detection, with no false alarms and the W32/Virut samples missed by the other products using the same engine causing no difficulties here – presumably due to a slightly later version of the detection data. However, one remaining item, a W32/VB worm variant, was missed, and although we are advised that detection was added to the product a week or so after the submission deadline, the missed detection prevents VirusBuster from attaining a VB100 award this month.

Last on the list of products comes Webroot’s SpySweeper. This is SpySweeper ’s second visit to the VB test bench, having made its debut – and gained VB100 status – in the June 2007 XP review (see VB, June 2007, p.10). The corporate version submitted here was considerably different from the home-user edition submitted previously.

After a rather drawn out installation and startup process, the product offers a fairly comprehensive interface with some apparently well-populated configuration pages. Unfortunately, these are initially greyed out, as the client system submitted is designed to cede all control to a management server. Some changes to the registry allowed access to the settings (after providing a password) and testing continued.

Problems did not end there however, as the on-demand scanner seemed to provide no option to scan only a given folder and the entire system had to be scanned – no small job in this case. On returning after leaving the scan running overnight I found that the test sets had been covered pretty thoroughly, and they were then replaced before attempting the on-access tests. These were again hampered by the product’s rather unusual implementation, with on-read scanning deactivated by default and only functioning rather flakily once enabled. This rendered any speed results gathered somewhat suspect, and only detection results were obtained by copying all test sets to the system across the network.

As far as can be judged by feeling alone, the protection did seem to slow the machine’s response time down noticeably, especially during the five or so minutes after a reboot when the system tray icon is whirring and the interface unavailable (presumably doing some sort of boot-up checks.) After several attempts yielded a usable log of detection, results turned out to be pretty good - close to the high level expected of the Sophos engine used in the product - bar a few file types not scanned with these settings. Without false positives either, Webroot earns another VB100 award this month.