VB comparative review: Microsoft Windows Vista Business Edition (32-bit)

I should perhaps start by saying, by way of excuse, that the products were not necessarily tested in the order in which they are presented here, and my thoughts may appear a little out of joint as a result. The main reason for this was Alwil coming so early in the alphabet; I couldn't face starting what I expected to be a difficult and complex batch of tests with a product which I knew was likely to cause difficulties. avast!'s on-access behaviour has never failed to baffle me, and its oddities cropped up once again in its Vista offering, but happily far less than I expected. Nevertheless, due to the product's strange strategies on access, the accuracy of some of the speed measurements may be a little misleading.

The super-simplified basic interface of avast! looks good and may well be fairly easy to use with some practice, but as ever allowed too little fine tuning to be of much use in many of the tests. The speed tests were completed with some ease, and files certainly seemed to be being processed in the on-access mode; on-demand scanning of the WildList and other infected sets was also simple and impressively speedy once I had refamiliarised myself with the complex and fiddly 'advanced' interface.

The changing of settings required much designing and creating of new 'tasks', including a copy of the 'Resident Protection' on-access scanner. On-access detection, the bane of many a previous outing with avast!, again had my eyebrows buried in my hairline, as numerous alert messages scrolled up the lower corner of the screen, but little blocking seemed to occur. As far as I can tell, documents and script-type files like VBS/Loveletter were mostly blocked when opened with my usual utility, while executables were mostly allowed through.

Resorting to copying files onto the machine across the network brought the sought-after happier results, although the logging of detections seemed entirely ineffective, despite the option for such logging being firmly checked. After several passes through the scanner, a check of remaining files revealed nothing of importance left behind, and without false positives aside from a single 'joke' in the clean set, avast! is the first product to qualify for the new-look VB100 award.

Please refer to the PDF for test data

CA's developers seem determined to keep me busy. For some time, VB comparatives have measured the performance of the two engines supplied with the eTrust product, with only the default Vet option qualifying for the VB100 award. This continued until the last set of tests, when the old InoculateIT engine was omitted due to time constraints. Now that it has finally been retired from the product, CA has found another way of lengthening my working days - by submitting both its home and corporate products for testing.

The home product was fairly typical of the genre, with much attention paid to attractive styling, in keeping with Vista itself. The installer seemed to take some time pondering its surroundings, before shutting itself down, unhappy that the admin user was also logged onto the machine. With this rectified, installation proceeded fairly simply, apart from CA's old trick of forcing the user to scroll through the EULA before it can be acknowledged, as if they'd actually read it. The product itself included various anti-spyware, anti-spam and firewall modules alongside the anti-virus under test, which was somewhat limited as to configuration options.

Speed tests were performed in the default mode only, as I could find no way of changing the settings for scanning file and archive types. It certainly seemed to be paying plenty of attention to the archive files on demand, at one point lingering so long over a particularly large installer that I impatiently rebooted and restarted the test. This second attempt proved more fruitful, getting through the file without further snagging, and scans of the infected sets showed good solid detection, perfectly adequate to earn the VB100 award.

Please refer to the PDF for test data

This new version of eTrust seems but little changed from previous editions. Installation followed the old pattern, with the blue-ish grey scheme suitably pastelly in the new environment of Vista. The main interface of the product, a Java thing displayed in a browser, has frustrated me considerably in the past with its slow reaction times, but this updated version showed no such tardiness, with the progress bar I have spent many a long hour staring at barely in evidence this time around. Some of the interface seemed different from my recollection, but not hugely so - perhaps a few new option boxes dropped in here and there. The drop-down for which engine to use is still in evidence, but is now populated only by the Vet option, with InoculateIT no more than a fast-fading memory.

Again, there was no clear way to tweak scanning settings, and zips seemed not to be scanned internally on access, but speeds in general were highly impressive, and detection good, although a handful of macro samples caught by the home version above were mysteriously missed by its big sister. These were not in the WildList set however, and without a whisper of a false positive, eTrust gains another VB100 award.

Please refer to the PDF for test data

Those wishing to install Quick Heal are advised to use the 'Run as Administrator' option, and also to run several of the component files with elevated privileges when required. This certainly seems necessary, as often when omitting these steps the options sections were inaccessible, or other oddities occurred. A few times after a reboot, access to the product, and even apparently on-access scanning, was prevented by Windows Defender - it also seemed to be blocking several Windows functions from operating, rather oddly.

Using great caution, I coaxed the product through some speed tests. There seemed to be no option to scan all files, but further types could be added manually to the rather sparse extension list, and even with standard settings speed was a little below my expectations from previous experiences with CAT products. Running over the infected sets, at first I foolishly omitted to deactivate the warning popups for the on-access mode, causing a barrage of alerts one on top of another which, when I returned to the machine some time later, had frozen it completely. A message warning me my performance seemed to be falling sat forlornly beneath the paralysed mouse cursor. After a reboot and a tweak to the settings, the test was run with more success, and results showed a few misses in the zoo but nothing in the wild, with no false positives; a VB100 award goes to CAT.

Please refer to the PDF for test data

The grey of NOD32's installation procedure suddenly looked rather dowdy and old-fashioned when surrounded by the flashy, colourful window borders provided by Vista. Somehow, however, despite the very un-Vista-like styling, the control centre maintained an air of aloof futuristic power with its separable windows, and some pleasantly fast-opening tooltips helped identify the modules otherwise only known by codewords. AMON zipped through the on-access speed tests, while the NOD32 scanner, looking very glossy in its stylish new window, was its usual pacey self in the on-demand tests.

I had quite forgotten that acquiring logs requires some rather unintuitive behaviour, opening the log in a viewer, selecting an individual entry, right-clicking and selecting export to drop the data into a parsable file. The log viewer had some scrolling issues, with the horizontal scroll bar disappearing before I could see the end of lines, and another problem arose when trying to open the on-access log from the infected files test; the product seemed to freeze entirely, although it is of course enormously unlikely that anyone outside a test lab would ever have so many detections on access all at once. Fortunately, I didn’t really need this log to complete my analysis of results, which as expected proved excellent. Not a single miss or false positive gives ESET another VB100 award, and the ever-impressive speed was barely affected by the addition of archives for the on-access test.

Please refer to the PDF for test data

Fortinet's FortiClient is a pretty complete product, with a broad range of features offered by the array of tabs for its various functions lined up down the side. As such, it was little surprise that during the installation, aside from requiring the administrator password at the start, the installation of no less than three drivers had to be confirmed as expected behaviour.

Once set up, the GUI looked much as ever - serious and option-rich, although the tone was lightened somewhat by the bright shiny outline provided by Vista.

Scanning over the various speed tests was reliable and impressively pacey. FortiClient was one of very few products in this test to scan all files by default both on demand and on access. Detection was similarly excellent, with the few misses in the zoo sets seen in the last couple of VB comparative reviews eradicated. Without false positives either, FortiClient once again earns its VB100 award comfortably.

Please refer to the PDF for test data

F-Secure's Vista product was still in Beta at the time of submission for the test, freely downloadable for trial purposes. Installation, featuring F-Secure's current colour scheme of flat, brilliant whites and cool blues, looked a little odd inside the more shimmery stylings of Vista, but functioned perfectly well, demanding an administrator log in after an initial reboot to 'complete the installation'. Unfortunately, it was unable to call home from my lab to 'validate' itself, and I was warned I only had seven days to complete my tests before it deactivated.

Fortunately, this proved just about enough time. The controls were familiar from previous versions, but I frequently found myself disconcerted by the greying-out of options in the configuration dialogues, and confused by the need to use the 'change' option before the 'configure' option had much power.

Speeds were decent in most of the tests, with extending the range and depth of scanning making little difference in the archive set scanning time on demand; on access, however, it was quite another story, with extensive examination slowing things to a snail's pace, proving that F-Secure developers were quite right to switch this off by default.

The scanning of large numbers of infected files was equally sluggish, and the log wizard displayed some bizarre behaviour when asked to show me details of a sizeable scan, popping up a pretty HTML log with a small subset of the detection, which varied wildly in size each time I clicked the button. Clearly, this sort of user-friendly wizard is not designed for such unusually large files, and a simpler version of the log was obtained easily for checking.

Some excellent scores, with only a few samples missed among the file types that the product deliberately avoids in default mode, more than made up for the extra time taken. F-Secure wins the VB100 award with some ease.

Please refer to the PDF for test data

G-DATA's AntiVirusKit has had a glossy redesign fairly recently, with its twinkly badges, fading colours and fancy icons sitting comfortably amongst the equally fancy Vista themes. Installation demanded logging in fully as the admin user, rather than just a confirming password, but once installed protection could be disabled by a standard user without prompting.

One of few products in this review to combine the efforts of two separate scanning engines, speeds were still reasonable, and despite a stern warning when I disabled the size limit on archive files, that it could seriously slow down my system, the overhead was not too great. Intensive scanning inside CHM files seemed to lengthen the time on the media set, but this was not extended much by adding further depth.

The usual excellent results were obtained over the infected sets, with the doubled engine ensuring complete coverage of all sets. But just as I was starting to think everyone would be passing cleanly this month, the ball was dropped; a false positive in the clean set, and another on the same file in zip format on demand, denies G-DATA a VB100 award this time.

Please refer to the PDF for test data

AVG's installer is a bare and simple thing, featuring some large and sparse artwork of folders and other computery things, with a single request for the administrator password and no reboot required.

The product itself was less straightforward, at least in the 'Advanced' mode required for my testing, with a wealth of windows appearing to control various tasks and options. An information page told me, rather cutely, that I was running 'Windows Longhorn Professional', which was the early codename for Vista.

While the styling remains simple, the convoluted design of AVG's controls had me baffled a few times, before calm and sober pondering of the menus led to the required dialogue. With the GUI's code cracked, tests were carried out fairly easily, with the speed tests looking fairly decent and coverage of viruses also reasonable.

With a fair chunk of the older polymorphics and file infectors missed, but nothing significant elsewhere, AVG can add another VB100 award to its set.

Please refer to the PDF for test data

Kaspersky's product, in Beta at the time of testing, also maintains the design and styling of previous versions; the familiar green and red of the installer provided some simple options, and required the admin password to complete. Applying updates was a little more troublesome, with the product taking some time to register a change of source; after removing the default and adding a network folder as its target, it persisted in trying to contact an ftp server somewhere in Europe for some time, before eventually registering the change and finding the correct update sources.

Once this was done, no further problems were encountered, with the interface providing all the options I needed quite easily, and scanning proceeding in a fairly rapid and thorough fashion. This was another product to allow deactivation of its monitors by a standard user.

On demand, the product defaults to scanning all files, although archives are normally missed on access, accounting for the unusual speed over the archive set. To achieve full scanning, an option to scan all files rather than only selected types was set, as were further check boxes for archives and installers, and the slowdown thus caused brought speeds down to more normal, but certainly not slow, levels.

As far as certification goes, Kaspersky reached the necessary standard with ease once more, with the only misses caused by not scanning zip files by default, and as a result Kaspersky is awarded another VB100.

Please refer to the PDF for test data

McAfee's latest product is also little changed to the naked eye, with just a few beautifications here and there. The installer spent some time pondering its new surroundings before getting going, but once off the mark got things set up fairly speedily, with no need for a reboot to get itself active. Some aspects of the GUI were a little fiddly, with some of the deactivation controls greyed out but available as options on the system tray icon.

Opening the console, like a few of the other products, required confirmation of my possibly dangerous actions, which makes the screen behind fade out, and a few times on clicking the 'reset to defaults' button on a configuration page, a similar effect occurred, leading me to think I had crashed out the console. However, all it needed was to close itself down and restart to apply the changes, and all was functional once more.

Speeds were pretty good, and the configuration logical and easy to follow; scanning over most of the test sets was fairly solid too, but both on access and on demand the product committed the ultimate sin and missed WildList viruses, thus spoiling McAfee's chances of a VB100 award on this occasion.

Update: After intensive investigation, VB has found that detection routines for the two malware samples missed were indeed included in the update package provided by McAfee. However, when McAfee's manual update procedure was run it failed to apply the update to the product, despite both on-screen messages and logs stating that the product had been updated successfully. This behaviour was reproducible throughout the review period and has continued to be the case in several subsequent retests.

The problem was found to be a result of the way in which McAfee VirusScan interacts with User Access Controls (UAC) included in Windows Vista. Despite being run by a user logged on with administrator rights, the update program designed for use in sealed environments like the VB test lab was also required to be executed with the 'Run as administrator' option to succeed, but did not report this to the user or display an error messages when the update failed.

'We feel justified in denying the product the VB100 in this case,' said John Hawes, Technical Consultant at Virus Bulletin. 'The product reported it had updated itself. A user who is fooled into thinking they are running up-to-date protection is in as bad a position as one who is running up-to-date but inadequate protection; a false sense of security is a dangerous thing.'

'Users with more standard update methods would apparently not have had the same issue we did,' continued Hawes. 'The problem we had can be put down not to an inability by McAfee to keep up with the latest malware, but rather to a failure to properly integrate all aspects of the product into the new Vista operating system, and most importantly the new security controls. Vista caused trouble for a lot of products and this, though seemingly a minor issue, had a major effect on the protection provided by McAfee's product.'

Please refer to the PDF for test data

This was my first experience of the Microsoft product, which is already in its second incarnation. Long amused by the name, which in the kindest light is reminiscent of a famous chocolate factory owner, I had been looking forward to trying it out for some time, and was almost denied the opportunity by a series of snags. The original submission was no more than a downloader, requiring Internet access to retrieve the bulk of the software. Some rapid explanation of the sealed-off nature of the VB lab brought a special version with some adjustments to the setup allowing it to be installed offline, which for a while sat untouched on the test bench, awaiting its turn. When I finally tried to get it going, the installer failed halfway through - a problem, I was told, due to access rights; running it as administrator got me slightly further, but in the end the UAC had to be completely disabled to get things up and running. I assume these steps are not necessary with the proper online installation process.

One look at the GUI lengthened my face considerably. There were not a lot of controls here, no tabs full of sliders and check boxes, no 'advanced mode' button for the serious user. My first glance at the settings page showed very few options indeed - 'On' and 'Off' seemed to be the extent of it, although closer examination revealed options to exclude certain files and areas, and also to inspect the quarantine area. A log was also available, which again I did not spot at first.

Looking back at OneCare's only previous appearance in a VB comparative (see VB, June 2006, p.11), I see my predecessor had similar problems, describing the product as 'a paranoid nanny'. His experiences back then were again mirrored after the on-access test, when the product ground to a halt, its interface fading to a pale pink with the ever-comforting '(Not responding)' appearing in the title bar. Even a reboot failed to solve this problem, and I ended up reimaging the machine and starting from scratch, although fortunately the results of the on-access scan, and some of the speed tests, were safely in. Again, I would assume that the unusual situation (the improbably large number of detections encountered in a short period) is probably at the root of this problem.

On-demand scans were similarly tricky. While the speed tests were fairly easy, producing good results, of course without the ability to change the settings it was difficult to tell how much scanning was going on; archives were clearly being delved into to some extent, on demand at least. Scanning the virus collections seemed to be going well, until the auto-cleaning began bludgeoning its way through the system32 folder to check for real infections. I began my first attempt mid-afternoon, and watched it climb fairly rapidly to 90%, where it remained for several hours and it was still hovering there when I returned next morning.

Another try at this finally got it through, and after getting some advice on acquiring logs for parsing, I finally got some results. The log contained a number of error messages for files in the system folder that had proved unscannable, in part explaining the trouble with completing the cleaning process. Detection of viruses, on the other hand, was generally decent, with a small handful of misses in the zoo sets, but more significantly numerous samples of one of the W32/Looked variants in the WildList set were missed in both modes, and so OneCare misses out on a VB100 award for now.

Please refer to the PDF for test data

Norman was again little changed from the user angle, a situation which disappointed me somewhat as I've always found the interface a little awkward. Installation was straightforward, with full admin login required but no extra demands for confirmation, and no reboot was called for; it seems to be required however, as at first the product exhibited some unusual behaviour, not least having no icon in the system tray from which to access the controls easily.

Restarting the machine rectified this and the scanning oddities, and testing proceeded, slowed only by the complicated and window-heavy task of setting up and running scan tasks. Speeds were more impressive on access, even with more complete settings switched on, than on demand, in which mode all files are scanned by default, although internal scanning of archives seemed to be eschewed at all times, with no option to enable such in-depth analysis.

On demand, Norman's usual handful of misses in the zoo sets were unsurprising, but a trojan detected in the clean set complicated issues somewhat; the file in question was the installer for a competitor's anti-rootkit product, the inclusion of which in the test set was made after some thought as to its appropriateness. The issue of failing a product after tricking it with a file known to be difficult became irrelevant, however, when several ItW viruses which had been detected with ease on demand, were missed repeatedly on access, and Norman misses out on another VB100 award.

Please refer to the PDF for test data

After several days awash in this sea of troubles, reaching the Sophos product was like the reassuring crunch of a sandy beach beneath the fast-eroding bit of driftwood that is my mind, with firm trees laden with plump fruit on the skyline. Suddenly it was as if Vista had never happened; Sophos's installer and components looked and felt just like they have done in the last half-dozen tests, since the last major redesign of the product a year or two ago.

Sophos made much, during the recent brouhaha over access to details of the inner workings of Vista, of how well prepared its developers have been for the launch, and playing briefly with this version shows the boasts were pretty justified. Installation was fast and slick, with just the one standard request for admin rights, and once installed the controls seemed properly suited to the UAC, with most configuration options blocked for the normal user and accessible only to the administrator. The GUI remains unchanged, not beautiful but functional, with not a cunningly hidden option to be rummaged for, and at last I had found a product where everything seemed just to work.

Speeds were pretty decent, and detection hit the usual solid levels, with a few file types and obscure older samples avoided. In the clean set a couple of files, both process manipulation utilities from SysInternals, were labelled as potential hacking tools, but as such definitions are allowed within the rules, Sophos earns a VB100 award, and a sigh of grateful relief from me.

Please refer to the PDF for test data

Symantec has yet another installation process that demands full administrator rights, but after a few false starts even this was not enough. Following some slightly inaccurate instructions in the readme, I changed some security settings in various MMC plugins, which enabled installation to proceed, disabling Windows Defender along the way, I noted.

Once set up, the product produced no further problems, with the normal GUI looking as serious and sensible as ever, wordy and adorned only with small, sober icons. Configuration was fairly straightforward, although I could find no option to scan zips internally on access for the new speed tests, making the speeds look even more impressive than they perhaps should, and detection across all sets was impeccable. Without false positives either, Symantec also earns another VB100 award.

Please refer to the PDF for test data