2006-12-01
Abstract
A diverse range of products was submitted for this month’s 64-bit comparative review. John Hawes has the details of how they all fared.
Copyright © 2006 Virus Bulletin
64-bit computing is once again the way of the future. After brief flashes of excitement in the 1990s, the DEC Alpha and various other proprietary 64-bit systems became confined mostly to specialist use, running their own proprietary UNIX versions, and even the Intel/HP collaboration the Itanium has become something of a niche player.
With the advent of the AMD64 architecture, however, 64-bit has moved out of the server farm and onto the desktop. Only a few years old and rapidly gaining popularity outside the sphere of hardened gamers, speed freaks and other early adopters, machines running on AMD64 (and Intel's version, EM64T) are becoming almost as common as their 32-bit counterparts, with their 32-bit compatibility making the upgrade a fairly painless one. A large part of the long-running row over the security of Windows Vista, concerning the PatchGuard kernel protection system, applies only to 64-bit platforms, proving the importance of this hardware in the eyes of both operating system and security providers.
A diverse range of products was submitted for this comparative review. Some regulars were notable for their absence – perhaps put off by the platform – while others submitted their standard products hoping that, by virtue of the built-in compatibility, they would work just as well as they do on 32-bit machines. The architecture is still somewhat on the young side however, and oddities of hardware and software are far from uncommon. Beside the usual difficulties associated with testing, I expected the occasional moment of bafflement as the platform, products and tests overlapped in strange new ways. An unusually large number of additions to the In the Wild (ItW) test set also seemed likely to cause a problem or two.
The x86-64 edition of Microsoft's Windows XP in fact has rather more in common with Windows 2003 Server, and this is immediately obvious from the user experience. Installing to the test lab's suite of 64-bit machines was a simple and remarkably fast process, with the high-powered dual-core AMD64 CPUs, ample RAM and zippy SATA hard drives making light work of the job.
Replicating samples for the VB test set was enlivened this month by the arrival of several file infectors in the August WildList, with which the ItW test set was aligned. W32/Detnat, W32/Looked, W32/Virut and W32/Polip, a polymorphic, are all fairly voracious infectors, dropping themselves into opened files or trawling filesystems for likely victims. This allowed several different samples of each to be included in the test set, making a change to the usual worms and bots which have dominated the lists for some time. These, of course, were also represented in some strength, with the expected swathes of W32/Mytob and W32/Areses, along with handfuls of W32/Bagle and other regulars. Most notable among the worms was the advent of W32/Stration, dozens of slightly adapted generations of which continue to be spread worldwide in wave after wave. Most of these I expected to cause little difficulty for the products; the file infectors, on the other hand – particularly the polymorphs – were expected to provide a more probing test of detection capabilities.
The avast! product has a resolutely home-user-friendly style about it. The basic GUI has a sleek and sexy appearance, the car-stereo styling providing simple 'Play' and 'Stop' buttons for scanning and a few other basic controls, while a more advanced interface is available for those requiring more fine tuning. This was reached through a small button providing various menu options (which I had ignored at first as it looked like an 'Eject' button, and I assumed it would shut the thing down). The 'Extended' interface provided most of the tools I required, along with a rather bizarre virus information section, featuring a table comparing various aspects of the malware described. While the table clearly showed which items belonged to which sub-grouping, affected which platforms and spread in which ways, the identities of the malware were hidden from the casual browser, and only revealed when an individual line of the table was selected.
With the interface mastered, the product ran along fairly well, although the disabling of scanning certain file types previously scanned by default resulted in several samples being missed (extreme speeds on certain parts of the clean set imply that zip files were among the extensions excluded).
As I have learned from testing Alwil products in the past, on-access scanning is not guaranteed to be activated by simple file opening, so some tests required copying test sets to the machine and having the product delete files as they arrived. Eventually avast! was cajoled through the tests, missing nothing important and finding nothing but a 'Joke' in the clean set, therefore becoming the first product to receive a VB 100% award this month.
Please refer to the PDF for test data
Avira's now-familiar shiny, happy style led me through a simple installation, past a warning to ensure I had a genuine copy of the software rather than a cheap rip-off, into the equally straightforward interface. Controls were where I expected to find them (perhaps through some familiarity with the product as much as judicious design), and the little umbrella in the system tray marking the status of the on-access protection opened and closed smoothly and quickly as I adjusted the settings for various tests.
Scanning speeds were fairly decent, and most of the collections were handled pretty thoroughly, with a smattering of zoo samples missed but nothing in the ItW set. In the clean set, the false positive spotted last time around has long since been fixed, so there was nothing to deny Avira a VB 100% this time.
Please refer to the PDF for test data
CA's eTrust product has been submitted in more or less the same form throughout my experience here at VB; with a new version looming, this could be the last appearance of this incarnation on the test bench. The large corporate installer, with its numerous EULAs, lengthy activation code and sizeable page of personal information to fill out, including access passwords for the configuration controls, took longer than most despite familiarity. As usual, I opted to install the agent parts only, without any of the extra network management tools, and after some time setting up was faced with the browser-based GUI. The testing itself also dragged over some time, with the GUI taking its time to respond when trying to switch between tabs. Displaying of logs was particularly drawn out; at one point, bored of watching the progress display telling me my logs would be ready to view in a moment, I wandered off to grab a drink, only to find on my return that my 'session' had timed out. Revisiting the logging tab and repeating the process, I was again distracted by other things, overestimating the length of the 'session' and finding myself once more back at the start.
In terms of scanning itself, things were quite different. Awesome speeds were achieved, both in the clean set and over infected areas, with detection pretty decent throughout – suggesting the engine, if not the interface, was making efficient use of the powerful hardware. The old InoculateIT engine, not used by default and therefore not eligible for the VB 100%, displayed some even quicker scanning speeds over some of the test sets, although detection was not as thorough as the Vet engine and some strange anomalies popped up when trying this option (including, for a brief moment, a file in the clean set locked by the on-access scanner – an event which could not be reproduced). With no false positives to report from the Vet engine, and little missed elsewhere, eTrust wins itself a VB 100%.
Please refer to the PDF for test data
The Quick Heal installation process included a quick scan of 'system areas' to ensure it was safe to install to my machine. After the setup and a reboot, a friendly message welcomed me to the product, and led me into the main GUI, a sharp and crisp affair with the shadowy image of a masked face barely visible in the background. The clean and simple controls hid no surprises, apart from a rather cute bug-in-gun-sights motif which seemed a little out of place amongst the seriousness shown elsewhere.
The generally well-designed interface did leave something to be desired when I couldn't figure out how to disable the pop-ups warning of on-access detections. A vast swathe of these overwhelmed my machine on one attempt, but eventually the on-access test was coaxed to completion. On demand, the product more than lived up to its name, zipping merrily through speed tests and virus collections, although OLE2 processing was not as impressive as other file types, and detection of some of the more obscure entries in the zoo collections was less than perfect. With nothing missed from the ItW test set though, Quick Heal earns a VB 100%.
Please refer to the PDF for test data
ESET's product had its usual fast and simple installation experience, sprinkled with green-tinged Matrix-style graphics and, at one point, a rather scary-looking eye I hadn't spotted on previous tests. Also along the way was an option to connect to ESET's ThreatSense system, to submit samples of detected malware to its researchers, and also the choice of whether or not to activate the on-access scanner by default on startup. Declining both of these, I played around with the GUI, having fun with separable and reconnectable panes, dragging them around the screen in various configurations only to be a little disappointed by the more standard XP-style of the main scanner. Now familiar with the rather obscure naming system of its modular functions, I found my way around easily, and the product powered through the tests with its usual highly impressive combination of speed and accuracy.
A few wobbles occurred, although my main annoyance, a momentary lingering after quitting from a scan job, would have seemed less noticeable on a product that ran at normal speed. A strange message shown on deactivating some monitors, telling me they would be completely uninstalled on reboot, seemed to have no lasting effect. With splendid and remarkably consistent speed, and irreproachable detection, NOD32 takes another VB 100% award in its stride.
Please refer to the PDF for test data
FortiClient has a somewhat sombre feel; its installation is fast and efficient and its interface grey and simple, light on graphics and heavy on text. The multi-tabbed controls left little to be desired, being easy to navigate and pretty comprehensive, giving me no problems in carrying out the tests. Speeds were very good over OLE2 files, though no more than decent elsewhere, and detection was pleasantly strong across the zoo sets. Just when all seemed to have gone well, checking the logs of the ItW test set showed that an entire variant of one of the newly added file infectors, W32/Looked, was not spotted, either on access or on demand, putting paid to FortiClient's chances of a VB 100% award.
Please refer to the PDF for test data
Next year's version of AntiVirusKit looked as futuristic as its title, with slick and shiny design and graphics, including the red-and-white shield logo, shimmering and glittering from the screen. After the zippy install and a reboot, the GUI itself was just as shiny and funky, with the usual clearly laid out controls given a zing and a fizz of colour. Setup was simple and straightforward, with the option to drop 'Engine A' or 'Engine B' ignored in favour of the default double-barrelled approach. As expected, this scanning style did not produce record times in the speed tests, but accuracy was beyond reproach, with only a 'Joke' in the clean set requiring me to make any further entries in my test notes. GDATA now has another VB 100% award for its trophy cabinet.
Please refer to the PDF for test data
Compared to its neighbours on the test bench, Grisoft's product looked positively dour, its greyish install process enlivened only by the rather useful option to create a rescue disk. The interface itself was also drab and grey and serious and, like many products aimed more firmly at the home user market, used the approach of providing a basic interface for the general user and an advanced one for those who require more specific settings. Tinkering away in here provided me with most of the configuration tools I needed to get through my tests, although when it came to saving logs I had some difficulty, and dumped numerous listings of the on-screen options to file before I discovered that the simpler interface was the way to go. Getting the results of my scans all on one screen enabled me to save them to file, and parsing showed solid detection, along with reasonable if unremarkable speeds. Missing nothing significant, and entirely without false positives, AVG also earns itself a VB 100% award.
Please refer to the PDF for test data
The Kaspersky interface for this product forms a major part of the company's Internet Security Suite, which I reviewed in some depth for these pages a few months ago (see VB, September 2006, p.16), so I expected to have no difficulties with it. With my brain swamped by so many AV products in recent months, it took me a few moments to refresh my acquaintance with the large, fist-friendly GUI, but had it doing my bidding in no time. Installation was very fast, with no reboot required, and testing passed in similarly painless fashion, running over the sets in respectable time and getting the expected impressive results. With the only samples missed being on-access, in file types not scanned by default in that mode, Kaspersky 6 is another worthy recipient of the VB 100% award.
Please refer to the PDF for test data
McAfee's VirusScan product, after 'recomposing' its constituent parts in a rather leisurely fashion prior to install, thanked me politely for making use of it as it set itself up. Once installed, the product was its usual unfussy self, its bare GUI and straightforward layout allowing for fairly simple adjustment of the appropriate options. Tests proceeded without problems, at a decent pace and with reliable detection, the product proving to be more than good enough to earn a VB 100%.
Please refer to the PDF for test data
Norman's product also has a multi-window approach, with various functionality provided by separate areas, but here it seemed somewhat disjointed, with some desired options falling between the gaps. The installation was simple enough, with the friendly green traffic-light man leading the way. Setup, configuration and running of scans was done via various control systems, with some options set globally and others as part of the scan 'task'. Running a scan, a separate window carried the results and hid away in a minimized state if nothing was found, quietly slipping away again at the end if the user didn’t demand to see it. On-access testing was equally fiddly, with unpredictable behaviour forcing me to resort to deletion. Scans were a little slow over some sets, but remarkably fast over OLE2 files, and detection rates were pleasantly regular in both on-access and on-demand tests. Unfortunately this consistency extended to the missing of three samples of W32/Detnat, added to the WildList used for this round of testing, thus denying Norman a VB 100% award.
Please refer to the PDF for test data
Installation of Sophos Anti-Virus was fast and simple, and using the product was equally unchallenging – until the point at which the result logs needed collecting. Configuration of this functionality seems limited in the end-user interface, perhaps moved to some higher level of the administration suite, but these issues were soon circumvented and useable logs acquired (although one Linux server I passed them to for parsing insisted they were in MPEG format). My only complaint apart from this was the progress bar, always more of an art than a science, which here seemed to either rush to 95% and hang around there for some time, or to complete the scan with the bar still on 10%. With its usual solid detection rates, Sophos also receives the VB 100% award.
Please refer to the PDF for test data
Symantec's product was almost ruled out of the game at a very early stage, when the supplied version announced it was not compatible with my processor, and a standby 32-bit version, spotting my swanky hardware, instructed me to install the 'Win64' product which had just brushed me off. On consultation, it emerged that an Itanium product had been provided in error, and I was pointed to the more appropriate AMD64 version, which ran without further difficulty. This product differed little at the user end from its counterparts, and setup and running of the tests was simple and rapid.
Scanning speed was decent, if not remarkable, over the clean sets, but a repeat of last month's issues of extreme slowdown over the infected collections threatened to upset things once more, especially as the deadline for this review drew rapidly closer. However, the problem had been diagnosed by Symantec techs as 'non file-related scanning', and a supplied utility to counter the effects of this got me my collection results at an impressive rate. Detection was even more impressive, and Symantec joins those at the top of the podium, not putting a foot wrong anywhere and earning its VB 100% award with ease.
Please refer to the PDF for test data
Nearing the end of my set of products, and the time allotted to my testing, Trend also presented me with 64-bit-related difficulties. When run on one of the machines set up for this review, the product seemed at first to have frozen during the installation, until switching windows revealed a message box lounging behind the drab green of the installer backdrop, informing me that the product could not be installed on my system. Checking with contacts at Trend, I learned that the 64-bit version could not be installed directly, but must be deployed via the management system, only available for 32-bit hardware. With time ticking by, I hurriedly set up a second machine with a Windows 2000 image from the previous comparative, installed the server product (which entailed, as in the earlier test, upgrading my browser), and from there was able to 'Notify' the client of the availability of a product. This installed via http, with half a dozen messages from the XP security system querying whether I really wanted to install, but with those dealt with I finally had a serviceable scanner.
Much of the administration was also carried out via the server, including changes to on-access settings and access to logs. Speed of scanning was very good, and after a few anomalous sets of results were cleared up by retesting, detection was fairly decent too, though a few sizeable sets of older polymorphic viruses were missed. More importantly, a single sample of W32/Detnat was not spotted in the WildList set, in either mode, spoiling the product's chances of an award.
Please refer to the PDF for test data
VirusBuster, last on the test bench, provided a 64-bit version of its product, but its looks and operation were more or less indistinguishable from other editions. The installation process presented various standard options, including where to install the product and whether to set up a desktop shortcut, before I could 'actualize the anti-virus protection.' I found the layout of the GUI somewhat fiddly, requiring a fairly lengthy process of designing scan tasks and then running them. The product had another rather misleading progress bar, often starting off at around 80%, and took a long time writing out its logs when asked to, but had no trouble with detection and got through the speed tests at a decent rate. Once again, some somewhat flaky results meant a second run over the tests was needed, but in the end VirusBuster proved itself capable of handling the ItW set without problems, and so also earns a VB100%.
Please refer to the PDF for test data
As expected, the test produced some upsets, with the new file-infector viruses causing trouble for several products. With few misses of ItW viruses over the first few months of my tenure here at VB, this proved a bumper crop, with three products failing to cover the whole list accurately, and one missing an entire variant – others missed only some samples, while detecting others spawned from the same source. False positives were less of a problem, after some cleaning out of the clean set, and overall coverage of the zoo collections has also improved almost across the board, since little new material was added for this test. The expected platform issues were limited to some confusion from vendors over which products to submit, and how they could be installed, and were soon overcome with a little investigation and advice from the providers.
Some considerable redesign of the VB 100% testing setup and processes is due, hopefully in time for the next comparative in two months' time. More details will be made available nearer to the time.
Test environment. All tests were run on identical AMD Athlon 64 3800+ dual core machines with 1GB RAM, 40GB and 200 GB dual hard disks, DVD/CD-ROM and 3.5-inch floppy drive, running Microsoft Windows XP Professional x64 edition.
Virus test sets. Complete listings of the test sets used can be found at http://www.virusbtn.com/Comparatives/Win64/2006/test_sets.html
Any developers interested in submitting products for VB's comparative reviews should contact [email protected]. The current schedule for the publication of VB comparative reviews can be found at http://www.virusbtn.com/vb100/about/schedule.xml.
