VB100 February 2009 - Red Hat Enterprise Linux 5.2

Alwil’s avast! product for Linux arrived as a trio of RPM packages, one of which included an attempt to adjust the crontab scheduler to automate updates. This seemed to be taking some time, so was aborted, but some initial tinkering with the product found it still to be inactive. The instructions provided by the developers revealed that this was not the result of our impatience, but rather the requirement for a licence file, provided along with the submission but which needed to be copied manually into the appropriate location, as indicated by a configuration file.

With these initial tasks complete, running the product proved straightforward, with the syntax of the command-line scanner a little esoteric but clearly laid out in the accompanying instructions. On-access scanning was similarly straightforward to administer, via standard and lucid configuration files, and everything ran pretty smoothly. Scanning speeds were quite excellent, both on access and on demand, and detection rates at their usual exemplary level.

The RAP scores showed a slight dip in the earliest week – which, logically, one would expect to have the best coverage, but the coinciding holidays in many territories may have affected the throughput of labs in this period. More in tune with predictions, a second dip was observed in the ‘week +1’ set compiled after update freezing, but detection remained pretty solid over these likely unseen samples.

Getting back to the VB100 certification requirements, with no trouble at all handling the diminished WildList set and not a whisper of a false positive, Alwil takes the first VB100 award of 2009 with considerable style.

Avira’s product came in the form of a single .tgz file. This raised some concerns initially, but on extracting, the file proved to contain an install script which performed all the necessary setup steps clearly and simply, with a series of simple questions allowing basic user configuration. With the locations of the licence file and dazuko module provided as part of the setup process (the dazuko module is developed and maintained by Avira), things were up and running in no time, and after perusing another well-documented, but again slightly eccentric set of command-line qualifiers, testing zipped along nicely.

Speed was highly impressive on demand, but on-access scanning seemed a little sluggish by comparison, and little difference was noted in speeds when archive scanning was activated (not the default setting). Indeed, it seemed the same kind of analysis was being performed – it took a considerable time to get through the control archive test set (consisting of the EICAR test file embedded at different depths inside a variety of archive formats) on access, without any detection being made, and not noticeably longer when full scanning was activated and access to the test files was correctly denied.

Detection rates were again superb, with over 90% across the board in all RAP sets including the ‘week +1’ set.

No detections were missed in the WildList set and no false alarms were generated in the clean sets, thus Avira starts 2009 with a VB100 award and great respect.

ESET’s long-standing dominance in VB100 testing has been challenged of late, both in terms of speed and detection rates, by some strong up-and-comers, with two of its most pressing rivals having already appeared in this month’s review. Installation of the product was in the form of a single, straightforward RPM package, with control of the program via a centralized configuration file and thorough, well-documented options to the main binary. The default settings were pretty thorough, covering all file types and a wide set of archive types, and speeds in both modes were as excellent as experience has led us to expect.

Detection rates were similarly strong – perhaps a fraction behind the excellent performers seen so far in the RAP tests, but close enough to put down to sample selection anomalies at this early stage. As usual for ESET, false positives were absent despite the product’s strong heuristics, and the set of WildList samples presented no problems, thus ESET continues its excellent run of success with another VB100 award and a performance worthy of respect.

Frisk’s F-Prot has shown itself in recent Windows comparatives to be the champion of pared-down, no-fuss protection, and here, once again, is an extremely basic piece of anti-malware kit – and none the worse for it. Installation consisted of little more than extracting an archive containing the required files, which could thus be located wherever the admin desires with relative ease. A simple install script is also provided to set up default paths to binaries and man pages. An initial problem was encountered when the submitted product turned out to be a simple workstation version with no on-access component. However, this issue was soon circumvented by grabbing the server version – available as a free trial download from the vendor’s website – and snaffling the required on-access components for interaction with the dazuko module, which proved more than adequate to provide the full level of protection.

As expected, given the pared-down nature of the product, speeds and overheads were exemplary. Detection rates were decent across the three reactive RAP sets, and in the newest set quite remarkable, producing somewhat eye-opening results. Further investigation showed that a fair proportion of the detections recorded when parsing the results were in fact rather vague – many of them being labelled simply ‘security risk’ or even ‘possible security risk’. Such detection labels would not be counted as false positives in the full test, so it was somewhat difficult to decide whether they should count as full detections in this case, but with time pressing and much of the processing of results already completed, we had no choice but to leave them in.

Moving on to the VB100 certification requirements, the WildList was once again covered without issues, but in the clean sets a single file from this month’s addition of web browsers and associated tools was erroneously flagged as a backdoor. While there is potentially some scope for the item in question – a cookie management tool – to be abused, the alert was judged sufficient to deny Frisk a VB100 award this month despite the product’s otherwise solid detection rates.

F-Secure’s product presented a much more professional aspect, with a .tgz file containing the required components alongside a thorough install script which leads the installer through all the required steps to get the product set up. This includes its own copy of the dazuko module – something which none of the other products so far have provided (despite requiring it for their on-access protection). It also comes with an attractive web-based interface, which plants its own desktop icon and provides configuration for much of the product. Along with numerous components in the init directory, a range of additional utilities are provided for the configuration and operation of the product, which provides a full protection suite, including firewall, alongside the standard anti-malware protection.

As expected, once the vagaries of the command-line interface had been decoded, helped along by clear documentation, detection rates were pretty solid, although less than perfect on some of the new families of polymorphic viruses. Scanning speeds were somewhat leisurely – which can partly be explained by the multiple engines in use by the product, which appear to contribute strongly to the depth of detection. Even using the default on-access settings, which ignored most archive types entirely, speeds were notably slower over the clean speed sets than for some of the other products. However, detection rates in the new RAP tests were once again excellent, with a notable dip in the ‘week +1’ set containing samples unlikely to have been seen by labs. For Windows users, F-Secure has made much of its additional ‘Deepguard’ protection with additional cloud-based black- and whitelists (which we have yet to be able to properly test under the requirements of the VB100); whether this layer is available for Linux users was not made clear.

Overall, the product’s performance was nothing to be sniffed at, with the WildList test set covered without a glitch, and the cleanliness of the clean sets was only called into question by a ‘potentially unwanted’ alert on the same file as was described as a backdoor by the Frisk product. As this is allowable under the VB100 rules, F-Secure earns a VB100 award, and extra praise for its solid and lucid design and usability.

Kaspersky’s Linux range has in the past eschewed the popular dazuko path in favour of the ‘Samba vfs object’ method, functioning only on file systems shared via Samba. In previous tests Kaspersky has proved to be one of the few vendors to utilize the technology to its full efficiency. This time, however, the vendor seems to have moved on to its own in-house technology, implementing full on-access detection without the need for any external software. Installation takes the form of an RPM, with a perl script to be run post installation to perform the necessary setup steps.

Operation of the product was less well streamlined, with lengthy commands needing to be issued from the command line and somewhat unpredictable syntax. Scanning speeds were not the fastest, but detection rates were at their usual solid level. The product showed another splendid performance in the RAP tests with, as was predicted for all products, a slight decline in the ‘week +1’ set. The diminutive WildList yet again presented no difficulties, and false positives were absent, thus earning Kaspersky Lab another VB100 award.

McAfee’s Linux product has, in previous comparative reviews, reflected the company’s reputation for professionalism and seriousness. Here, that impression was bolstered, with a broad collection of PDF-format documentation needing to be read before the required installation order of the RPM packages could be ascertained. This done, and after a standard array of installation questions, the product was quickly up and running, with administration performed via a fairly clear and thorough web interface.

A few changes have clearly taken place since my previous encounter with the product however, and a chink in the company’s armour emerged when it became clear that these changes had yet to filter through to the online documentation, knowledgebase and even the staff submitting the product. The update method – admittedly not the standard online method, but one certain to be preferred by many Linux systems administrators who may be running their servers behind all kinds of protective barriers – had been adjusted with a recent iteration of the product, rendering previous techniques ineffectual and online instructions inaccurate. Eventually, after much discussion with tech support personnel, the problem was diagnosed and the correct form of updates provided, albeit not quite the freshest possible from the submission date. A more accurate set of instructions was also provided, and testing continued.

The running of on-demand scans required the use of the interface from which scan ‘tasks’ could be designed and run; these same tasks could also be kicked off from the command line, allowing for some scripting and the use of the standard cron scheduler. However, the lack of ability to configure the tasks from the bare console, even to the extent of providing a scan target, seemed a rather glaring omission which the diehard command-line-loving Linux administrator may find hard to forgive.

Once the scans were set up and run, scanning speeds were surprisingly good, overheads not too heavy, and detection rates in the standard test sets reached the expected level. In the RAP tests, scores were generally pretty good, with that telltale dip in the ‘week +1’ set demonstrating the superior performance of signature detections over heuristic and generic methods, but a worthy performance nevertheless. As far as certification requirements were concerned, nothing was missed in the WildList, but in the clean sets a single file – which has been included in the set since the summer of 2007 – was alerted on with a generic trojan identification, thus spoiling McAfee’s recent run of success and denying the vendor a VB100 on this occasion.

Quick Heal’s product is another dazuko-based setup, with a nice, simple installer inside a .tgz file which, for once, utilizes colour to improve clarity and ease of use. With the setup completed quickly and easily, a proper desktop interface was another pleasant surprise, but although easy on the eye it provided little in the way of in-depth configuration. Some was available in more traditional configuration files, but even here some functions, such as enabling of archive scanning on access, seemed impossible.

Of course, all of this helped with Quick Heal’s famous speediness, which was once again up there with the best. Although detection rates were a little behind the best scores recorded so far in this review, in most sets they were decent, and the WildList presented no problems. With no false positives either, Quick Heal reaches the required standard and earns another VB100 award.

Another company with a solid reputation in the enterprise market, Sophos also ignores the availability of the dazuko module and goes for its own in-house technology to provide on-access scanning. The product installs simply and smoothly on this platform, which is a prime target as one of the leading Linux setups in use in enterprise. The absence of any recompilation, dependencies or other fiddly tasks counts strongly in the product’s favour as far as initial installation goes. Post-install operation is also something of a breeze, with a well-documented and pleasantly usable product. Alongside the standard command-line operation and configuration files, another web interface is provided, which, again, is very well laid out and simple to use.

Scanning speeds were similarly pleasing, particularly with the default settings, and lag times were among the best on offer. Detection rates across all test sets left little to be desired, with a few misses in the polymorphic and trojan sets more than made up for by an excellent showing across the new RAP sets, although the dip in the ‘week +1’ set was perhaps a little more pronounced here than elsewhere. With no false positive issues and nothing missed in the WildList set, Sophos comfortably achieves a VB100 award.

Somewhere in the deeper circles lies a special hell, where testers who have devoted their lives to the more unforgiveable sins must forever wrestle fruitlessly with the Symantec Linux product. After two previous encounters with the product, and despite being given some insight into its intricacies by a gifted support engineer, it remains opaque and bizarre. The company’s support forums are littered with desperate cries for help and simple requests for an explanation of the Machiavellian layout of the configuration process, while the accompanying documentation provides only the vaguest outline of how the controls actually work.

For those happy to go with the defaults, perhaps things are not so bad. The install process consists of a batch of RPM packages along with setup instructions buried in a PDF, and once these have been followed the product is quickly up and running. An interface is even provided, but here there is little more than a summary of the product’s version information and running status, as well as a button marked ‘update’. Manual updating is also possible, with the definitions provided in the form of a self-extracting install file. On the test platform, this required several extra packages to be installed to support its extraction processes, but with these tasks carried out it worked without a hitch. The product was rendered fully operational, including the on-access scanning provided by the company’s own technology, fairly easily.

It is only when the default settings must be changed that things become difficult. The configuration is not stored, as is standard in Unix/Linux, in a nice, humanly readable and easily adjusted configuration file. Instead, a database in the style of the Windows registry is used, and any changes must be passed into this using a dedicated configuration tool. This tool responds equally blankly to both accurate and errant attempts to render the lengthy, syntactically complex commands required. Frequent rechecking of the full list is a must to ensure the proper changes have been made, while documentation of the numerical codes representing such options as on-detection actions seems non-existent.

With the required tweaks assumed to have been made, the process of running command-line scans is a little less arduous, but by no means straightforward, and is similarly lacking in any form of feedback from the product. An option was found which would at least retain control of the command line, returning it when the scan completed, which enabled speed tests and monitoring of progress without recourse to checking the logging. This took the form of complex, barely readable output via the syslog facility, and in the main proved sufficiently usable to produce the required results.

In the RAP tests, eccentric and uneven figures hinted at a possible error in the multi-stage process of extracting information from the multi-record-per-line confusion of the logs, and a retry did produce different, but similarly erratic, results. In a slight bending of the VB100 ‘three attempts’ rule, the scan was run multiple times and detections for each scan, varying by up to 10% each time, merged together to produce the final figures displayed here. It seems more than likely that this may not reflect the true detection capabilities of the product – for which I can only apologize to the vendor, but it was the best that could be achieved under the trying circumstances. If a more accurate way of procuring results can be found, we will strive to achieve it and update these figures – watch this space.

On a happier note, scanning speeds were pretty good and detection rates in the standard sets very good, with no difficulty handling the WildList and no false positives; a VB100 is duly awarded.

Update - 4 February 2009: Following some deeper investigation, the cause of the anomalies noted in our original report has been uncovered. After some initial difficulties obtaining usable data, an error was introduced into the scripts used for parsing results. This, regrettably, led to a level of categorization being skipped. In all the data recorded in VB100 comparative reviews, we endeavour to group samples together where they represent a single item - an example would be a polymorphic virus where several hundred samples are included in our test set, but treated as a single entity. In the RAP test sets this month, several groups of trojans were also clustered in this manner - large numbers of samples displaying identical properties despite each sample presenting a unique MD5 hash. An adjustment made to the data processing had the result, in Symantec's case, of ignoring two such groupings and treating each of the samples as separate items. As the undetected samples in the groups numbered several hundred in each case, the change significantly skewed Symantec's RAP scores. These have now been recalculated, with the offending sample sets properly classified. VB extends its sincere apologies to Symantec for this misrepresentation, and will, of course, endeavour to ensure such classification errors are eradicated in future.

Figure 1. Original (incorrectly calculated) RAP graph.

Figure 2. Corrected (true) RAP graph for Symantec.

The final product on the test bench brings what I had expected, based on past experience, to be the main rival to the dazuko setup in terms of on-access file-hooking: the Samba vfs object. VirusBuster’s product provides a series of .tgz files with an install script, making the installation reasonably straightforward despite a few rather vague passages of text. The setup of the Samba protection must be done manually, with instructions provided, but a slight inaccuracy in the guidelines led to the Samba share in question being rendered completely inaccessible – safe from malware perhaps, but hardly the ticket. A small tweak soon had things operational however, and testing continued.

Scanning speeds were fairly decent, and overheads pretty good too, but detection rates lagged a little behind this month’s very strong field. Logging proved a particular issue, with complex multi-line logs not the most comfortable to parse – for Linux administrators with large amounts of text-based data to handle, such inelegancies weigh heavily, just as they do for testers. However, after a period of hair-pulling and text-mangling, usable results were obtained, showing some fairly decent scores in the RAP sets, which improved considerably when the non-default ‘grayware’ option was enabled. Moving on to the VB100 certification requirements, the WildList was once again covered thoroughly and with no false alarms generated in the clean test set, VirusBuster is awarded a VB100.