VB100 comparative review on SUSE Linux Enterprise Server 11

Main version: v3.2.1

Update versions: 121219-1, 130121-1, 130123-1, 130128-0

Avast’s developers were among the first to warn us of potential problems with the dazuko subsystem, but once we informed them that we were happy to change the kernel in use, everyone was happy. The submission was sent in as a 160MB archive, which contained everything needed to roll back the kernel and install the dazuko and redirfs modules, as well as the three RPM packages comprising the product itself. These were installed using the usual method, and everything else was found to be fairly standard, much to our satisfaction: configuration was adjusted using plain files in /etc, the location of the main executables was added to the search path, and ample man pages were provided describing the usage of the product in clear and simple terms. The only action required initially was the adjustment of one config file to point to a licence key, and then everything was up and running. The whole process, including the kernel adjustments and the reboot this necessitated, took only a couple of minutes. Updates were fairly speedy too, rarely taking more than 30 seconds or so.

The product ran without issues, proving just as stable to run as it was simple to operate. Scanning speeds were not bad – very fast indeed over our sets of media and miscellaneous files – with overheads showing a similar pattern. Memory use was low, but CPU use fairly high during busy times, not surprisingly. Detection was decent too – looking a little below par in some of our graphs, but this is entirely thanks to the quality of the opposition. Detection in the Response sets started very respectably and showed just a slight downward trend into the more recent days. The RAP sets were mostly well covered, but a rather sharp blip in the oldest week brought the average down somewhat.

The clean sets brought no surprises, and the WildList sets seemed to be well covered too, until the third of three test runs showed a loss of detection for a pair of samples in the standard set. The issue remained during further checks over the next few days, and was reported to the developers as a matter of urgency. Their feedback implied that the problem was thanks to the aged nature of the Linux product, lagging several cycles behind its Windows cousins (which were unlikely to be affected by the issue). Nevertheless, despite a generally strong showing Avast doesn’t quite make the grade for VB100 certification, putting paid to our hopes for a clean sweep before we’d really got started. Our test history for Avast now shows one fail and five passes in the last six tests; ten passes and two fails in the last two years. Stability was fine this month once again, with a ‘Solid’ rating easily earned.

ItW Std: 99.88%

ItW Std (o/a): 99.88%

ItW Extd: 100.00%

ItW Extd (o/a): 100.00%

False positives: 0

Stability: Solid

Main version: 12.0.1795

Update versions: 2637/5465, 2639/5549, 2639/5553, 2639/5565

Clearly having received some recent attention from its developers, AVG’s product managed to circumvent the problems with dazuko, which remains supported where appropriate but on more recent kernels the fanotify system has been implemented. Installation of the product was fairly straightforward, with a single RPM of just over 100MB provided. This put everything in place nice and simply, but configuration was a little more complex than might be desirable, with a fairly lengthy syntax required to pass even fairly minor changes into the product. Figuring out the right wording was by no means straightforward, but was helped by some comprehensive man pages, and things were up and running in a decent amount of time. Initial updates seemed rather start-stop, taking over eight minutes to run through on the first attempt, but later runs were much smoother, all completing in under a minute.

Operation was generally fairly simple once the initial set up had been completed, and things ran without problems. Scanning speeds were pretty good, and overheads not bad at all either, with a fair amount of RAM use but pretty low use of CPU cycles. Detection was excellent, with consistently high rates throughout our Response sets and RAP scores also starting very high indeed, tailing off rather more than most into the proactive week, but remaining pretty decent.

There were no problems in the core sets and a VB100 is comfortably earned by AVG; the vendor stands at one fail and five passes in the last six tests; two fails and ten passes in the last two years. This was another very smooth run, meriting a ‘Solid’ rating.

ItW Std: 100.00%

ItW Std (o/a): 100.00%

ItW Extd: 100.00%

ItW Extd (o/a): 100.00%

False positives: 0

Stability: Solid

Main version: 3.1.2

Update versions: N/A

Bitdefender’s Samba-focused product also came as a single item, an executable-wrapped RPM installer which set things up simply and quickly, with minimal extra work needed to get everything working nicely. Updates were swift too, but the design shies away from standard practice, using a complex system of commands passed into the main executable to make adjustments rather than the more common process of editing a plain text configuration file. Nevertheless, with a little trial and error it soon became reasonably usable. Help was provided in the form of a set of man pages, but the set up did not appear to have functioned properly so these were not discovered until late in the day, when rummaging through the product’s install directory for some ideas. The main executable was not added to the search path automatically, either.

On-access protection covers Samba shares only, by means of a VFS object pointed to by some tweaks to the Samba configuration file, which are put in place automatically during the set up. Logging was a little verbose but clear enough and pretty reliable, while the basic syntax of the scanner was fairly straightforward once the schema had been worked out. Scanning speeds were on the slow side, and on access overheads perhaps a little higher than some this month, with fairly high use of RAM but low CPU use.

Detection was superb though, continuing a recent string of excellent scores from Bitdefender, with a distinct but fairly minor decline through the days of the Response sets and RAP numbers consistently excellent, even the proactive set very well covered. The certification sets caused no problems, and a VB100 award is easily earned, maintaining Bitdefender’s recent run of success and keeping it on 12 passes in the last two years of comparatives. No serious problems were noted, but the failure of the set up of the man pages was just enough to affect our scoring, meaning only a ‘Stable’ rating is earned.

ItW Std: 100.00%

ItW Std (o/a): 100.00%

ItW Extd: 100.00%

ItW Extd (o/a): 100.00%

False positives: 0

Stability: Stable

Main version: 5.6.1

Update versions: N/A

The developers of eScan provided a fairly comprehensive submission: the four main RPM packages, weighing in at 190MB, were accompanied by a licence file, a dependency list and a PDF user guide. The dependency list proved unnecessary, with all of the requirements already in place on a fairly basic system, and the user guide was only really needed to confirm the order in which the RPMs should be run, and to provide the syntax for changes needed in the Samba configuration file. Man pages were provided for the main executable components, but not for the daemons, however configuration was mostly performed via a browser-based GUI, which proved simple to access and operate, providing ample information and controls for most purposes. Updates were fairly speedy, completing in little over a minute on most runs, although the ClamAV-based part regularly reported issues which could only be resolved by updating core components manually; the Bitdefender part of the product operated smoothly throughout.

Speeds were not too bad and overheads just a little high, with RAM use around average and CPU use fairly low. Detection scores, as expected, were excellent just about everywhere, showing slight declines into the most recent sets but from very high starting points. No issues were encountered in the core sets, earning eScan a VB100 award and putting it on a perfect 12 passes in the last dozen tests. With no stability issues to report, a ‘Solid’ rating is earned.

ItW Std: 100.00%

ItW Std (o/a): 100.00%

ItW Extd: 100.00%

ItW Extd (o/a): 100.00%

False positives: 0

Stability: Solid

Main version: 4.0.5

Update versions: Scanner 7818, 7919, 7926, 7943

ESET’s submission was a nice and compact 60MB single RPM installer, which ran through quickly and simply; final set up of the product included adjustment of just a single line in the Samba resource control script and a few tweaks to the product’s comprehensive and clearly documented configuration file. Support is also provided for dazuko protection if required. Updates were rapid at around two minutes on average.

Operation is simple and conforms to standard practices, with clear and thorough man pages detailing all components and everything positioned more or less where one would expect to find it. Scanning speeds were decent for the most part, blazing fast over our media sets, and overheads were very low indeed, although we could find no option which would effectively activate archive scanning on access. RAM and CPU use were both around average, and detection was solid, with very consistent and highly respectable scores through the Response sets, a good start in the RAP sets and not too steep a decline into the latter weeks.

The core sets presented no problems as usual, and ESET’s epic string of successes continues, with no tests failed or even skipped for many years. No stability issues were encountered, earning the product another ‘Solid’ rating.

ItW Std: 100.00%

ItW Std (o/a): 100.00%

ItW Extd: 100.00%

ItW Extd (o/a): 100.00%

False positives: 0

Stability: Solid

Main version: 9.20.2520

Update versions: 2012-12-19_07, 2013-01-22_06, 2013 01-24_06, 2013-01-29_04

The F-Secure product was provided as a slimline 66MB installer, although this lacked updates and had to be installed on the deadline day to take snapshots of the latest definitions for the RAP tests. The set up process involved unpacking an archive and running a script contained within it. This seemed to run smoothly, with no major alerts, but on closer scrutiny of the scrolling output, some errors were observed in compiling the dazuko and redirfs modules. Knowing these were not compatible with the kernel in use, we set up a second system with an older kernel. Here, there were no issues and on-access protection operated well, but on the original machine the product’s browser-based GUI seemed unaware of any installation issues, insisting that protection was fully functional. We also saw some error messages concerning an alternative GNOME desktop interface, which seemed unable to run properly.

With the initial confusion ironed out (the developers informed us that the platform version we were testing on is not officially supported, which makes the problems we noted rather less severe), things moved along nicely. The GUI provided decent levels of information and configuration, and the syntax for the command-line scanner was clear and sensible. Scanning speeds were a little underwhelming, but overheads were mostly decent, although a few sets did slow down considerably when the settings were turned up to the most thorough. Use of memory was very low throughout testing, but CPU use was well up at busy times.

Detection was solid, as we would expect from the Bitdefender engine included in the product alongside the company’s own work, with splendid scores everywhere. The core sets were brushed aside comfortably, and a VB100 award is earned. F-Secure has a somewhat erratic pattern of submissions of late, and now stands on three passes and one fail from four entries in the last six tests; eight passes and just that single fail in the last two years. There were a few wobbles observed, including the erroneous assertion that protection was in place when it was not, meaning a stability rating of no more than ‘Fair’.

ItW Std: 100.00%

ItW Std (o/a): 100.00%

ItW Extd: 100.00%

ItW Extd (o/a): 100.00%

False positives: 0

Stability: Fair

Main version: 8.0.2-160

Update versions: N/A

Of all the products in this month’s line up, Kaspersky’s inspired the most dread among the test team thanks to difficulties encountered in previous Linux tests. The initial set up, from a single 57MB RPM file, was fairly painless and speedy, with a set up script provided to do the work of compiling and implementing on-access protection, licensing and so on. The on access component appears to be a custom offering loosely based on the redirfs approach, but there were no problems with getting it set up, and options to protect Samba shares explicitly were also provided. Once these first stages were complete, the work of figuring out the configuration and operation process began in earnest, first of all to apply the updates downloaded separately for the RAP tests. This involved lengthy perusal of an epic 178-page user manual (no man pages are provided), and considerable trial and error as the details of adjusting the update process were less than perfectly clear. With no configuration file as such, all changes to settings had to be performed by getting the product to dump the settings of the module in question to a file, tweaking that file and then reading it back in again. In most cases this took some time and a fair amount of brain power. After a while it all became reasonably intuitive, once the basic processes were memorized, but mistakes still occurred disappointingly frequently, meaning testing took far longer than for any of the other products.

In addition, we had some problems getting through the RAP tests, with scans crashing out with errors which hinted at problems in processing log data. This was confirmed by the developers, who provided an alternative syntax to the scan command. This proved more effective, but only after much extra work had been put into trying to nurse the scanner through the sets. In the end, we saw some fairly slow scanning speeds, especially over our set of archives (which were scanned in-depth by default) and the set of Linux files, which included a large number of small files. On access overheads were excellent though, with low measures across the board, and archive handling still very reasonable when settings were turned right up. RAM use was around average, but CPU use distinctly high.

Detection rates were very good, remaining fairly steady through the Response sets, actually improving in the latter parts, and starting very well in the RAP sets, although dropping quite sharply into the proactive week. The core sets were dealt with properly, and a VB100 award is well deserved, putting Kaspersky on six passes in the last six tests; longer term things are a little more shaky, with nine passes and two fails in the last two years. There were a few fairly minor problems this month, which only occurred under heavy stress, earning the product a ‘Stable’ rating.

ItW Std: 100.00%

ItW Std (o/a): 100.00%

ItW Extd: 100.00%

ItW Extd (o/a): 100.00%

False positives: 0

Stability: Stable

Main version: 7.0.20

Update versions: 16375682, 17218629, 17285786, 17472528

Norman’s Linux solution was submitted as a tiny 6.4MB main installer, with 170MB of offline updates for the RAP sets. This hefty update size was reflected in standard installs, which seemed to complete very rapidly but required considerably more time doing background set up tasks, much of which seemed to revolve around downloading and implementing updates. Indeed, on at least a couple of installs, the product remained non functional even after leaving it overnight and giving the entire system a reboot. Most proved reasonably successful after a decent wait though. Configuration was mostly done through another browser GUI, which closely resembled the company’s Windows products, making for a pleasant continuity of usage patterns (at least when the GUI could be persuaded to open properly).

On-access protection is provided using an unusual FUSE based approach, which worked fine with the default layout, but we were unable to extend this to cover additional areas. As a result, all our on access tests had to be re-purposed to operate in the /home directory – the only area which seemed to be protected. On demand scans were a little slow over archives, Windows binaries and Linux files, but reasonable elsewhere. On access overheads, on the other hand, were fairly heavy for the most part but huge over the set of binary files, with the product’s sandbox component clearly adding heavily to the time required to mark files as safe. Some sort of caching of recent results appears to be needed, as much time was spent re-analysing items that had been checked only a few minutes earlier. RAM use was fairly high, and CPU use a little above average too.

Detection was very good though, at least over the older sets, with a fairly sharp drop in the second halves of both the Response and RAP tests. The core sets presented no issues however, and Norman earns a VB100 award. The company now boasts a fairly steady five passes and one fail in the last six tests; ten passes and two fails in the last two years. There were a number of install problems this month, most significantly with trying to apply on-access protection to non-default areas, meaning only a ‘Fair’ stability rating.

ItW Std: 100.00%

ItW Std (o/a): 100.00%

ItW Extd: 100.00%

ItW Extd (o/a): 100.00%

False positives: 0

Stability: Fair

Main version: 4.84.0

Update versions: 2.03.038/3.38.1/4.84, 3.39.0/4.85

The last of this month’s rather slender field, Sophos provided its solution as a 255MB bundle, with separate updates of 6MB. The install runs from a script inside the main archive package, which ran through the set up, licensing and initial configuration in a clear and simple manner. On access protection from the company’s own ‘Talpa’ service compiled automatically and without problems, and updates were zippy after some initial confusion over an expired licence code. A web admin tool is available but mainly provides information, with most configuration performed by passing commands into the main control binary – a process which is a little fiddly, but reasonably easy to work out thanks to the copious man pages provided. One adjustment, disabling the ‘live protection’ cloud look up system for the RAP tests, could not be found in the official processes but was simple enough to do by editing an XML config file.

Scanning was very fast indeed, at least with the default settings, slowing down considerably when settings were extended to cover all file and archive types. Overheads were very light, although it should be noted that the on-access mode is limited to exclude archives. RAM use was average and CPU use very low.

Detection was a little below par in the RAP sets, looking further behind the rest than one might expect thanks to an exceptionally strong field this month, but in the Response sets scores were excellent, demonstrating the effectiveness of the cloud system which should be in place in most cases. The WildList and clean sets were handled without incident, and Sophos is a worthy winner of another VB100 award, putting it on five passes and one fail in the last six tests; ten passes and two fails in the last two years. Stability was excellent this month, earning a ‘Solid’ rating.

ItW Std: 100.00%

ItW Std (o/a): 100.00%

ItW Extd: 100.00%

ItW Extd (o/a): 100.00%

False positives: 0

Stability: Solid