Unveiling the kernel: rootkit discovery using selective automated kernel memory differencing

Friday 26 September 14:30 - 15:00, Green room.

Ahmed Zaki Sophos
Benjamin Humphrey Sophos

This paper is available online (HTML, PDF).

As an increasing number of automated malware analysis systems become mainstream, the emphasis on the relevance of the data extracted from the analysis task increases. Conceptually, automated malware analysis systems provide information about a sample and also identify modifications caused by the sample to a computer system. Traditionally, the focus of such systems has primarily been on monitoring process, disk and network-level behaviour with varying levels of granularity. While offering a varied set of information, these systems offer limited ability to identify and classify rootkits. The very nature of rootkits makes it hard to classify, and in some cases even detect a sample using these scanning techniques. Kernel memory modifications can indicate that samples are trying to conceal information or hijack execution paths, thus exhibiting malicious behaviour. In an environment with a large throughput of analysis jobs, a need arises for an efficient and accurate way to identify these complex threats that could otherwise be misclassified or pass unnoticed.

We present a system for identifying rootkit samples that is based on an automated analysis system, set in a cluster environment. In this system, we recognize the performance and memory constraints of a high throughput environment; instead of monitoring modifications to the whole memory, we capture changes to data structures and memory regions that, on a Microsoft Windows operating system, have been known to be targeted by rootkits in the past. We explain the reasons behind our design decisions and how they have reflected on identifying different classes of rootkits. In our research we also explore the effectiveness of using this model as a standalone component to identify malicious behaviour. To answer this question we run a large set of known clean vs. malicious files to identify traits that could be indicative of malicious activity.

Ahmed Zaki

Before joining Sophos, Ahmed briefly worked as an incident handler for the Egyptian national computer emergency response team (EG-CERT) after completing his Master's degree in information security at Royal Holloway, University of London. His interest in rootkit research started with his Master's, when he wrote his thesis on the diverse rootkit techniques while assessing methods of detection and prevention available at the time. Ahmed joined Sophos in 2009 as a threat researcher, where he spends most of his time reversing rootkits with the objective of writing detection and remediation routines for them. His efforts concentrate on tackling detection and remediation for kernel-mode rootkits and bootkits. More recently, he has also become interested and involved in developing systems for behavioural analysis automation. Ahmed enjoys playing badminton and reading in his free time.

Benjamin Humphrey

Benjamin started life as a black-box software tester. After four years of mindlessly breaking software, and with no real formal qualifications to his name, he enrolled in a foundation course at Oxford Brookes University in 2006 where he went on to attain a BSc in computer science. In his final year of study, Ben developed a penchant for computer security and, after graduating, sought to further his knowledge at the University of Kent with an MSc in networks and security. Upon completion of his Master's, Ben joined SophosLabs, where he has been for the past two-and-a-half years. Since joining Sophos, Ben has acquired and developed his tools of the trade as a malware reverse engineer. After trifling with various forms of malware he developed an infatuation for rootkit analysis and remediation. Now, Ben is happiest when hiding far away, deep within the windows kernel, reams of WinDbg script displayed on a vast array of monitors, and ambient electronica soothing his ears. When Ben is not working, you are likely to find him playing darts for his local pub team or hanging off the Oxford Brookes bouldering wall.