Darknet monitoring

The Internet has faced a sustained and significant threat from network malware since the emergence of the global Windows network worm in 2001. For instance, in September 2001, the Nimda worm disrupted global BGP routing tables for hours at major Internet peering points, and in January 2002 the SQL Slammer worm caused significant outages and slowdowns. DDoS attacks directed at the root DNS servers, most recently in February 2007, were launched only with the help of Internet-scale malware and botnets.

Discovering this malware and other malicious activity is key to any global monitoring approach, especially an early warning system. Honeypots are often an excellent source of data, but they rely on the attacker encountering the system. By the same token, absorbing all of the data by active client collection techniques simply doesn’t scale. Clearly a balance must be found, one that can be used to highlight possible new sources of activity.

A new approach to monitoring malicious Internet traffic on a global basis is to utilize darknets – unallocated IP space owned by service providers or enterprises. This is the portion of the Internet that has not been assigned to customers, and every network has some portion of darknet space. Because it is unallocated, there are only two reasons for traffic to be going into darknet space: either due to some form of misconfiguration or because it is malicious. Darknets have little background traffic, meaning that the data captured is purely signal and can be analysed easily. A key facet of any darknet is that it is globally routed and reachable, so a host anywhere on the Internet sending traffic to it will register from any source.

Darknets work because network-scanning malware is unable to predict which addresses are in use on every level of the Internet. Bots and malware are not intelligent enough to pick and choose where they go, they will simply attempt to spread to as many hosts as possible. Monitoring darknet traffic yields great visibility into what threats are present.

By some estimates, only about one third of the Internet is in active use at any time, counting from the subnet that is DHCP allocated up through the BGP allocations given by organizations such as ARIN. This leaves tremendous room for darknets to be deployed.

Over the years, we have seen only a handful of Internet worms try to avoid the largest of darknet monitors by carrying a list of networks to scan. This didn’t work as well as the authors had hoped, and since then very few other malware authors have tried this. The bulk of bots and malware these days use ‘island hopping’ strategies to bias their scanning and attacks locally, either by hardcoding such an algorithm (first made popular with Code Red II and Nimda) or through botnet scan commands focusing on the local networks. Even in these cases darknets observe the malware due to the sparseness of IP address assignments on the local network.

Multiple levels of data can be analysed in darknets, including NetFlow-based approaches through honeypots. At the NetFlow level, routers and switches show what traffic is destined to a darknet by generating traffic summaries called ‘flows’. This provides a lightweight data representation of the traffic by omitting payloads and aggregating packets into a single flow record. These records are great for trend analysis and useful in analysing global scan patterns. Packets can be collected, which, in combination with a honeypot system, can be used to discover the nature of the attack and provide further characterization.

Darknets are the network equivalent of email spam traps, dummy IM accounts and other such data collection points. The major difference between a darknet and a typical honeynet, however, is the scale of data collection. Darknets are composed of hundreds of addresses, rather than one or two hosts. This means that data analysis techniques must scale up dramatically, focusing on trends and patterns instead of deep specifics.

At Arbor Networks, we have found that a distributed darknet monitoring system provides global visibility into malicious traffic and probes. The scan and attack patterns indicate the prevalence of bots and malware and a network of sensors collects new malware samples continuously. Because of this, there is usually an indication of a large-scale attack before it impacts customers dramatically.

Latest articles:

Bulletin Archive