IPv6 mail server whitelist declaring war on botnets

Although estimates differ between sources, around 95% of all email traffic currently consists of spam. Despite there having been some decreases in spam volumes recently (see http://www.virusbtn.com/virusbulletin/archive/2011/07/vb201107-news1), we are not likely to see a significant drop in spam levels any time soon, as spammers still earn a lot of money from their activities. Highly organized gangs operating from numerous countries make a professional living from sending spam and invest serious amounts of money and resources into their businesses to remain on top of their game – just like any legitimate industry. Since spam is a global problem, it can be difficult to track down and take legal action against these gangs.

A large proportion of spam is sent out by botnets. A botnet consists of a network of many infected computers that are controlled by a ‘bot master’ and may be used for any type of online crime, including sending spam. Botnets are generally made up of home computers whose owners do not realize that their machines have been infected and are being used as part of a botnet. Such infected machines can send out thousands of spam messages per day, and until the malware is cleaned from the computer, or the machine is disconnected from the Internet, it will continue to send spam via the control of the bot master.

Within the security industry, the spam problem is tackled in a number of different ways. For example, anti-virus companies provide software to detect the malware responsible for turning computers into bots, and firewall providers attempt to identify and block suspicious traffic coming from the computer. Anti-spam companies, meanwhile, resort to different methods to try and stop these bots from delivering spam.

Each computer on the Internet is assigned a unique number which is used for all Internet communication, the so-called IP address. When a computer visits a website or sends an email, the IP address is revealed to the destination server. Anti-spam companies monitor the activity of these IP addresses, and if they suddenly detect a stream of spam from a particular address, they add it to an IP blacklist. All email from that IP address will then be blocked, stopping the flow of spam to the recipient server. Removal of the IP address from the blacklist must be requested manually once the spam issue has been resolved.

There are a few problems with this method. First, spam has to be detected before the system can make a proper judgement as to whether or not to block the IP address. Spammers often send out small bursts of spam messages to try and keep the volume below the threshold that would trigger such a listing. Secondly, spammers can keep infecting new machines to gain access to new IP addresses which have not yet been listed.

In total there are around four billion IP addresses in IP version 4. Because there are an increasing number of devices on the Internet in need of an IP number, this pool of addresses is rapidly running out and will soon be exhausted. To get around this problem, a new version of the numbering system (IP version 6) has been introduced. To avoid running out of IP space again, this new standard will create a pool of approximately 340 undecillion (2¹²⁸) addresses. It is hard to comprehend such an enormous number, but to give an idea, it’s greater than the number of stars in the sky.

Thanks to the introduction of IPv6, spammers will have access to a much larger pool of unique IP addresses, making it almost impossible for anti-spam companies to maintain useful blacklists. It will be a lot harder to accurately stop spam at an early stage, because there will be too many different IP sources from which spam can be delivered. Blacklists will grow too large for computers to handle efficiently, and spammers will be able to switch to a new address as soon as the current one gets blocked.

The not-for-profit project ‘IPv6whitelist.eu’ was founded in 2010 in The Netherlands by Dreas van Donselaar (SpamExperts), Ruud van den Bercken (XS4ALL Internet/Stay-Secure) and Raymond Dijkxhoorn (Prolocation/SURBL) to try to solve the quantity problem IPv6 introduces. Until now the mechanism has been to assume that computers don’t send out spam, and then to blacklist them when they do. The Ipv6whitelist.eu project, however, assumes that all computers send out spam, unless they have been registered on the list. All IPv6 addresses are simply blacklisted unless they appear on the whitelist – addresses must be added to the whitelist manually via a simple web form.

The project is controversial because it goes against the openness of the Internet by obliging mail server administrators to register in a central database before sending out email. The situation is turned around and instead of the recipient deciding whether or not to accept email from a specific system, the sender is now obliged to specify that he/she would like to send email from a specific system.

The initiative will only succeed if sufficient recipient mail servers enforce the requirement for senders to join the IPv6 whitelist. If not enough recipients enforce the rule, senders will simply ignore it and not bother registering their mail servers. The project currently only applies to IPv6 addresses assigned to the Netherlands. Thanks to the close collaboration of many IPv6-enabled access providers and web-hosting companies in the Netherlands, a critical mass of enforcing recipients has quickly been established, ensuring that IPv6 senders are forced to comply.

Email from any mail server in the Netherlands which is not yet registered to the central database is automatically temporarily rejected by recipient mail servers until the sending server has been registered (free of charge) via the API or website. More often than not, unregistered servers are hacked computers which are being used to send spam without their owners’ knowledge.

In the long term, we foresee a significant reduction in spam originating from the Netherlands. Because this is a completely cost-free system, there has been little resistance from the market – people understand that the small inconvenience of having to register their mail servers resolves a major issue on the receiving side, keeping incoming spam under control.

The system is vulnerable to abuse though, since spammers could simply start registering their mail servers on the list as well. Besides verifying that the registration has been made by a human, there is no further control or judgement on an IP whitelisting. The IP netblock owner does have the option to delist certain IPs, if required. However, we do not envisage a problem if spammers start registering IPv6 addresses – even if there are millions of bad registrations that is still a very small number compared to the overall IPv6 pool. Thanks to that reduction, anti-spam companies can easily keep track of the reputation of sending servers as they currently do.

At the moment the volume of spam is so high that anti-spam companies will continue to play a vital role. The initiative will ensure that the problem remains manageable, not only now but also in the future.

All IPv6whitelist.eu software, APIs, systems and data are open to the public. There is no commercial incentive and the association is run by volunteers. Since the rollout of IPv6 has only just started, the effect of the project on live mail streams is currently minimal. However, because of the early launch, easy adoption on the recipient side has been ensured, and it is hoped that many more countries will either join the project or launch similar initiatives. A critical mass on the recipient side is the only requirement to be able to force senders to make changes to their sending behaviour – and there are no technical limitations or restrictions involved in the registration process, meaning that there are no barriers to making this a new standard requirement for email senders.