Posted by Virus Bulletin on Apr 15, 2014
Collateral damage in instruction to reject emails with invalid DKIM signatures.
A change in Yahoo's DMARC policy has caused frustration among operators of many mailing lists and their subscribers.
On its official website, DMARC is described as standardizing "how email receivers perform email authentication using the well-known SPF and DKIM mechanisms". It was set up in 2012 by some of the bigger players in the email field, including Yahoo, Gmail, Facebook and PayPal.
Both SPF and DKIM are used primarily to authenticate email and to help ensure its correct delivery: SPF can confirm that the mail was sent from one of the mail servers belonging to the sender's domain, while DKIM adds a domain-based signature to the email. If an email was sent from a mail server belonging to example.com and signed with example.com's DKIM signature, it is very likely that it was indeed sent by example.com and not by someone spoofing that domain.
Although neither standard makes a statement on whether authenticated emails are spam, when used together with domain-based reputation, the standards can help with the correct delivery of emails.
What DMARC adds to DKIM and SPF is to give domain owners a say over what receivers should do with emails that claim to have been sent from their domain, but which fail both SPF and DKIM checks. Depending on how much they trust their own records and how important they consider the risk of phishing compared with the risk of legitimate emails not being delivered, they can tell receivers to accept the emails anyway, to turn up the filters, or to reject the emails.
A recent survey among participants of our VBSpam tests showed that very few spam filters are currently checking the DMARC status of emails. Yet its adoption by many of the bigger players means that DMARC cannot be ignored in the current email landscape.
Yahoo recently changed its DMARC policy to address a security problem. But, as John Levine has pointed out, in doing so it broke every mailing list in the world.
Yahoo apparently saw many phishing emails that forged the From: address to appear to have been sent from its servers. They weren't, and they did indeed lack a valid DKIM signature. Yahoo's decision to add a p=reject flag to its DMARC policy thus may seem reasonable: it told receivers to reject emails that claim to come from Yahoo yet which lack a valid DKIM signature.
However, there is one important case in which emails lack such a signature for a good reason: email discussion lists, more commonly known as mailing lists.
Popular since the early days of the Internet, and still used by millions, mailing lists involve a central server that forwards emails from one subscriber to the list address to all other subscribers. While forwarding, most list servers modify the subject line of the email to include a 'list tag' and add a footer containing unsubscribe information to the bottom of the email. In doing so, they break any existing DKIM signatures.
To understand why this matters, imagine that Alice sends an email from her Yahoo address to a list server. The server adds a footer, modifies the subject and sends the email to the list's subscribers, including Bob's Gmail address.
Gmail's mail servers notice that the email sent to Bob has an invalid DKIM signature, yet it claims to come from Alice's Yahoo address (list servers typically don't modify the From: header). Following Yahoo's p=reject DMARC flag, rather than deliver the message to Bob, they bounce it back to the list server. The server notices the bounce and does what has long been best practice when messages bounce: it unsubscribes Bob from the list.
This issue isn't new: the same problem arose with the optional and now historic DKIM extension ADSP. Yet, while smaller domains caused problems for a few mailing lists, Yahoo is one of the largest email senders and there are very few lists that don't have any subscribers with a Yahoo address.
The issue has long divided the email community. Some argue that mailing lists have worked this way for decades and it is unreasonable to ask them to change because the relatively recent addition of DKIM has started to cause issues. Others say that leaving the From: address unchanged, while modifying subject and body, simply isn't desirable any more in today's email landscape. Indeed, this is what Yahoo said in defence of its policy.
I have a lot of sympathy for the former argument, which tends to be backed up by the RFCs. Yet Yahoo's changes (assuming they don't change their policy) may see the balance shift the other way. And perhaps that will be better in the long run.
So why did Yahoo decide to ignore those of its users who subscribe to mailing lists?
Some have argued that it did so because it runs its own mailing lists (Yahoo Groups). I personally think the answer is more mundane. As Al Iverson suggests, it is likely that there are simply too few email users subscribing to mailing lists today to make them worth caring about in the face of significant abuse of their domain.
It is unlikely that the last words on DKIM and DMARC have been said. To find out more about the method's strengths and weaknesses, an article written by the aforementioned John Levine for Virus Bulletin provides a good introduction. At VB2014, Microsoft's Terry Zink will give a presentation on DMARC.
Posted on 15 April 2014 by Martijn Grooten
