A tip for mailing list operators to interoperate with DMARC to avoid failures
One of the problems with DMARC is how mailing lists deal with participants that publish p=reject records. The domain owner has published a policy to prevent spoofing, but all of the various participants on the mailing list may be affected. This includes people within an organization that previous were taking part in technical discussions, but suddenly find themselves unsubscribed, or seeing all of their messages going to Junk. Why does this happen?
Suppose that someone, John Doe , is on a technical discussion list. He is discussing Internet of Things Security, and the mailing list owner relays email from security-iot.org. When John sends to the discussion list, he sends an email this way:
SMTP MAIL FROM: <joe@example.com>
From: John Doe <joe@example.com>
To: Internet of Things Security
Subject: How are we going to deal with this?
DKIM-Signature: d=example.com
What's the latest on the next big vulnerability?
John is completely unaware that his messaging admin has set up DKIM-signing on his mail server. He just knows that he participates in a mailing list. The mailing list receives the message, and if it just ended up in the mailbox lists@security-iot.org, it would look like this:
Authentication-Results: spf=pass (sender IP is xx.xx.xx.xx)
smtp.mailfrom=example.com;
dkim=pass header.d=example.com;
dmarc=pass action=none header.from=example.com
From: John Doe <joe@example.com>
To: Internet of Things Security
Subject: How are we going to deal with this?
DKIM-Signature: d=example.com
What's the latest on the next big vulnerability?
But, the message doesn't stay there, it gets relayed out to the rest of the mailing list participants. The mailing list does many things to the message - it re-signs the message with a new DKIM-signature, it adds a footer to the message, and modifies the subject:
SMTP MAIL FROM: <lists@iot-security.org>
From: John Doe <joe@example.com>
To: Internet of Things Security <lists@iot-security.org>
Subject: [Internet of Things Security Discussion] How are we going to deal with this?
DKIM-Signature: d=security-iot.org
DKIM-Signature: d=example.com
What's the latest on the next big vulnerability?
----------------------
To unsubscribe from this list, click here: https://unsubscribe.example.com
When the message is replayed back to John, both SPF and DKIM pass. However, SPF passes with the mailing list's domain, and DKIM passes with the mailing list's DKIM-signing domain. The original DKIM signature from d=example.com no longer verifies because the message contents have been modified.
Since the domain in the From: address example.com does not align with either the domain that passed SPF (security-iot.org) or DKIM (also security-iot.org), it fails DMARC back at Joe's email server:
SMTP MAIL FROM: <lists@iot-security.org>
Authentication-Results: spf=pass (sender IP is yy.yy.yy.yy)
smtp.mailfrom=security-iot.org;
dkim=pass header.d=security-iot.org;
dkim=fail (body hash failure) header.d=example.com;
dmarc=fail action=none header.from=example.com
From: John Doe <joe@example.com>
To: Internet of Things Security <lists@iot-security.org>
Subject: [Internet of Things Security Discussion] How are we going to deal with this?
DKIM-Signature: d=security-iot.org
DKIM-Signature: d=example.com
What's the latest on the next big vulnerability?
----------------------
To unsubscribe from this list, click here: https://unsubscribe.example.com
As long as the DMARC policy is p=none, nothing bad happens [1]. John gets all his email from the mailing list.
The problem occurs when John's messaging admin updates the DMARC policy from p=none to p=quarantine, and then the problem becomes even worse when the policy is p=reject.
Why?
Because when the policy is p=quarantine, then every message that John sends to the list will fail DMARC when it is relayed back to other receivers. That means all of poor John's email will end up in the Junk folder of all these destinations (unless the recipient keeps hitting "This is not spam" or adds John to a safe senders list).
When the policy is p=reject, then John's messages are no longer getting sent to Junk. Instead, they are getting rejected by the receiving email server. There is no chance for any recipient to add John to a safe sender or hit "This is not Junk", it just bounces back to the mailing list server. And, worse yet, some mailing lists will keep track of which sender email is causing all of these bounces and will unsubscribe John from their mailing list.
The net result is that John can't send emails to mailing lists.
The admin is trying to control phishing and spoofing, but poor John has been caught in the cross-fire.
A way forward
The official way that the industry is trying to resolve this is with a new protocol called ARC - Authentication Received Chaining. I'm not going to discuss that here. Instead, I'm going to talk about a quick and immediate fix.
If the mailing list server wants to avoid DMARC bounces like this, all it needs to do is rewrite the From: address. Instead of relaying the From: of the original sender, do some creative rewriting and send with the From: belonging to the mailing list:
SMTP MAIL FROM: <lists@iot-security.org>
From: "John Doe via Internet of Things Security mailing list" <lists@iot-security.org>
To: Internet of Things Security
Subject: [Internet of Things Security Discussion] How are we going to deal with this?
Reply-To: john.doe@example.com
DKIM-Signature: d=security-iot.org
DKIM-Signature: d=example.com
In this example, the mailing list now sends from itself as the general From: address, puts the user's Display Name (and its own name) into the Display Name, and the original sender email address has been retained in the Reply-To address. This works for many people, although some email clients only send to the Reply-To address when you hit "Reply" or "Reply All" which is not what you want, you want to reply to the list and not the original author. That's an easy enough workaround, just copy/paste the From: address into the To: field when replying. Or, as a list maintainer, drop the Reply-To.
Some list maintainers say that this messes up address books, since you'll have multiple Display Names associated with a single From: address (lists@security-iot.org) if everyone's name is inserted into the From: address. The workaround for this is to hash the original email address and include it into the email address:
SMTP MAIL FROM: <lists@iot-security.org>
From: "John Doe via Internet of Things Security mailing list" <am9obi5k+lists@iot-security.org>
To: Internet of Things Security <joe@example.com>
Subject: [Internet of Things Security Discussion] How are we going to deal with this?
Reply-To: john.doe@example.com
DKIM-Signature: d=security-iot.org
DKIM-Signature: d=example.com
I created the above by encoding john.doe@example.com and converting it to Base64, taking the first 8 characters, and putting it in front of a + sign in the localpart of the email address. That gives everyone a unique sending From: address so as not to collide multiple Display Names on a single email address.
This lets senders with strong DMARC policies to sign up to your mailing list. They can still relay to receivers because the DMARC check is done against your domain in the From: address, and you control both the SPF, DKIM, and DMARC policies. Everything is under your control, an administrator of a 3rd party domain doesn't need to be aware of which of their users are on mailing lists, and a user notices no change (almost no change).
SMTP MAIL FROM: <lists@iot-security.org>
Authentication-Results: spf=pass (sender IP is yy.yy.yy.yy)
smtp.mailfrom=security-iot.org;
dkim=pass header.d=security-iot.org;
dmarc=pass action=none header.from=security-iot.org
From: "John Doe via Internet of Things Security mailing list" <am9obi5k+lists@iot-security.org>
To: Internet of Things Security <lists@iot-security.org>
Subject: [Internet of Things Security Discussion] How are we going to deal with this?
DKIM-Signature: d=security-iot.org
DKIM-Signature: d=example.com
What's the latest on the next big vulnerability?
----------------------
To unsubscribe from this list, click here: https://unsubscribe.example.com
You can see it all passes from a security perspective. The user is happy, email receivers are happy, and the mailing lists are... a little less happy than they were before because they had to do some extra work... but are happy to get things working.
Conclusion
This is how I would run a mailing list server if it were up to me because it allows senders with p=reject DMARC records to sign up, and is relatively quick to deploy.
The DMARC.org maintainers have a DMARC FAQ and this is covered in more detail with specific implementations. It's available here: I operate a mailing list and I want to interoperate with DMARC, what should I do?
[1] If sending to Office 365, it'll get marked as spam with a red safety tip. So, Office 365 treats email "from" yourself "to" yourself as if you have a DMARC p=quarantine record regardless of if you have one published or not. It has the same issues with mailing lists, see https://blogs.msdn.microsoft.com/tzink/2016/11/02/troubleshooting-the-red-suspicious-safety-tip-for-fraud-detection-checks/
Comments
- Anonymous
March 24, 2017
BTW, this is not a post about ARC, but instead a post about how to modify the message to make it DMARC compliant by rewriting the From: address. ARC lets you be DMARC compliant and not rewrite the From: address while also modifying content.