[Rwp] Why the From: field is set to the list itself

[Jayson Smith]

Hi,

I know about the From: field of list messages being set to the list 
itself. This is yet another thing legitimate list owners have had to do 
in the ongoing fight against spam. I'll explain why, but hold on to your 
hats, this is going to get pretty technical.

First, some terms you need to know before we get started. Many of you 
probably already know some of these, but just so everybody is on common 
ground, here we go.

Internet Protocol address or IP address: This refers to a specific point 
on the Internet. It's sort of like a telephone number computers use to 
contact each other. An example is 96.126.127.231.

Domain name: This is the human-friendly representation of a computer's 
address on the Internet. Examples are Google.com, whitehouse.gov, and 
barbershop.org. Most if not all Internet users use these every day 
without even thinking about it.

Domain Name System or DNS: This is sort of like a phone book for 
computers. A computer can contact DNS and say, "I have a user who wants 
to go to google.com, what's their IP address?" DNS then returns the IP 
address or a response saying there's no such domain name. The computer 
then contacts the IP address directly. This goes on behind the scenes, 
and the user is not involved. Additionally, domain owners can publish 
other types of records in their DNS, which we're about to see.

Sender Policy Framework or SPF: A record published by a domain owner in 
DNS, specifying which IP addresses are permitted to send mail from that 
domain. Mail servers can check incoming mail against such a list as part 
of their determination of whether any given message is spam. For 
example, my domain, bluegrasspals.com, specifies in its SPF record that 
96.126.127.231 is the only server allowed to send mail on its behalf. If 
a message comes in somewhere claiming to be from bluegrasspals.com, the 
receiving mail server can check the SPF record, and if the sending IP 
address isn't on that list, the from: line is probably forged, which 
means the message is more likely to be spam.

DomainKeys Identified Mail or DKIM: Another method of message 
authentication. In this system, a domain owner who wishes to send mail 
generates a pair of cryptographic keys, one public, the other private. 
They publish the public key in their DNS record, and keep the private 
key on their server. After that, they attach a cryptographic signature 
to the headers of every message they send from that domain, based on 
that private key. Mail receivers can then check that signature against 
the public key stored in DNS, and if the signature checks out, this 
tells the receiving system two things. First, that the domain claiming 
to send the message actually sent it, since presumably only servers 
under the control of the domain owner have the private key needed to 
generate signatures. Second, that the message has not been modified in 
any way since it was signed. This is yet another way to estimate the 
likelihood of a message being spam.

Domain-based Message Authentication, Reporting and Conformance or DMARC: 
Finally! The reason for this entire discussion! DMARC is a system 
whereby domain owners can publish a policy in their DNS records, 
specifying how compliant receiving systems are to handle mail claiming 
to originate from their servers, but which fails SPF or DKIM checking. 
The available actions are to ignore the problem, quarantine such mail, 
or outright reject such mail.

And now we come to the problem for list owners. Certain large ISP's in 
particular, Yahoo! being one of them, have published DMARC records in 
their DNS which instruct receiving mail systems to reject mail claiming 
to come from those domains, but which doesn't pass SPF or DKIM checks. 
An example might help.

You have a Yahoo! account. You send a message to my mailing list that 
lets the From: field of its outgoing messages go through unmodified. My 
server receives the message, processes it, and sends it to all my list 
members. So off the message goes on its way, but then it hits a provider 
which uses DMARC as a way to authenticate messages.

Okay, we have a message claiming to come from Yahoo! Let's check to see 
if it's legitimate. Does the sending IP address appear on Yahoo's SPF 
record? Um, let's see. Wait, no, it doesn't! Okay, what about DKIM, is 
that valid? Checking, looking... Nope, that's not valid either! Now, 
does Yahoo! tell us to do anything special when this happens? Let me 
see...here it is, yes, I should reject this message!

And it tells my server, "Sorry, but I'm not going to accept this message 
from you because you're probably a spammer trying to forge Yahoo! 
addresses."

For clarification, the reason the DKIM check didn't pass is that, while 
Yahoo! almost certainly signed the message when it was originally sent, 
my mailing list software messed with it in the process of getting it 
ready to send to the list, so the signature was no longer valid.

The solution to this problem is to set the From: address of outgoing 
list messages to the address of the list itself, which is safe. When 
this happens, any authentication policies put in place by the original 
sender of the message never come into play, since  my server is no 
longer seen as trying to forge those addresses.

Unfortunately, Mailman, the mailing list software I use, requires a list 
to either allow the From: field to go through intact, or change it to 
the list address, for all list posts. You can't isolate specific problem 
domains, it's either all or nothing. Another option I considered was 
putting the sender's Email address in the list footer, so people could 
write privately if desired, but guess which bit of text Mailman does 
*not* allow as a substitution? That's right, the sending address!

I could turn this feature off on a trial basis and just see how it 
works, but this would probably cause some mail to start bouncing. What 
do you guys think?

Sorry this has been so long, but I felt it necessary to insure everyone 
had the proper background knowledge to understand the problem, and the 
solution.

Jayson