When Battling Spam, There Are Few Real Choices
The past two years have seen several magic bullet spam solutions. Whether they've panned out, and whether you should trust them, is up for debate.
Recently people have been led to believe that the solution to spam is just around the corner. In the top running, we have SPF, Sender-ID and Domain Keys, but will any of them actually help? The answer is: only slightly. We'll explain why and cover how each of these technologies work.
All of the solutions are very similar, and they tend to restrict e-mail in a way that nobody in the history of e-mail has experienced. The first and most important point to understand is that e-mail is used in very strange ways. The proposed spam fighting solutions limit just a few servers to being the originators of e-mail from a domain, but people frequently send e-mail in such a way that violates this. They're using e-mail legitimately, too.
Configuring your e-mail client of choice normally involves a few steps, namely identifying the incoming POP or IMAP server and the outgoing SMTP server. This is usually a straightforward process, and so is the next step. To add a second account, perhaps so that you can check mail from your university or work account, only the incoming information is generally provided. End result? People are sending mail from user@domain2 via their SMTP server for user@domain1. Often times people do this on purpose to get around filtering issues that crop up because of misguided administrators. This is the simplest example, but rest assured there are more, for example forwarding messages from one account to another (using a .forward file for us Unix people). Many businesses also rely on e-mail tricks when conducting B2B transactions.
The second most important point, and we hope the most valuable take-away from this article, is that you should never, ever straight-out reject e-mail based on these criterion. It is perfectly acceptable to use these tools to aid in the decision process, but if your mail server identifies a message as having invalid SPF information and simply rejects the e-mail, you're in for a world of hurt. A reasonable decision process involves your spam- or virus-filtering software checking these DNS records, not your mail server software itself. Most spam filtering applications use weights, and each piece of evidence will count toward a final score used to determine if an e-mail is spam. If your mail server is configured to run these checks, generally it will completely reject the e-mail based on the information it receives.
Speaking about the act of "using" these protocols can be a bit confusing. You can use (or misuse) them in three different ways. All three are just DNS records published for your domain, so the first concept of use is simply publishing a record that gives other people some information. The published information is a list of IP addresses that are authorized to be a mail server for your domain. When another server receives e-mail with a "From:" header that claims to be your domain, they can check with your published information to make sure they are receiving it from an authorized server.
The act of checking, or "turning on checks" is the second way to use these protocols. Hopefully, as we discussed earlier, the receiver will not simply discard the message if it's coming from an "unauthorized" source. The third method is to both publish your own, and also check all incoming mail for these records.
Our first contender, SPF, or Sender Policy Framework, was the first kid on the block. SPF is simply a TXT DNS resource record that can be queried by remote sites to see who you authorize mail to be sent from. This doesn't work, because of the reasons cited above.
The MARID working group of the IETF was tasked with finding a solution. It failed, but produced some interesting analyses in the process. The second on our list of buzzwords is Microsoft's Sender-ID. Sender-ID is the same as SPF, with a few minor enhancements. It's functionally equivalent, for our purposes. Microsoft joined the MARID working group late, and tried to push its protocol on everyone, arguing it was the best solution. Everyone who runs Microsoft servers and doesn't care about patent issues tended to agree, but the majority of the world did not. Neither SPF nor Sender-ID will ever be widely used. While many sites started using SPF, most have also since backed down.
Domain Keys is the final contender. It, too, is a DNS record that provides information about a domain's policy regarding where e-mail should originate. There is a commonly used tactic in the computing world to fake progress: wrap it with crypto. Domain Keys is providing the same information as SPF and Sender-ID, but it's cryptographically signed so you know 100% that it's correct. Google publishes Domain Keys, but uses it properly: as a tool to help, knowing full well that it isn't the final solution.
To be fair, Domain Keys can also be used to verify a specific e-mail's integrity, but we're really trying to deal with the spam issue here. The consensus, regardless of the author's obvious bias in the same direction, is really: "domain verification breaks too much." Especially when used improperly, i.e. dropping all e-mail that gets shipped through an unauthorized server. Domain Keys aren't as bad as other cryptographic nightmares, like DNSSEC, because they don't require a central certificate. What this really means is that people can implement this without having to rely on a third party's stamp of approval and subsequent collection of funds.
The moral is that these solutions are all the same. It's called domain verification, and it simply won't work. E-mail has been around since the dawn of the Internet, and its model has always been to allow everyone access. Quite literally, your e-mail is a file that everyone on the Internet is allowed to append things to. To get rid of spam, the whole system is going to have to be redone, and a complete solution will have to involve a trusted third party.