Principles of operation
The Simple Mail Transfer Protocol permits any computer to send email claiming to be from any source address. This is exploited by spammers who often use forged email addresses, making it more difficult to trace a message back to its source, and easy for spammers to hide their identity in order to avoid responsibility. It is also used in phishing techniques, where users can be duped into disclosing private information in response to an email purportedly sent by an organization such as a bank.
SPF allows the owner of an Internet domain to specify which computers are authorized to send mail with “from” addresses in that domain, using Domain Name System (DNS) records. Receivers verifying the SPF information in TXT records may reject messages from unauthorized sources before receiving the body of the message. Thus, the principles of operation are similar to those of DNS-based blackhole lists (DNSBL), except that SPF uses the authority delegation scheme of the Domain Name System.
The “from” address is transmitted at the beginning of the SMTP dialog. If the server rejects the domain, the unauthorized client should receive a rejection message, and if that client was a relaying message transfer agent (MTA), a bounce message to the original from address may be generated. If the server accepts the domain, and subsequently also accepts the recipients and the body of the message, it should insert a Return-Path field in the message header in order to save the from address. While the address in the Return-Path often matches other originator addresses in the mail header such as from, this is not necessarily the case, and SPF does not prevent forgery of these other addresses such as sender.
Spammers can send email with an SPF PASS result if they have an account in a domain with a sender policy, or abuse a compromised system in this domain. However, doing so makes the spammer easier to trace.
The main benefit of SPF is to the owners of e-mail addresses that are forged in the Return-Path. They receive large amounts of unsolicited error messages and other auto-replies. If such receivers use SPF to specify their legitimate source IP addresses and indicate FAIL result for all other addresses, receivers checking SPF can reject forgeries, thus reducing or eliminating the amount of backscatter.
SPF has potential advantages beyond helping identify unwanted mail. In particular, if a sender provides SPF information, then receivers can use SPF PASS results in combination with a white list to identify known reliable senders. Scenarios like compromised systems and shared sending mailers limit this use.
Reasons to implement
If a domain publishes an SPF record, spammers and phishers are less likely to forge e-mails pretending to be from that domain, because the forged e-mails are more likely to be caught in spam filters which check the SPF record. Therefore, an SPF-protected domain is less attractive to spammers and phishers. Because an SPF-protected domain is less attractive as a spoofed address, it is less likely to be blacklisted by spam filters and so ultimately the legitimate e-mail from the domain is more likely to get through.
FAIL and forwarding
SPF breaks plain message forwarding. When a domain publishes an SPF FAIL policy, legitimate messages sent to receivers forwarding their mail to third parties may be rejected and/or bounced if all of the following occur:
- The forwarder does not rewrite the Return-Path, unlike mailing lists.
- The next hop does not whitelist the forwarder.
- This hop checks SPF.
This is a necessary and obvious feature of SPF – checks behind the “border” MTA (MX) of the receiver cannot work directly.
Publishers of SPF FAIL policies must accept the risk that their legitimate emails are being rejected or bounced. They should test (e.g., with a SOFTFAIL policy) until they are satisfied with the results. See below for a list of alternatives to plain message forwarding.
SPF FAIL policies can be an effective but problematic tool. A typical example is a user that wishes to send an email from a private PC or a mobile phone: the user uses their corporate email address but may use a different outgoing SMTP server which is not listed in the SPF record. The corporate domain may therefore be secure by blocking all email that does not originate from themselves, but have thereby limited some of their own users. Many organizations consider this compromise acceptable and even desirable to avoid spoofing.
SPF PASS is useful for authenticating the domain for use as a parameter to a spam classification engine. That is, the domain in the sender address can be considered to be authentic if the originating IP yields an SPF PASS. The domain can then be referenced against a reputation database.
SPF results other than PASS (used in combination with a reputation system) and FAIL cannot be meaningfully mapped to PASS and FAIL. However, a reputation system can easily track independent reputations for each SPF result, i.e. example.com:PASS and example.com:NEUTRAL would have different reputations, and ditto for the other results. This approach is useful even without whitelisting plain forwarders, because the FAIL results from the plain forwarders simply accrue an independent reputation.
The meaning of PASS, SOFTFAIL, FAIL is sometimes incorrectly interpreted to mean “not-spam”, “maybe-spam”, “spam” respectively. However SPF does nothing of the sort. SPF merely offers an organization firstly the means to classify emails based on their domain name instead of their IP address (SPF PASS); and secondly, the means to block unauthorized use of their domain (SPF FAIL).
In a naive implementation, SPF does not prevent a user with the same domain sending an email on behalf of another user because only the domain part of the address is used to locate the SPF policy record. In more sophisticated implementations, the domain owner can specify separate policies for each user by means of SPF “macros” that reference the “localpart” (user), or simply require all mail submissions for the domain to use SMTP AUTH. The latter is highly recommended anyway for many reasons.
SPF needs to operate on the host indicated by the receiving domain’s MX record. This means the host(s) that are the direct recipient of remote TCP connections, because such a host can easily deduce the originating IP address from the TCP session. These hosts are able to block the email during the SMTP session, avoiding the necessity to generate bounce messages which could be backscatter.
Other downstream hosts, for instance in a forwarding scenario, can only perform SPF checks based on “Received” headers. This is cumbersome and error-prone. A better approach is for the MX host to check SPF without blocking any email, and then add a “Received-SPF” header field or the newer “Authentication-Results” header. Downstream hosts can then look at these trace headers and set their own policy of whether to reject, accept, or quarantine based on the SPF result and other factors.
An Internet draft discussed concerns related to the scale of an SPF answer leading to network exploits as a means to corrupt the DNS. The SPF project did a detailed analysis of this draft and claimed that SPF does not pose any unique threat of DNS DoS, citing example attacks using NS and MX records and identifying void DNS lookups (negative caching) as the key DNS weakness.
An SPF based attack can generate more than 40KB of traffic per message originating completely from recipient resources once a spam campaign containing MAILFROM with unique local-parts exceed the recipient’s negative caching limits commonly imposed. Often Windows based services impose rather low limits and other resolver offerings also permit low negative caching limits to mitigate access problems following service disruptions. SPF includes the “l” macro able to combine components of the email-address local-parts to construct unique DNS requests generated by recipient resources. The SPF result for “email@example.com” may not be the same as that for “firstname.lastname@example.org” which always requires repeating potentially long sequences of more than 100 DNS transactions based upon the same cached SPF record.
SPF provides several advantages for malefactors:
- SPF based attacks reflect off recipient resources which obfuscates likely compromised systems initiating the abuse where Authentication-Results omit the authorized IP addresses.
- SPF based attacks, even without short negative caching, offers a means to obtain significant network amplification.
- SPF based attacks are seldom logged.
- When based upon the MAILFROM, the common case, authentication is not achieved so referenced domain’s involvement may not extend beyond mere authorization.
Relationship with DKIM
SPF validates the message envelope (the SMTP bounce address), not the message contents (header and body) – this is the distinction between SMTP and Internet Message Format. It is complementary to DomainKeys Identified Mail (DKIM), which signs the contents (including headers).