Background

In this post we provide an overview of the Sender Policy Framework and Error Conditions with a focus on Google’s enforcement of the SPF 10 Lookup Limit.  You may be surprised by what we’ve found!

Sender Policy Framework (SPF) is a core protection against email spoofing and sophisticated phishing attacks.  SPF records are published in DNS to authorize IP addresses permitted to send email on a domain’s behalf.  The Sender Policy Framework standard defines specifications for formatting valid SPF records, and provides several methods for administrators to control the list of authorized IPs.  These methods are known as mechanisms, with the most common ones being a, mx, and include, all of which allow domain administrators to reference IP addresses maintained in other DNS records.  Each of these mechanisms cause receiving email infrastructures to launch DNS lookups during the SPF authorization process.  When all goes well, SPF record evaluation provides assurance that the email was sent by an authorized IP address or identifies that the message was sent by an unauthorized IP address.  But, what happens when there are errors?

SPF Errors

The SPF standard provides two categories of error, temperror and permerror.

temperror results mean a transient error was encountered during SPF record evaluation.  This is generally rooted in connectivity issues, which could be as simple as a delay in DNS server response.

permerror results mean the published SPF record could not be correctly interpreted.  This may occur due to syntactical errors, or other violations of the SPF standard.

Handling of both temperror and permerror are managed by the receiving infrastructure’s SPF implementation and related policies.  At a minimum, we can say receiving infrastructure is unable to rely on SPF results if either error is encountered.  The receiver may use other message security standards and filtering to decide if the message should be delivered or not, and in some cases may simply reject the message, deliver to the spam folder, or take other actions according to receiver policies.

It is in the domain administrator’s best interest to be sure their SPF record is well-configured to avoid permerror conditions.

SPF 10 Lookup Limit and permerror

A common violation often overlooked by the domain administrator is exceeding the 10 Lookup Limit defined by the Sender Policy Framework.  The SPF standard sets an upper limit of 10 DNS lookups that may be initiated when evaluating an SPF record.  The standard further states that exceeding the 10 Lookup Limit must return a permerror result.  Below is an extract from the SPF standard.

SPF implementations MUST limit the total number of those terms to 10 during SPF evaluation, to avoid unreasonable load on the DNS.  
If this limit is exceeded, the implementation MUST return "permerror".

Reference: https://datatracker.ietf.org/doc/html/rfc7208#section-4.6.4

SPF evaluations returning a permerror provide no true indication if the message being evaluated is authorized by the sending domain or not.  Receivers are free to implement their own policy for handling permerror conditions, and may choose to outwardly reject the message or passively accept.

Google’s SPF 10 Lookup enforcement.  What does Google do?

Well, Google does what Google wants to do, of course! 

According to our testing, Google’s SPF implementation as it pertains to the 10 Lookup Limit does not comply with the Sender Policy Framework Standard.

Google will return a permerror result when SPF record evaluation exceeds 20 DNS Lookups, but not before.  Therefore, Google’s SPF implementation does not respect the SPF 10 Lookup Limit.  After exceeding 20 Lookups, a permerror result is returned and Google servers outwardly reject the message with the following status:

bounced (host gmail-smtp-in.l.google.com[142.250.31.27] said: 550-5.7.26 The MAIL FROM domain [REDACTED] has an SPF record with 550-5.7.26 a 
hard fail policy (-all) but it fails to pass SPF checks with the 550-5.7.26 ip: [REDACTED]. To best protect our users from spam and 
550-5.7.26 phishing, the message has been blocked.

We could suspect that Google made this implementation decision as an attempt at balancing email delivery with poor SPF record administration, as the SPF 10 Lookup Limit is often overlooked and commonly exceeded given increased dependency on third-party email senders.  However, we should also recognize Google’s recent change in Email Sender Guidelines requires senders to setup SPF authentication.  This hard requirement may foreshadow Google’s stricter interpretation of the Sender Policy Framework standard moving forward.

What should Domain Administrators do?

If you are a Domain Administrator, you should verify that your SPF record is well-configured and respects the SPF 10 Lookup Limit.  Take advantage of our Domain Inspector for a free SPF record evaluation, or book a meeting with one of our experts for a free and personalized review.

If you are exceeding the 10 Lookup Limit, you should launch improvement efforts designed to scale with third-party email sending dependencies.  Below are a few tips to get you started.

Look for Consolidation Opportunities

Often, we see duplicative mechanisms referencing the same target IP Addresses.  Evaluate your include references, and your include references references, and so on to identify any overlap.  It is common to find overlapping IP ranges across multiple include statements that inflate your Lookup Count.  It is also common to find overlapping references within a single SPF record, such as an mx reference and include reference to a hosted secure email gateway.  Consolidation may bring you back into compliance with the SPF 10 Lookup Limit.

SPF Flattening

SPF Flattening consists of evaluating your SPF record to identify all referenced IP address ranges, and then creating a nested SPF structure using include mechanisms to publish only the authorized IP Addresses.  This approach buys time if a domain is over the SPF 10 Lookup limit, but is truly kicking the can down the road.  You will likely exceed the 10 Lookup limit at some point in the future if you follow this approach.  There are other reasons why you would want to avoid SPF Flattening.  We cover those n our SPF Flattening is BAD blog post, and also provide better alternatives.

Separate to Subdomains

Many organizations use subdomains to work around the SPF 10 Lookup Limit.  This allows a more granular control of which domains may use which senders, which is desirable in many cases; however, it also introduces increased administrative burden by distributing policy management across many SPF records.

SPF Macro Expansion

SPF Macro Expansion uses macros as defined in the SPF RFC.  These macros act a bit like function calls, allowing SPF request to be dynamically constructed based on sender detail.  When properly structured and managed, SPF Macro Expansion offers a scalable solution to the SPF 10 Lookup Limit.

The spfXio.com Effortless Email Authentication platform uses SPF Macro Expansion, allowing organizations to easily authorize a virtually unlimited number of third-party SPF senders. With customizable SPF policies on a per-sender basis and Email Address Auditing capabilities, you can tailor each authorized sender record to your specific needs.  To learn more, schedule a spfXio.com Managed Email Authentication Demo today!  

Conclusion

SPF provides a method for receiving email infrastructure to verify that received messages were sent by authorized IP Addresses.  While the premise of this blog was rooted in Google’s implementation of the SPF 10 Lookup limit and handling of permerror  conditions, having a well formatted SPF record that respects the SPF 10 Lookup limit is necessary to protect your domain against spoofing.  Be sure you understand the current state of your domain’s SPF record and have the expertise necessary to enact improvements.  Also understand DKIM and DMARC email security standards, and how they work in conjunction with SPF to provide best-in-class protection against email spoofing.