ok it only took a few minutes on wikipedia to get a handle on the "v=..." part.

That SPF record says that mail is allowed to be sent from the "A" and "MX" systems, and all others should be forbidden.

So does this mean that the IP of the "sender's address" doesn't match the IP in the A record? And that's why the message is bouncing?

I'm sending my mail through the SMTP defined as "mail.(mydomain).com", authenticated by password.

The DNS record for the "mail" subdomain is a CNAME to (mydomain).com.

The MX record for (mydomain).com is [0] (mydomain).com.

The A record for (mydomain).com is the IP address of my server.

As far as I can tell, everything's set up properly. And... it's worked fine until just a few days ago when I started getting occasional messages bouncing back.

A test

As suggested by the SPF tools page:
[openspf.org...]

I sent an email to spf-test@openspf.org

The result: it bounced back (as it's designed to do), but the news wasn't good

This message was created automatically by mail delivery software.

A message that you sent could not be delivered to one or more of its
recipients. This is a permanent error. The following address(es) failed:

spf-test@openspf.org
SMTP error from remote mail server after RCPT TO:<spf-test@openspf.org>:
host mailout02.controlledmail.com [72.81.252.18]:
550 5.7.1 <spf-test@openspf.org>: Recipient address rejected:
SPF Tests: Mail-From Result="fail": Mail From="**bleep**@***bleep***.com" HELO name="vps.***myVPSdomain***.com" HELO Result="none" Remote IP="***bleep***"

What's the problem?

the problem is that the Remote IP - as echoed by the response from openspf - doesn't match the IP defined as the "A" record of my domain.

They look very similar. But the last octet is slightly different. Definitely something funky going on with my host.

What's the problem?

"bleep" seems to sum it up ;)

The sender is not authorized to send to the destination. This can be the result of per-host or per-recipient filtering.

But I like the generic x.7.x explanation better:

The security or policy status codes report failures involving policies such as per-recipient or per-host filtering and cryptographic operations. Security and policy status issues are assumed to be under the control of either or both the sender and recipient. Both the sender and recipient must permit the exchange of messages and arrange the exchange of necessary keys and certificates for cryptographic operations.

In other words, you need to get together with the recipient-- by carrier pigeon, I guess-- and agree on, uhm, something. Seems a bit extreme if all you were trying to say was "Yes, we have that in stock".

Actually the mail in question is mission-critical for the normal functioning of my app, and the SPF dysfunction has worse consequences - I noticed that some hosts have added my domain to their spam blacklist. So, this not a trivial issue.

Thankfully, I've solved the problem and it wasn't difficult to do.

I'm running this site on a VPS, which hosts several domains. The root IP for the VPS is A.B.C.X (for example)

this one site in question has a dedicated IP, A.B.C.Y (for example)

So, the "A" record for the domain is A.B.C.Y, but the mail service is all hosted on the VPS, which is A.B.C.X. They don't match, and that's why the SPF Test failed.

My SPF record states "v=spf1 a mx -all" , meaning the IP of the sender-host must match the IP in the "A" record. It didn't.

Rather than fuss with the IPs, the solution was far simpler. I changed the SPF record to:

"v=spf1 a mx ip4:A.B.C.X -all"

Thus allowing the root VPS mail service to send mail originating from my domain.

Wait an hour for the DNS to propagate... then I sent another test message to spf-test@openspf.org

the response:

spf-test@openspf.org
SMTP error from remote mail server after RCPT TO:<spf-test@openspf.org>:
host mailout02.controlledmail.com [72.81.252.18]:
550 5.7.1 <spf-test@openspf.org>: Recipient address rejected:
SPF Tests: Mail-From Result="pass": Mail From="***bleep***@***bleep***.com" HELO name="vps.***myVPSdomain.com" HELO Result="none" Remote IP="***bleep***"

Now the SPF test passes.

SPF is an arcane art; I can appreciate how easy it is to ignore, or configure badly. But now I know the importance of doing it, and doing it right. The first time around I was merely pasting in an ignorant suggestion, and without knowing it my SPF test has been failing for a long time. It took me a couple of hours to read a little deeper, understand how SPF works, how to test if it is working, and figure out a solution.

I hope this thread helps someone overcome the same problems.

As an email admin by day and webmaster by night I vote that this thread be placed in the WW Library.

btw, the tongue-in-cheek title of this thread alludes to a more common meaning for the acronym SPF, "Sun Protection Factor", which is a number they stamp on sunscreen and lotion products in North America. Readers unfamiliar with that other meaning of SPF might be puzzled by that.

Actually the mail in question is mission-critical for the normal functioning of my app

Do you have DomainKeys set up?

I recently added a forum to one of my domains (mostly to handle user registrations for tools on the site rather than for the forum itself) but it soon became clear that despite correct SPF and reverse DNS having been setup, certain email providers (including Yahoo) were bouncing mails from the server. Investigation indicated that lack of DomainKeys was the issue.

After a few searches I decided life was too short to include setting up DomainKeys on my current server and went with a third party, self-service, cloud based emailer ($0.001 per email - at my volumes $20 will last all year, maybe more :( ). Note if you're using PHP then the PEAR Mail package allows sending through 3rd party SMTP servers.

No I don't have DomainKeys set up.

From the looks of their site, I don't think I'll be setting it up in this decade, either.

Look in the docs, under the heading "Implementation":

The signer needs to add code in the appropriate agent, to perform signing, and they need to modify their DNS administrative tools to permit creation of DKIM key records.

A validator needs to add code to the appropriate agent and then feed the result into the portion of their system needing it, such as a filtering engine.

The mere existence of a valid signature does not imply that the mail is acceptable, such as for delivery. Acceptability requires an assessment phase. Hence the result of signature validation must be fed into a vetting mechanism that is part of the validator's filter.

Seriously, that's it. The whole chapter. That's all you get.

DKIM = Yet another pile of esoteric technology that no one is going to use

DKIM = Yet another pile of esoteric technology that I had to use (in some way) if I wanted people with yahoo email address to be able to register to use my tools!

Actually, my sunconcious has obviously been mulling this over for the last day I've got vague memories of the initial bouncing being related to switching forum software.

I emailed all currently active users ( I think it was approx 30 at the time!) that I was changing software, that the tools would be down on such and such a date, and if they couldn't log in when they were back up to let me know.

With the way my server was set up at the time I think it took only 6(!) identical emails to Yahoo addresses without DomainKeys for them to blacklist my web server as an email sender!

Take heed: this SPF stuff requires that you change the DNS records for your domain. If you're familiar with DNS and how it works and how to change it without breaking everything, then all is well. But if you're new to DNS shenanigans, it's good to do a little reading & refresher before fiddling with it.

The SPF record has a type "TXT". Some of my servers are running WHM - and with that I'm able to add a "TXT" records to the DNS, no problem.

But I also have a couple of Rackspace Cloud instances, and using their management tool you can add an "A", "CNAME", "MX", but... not a "TXT". To add an SPF record to a Rackspace VPS, you have to open a support ticket and get one of their support staff to do it for you.

My app sends notification emails to buyers giving them instructions for completing a sale. So, that's mission-critical, and when many people buy the same product the emails are pretty much identical. This may be more crucial than I thought.

To set up DomainKeys it's another "TXT" record added to the DNS.

Here's what an online tool generated for me, I have no idea yet if it's correct:

/._domainkey.example.com IN TXT "v=DKIM1; p={public key goes here}; s=email; t=y"

(yes there's a dot after the TLD. Is that normal?)

yes always

Two point worth mentioning for those on cPanel hosts:

The sending email server *may* be bound to a different IP than the webserver. If so add an additional ip4 server to setup.

Adding the DK/Sender ID/SPF records is easy - all forms and buttons.

I recently had the same issue, mails getting bounced a lot, and ended up learning about SPF, then applying a similar rule to my dns which I must admit has helped greatly.

SPF has been around for sometime, but it does appear it is starting to be used quite heavily now.

I still have a problem with aol though, they seem quite happy in blocking too much. I have quiet a few important emails lost in the aol black hole, even though I'm not on any public blacklists.

AOL has a surprisingly good help page [postmaster.aol.com ] visit it and apply for whitelisting to fix your blocking issue.

I usually use the SPF Wizard on the MS site, works real nice
[microsoft.com...]

that address for checking the validity of SPF (spf-test@openspf.org) was really handy. Without it I wouldn't have known for sure that everything was OK.

Is there such a thing for testing a Domain Key?

Um I went into CPANEL and there's a button that said "Enable Domain Keys" and I clicked it. then the page refreshed and now it says "Status: Enabled & Active (DNS Check Passed)"

That's dandy. I don't know if it's really working. How can I check?

And then I'll need to do the same thing to some servers that don't have CPANEL. It'll be good for my soul to learn how to configure this from scratch.

Any tips for getting Domain Keys working on a Fedora box running postfix?

I don't know if it's really working. How can I check?

Send yourself an email to a freemail account and check the raw headers. You should find a line starting with: DKIM-Signature

an email from the account in question, to my hotmail address.

after bleeping out personal details:

Authentication-Results: hotmail.com; sender-id=temperror (sender IP is --.--.---.---) header.from=------@-------.com; dkim=none header.d=------.com; x-hmca=none

X-Message-Status: n:0:n

X-SID-PRA: ----- ----- <-----@-----.com>

X-DKIM-Result: None

X-AUTH-Result: NONE

X-Message-Delivery: Vj0xLjE7dXM9MDws5TAhYT01O0Q9MTtTQ0w9Mw==

X-Message-Info: JGTYoYF78jGzuoPRzxQ33HYff5bTZMxB31RtcDy/c5MVSw1+ufv+rbWCLt7lLv+uM4fTNhHx0vpvpta+
MNDYBtFGrcpxjP9W1IIDSsU4EfvDFZsOucGHRBtjyQKWBecU5xomFn8cxbI=

Received: from ---.-------.com ([--.--.---.---]) by snt0-mc4-f12.Snt0.hotmail.com with Microsoft SMTPSVC(6.0.3790.4675);

Fri, 23 Sep 2011 13:03:48 -0700

Received: from --------.--------.--------------.net ([--.---.--.--] helo=[10.10.10.217])

by ---.-------.com with esmtpa (Exim 4.69)

(envelope-from <-----@------.com>)

id 1R7ByI-0003gg-Sb

for -----------@hotmail.com; Fri, 23 Sep 2011 13:03:43 -0700

From: --- ---- <-----@-------.com>

Content-Type: text/plain; charset=us-ascii

Content-Transfer-Encoding: 7bit

Subject: testing DKIM

Date: Fri, 23 Sep 2011 16:03:32 -0400

Message-Id: <92E7E47D-E0AD-4145-9011-4AC22D448D5F@scubbly.com>

To: --- ---- <----------@hotmail.com>

Mime-Version: 1.0 (Apple Message framework v1082)

X-Mailer: Apple Mail (2.1082)

X-AntiAbuse: This header was added to track abuse, please include it with any abuse report

X-AntiAbuse: Primary Hostname - ---.-------.com

X-AntiAbuse: Original Domain - hotmail.com

X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12]

X-AntiAbuse: Sender Address Domain - -------.com

Return-Path: -----@-------.com

X-OriginalArrivalTime: 23 Sep 2011 20:03:48.0470 (UTC) FILETIME=[E8326160:01CC7A2B]

testing!

[edited by: phranque at 12:54 pm (utc) on Sep 24, 2011]
[edit reason] fix thread width [/edit]