Forum Moderators: phranque

Message Too Old, No Replies

Trying to get /sendmail emails delivered to Gmail

         

csdude55

5:27 am on Dec 6, 2023 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



I have a form on my site for the user to request a forgotten password to be emailed to them. It's processed in Perl (sendmail) and sends an email to them.

The problem I'm having is that Gmail (and presumably Yahoo) always sends that email to the recipient's spam folder! And 99 times out of 100 the user doesn't understand to look there.

I have SPF, DMARC, and DKIM in the DNS. Gmail's headers show the SPF and DMARC, but I don't see any reference to DKIM in the header. This is from Gmail:

Authentication-Results: mx.google.com;
spf=pass (google.com: domain of srs0=3dc0=hr=my.server.com=nobody@mysite.com designates 123.45.67.89 as permitted sender) smtp.mailfrom="SRS0=3dc0=hr=my.server.com=nobody@mysite.com";
dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=mysite.com


The Perl script looks like this:

open (MAIL, "|/usr/sbin/sendmail -t") || die "Can't open /usr/sbin/sendmail!\n";

# $found_email is set and verified earlier
print MAIL "To: $found_email\n";
print MAIL "From: info\@mysite.com (Me)\n";
print MAIL "Subject: Blah blah blah\n\n\n";

print MAIL "yada yada yada";
close (MAIL);


Is there a trick to getting sendmail to send the DKIM header?

engine

9:53 am on Dec 6, 2023 (gmt 0)

WebmasterWorld Administrator 10+ Year Member Top Contributors Of The Month



I'd be interested to know solutions as I've seen similar issues. The most successful solution I've found to date is not technical, but is simply to put clear instructions on the form to check spam folder.

csdude55

9:45 pm on Dec 6, 2023 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Well, I can't find a way to authenticate with sendmail, so I've spent today rebuilding using Net::SMTP. I had to install Net::SMTP, IO::Socket::SSL, and MIME::Lite to fit the example I found:

[skipser.com...]

And here's my final test script:

use Net::SMTP;
use MIME::Lite;

$login_user = 'noreply@mysite.com';
$login_pass = <PASSWORD>;

$to = 'me@gmail.com';

$msg = MIME::Lite ->new (
From => 'info@mysite.com',
To => $to,
Type => 'text/html',
Subject => 'Testing Net::SMTP',

Data => qq~
The <i>body</i> of the <b>email</b> Net-SMTP example
~
);

$smtp = Net::SMTP->new(
'mail.mysite.com',
Port => 465,
SSL => 1,
Debug => 1,
);

$smtp->auth($login_user, $login_pass) or die('Could not authenticate');

$smtp->mail($login_user);
if ($smtp->to($to)) {
$smtp->data();
$smtp->datasend( $msg->as_string() );
$smtp->dataend();
}

else {
print "Error: ", $smtp->message();
}

$smtp->quit;

print "Done";
exit;


The email successfully went through... but it still went to Gmail's spam folder :-/

When I "Show original", though, it does show DKIM:

Message ID<6570e753.500a0220.1ee56.8ea5SMTPIN_ADDED_MISSING@mx.google.com>
Created at:Wed, Dec 6, 2023 at 4:27 PM (Delivered after 2 seconds)
From:info@mysite.com Using MIME::Lite 3.033 (F2.84; T2.24; A2.21; B3.16; Q3.16)
To:me@gmail.com
Subject:Testing Net::SMTP
SPF:PASS with IP 123.45.67.89 Learn more
DKIM:'PASS' with domain mysite.com Learn more
DMARC:'PASS' Learn more


WHM's Delivery Report shows that the email has a spam score of 2, so maybe that's why it was filtered? I honestly don't know.


The most successful solution I've found to date is not technical, but is simply to put clear instructions on the form to check spam folder.

I have that, too, but I'm pretty sure that my users can't read! LOL I have AT LEAST 2 emails a day from users that requested the password and didn't get it in their Inbox. The wide majority of the time, it was in their spam folder :-/

haramamba

9:28 am on Dec 7, 2023 (gmt 0)

Top Contributors Of The Month



Did you check the ip address on spamhaus?
Two years ago Gmail decided to reject all my emails on a new ip address until I cleared it on spamhaus.
PS I have only SPF.

engine

11:00 am on Dec 7, 2023 (gmt 0)

WebmasterWorld Administrator 10+ Year Member Top Contributors Of The Month



>users can't read!
Often don't read. Been faced with that for decades!

Keep experimenting with the notification and prominence of the message for people to check their spam folder.

mack

1:44 am on Dec 8, 2023 (gmt 0)

WebmasterWorld Administrator 10+ Year Member Top Contributors Of The Month



I would say try and migrate your script to use SMTP if possible. I know this will be very dependent on your host and what they allow but it does solve a lot of email problems that tend to be common with both sendmail and phpmail (on the PHP side).

I had a similar problem with a number of outgoing emails.. some were for forgotten password reminders others were for email verification upon sign-up. Using SMTP certainly resulted in no bounces and a better chance of getting into the main inbox.

Mack.

csdude55

7:05 am on Dec 8, 2023 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Did you check the ip address on spamhaus?

Yep, I get a regular report from MXToolbox on that.

I tested the email with mail-tester.com and it said that I had a perfect spam score. Then I tested with dkimvalidator.com, and it passes everything and says that my SpamAssassin score is -5.106. But I just tried again, and the exact email that the testers approved STILL went to Gmail's spam folder :-/

I've created a Message ID (way more work than anticipated!), but that didn't help. I also figured out to use MIME::Lite's scrub() to remove the X-Mailer and Content-Disposition tags, but that also didn't help.

My only remaining theory:

The POP account I'm sending from is noreply@mysite.com, and the From address is info@mysite.com. Maybe Gmail doesn't like the noreply, or that they're not the same?

I would say try and migrate your script to use SMTP if possible.

My server uses Exim for regular emails, but the docs for Mail::Transport::Exim were super vague and I kept finding references to using Net::SMTP instead. Which I think is the same thing that you're suggesting? Or is there a better way to use SMTP?

haramamba

12:09 pm on Dec 8, 2023 (gmt 0)

Top Contributors Of The Month



My dkimvalidator test results:

SpamAssassin Score: -5.01
Message is NOT marked as spam
Points breakdown:
0.0 RCVD_IN_ZEN_BLOCKED_OPENDNS RBL: ADMINISTRATOR NOTICE: The query
to zen.spamhaus.org was blocked due to
usage of an open resolver. See
https://www.spamhaus.org/returnc/pub/
[x.x.x.x listed in zen.spamhaus.org]
-5.0 RCVD_IN_DNSWL_HI RBL: Sender listed at https://www.dnswl.org/,
high trust
[x.x.x.x listed in list.dnswl.org]
-0.0 SPF_HELO_PASS SPF: HELO matches SPF record
-0.0 T_SCC_BODY_TEXT_LINE No description available.

I'm sending only plain text, no MIME. My headers:

Received: from localhost (localhost [127.0.0.1])
From:
To:
Subject:
Message-Id:
Date:

Hello User,
Some text

I'm using Postfix, it does not include any mailer-related info into the headers.

csdude55

5:02 pm on Dec 8, 2023 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Mine is similar. I was originally sending in plain text, too, but mail-tester took off a few 0.x because I didn't have HTML. So I modified the test to include HTML.

SpamAssassin Score: -5.106
Message is NOT marked as spam
Points breakdown:
0.0 RCVD_IN_ZEN_BLOCKED_OPENDNS RBL: ADMINISTRATOR NOTICE: The query
to zen.spamhaus.org was blocked due to
usage of an open resolver. See
https://www.spamhaus.org/returnc/pub/
[123.45.67.89 listed in zen.spamhaus.org]
0.0 URIBL_DBL_BLOCKED_OPENDNS ADMINISTRATOR NOTICE: The query to
dbl.spamhaus.org was blocked due to usage
of an open resolver. See
https://www.spamhaus.org/returnc/pub/
[URIs: gonc.net]
0.0 URIBL_ZEN_BLOCKED_OPENDNS ADMINISTRATOR NOTICE: The query to
zen.spamhaus.org was blocked due to usage
of an open resolver. See
https://www.spamhaus.org/returnc/pub/
[URIs: gonc.net]
-5.0 RCVD_IN_DNSWL_HI RBL: Sender listed at https://www.dnswl.org/,
high trust
[123.45.67.89 listed in list.dnswl.org]
-0.0 SPF_HELO_PASS SPF: HELO matches SPF record
0.0 HTML_MESSAGE BODY: HTML included in message
-0.1 DKIM_VALID Message has at least one valid DKIM or DK signature
0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily
valid
-0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from
author's domain
-0.0 T_SCC_BODY_TEXT_LINE No description available.
0.0 TVD_SPACE_RATIO No description available.


For my test, the subject is Testing Net::SMTP and the body is:

This is a multi-part message in MIME format.

--_----------=_170201855399430
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain


This is the ID:
1702018553-7246cc4ded783e20548955f80f98dbc7-4165f4608534e7e15bb78aa956f6210=
c@mysite.com

--_----------=_170201855399430
Content-Transfer-Encoding: 8bit
Content-Type: text/html


<body>
<b>ID:</b><br>
1702018553-7246cc4ded783e20548955f80f98dbc7-4165f4608534e7e15bb78aa956f6210c@mysite.com
</body>

--_----------=_170201855399430--


At this point I'm at a loss, I just don't know how to make it better than -5.106.

not2easy

5:32 pm on Dec 8, 2023 (gmt 0)

WebmasterWorld Administrator 10+ Year Member Top Contributors Of The Month



Have you looked at gmail's requirements/specs for email? I know they changed something mid-2022 and it was suddenly quite a pain to get my gmail via ThunderBird email client. I realize the changes affected receiving, not sending, and the instructions I got were for ThunderBird client but here is what they required to receive email:
The correct servers are imap.gmail com and smtp.gmail.com, and the authentication method for both should be OAuth2. Check that cookies are allowed in TB Options/Privacy & Security, remove the Saved Passwords in Options/Privacy & Security, restart TB, enter the account password in the OAuth browser window when prompted.

For a long time after that (about 10 months, all my automated notifications sent from my sites were routed to the spam folder. I kept marking them as not spam and finally they stopped sending them to the spam folder.

Have you tried sending via IMAP instead of POP?