Forum Moderators: phranque

Message Too Old, No Replies

Combatting Email Spam

Can someone explain how to set up a reasonable system?

         

old_expat

9:42 am on Jul 26, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I don't get very much *real* email, but some of what I get is important.

I am using Formmail.

Lately my SPAM on 3 websites is about 1,000/day and it started driving me crazy. My sites are 4-6 years old so a lot of Spammers have passed my URLs around, I guess.

So I looked into the options.

The CPanel boxtrapper is just about as bad, since the "your autoresponder message bounced" is just as bad as the SPAM.

If I use the "Forward all Unrouted Mail" to :blackhole:, I get no spam, but no messages either.

Can someone explain how to set up a reasonable system?

Or is it a lost cause?

lammert

10:06 am on Jul 26, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



I am using SpamAssassin, one of the Apache projects.

SpamAssasin in my configuration distributes the email over two mailboxes, one that I access directly, and a second one that contains spam. The spam mailbox is accessible with webmail software and I can in bulk transfer those messages to two other mailboxes with just two clicks (Select All and Move). One of those mailboxes contains the confirmed spam messages, the other is used for the false positives (real messages marked as spam). Each night a cron job scans these two directories and uses the messages to retrain the SpamAssassin algorithms.

About once a month I also feed a group of my valid messages to SpamAssassin for optimal algorithm training.

In the last six months with this configuration I haven't had one false positive (real message for me that was labeled as spam) and the false negative ones (spam messages slipping through to my normal mailbox) have a rate of 1%. These figures are at a spam rate of about 200 messages per day.

With this false positive rate, I could configure SpamAssassin to directly delete my spam mails, but I feel safer checking the contents of the spam directory before actually removing them.

BeeDeeDubbleU

1:37 pm on Jul 26, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



I use Spambayes but have you considered changing your email addresses and ditching the old ones?

rocknbil

5:52 pm on Jul 26, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I am using Formmail.

This is probably the root of your problem, but there is another possibility too, see below.

Quite often spammers aren't even visiting the page when they abuse mailer scripts; they figure out what fields work for the form, then from a command line, point an automated program at it, and fire away. It probably takes all of 3 seconds to ram thousands of requests to your script.

It gets worse: it's been a long time since I messed with Formmail, but if it's an old version or doesn't have the proper screening, you may only be seeing the tip of the iceberg. What a lot of these guys do is sent a data stream that creates a BCC field and inserts it into one of your vaild fields. So instead of putting just me@example.com in the "to" field, it inserts a newline character, then does "BCC:one@aol.com,two@aol.com . . . " thousands. And being a BCC, you never know it.

If a lot of this spam is "returned undeliverable" this will definately be the case. The only way you'd know is if it sends to something@somewhere.com and it doesn't exists, so it bounces back to the sender. You.

One way to fight this is to LOG everything sent to this script. Review the logs. Email logs sometimes reveal what's happening, but this logs at the server level. You should log at the script input level, see what crap people are sending to your script. Once that is in place, you can begin banning IP addresses from sending anything at all to your form. This is still a tedious and time consuming process - having to add another IP every time someone hacks your script - but eventually they give up.

Also review your code, do something to stop the BCC hack, if it's being done - like I say the only way to truly know is log everything sent to the script. If the only email address they can send to is you, well there's not much point in bothing to hack your script, is there?

As for stopping it at the mail server level - the suggestions here are better than what I would have, with one exception: does your mail service contain a "nobody" account? A nobody account allows "anything@yourserver.com" to be piped in to you. So when someone throws two@yourdomain.com, one@yourdomain.com, and three@yourdomain.com at you, it puts it wherever your nobody alias tells it to (admin@yourdomain.com?) DELETE THE NOBODY ACCOUNT and only use valid addresses, this can help cut down what actually comes into your box.

Hope my blatherings are somewhat helpful.

old_expat

12:27 am on Jul 27, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



# I have used SpamAssassin, pretty easy to set up in cPanel. But, I get just as much SPAM. It is labled so it can be easily deleted, but it still has to be downloaded before it can be sorted.

# The SPAM is not coming to my "proper" email address <username@mysite.com> or <contact@mysite.com>. But anything in combination with mysite.com seems to get through.

Ex. SPAM addressed as admin@mysite.com, abcd@mysite.com gets added to the mail file on the server.

I tried changing the username on one acount. Didn't make a bit of difference.

I really would like something that, unless email is sent to username@mysite.com, it gets deleted at the server. I could live with that.

rocknbil

3:00 am on Jul 27, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



addressed as admin@mysite.com, abcd@mysite.com gets added to the mail file on the server.

Correct! I may be using the wrong terminology, but every mail server I've worked with called this a "nobody" account. It usually is the account nobody@yourdomain.com. Sometimes it's called a catchall account. Delete the nobody account and this will put a stop to some of that.

If someone is abusing your web form though, all this will do is stop the spam from coming to you through anonymous email addresses - it won't stop them from abusing your copy of Formmail to spam thousands of addresses, see my comments on that part relating to BCC. What I am trying to say is that all this spam may be an indicator of a larger problem, one which you may not be aware of.

<snip>

[edited by: physics at 10:00 pm (utc) on July 27, 2006]
[edit reason]
[1][edit reason] removed commercial info. [/edit]
[/edit][/1]

BeeDeeDubbleU

11:49 am on Jul 27, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



I just figured that the "¥" character is almost always found within these emails so I set up an Outlook rule to pemanently delete messages with this in the body.

Seems to be working :)

rocknbil

7:25 pm on Jul 27, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I'm only going to repeat this because I have a personal "thing" against this kind of attack. I am available to help in any way I can for anyone who interested in my assistance in stopping it as best as we can.

I am using Formmail.

If you are receiving these emails as a direct result of anny kind of mail-to form processor on your website, it may be indicative of a much larger problem. Filtering them at your inbox only closes your eyes to this problem and they will persist in using your form mailer to spam, possibly sending thousands of emails from your website without your knowledge.

lorax

9:16 pm on Jul 27, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



I use POPFile in conjunction with Pegasus mail. POPFile is an automatic mail classification tool. You set it up so that your email client requests mail through it and it examines all inbound email and classifies it based on it's algo - which you have some control over. The big advantage is that it learns! You can teach it what to accept and reject and you can even tell it to always accept mail based on certain criteria like domain or header info. Mine is currently running at about 99.98% accurate. Then I have rules set up in Pegasus that are tripped by the classifications I added to POPFile.

For example. POPFile adds the label [spam] to the subject line of any email it feels is spam. WHen I open my new mail folder the color of the subject text with this label is red. Pegasus has been set up to delete all email that has that label when I close the new mail window. I can visually scan the list of inbound email to look for mistakes then close the window to get rid of the trash. If I find a mistake I open up POPFile and reclassify it. You do need to keep your POPFile History screen clean. I clean mine out once a day but the program has saved me countless hours of sifting through all the crap.

nancyb

10:27 pm on Jul 27, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



try M_ailwasher. It won't automatically delete from the server (unless you set it up that way with filters - a little dangerous for me), but you can prescreen emails on the server (even answer them and download later) and then delete them without downloading.

netchicken1

10:41 pm on Jul 27, 2006 (gmt 0)

10+ Year Member



To stop spam hitting my admin accouts I do this...

An email coming to this account gets an autoreply saying that because of spam please email abc@website.com instead. The emails in are auto deleted, and the reply give the send, if its a human, a chance to send me real info.

If abc@ gets spammed, just change it to another account 123@

This works for me and means I see no spam.

The webhost set "something" in the emails to "kill" so that messages don't generate error messages.

BeeDeeDubbleU

7:55 am on Jul 28, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Sorry, my last message should have been in this thread, [webmasterworld.com...]

(senior moment)

sandyeggo

8:05 am on Jul 28, 2006 (gmt 0)

10+ Year Member



i just started using postini
spamassasin wasnt keeping up with my junk mail
postini has deleted 99% for me
guess what? its $2 a month with a 60 day trial

2 bucks was worth it 4 sure - for me anyway

lammert

10:05 am on Jul 28, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



spamassasin wasnt keeping up with my junk mail

This is a common problem after the first installation of SpamAssassin. The default settings of the filters let pass quite an amount of spam. For a good filtering, you need to feed the Bayesian filter in SpamAssassin with a group of known spam, and known clean messages. The filter will adapt itself and after some runs the number of spam messages comming through will drastically decrease.

The Bayesian filter in SpamAssassin won't work at all, unless it has been fed with a statistical significant amount of spam, and clean messages. 1000 messages of each kind would be optimal.

fabricator

1:53 am on Jul 31, 2006 (gmt 0)

10+ Year Member



The form mail is the real problem here, firstly rename the script. Instead of calling it formmail.cgi call it e578hcw.cgi, spammers look for scripts with common names.

If your able to re-write your formmail script, have it log ip and timestamp, then use the log to only allow one message per minute/hour/day. You can then add anyone that gets rejected several time a ban list, and have the script ignore anyone on that list. Just remember to put a nice message for rejected messages, "sorry due to abuse you are only allowed to post 1 message per hour."