Linux Today: Linux News On Internet Time.

Putting the Kybosh on Spam (Part 1)

Oct 13, 2005, 00:15 (11 Talkback[s])
(Other stories by Brandioch Conner)

By Brandioch Conner

Here's the short answer on how you ("you" being the recipient) can stop spam: you can't. The real problem is the other email admins and ISP's do not see the need to correctly configure and maintain their own systems.

Spam isn't a problem just because it means you get a lot of mail. Spam eats up bandwidth and there's an increasing probability that you'll delete a legitimate message while wading through the spam.

I'm not even going to define what "spam" is because it means different things to different people and there's always an argument about "what if such and such and such happened, would it still be 'spam'?" Even if you only consider the machine sending it you still run into problems. Right now, I know of five different ways to send spam (listed in order of annoyance to me).

  • Spam zombies
  • Open relays
  • Free email accounts (phisher's phavorite)
  • Legitimate companies that get stupid
  • Spam companies/"bulletproof" ISP's that use their own servers/addresses

"Spam zombies" are bad, right? But how do you identify a spam zombie during the SMTP connection? You can't check the machine itself for infections. All you can check is the data that is published (or the absence thereof) and its actions. Here are some of the characteristics of a "spam zombie."

  • It is in an ISP's "home" or dynamically assigned IP range.
    • But so are some legitimate mail servers run by people who don't understand why they should spend money for a static IP address (and "static" in this context does not mean "I haven't seen it change").
    • Not all ISP's correctly identify those ranges.
  • It does not have a correct DNS/rDNS configuration.
    • But there are also email admins who don't have that either. (And one of those admins also has a box in a dynamic range from their ISP.)
  • Currently, it fails to re-send if it gets a non-fatal error from the receiving server.
    • This means that "greylisting" is, currently, very effective against these.
    • But there are also email admins who have not correctly configured their systems to handle non-fatal errors, either.
    • Not everyone is happy with the delay on legitimate messages.

I'm sure everyone can see where this is going now. I work for a small insurance company and I've still had to make seven specific exceptions for clients who fit one or more the above cases. I'm sure it only gets worse as your company gets larger. And that isn't even counting the spammer's ability to adapt faster than many of the email admins. Spammers are one of the first groups to adopt SPF. I know the problems with SPF so don't bother with that. The point is that the spammers will adopt anything that they believe will get their spam through your defenses.

Spam zombies would be almost completely wiped out if the ISP's would:

  • Correctly identify their sub-nets
  • Completely block outgoing port 25 on their dynamic sub-net.(Businesses should already be blocking outgoing port 25 at their firewall, except for their legitimate mail server.)

But that's not going to happen soon because it isn't financially reasonable for them to do so. For each person who used to send via port 25, the ISP's tech support people would have to deal with a support call from that person. Not to mention the additional training those support techs would need. Not to mention any infrastructure changes that they'd have to implement. Not to mention that some of them do not have the expertise necessary to do so. Long story short, it's not going to happen. Some will (Comcast does), many won't.

Right now, I've dramatically reduced the spam we receive with a combination of tests based upon the above three criteria that cumulate in an unknown box being forced into greylisting. But this is only a temporary and very brittle fix. All it would take for the spam zombies to defeat it would be for them to re-send their spam after 1 hour. That should be a minor code change and it should be easy to push out to all the zombies. Like I said, the spammers are faster to adapt than a lot of the legitimate email admins.

In preparation for that day, I'm collecting statistics on all our sent/received email. When worse comes to worst, I'll fall back to personal white/black lists. Yes, they suck. They suck badly. They require far too much manual labour to maintain or they are too easily defeated if scripted.

The good news is that it looks like 80% (by item, not size) of our email is from/to the same servers, day after day. If you work for an Internet sales company, your numbers are probably a lot different. The bad news is that not all of that 80% is business related. Most of it seems to be personal messages. So the remaining 20% may be more important from a business aspect than the 80%. So the good news may not be that good.

On the other hand, the Spamassassin tests are getting better every day. So even if I accept email from a spam zombie v2.0, there is still a very good chance that it will be correctly identified as spam. The downside is that once my system has accepted the email, I cannot risk deleting it before the user sees it, just in case my system is wrong. So the end user still has to wade through the spam. But the more accurate my system is, the more my users will come to accept its evaluation and so legitimate email may be deleted by them just because they didn't have time to verify it and took the machine's opinion as fact. Not to mention that a lot of the tests being run depend upon other people already receiving the spam and correctly identifying it and reporting it to the sites that my system will check. This is great as long as I'm last on the spammer's list, but when I'm first, the junk comes through.