Email Blacklisting Considered Harmful.

Saw another legitimate email bounced as spam today:

This message was created automatically by mail delivery software.

A message that you sent could not be delivered to one or more of its
recipients. This is a permanent error. The following address(es) failed:

  myfriend@myfriendsdomain.com
    (generated from myfriend@domain-on-shared-server.org)
    SMTP error from remote mail server after
    RCPT TO:<myfriend@myfriendsdomain.com>:
    host mx.service-myfriendsdomain-uses.com [216.122.171.54]:
    554 5.7.1 Service unavailable;
    Client host [67.152.129.89] blocked using
    hostkarma.junkemailfilter.com=127.0.0.2;
    Black listed at hostkarma
    http://ipadmin.junkemailfilter.com/remove.php?ip=67.152.129.89

In other words, a completely legitimate mail was bounced because people who use the same mail server as the recipient (or for that matter, the sender) receive too much spam.

Sound surprising? Here’s the scenario:

  1. Sender spammer@spammyspamspam.com sends bad (even virus-laden) email to innocentvictim@domain-on-shared-server.com.
  2. The innocentvictim@ account is configured to forward automatically to innocentvictim’s real email address, like ivictim@gmail.com or innovic@somepersonaldomain.com or whatever.
  3. The recipient domain (gmail or somepersonaldomain) is protected by a spam-filter (in gmail’s case, their own custom filter, in the latter case, a filter like junkemailfilter.com’s service).
  4. The spam filter simply sees spammy mail coming from the shared server.
  5. The shared server gets docked points for sending spam!
  6. Lather. Rinse. Repeat.
  7. After a while, legitimate people get bounced for sending legitimate mail to innocentvictim@domain-on-shared-server.com, because the filtering service that protects the recipient’s final account treats all the forwards as spam, without unpacking them.
  8. Furthermore, mail from innocentvictim@domain-on-shared-server.com starts getting auto-rejected by some recepients, because those recipients use the same filtering services as innocentvictim and, as we already know, innocentvictim’s mail server is being docked points because of all the spammy mail innocentvictim receives and auto-forwards.

In other words, a server from which many people forward mail tends to get blacklisted not because that server originates any spam, but because addresses there receive spam. And who doesn’t receive spam? Right. You begin to see the problem :-). Furthermore, it’s very hard for the filtering service to do better: if the spam-filtering service were not to dock points in that scenario, then the spammers would get clever and structure their original mails to just look like forwarded mails. They don’t care. In fact, they already do that sometimes.

So as far as I can tell, blacklists are kind of inherently broken. I’ve personally had to deal with this problem many times. What I did in this case was go to the URL mentioned in the bounce message and removed our shared server’s IP from the blacklist, using the procedure offered by junkemailfilter.com. But they’ll just re-add us soon, because the source of the problem isn’t going away.

One solution would be for the forwarding source address to insert a special header (containing a unique code) into the mail before it passes the mail along to the final destination. Then on the junkemailfilter.com side, that person would configure their filtering to allow mails with that code through — never treat them as spam. However, that would be a lot of work for most email users, due to the heterogeneity of mail delivery software; I don’t see it as a generally applicable solution.

Another solution would be an interface at junkemailfilter.com whereby users could tell it “I’m auto-forwarding mail to you from domain-on-shared-server.com. Please keep that in mind when deciding whether domain-on-shared-server.com is an originating source for spam.”

Any other ideas?

5 Responses to “Email Blacklisting Considered Harmful.”

  1. dave glasser Says:

    Well, one answer is that innocentvictim@domain-on-shared-server.com should be running its own spam filter before blindly propagating mail across the net.

  2. Karl Fogel Says:

    The whole point is that Innocent Victim wants to use the spam filters offered by the service she forwards to, though. She just doesn’t want all the domains on her shared server penalized because she receives a lot of spam.

    I don’t think “blindly propagating mail across the net” is an accurate description of what’s going on at all. When I forward mail to a service, I know exactly what I am doing and where the mail is going. No normal user would ever think of themselves as behaving irresponsibly when they auto-forward emails from one address to another… nor should they have to.

  3. dave glasser Says:

    Yeah, it really seems that there needs to be an authenticated channel between domain-on-shared-server and gmail (or whatever) to make this work.

  4. Karl Fogel Says:

    Agreed. Not necessarily a secure channel, just an authenticated one. The authentication scheme can be pretty trivial, as it doesn’t take much to raise the cost of doing business (per email) to something beyond what a spammer would be willing to pay.

    Speaking of spam prevention: my reCaptcha for this comment is “emergency flavors”. Oh yeah, gotta get me some o’ those…

  5. Karl Fogel Says:

    I finally solved at least the sending-mail-via-shared-server part of this problem like this:

    http://viewvc.red-bean.com/kfogel?view=revision&revision=3310

    Not ideal, needless to say :-(.

Leave a Reply