May 2010

Saw another legitimate email bounced as spam today:

This message was created automatically by mail delivery software.

A message that you sent could not be delivered to one or more of its
recipients. This is a permanent error. The following address(es) failed:

  myfriend@myfriendsdomain.com
    (generated from myfriend@domain-on-shared-server.org)
    SMTP error from remote mail server after
    RCPT TO:<myfriend@myfriendsdomain.com>:
    host mx.service-myfriendsdomain-uses.com [216.122.171.54]:
    554 5.7.1 Service unavailable;
    Client host [67.152.129.89] blocked using
    hostkarma.junkemailfilter.com=127.0.0.2;
    Black listed at hostkarma
    http://ipadmin.junkemailfilter.com/remove.php?ip=67.152.129.89

In other words, a completely legitimate mail was bounced because people who use the same mail server as the recipient (or for that matter, the sender) receive too much spam.

Sound surprising? Here’s the scenario:

  1. Sender spammer@spammyspamspam.com sends bad (even virus-laden) email to innocentvictim@domain-on-shared-server.com.
  2. The innocentvictim@ account is configured to forward automatically to innocentvictim’s real email address, like ivictim@gmail.com or innovic@somepersonaldomain.com or whatever.
  3. The recipient domain (gmail or somepersonaldomain) is protected by a spam-filter (in gmail’s case, their own custom filter, in the latter case, a filter like junkemailfilter.com’s service).
  4. The spam filter simply sees spammy mail coming from the shared server.
  5. The shared server gets docked points for sending spam!
  6. Lather. Rinse. Repeat.
  7. After a while, legitimate people get bounced for sending legitimate mail to innocentvictim@domain-on-shared-server.com, because the filtering service that protects the recipient’s final account treats all the forwards as spam, without unpacking them.
  8. Furthermore, mail from innocentvictim@domain-on-shared-server.com starts getting auto-rejected by some recepients, because those recipients use the same filtering services as innocentvictim and, as we already know, innocentvictim’s mail server is being docked points because of all the spammy mail innocentvictim receives and auto-forwards.

In other words, a server from which many people forward mail tends to get blacklisted not because that server originates any spam, but because addresses there receive spam. And who doesn’t receive spam? Right. You begin to see the problem :-). Furthermore, it’s very hard for the filtering service to do better: if the spam-filtering service were not to dock points in that scenario, then the spammers would get clever and structure their original mails to just look like forwarded mails. They don’t care. In fact, they already do that sometimes.

So as far as I can tell, blacklists are kind of inherently broken. I’ve personally had to deal with this problem many times. What I did in this case was go to the URL mentioned in the bounce message and removed our shared server’s IP from the blacklist, using the procedure offered by junkemailfilter.com. But they’ll just re-add us soon, because the source of the problem isn’t going away.

One solution would be for the forwarding source address to insert a special header (containing a unique code) into the mail before it passes the mail along to the final destination. Then on the junkemailfilter.com side, that person would configure their filtering to allow mails with that code through — never treat them as spam. However, that would be a lot of work for most email users, due to the heterogeneity of mail delivery software; I don’t see it as a generally applicable solution.

Another solution would be an interface at junkemailfilter.com whereby users could tell it “I’m auto-forwarding mail to you from domain-on-shared-server.com. Please keep that in mind when deciding whether domain-on-shared-server.com is an originating source for spam.”

Any other ideas?

[Reblogged from my post at Talking Points Memo.]

Bob Ostertag has a short but lancing piece in the Huff Post today about how the New York Times got astroturfed by an organization calling itself the “Gulf of Mexico Foundation”. The NYT describes them as a “conservation group” when the evidence is that they are, essentially, an oil industry front.

I understand that the pressures of reporting a story as it happens are real and sometimes require cutting corners, but if you don’t have time to do fact checking, then why not simply avoid making any factual assertions you don’t have to make? Nothing about the story requires labeling them a “conservation group”. Just, you know, leave off the word “conservation” — that’s all it takes.

(Of course, doing some elementary digging into the group’s governance would have been even better, but failing that, the NYT could at least avoid doing their PR work for them.)