[TAG] State of the antispam regime (was: Closed account question)

[Rick Moen]

I wrote:

> Herewith, a brief report on the state of system spam-rejection.  
> 
> As a reminder, TAG's setup is challenging, in that list policy
> precludes requiring subscription before posting.  Thus, a prime antispam
> tool isn't available.

[Description of the existing toolset:  Exim 4.60, sa-exim 4.2,
EximConfig 2.0, and SpamAssassin 3.1 -- along with its design goals, 
current weaknesses, and reasons why I've been reluctant to implement
some optional extensions and in general am very careful about breakage.]

Warning: Following paragraphs include opinions, which you are welcome to 
either adopt, take home, and admire, _or_ scowl at and hurl imprecations
towards, as local cerebral wiring policy dictates.  Readers looking for
guaranteed objective truth should stick to mathematics -- and even then
stay far away from anything touched by Kurt G?del.


Spam defence is endlessly controversial for lots of reasons, including
inherent drawbacks (false positives, false negatives, other types of
collateral damage) in even the best regimes.  No matter what tactics you
use, you annoy someone -- and mailing lists (being glorified
mail-forwarding devices) have proven to be Ground Zero for the spam
problem and resulting controversies.

The "luv-main" mailing list, of the Linux Users of Victoria (Australia) has
recently erupted in one such donnybrook, where the overwhelming majority 
(and pretty much _all_ of the more-technical members) wish to convert
the mailing list to publicly searchable archives, while a vocal minority 
stand in the way, protesting their right to continue hiding their
mailing addresses from spammers.  Many innocent electrons have been
killed in the resulting discussion, but at least it was established to
the satisfaction of most that the only reasons hide-from-spammers tactics 
"work for many people" (as a proponent put it) are short usage periods
and dumb luck:  Over time, any address used for mail will become
discovered by spammers through any of several diverse means, including
exposure on other people's virus-infected Windows boxes.

LUV's likely compromise solution will be creation of a fully public
"luv" mailing list alongside the obscured "luv-main" one, with the
expectation that the latter ghetto will wither and die.  (Naturally, the
minority aren't happy with that proposal, either.)

On the other side of the Pacific Ocean, the Silicon Valley Linux User
Group's mailing lists are gradually becoming more spammy, despite using
(modulo versions) _exactly_ the same MTA and spam-rejection software 
I use, because SVLUG's Linux server has been completely unmaintained for
2+ years.

I'm not unsympathetic towards hide-from-spammers people:  As they often
point out, most use mail facilities over which they have no
administrative control -- often, in fact, their work mailboxes.  A
deluge of spam would make their lives miserable and might even interfere
with their professional lives.  This perceived loss-of-control threat 
increases their stress levels immensely, and impels them to make
sometimes emotional demands on listadmins and others.  (I frequently 
get mail saying "I [/someone else] inadvertantly revealed my private
e-mail address on mailing list $FOO.  Would you mind please removing it
from the public archive?"  I always _do_ help such people.  Even though
I don't share their general approach, it's the kind thing to do.)


In any event, I'd been pondering both the slow deterioration of SVLUG's
spam-defences (from my front-row seat as its mailing list moderator) and
my own system's occasional slippage on TAG mail -- e.g., the
aforementioned twelve spams over the last six weeks: 5 advance-fee
frauds, 4 phishing frauds, 1 in Russian, 1 Windows malware, and 1 UCE.
As with SVLUG's system, I realised that I could probably do better,
after a bit of updating.

So, a couple of days ago, I _did_ some.  You'll maybe have noticed that
we've had absolute, blissful silence on the spam front, since then -- 
which might be coincidence, or maybe not.

I figured out that one easy path to the low-hanging fruit was:  beef up
the SpamAssassin rulesets.  In an ideal world, this would not be my
preferred approach:  SA is a very beefy and slow Perl app, and
those _same_ heuristics, implemented as improvements to the Exim4
front-end rulesets would, mutatis mutandis, be much more desirable.
However, those aren't at hand, while there's a veritable bazaar in
third-party ruleset files for SA, right here:

http://wiki.apache.org/spamassassin/CustomRulesets

The ones I dropped into /etc/mail/spamassassin, a couple of days ago,
were as follows:

70_zmi_german.cf     Catches German language spam.
Chinese_rules.cf     Rules to catch spams written in Chinese.
mime_validate.cf     Finds MIME errors common in mails sent by bulk mailers.
blacklist_ro.cf      Catches spams written in Romanian or by Romanian spammers.
evilnumbers.cf       Phone #s, PO boxes, & street addresses harvested from spam.
chickenpox.cf        Looks for words broken up by extraneous symbols.
french_rules.cf      Catches spams written in French.
malware.cf           Detects URLs known to point to malware.

There's a standard cronjob (Rules du Jour) to keep these and others
updated; I haven't implemented that yet, as I'm still testing.  

Initially, I also installed this one:  

  sa-blacklist:  a large set of blacklist entries of domains and IP addresses. 

This turned out to an 11MB(!) ruleset file.  For a Perl script.  I
rather recklessly _did_ give it a try, resulting in the kernel
out-of-memory killer going on a shooting spree (on my antique 256MB RAM
PIII server) about five minutes later.

In addition, _also_ quite usefully, I'm sure (if not more so than the SA
improvements), I dropped in updated nine updated ACL files for the
J.P. Boggis's EximConfig suite of Exim4 rules (and other things):
http://www.jcdigita.com/eximconfig/#ACLs

...which brings us to the present, and the (for now) absence of new spam
arrivals.  Moral of the story:  It's possible (at least, if you're an MTA
operator) to have a credible, livable alternative to the venerable
"hide from spammers" stance:  Use better technology, apply it
intelligently, and be aware that some ongoing maintenance is required.

Personally, I would regard it as beneath my dignity to do otherwise.  
Internet presence is my community's core competency, so I'll be damned if
I'm going to surrender even an inch of it.

One caveat:  Silence isn't necessarily blissful.  There will always be
collateral damage, and one must keep an eye out for addresses, IPs,
hostnames, etc. that should be whitelisted.

-- 
Cheers,                   I have yet to see any problem, however complicated, 
Rick Moen                 which, when you looked at it in the right way, did 
rick at linuxmafia.com       not become still more complicated. -- Poul Anderson