Tuneups and Tweaks for the Better Spam-Trap

By Carla Schroder | Sep 7, 2004 | Print this Page
http://www.enterprisenetworkingplanet.com/netos/article.php/3404561/Tuneups-and-Tweaks-for-the-Better-SpamTrap.htm

In parts 1 and 2 of our thrilling "Building an Anti-Virus/Anti-Spam Gateway" series, we covered the basic steps for setting up SpamAssassin and Clam Anti-virus with Postfix. This installment is devoted to testing and tweaking, and creating whitelists in Amavisd-new. Whitelists are essential when you set up any kind of spam filtering: It's the best way to make sure your wanted mail gets through.

Postfix Soft Bounces

The first thing to do is configure Postfix for soft bounces. Add this line to /etc/postfix/main.cf:
soft_bounce = yes

"When you're running a server for other users ... filter messages tagged as spam into a special folder, for human eyeball review."
This converts 5xx response codes to 4xx. 4xx codes are temporary failures, and the sending server is told to keep trying. 5xx response codes are permanent failures, so messages are rejected and the sending server is told to go away, and to not try again. Then you must watch your Postfix and Amavisd-new logs like the proverbial hawk to see what is happening. (See "Logging Fun", below.)

For a complete list of SMTP response codes, see RFC 2821.

Setting Up Whitelists In Amavisd-new

There are a couple of ways to do this. One is to enter your list directly into amavisd.conf, like the example in amavisd.conf in Section V:
map { $whitelist_sender{lc($_)}=1 } (qw(
  nobody@cert.org
  owner-alert@iss.net
  slashdot@slashdot.org
  bugtraq@securityfocus.com
  NTBUGTRAQ@LISTSERV.NTBUGTRAQ.COM
.....
  owner-textbreakingnews@CNNIMAIL12.CNN.COM
  yahoo-dev-null@yahoo-inc.com
));

Or you can create a separate file, like /etc/amavisd/whitelist.txt, using the same one-address-per-line format, and call it from amavisd.conf like this:

read_hash(\%whitelist_sender, '/etc/amavisd/whitelist.txt');

If you're a Perl guru, you can also set up ACLs using typical mystical Perl syntax. Read Section V of amavisd.conf for examples.

Tweaking SpamAssassin
Remember, when using Amavisd-new with SpamAssassin, don't bother with SpamAssassin's configuration file. Do all the configurations in amavisd.conf, in Section VII.

This setting controls whether SpamAssassin will run all tests, or omit the ones that require Internet access. Disabling the Internet tests, which include RBLs (Realtime Blackhole Lists), open relay databases, some DNS checks, and optionally Pyzor and Vipul's Razor, will let more spam through.

Vipul's Razor and Pyzor are distributed spam databases of actual spams. Spammers spew forth millions upon millions of the same message; Vipul's Razor and Pyzor use a distributed system of reporting and spamtraps to keep their databases current, and then generate digital signatures of the messages to enable fast checking.

But if you're on a slow or unreliable Internet connection, this will speed up performance:

$sa_local_tests_only = 0;  #set to 1 to disable Internet tests

Don't bother checking small messages, because most spam is over 64K in size:

$sa_mail_body_size_limit = 64*1024; 

The most important and useful SpamAssassin setting is the level at which it tags messages as spam.

$sa_tag2_level_deflt = 6.0; # add 'spam detected' headers at that level

You'll want to watch this closely for several months. Higher numbers let more spam through. Lower numbers are more aggressive, so you run the risk of having legitimate mail tagged as spam.

E-mail newsletters and HTML mail from friends are likely candidates for false positives, or even people with an inexplicable fondness for too many exclamation points, or all-caps. I'm old and grumpy, and I don't care if I lose some messages. Anyone who really wants me will find a way. You know, with archaic communcations devices like telephones and paper letters. But when you're running a server for other users, you probably can't afford to be that cavalier. The safest method is to filter messages tagged as spam into a special folder, for human eyeball review.

Continued on page 2: Some More Fine Tuning

Continued From Page 1

Just for fun, you may customize the spam subject header to suit yourself. The default is this:

$sa_spam_subject_tag = '***SPAM*** ';

Feel free to use words like "loathsome", "parasitical", "evil", "steaming heap", whatever you need to express your true feelings.

Logging Fun

Every piece of this puzzle has its own logfile. These are the logfiles to pay the most attention to:
  • /var/log/syslog
  • /var/log/amavis.log
  • /var/log/mail.log

Refer to /etc/syslog.conf to find the locations for your various syslogs. Amavisd-new should be configured to have its own logfile, instead of dumping everything into /var/log/syslog. See Section III in amavisd.conf:

# true (e.g. 1) => syslog;  false (e.g. 0) => logging to file
$DO_SYSLOG = 0;                 # (defaults to false)


# Log file (if not using syslog)
$LOGFILE = "/var/log/amavis.log"; 
$log_level =0;

The higher the log level, the more verbose. If you are having problems, kick it up to level 3 for debugging. It goes all the way up to level 5, if you really get stuck. Once everything is working smoothly, level 0 is fine. This records startup, exit, and failure messages, and lists viruses detected. You don't want to leave the log level high, unless you enjoy rapidly filling your drives, and generating oceans of output to wade through.

/var/log/mail.log is for Postfix. The different Postfix processes each generate their own log entries: master, smtpd, cleanup, qmgr, cleanup, postfix-script, and smtp. Here's snippet showing Amavisd-new and ClamAV starting up:

Aug 21 12:54:25 windbag amavis[973]: Using internal av scanner code for (primary) Clam Antivirus-clamd
Aug 21 12:54:25 windbag amavis[973]: Found secondary av scanner Clam Antivirus - clamscan at /usr/bin/clamscan
Aug 21 12:54:39 windbag postfix/postfix-script: starting the Postfix mail system
Aug 21 12:54:39 windbag postfix/master[1195]: daemon started -- version 2.1.3

Postfix has the loveliest way of setting your logging levels: if you have problems with a particular domain, you can increase the log level for that domain only. Let's say that mail from a user with a yahoo.com address is not getting through. In main.cf , use the debug_peer_level directive :

debug_peer_list = yahoo.com
debug_peer_level = 3

Run postfix reload, then send messages to your server from yahoo.com and see what happens. Use this simple regexp to quickly find errors in the Postfix log:

$ egrep '(reject|warning|error|fatal|panic):' /var/log/mail.log

See Postfix's DEBUG_README for a complete debugging howto.

Resources