Anti-Spam Email server

In this post I'll show you how to install an anti-spam smart host relay server, based on Ubuntu 12.04 LTS, that will include:

Postfix w/Bayesian Filtering and Anti-Backscatter (Relay Recipients via look-ahead), Apache2, Mysql, Dnsmasq, MailScanner (Spamassassin, ClamAV, Pyzor, Razor, DCC-Client), Baruwa, SPF Checks, FuzzyOcr, Sanesecurity Signatures, PostGrey, KAM, Scamnailer, FireHOL (Iptables Firewall) and Relay Recipients Script.

Continue reading for the instructions.

Install main packages:

First we need to add Baruwa's repo to /etc/apt/sources.list

deb http://apt.baruwa.org/ubuntu precise main

To use this repo add the gpg key:

wget -O - http://apt.baruwa.org/baruwa-apt-keys.gpg | apt-key add -

Now we'll install some dependencies:

apt-get install binutils cpp fetchmail flex gcc libarchive-zip-perl libc6-dev libcompress-raw-zlib-perl libcompress-bzip2-perl libpcre3 libpopt-dev lynx m4 make ncftp nmap openssl perl perl-modules unzip zip zlib1g-dev autoconf automake1.9 libtool bison autotools-dev g++ build-essential telnet wget gawk

Use bash as default shell:

dpkg-reconfigure dash

Caching Dnsmasq:

apt-get install dnsmasqvi /etc/dnsmasq.conf

and make Dnsmasq listen on localhost:

listen-address=127.0.0.1

Install Mysql:

apt-get install mysql-client mysql-server libdbd-mysql-perl

Install and configure Postfix:

apt-get install postfix postfix-mysql postfix-doc procmail

You will be asked two questions. Answer as follows:

General type of mail configuration: --> Internet Site
System mail name: --> server1.example.com

Stop Postfix

postfix stop

We’ll want to edit Postfix with the below:

vi master.cf

We need to add two items below the pickup service type. The pickup service "picks up" local mail (local meaning "on this machine") and delivers it. This is a way to bypass content filtering for mail generated by this machine.

It should look like this when you are done:

pickup fifo n - - 60 1 pickup -o content_filter= -o receive_override_options=no_header_body_checks

Edit main.cf with a script:

vi /usr/src/postfix.sh

#!/bin/sh
postconf -e "alias_maps = hash:/etc/aliases"
newaliasespostconf -e "myorigin = domain.tld"postconf -e "myhostname = server1.domain.tld"postconf -e "mynetworks = 127.0.0.0/8, 192.168.0.0/24"postconf -e "message_size_limit = 10485760"
postconf -e "local_transport = error:No local mail delivery"
postconf -e "mydestination = "
postconf -e "local_recipient_maps = "
postconf -e "relay_domains = mysql:/etc/postfix/mysql-relay_domains.cf"
postconf -e "relay_recipient_maps = mysql:/etc/postfix/mysql-relay_recipients.cf"
postconf -e "transport_maps = mysql:/etc/postfix/mysql-transports.cf"
postconf -e "virtual_alias_maps = hash:/etc/postfix/virtual"
postconf -e "disable_vrfy_command = yes"
postconf -e "strict_rfc821_envelopes = no"
postconf -e "smtpd_banner = $myhostname ESMTP SpamSnake"
postconf -e "smtpd_delay_reject = yes"
postconf -e "smtpd_recipient_limit = 100"
postconf -e "smtpd_helo_required = yes"
postconf -e "smtpd_client_restrictions = permit_sasl_authenticated, permit_mynetworks, permit"
postconf -e "smtpd_helo_restrictions = permit_sasl_authenticated, permit_mynetworks, permit"
postconf -e "smtpd_sender_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_non_fqdn_sender, reject_unknown_sender_domain, permit"
postconf -e "smtpd_recipient_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_unknown_recipient_domain, reject_unauth_destination, whitelist_policy, grey_policy, rbl_policy, spf_policy, permit"
postconf -e "smtpd_data_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_unauth_pipelining"
postconf -e "smtpd_restriction_classes = spf_policy, rbl_policy, grey_policy, whitelist_policy"
postconf -e "spf_policy = check_policy_service unix:private/policy"
postconf -e "rbl_policy = reject_rbl_client zen.spamhaus.org, reject_rbl_client bl.spamcop.net"
postconf -e "grey_policy = check_policy_service unix:private/greyfix"
postconf -e "whitelist_policy = check_client_access mysql:/etc/postfix/mysql-global_whitelist.cf, check_sender_access mysql:/etc/postfix/mysql-global_whitelist.cf"
postconf -e "header_checks = regexp:/etc/postfix/header_checks"
touch /etc/postfix/virtual
echo "root administrator@example.com" >> /etc/postfix/virtual && echo "abuse administrator@example.com" >> /etc/postfix/virtual && echo "postmaster administrator@example.com" >> /etc/postfix/virtual
postmap /etc/postfix/virtual
touch /etc/postfix/header_checks
echo "/^Received:/ HOLD" >> /etc/postfix/header_checks
postmap /etc/postfix/header_checks
cat > /etc/postfix/mysql-global_whitelist.cf <<EOF
#mysql-global_whitelist
user = baruwa
password = password
dbname = baruwa
query = select concat('PERMIT') 'action' from lists where from_address='%s' AND list_type='1';
hosts = 127.0.0.1
EOF
cat > /etc/postfix/mysql-relay_domains.cf <<EOF
#mysql-relay_domains
user = baruwa
password = password
dbname = baruwa
query = select concat(address, ' ', 'OK') 'domain' from user_addresses where user_addresses.address='%s' and user_addresses.enabled='1';
hosts = 127.0.0.1
EOF
cat > /etc/postfix/mysql-relay_recipients.cf <<EOF
#mysql-relay_recipients
user = baruwa
password = password
dbname = baruwa
query = select concat('@', address, 'OK') 'email' from user_addresses where user_addresses.address='%d';
hosts = 127.0.0.1
EOF
cat > /etc/postfix/mysql-transports.cf <<EOF
#mysql-transports
user = baruwa
password = password
dbname = baruwa
query = select concat('smtp:[', mail_hosts.address, ']', ':', port) 'transport' from mail_hosts, user_addresses where user_addresses.address = '%s' AND user_addresses.id = mail_hosts.useraddress_id;
hosts = 127.0.0.1
EOF

Now make it executable:

chmod +x /usr/src/postfix.sh

and run it using

/usr/src/postfix.sh

Postfix Recipient Callout:

This feature queries the recipient server to see if the recipient exists. If not, it replies with a 550 error to the sending server and drops the connection. If the user does exist, the SpamSnake will continue processing the email. This is just another method to prevent backscatter, but comes at a price. Read up on it at http://www.postfix.org/ADDRESS_VERIFICATION_README.html. You can skip this method and use the script method (later on in this guide) if you decide it will bog down your server.

vi /etc/postfix/main.cf

and add the following:

verify_recipient = reject_unknown_recipient_domain, reject_unverified_recipientlook_ahead = check_recipient_access hash:/etc/postfix/accessunverified_recipient_reject_code = 550address_verify_map = btree:/var/lib/postfix/verify

Add this to your smtpd_restriction_classes:

verify_recipient, look_ahead

Add this to smptd_recipient_restrictions:

smtpd_recipient_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_unauth_destination, look_ahead, whitelist_policy, grey_policy, rbl_policy, spf_policy, permit

Create the access file:

touch /etc/postfix/access

and add your domains:

domainA.com verify_recipientdomainB.com verify_recipient

Note: Make sure to add valid domains you're filtering for.

Postmap it:

postmap /etc/postfix/access

Final look at the Postfix install:

less /etc/postfix/main.cf

Check the contents of the file for errors and repair if needed. Fire up Postfix:

Start Postfix

postfix start

Check that Postfix responds:

telnet 127.0.0.1 25

You should see:

220 [yourFQDNhere] ESMTP Postfix (Ubuntu)

Install MailScanner (Apparmor, Clamav, DCC, Pyzor, Razor and Spamassassin):

aptitude install razor pyzor clamav-daemon libclamav6 apparmor mailscanner libltdl7

Apparmor configuration for Clamav

Add clamav to the www-data group so that it can access the directory:

usermod -a -G www-data clamav

Now edit the profile for clamd:

vi /etc/apparmor.d/usr.sbin.clamd

and add the Incoming folder to the list of folders:

/usr/sbin/clamd {
#clamav
/var/spool/MailScanner/** rw,
/var/spool/MailScanner/incoming/** rw,
}

Reload apparmor:

/etc/init.d/apparmor reload

DCC 32bit/64bit Configuration:

Install DCC from .deb source, you can just download the necessary packages:

cd /tmp
wget http://ppa.launchpad.net/jonasped/ppa/ubuntu/pool/main/d/dcc/dcc-common_1.3.130-0ubuntu1~ppa2~precise1_$(uname -m | sed -e 's/x86_64/amd64/' -e 's/i686/i386/').deb && dpkg -i dcc-common_1.3.130-0ubuntu1~ppa2~precise1_$(uname -m | sed -e 's/x86_64/amd64/' -e 's/i686/i386/').deb
wget http://ppa.launchpad.net/jonasped/ppa/ubuntu/pool/main/d/dcc/dcc-client_1.3.130-0ubuntu1~ppa2~precise1_$(uname -m | sed -e 's/x86_64/amd64/' -e 's/i686/i386/').deb && dpkg -i dcc-client_1.3.130-0ubuntu1~ppa2~precise1_$(uname -m | sed -e 's/x86_64/amd64/' -e 's/i686/i386/').deb

Or you can add the PPA and install:

add-apt-repository ppa:jonasped/ppaaptitude updateaptitude install  dcc-client dcc-common

Test our installation with:

cdcc info

You should get 'requests ok' from the servers.

Pyzor Configuration
Because pyzor doesn’t work with python2.6 very well, the workaround is to append the following to the first line of /usr/bin/pyzor to make it look like:

#!/usr/bin/python -Wignore::DeprecationWarning

Here we supply the IP address of the Pyzor server to Pyzor. This will create the server's IP address in a servers file therein. Then it will test the connection. If you are behind a firewall, open port 24441/udp in and out to your server. While you're at it also open up 6277/udp for DCC, 2703/tcp for Razor and 783/tcp for SpamAssassin:

mkdir /var/lib/MailScannerpyzor --homedir=/var/lib/MailScanner discover
pyzor ping

Razor Configuration

Create the .razor configuration:

cd && rm /etc/razor/razor-agent.confmkdir /var/lib/MailScanner/.razorrazor-admin -home=/var/lib/MailScanner/.razor -createrazor-admin -home=/var/lib/MailScanner/.razor -discoverrazor-admin -home=/var/lib/MailScanner/.razor -register

Edit the razor agent conf.

vi /var/lib/MailScanner/.razor/razor-agent.conf

And set:

debuglevel = 0razorhome = /var/lib/MailScanner/.razor/

Install dependencies:

apt-get install libconvert-tnef-perl libdbd-sqlite3-perl libfilesys-df-perl libmailtools-perl libmime-tools-perl libmime-perl libnet-cidr-perl libsys-syslog-perl libio-stringy-perl libfile-temp-perl libole-storage-lite-perl libarchive-zip-perl libsys-hostname-long-perl libnet-cidr-lite-perl libhtml-parser-perl libdb-file-lock-perl libnet-dns-perl libncurses5-dev libdigest-hmac-perl libdigest-sha-perl libnet-ip-perl liburi-perl libfile-spec-perl spamassassin libnet-ident-perl libmail-spf-perl libmail-dkim-perl dnsutils libio-socket-ssl-perl gdebi-core

Edit Crontab:

vi /etc/crontab

And add:

#Mailscanner
37 5 * * * /usr/sbin/update_phishing_sites &> /dev/null
07 * * * * /usr/sbin/update_bad_phishing_sites &> /dev/null
#58 23 * * * /usr/sbin/clean.quarantine &> /dev/null
42 * * * * /usr/sbin/update_virus_scanners &> /dev/null
3,23,43 * * * * /usr/sbin/check_mailscanner &> /dev/null

Install SpamAssassin

First we need to disable the default SpamAssassin configuration file:

mv /etc/spamassassin/local.cf/etc/spamassassin/local.cf.disabled

Now let's backup the SpamAssassin configuration file in MailScanner then edit:

cp /etc/MailScanner/spam.assassin.prefs.conf /etc/MailScanner/spam.assassin.prefs.conf.back

SpamAssassin SQL Bayes
Pre-requisities: You'll need the perl-DBI and perl-DBD-MySQL modules installed.

Assumptions and Variables:

SpamAssassin Bayes Database Name: sa_bayesSpamAssassin Bayes Database UserName: sa_userSpamAssassin Bayes Database Password: sa_password

Create the MySQL database on the server where you intend on storing the bayesian information.

mysql -u root -pmysql> create database sa_bayes;mysql> GRANT ALL ON sa_bayes.* TO sa_user@localhost IDENTIFIED BY 'sa_password';mysql> flush privileges;

Import database structure:

mysql -u sa_user -p sa_bayes < /usr/share/doc/spamassassin/sql/bayes_mysql.sql

To enable DCC:

vi /etc/spamassassin/v310.pre

And add:

loadplugin Mail::SpamAssassin::Plugin::DCC

Edit Spam Assassin's conf file:

vi /opt/MailScanner/etc/spam.assassin.prefs.conf

and add the following to the top:

#pyzoruse_pyzor 1pyzor_options --homedir /var/lib/MailScanner/
#razoruse_razor2 1razor_config /var/lib/MailScanner/.razor/razor-agent.confFix DCC path:
dcc_path /usr/bin/dccproc

Update header string:

bayes_ignore_header X-YOURDOMAIN-COM-MailScannerbayes_ignore_header X-YOURDOMAIN-COM-MailScanner-SpamCheckbayes_ignore_header X-YOURDOMAIN-COM-MailScanner-SpamScorebayes_ignore_header X-YOURDOMAIN-COM-MailScanner-Information#use_auto_whitelist 0

"YOURDOMAIN-COM" should be replaced with whatever you used for "%org-name%" in the MailScanner.conf file. Leave the "X-" in place. This is the same orgname used in the MailScanner.conf above.

Add sql connection string to the bottom:

bayes_store_module Mail::SpamAssassin::BayesStore::SQLbayes_sql_dsn DBI:mysql:sa_bayes:localhostbayes_sql_username sa_userbayes_sql_password sa_passwordbayes_sql_override_username rootvi v310.pre

and comment out domainkeys since DKIM has superseeded it:

#loadplugin Mail::SpamAssassin::Plugin::DomainKeys

Add it to cron:

30 01 * * * /usr/bin/sa-learn --force-expire --sync -p /etc/MailScanner/spam.assassin.prefs.conf

Install Apache and mod-passenger:

aptitude install apache2 libapache2-mod-passenger

Create the following to prevent an error in a lint test:

mkdir /var/www/.spamassassin

Install missing perl packages:

aptitude install libencode-detect-perl libcrypt-openssl-rsa-perlperl -MCPAN -e shellinstall IP::Country::Fastexit

Set permissions to bring it all together:

chown -R postfix:www-data /var/spool/postfix/holdchmod -R ug+rwx /var/spool/postfix/hold

Test out the setup:

spamassassin -x -D -p /opt/MailScanner/etc/spam.assassin.prefs.conf --lint

Check for lines like:

debug: bayes: Database connection establisheddebug: bayes: found bayes db version 3debug: bayes: Using userid: 2

You should see lines come up with DCC, Pyzor and Razor that say loading plugin and hopefully no errors.

MailScanner Configuration:

We need to make a directory for SpamAssassin in the spool and give postfix permissions to it, if you run sa-learn --force as root, bayes databese that is stored in these directories will change to root:root and spamassassin will error looking at the db. Just keep an eye on the mail.log and you'll remember to change the permissions back. Also disable the MailScanner default configs:

mkdir /var/spool/MailScanner/spamassassin

Backup your MailScanner.conf file:

cp /etc/MailScanner/MailScanner.conf /etc/MailScanner/MailScanner.conf.dist

Change the following parameters in MailScanner.conf with the following script:

vi /usr/src/mailscanner.sh

Add:

sed -i "/^%org-name% =/ c\%org-name% =orgname" /etc/MailScanner/MailScanner.conf
sed -i "/^%org-long-name% =/ c\%org-long-name% = longorgname" /etc/MailScanner/MailScanner.conf
sed -i "/^%web-site% =/ c\%web-site% = www.domain.tld" /etc/MailScanner/MailScanner.conf
sed -i "/^Run As User =/ c\Run As User = postfix" /etc/MailScanner/MailScanner.conf
sed -i "/^Run As Group =/ c\Run As Group = www-data" /etc/MailScanner/MailScanner.conf
sed -i "/^Incoming Work Group =/ c\Incoming Work Group = clamav" /etc/MailScanner/MailScanner.conf
sed -i "/^Incoming Work Permissions =/ c\Incoming Work Permissions = 0640" /etc/MailScanner/MailScanner.conf
sed -i "/^Incoming Queue Dir =/ c\Incoming Queue Dir = /var/spool/postfix/hold" /etc/MailScanner/MailScanner.conf
sed -i "/^Outgoing Queue Dir =/ c\Outgoing Queue Dir = /var/spool/postfix/incoming" /etc/MailScanner/MailScanner.conf
sed -i "/^MTA =/ c\MTA = postfix" /etc/MailScanner/MailScanner.conf
sed -i "/^Quarantine User =/ c\Quarantine User = root" /etc/MailScanner/MailScanner.conf
sed -i "/^Quarantine Group =/ c\Quarantine Group = www-data" /etc/MailScanner/MailScanner.conf
sed -i "/^Quarantine Permissions =/ c\Quarantine Permissions = 0660" /etc/MailScanner/MailScanner.conf
sed -i "/^Quarantine Whole Message =/ c\Quarantine Whole Message = yes" /etc/MailScanner/MailScanner.conf
sed -i "/^Virus Scanners =/ c\Virus Scanners = clamd" /etc/MailScanner/MailScanner.conf
sed -i "/^Monitors for ClamAV Updates =/ c\Monitors for ClamAV Updates = /var/lib/clamav/*.cld /var/lib/clamav/*.cvd" /etc/MailScanner/MailScanner.conf
sed -i "/^Clamd Socket =/ c\Clamd Socket = /var/run/clamav/clamd.ctl" /etc/MailScanner/MailScanner.conf
sed -i "/^Clamd Lock File =/ c\Clamd Lock File = /var/run/clamav/clamd.pid" /etc/MailScanner/MailScanner.conf
sed -i "/^Spam Subject Text =/ c\Spam Subject Text = ***SPAM***" /etc/MailScanner/MailScanner.conf
sed -i "/^Spam Actions =/ c\Spam Actions = deliver store" /etc/MailScanner/MailScanner.conf
sed -i "/^High Scoring Spam Actions =/ c\High Scoring Spam Actions = store delete" /etc/MailScanner/MailScanner.conf
sed -i "/^Non Spam Actions =/ c\Non Spam Actions = deliver store" /etc/MailScanner/MailScanner.conf
sed -i "/^SpamAssassin User State Dir =/ c\SpamAssassin User State Dir = /var/spool/MailScanner/spamassassin" /etc/MailScanner/MailScanner.conf

Make the script executable:

chmod +x mailscanner.sh

and run using

/usr/src/mailscanner.sh

Edit /etc/default/mailscanner. There you will have to set the variable run_mailscanner to 1.

Install and configure Saslauthd:

aptitude install sasl2-bin libsasl2-modules-ldap

edit the configuration:

vi /etc/default/saslauthd

change:

MECHANISMS="pam"

to:

MECHANISMS="rimap"

Set MECH_OPTIONS to the ip address of your imap server

And at the bottom change:

OPTIONS="-c -m /var/run/saslauthd"

to:

OPTIONS="-c -m /var/spool/postfix/var/run/saslauthd -r"

Set the correct permissions:

chown postfix /var/spool/postfix/var/run/saslauthdchgrp sasl /var/spool/postfix/var/run/saslauthdchmod 710 /var/spool/postfix/var/run/saslauthd

Start the system:

/etc/init.d/saslauthd start
/etc/init.d/mailscanner start
/etc/init.d/postfix start

Check your logs for errors:

tail -f /var/log/mail.log

Check your mail.log (tail –f /var/log/mail.log) and you should see the following:

Jun 13 12:18:23 hoshi MailScanner[26388]: MailScanner E-Mail Virus Scanner version 4.81.4 starting...

Install Baruwa:

First you need to install rabbitmq-server

aptitude install rabbitmq-server

And configure it:

rabbitmqctl add_user baruwa passwordrabbitmqctl add_vhost baruwarabbitmqctl set_permissions -p baruwa baruwa ".*" ".*" ".*"/etc/init.d/rabbitmq-server restart

Now install Baruwa:

aptitude install baruwa

You'll be prompted to setup the baruwa db for logging, including an access username and password, which will be used by the Baruwa scripts. You'll also be prompted to setup the admin user information for the Baruwa frontend.

Edit Baruwa's setings:

vi /usr/share/pyshared/baruwa/settings.py

Change the Quarantine hosturl to your liking:

QUARANTINE_REPORT_HOSTURL = 'http://baruwa-alpha.local'

Setup the Baruwa DB:

baruwa-admin syncdb --noinputbaruwa-admin migratebaruwa-admin syncdb --noinputbaruwa-admin migrate

Set MailScanner to use Baruwa's DB:

vi /etc/MailScanner/MailScanner.conf

You need to make sure that the following options are set:

Always Looked Up Last = &BaruwaSQLIs Definitely Not Spam = &BaruwaWhitelistIs Definitely Spam = &BaruwaBlacklistRequired SpamAssassin Score = &BaruwaLowScoreHigh SpamAssassin Score = &BaruwaHighScore

Enable Baruwa in Apache

a2dissite 000-defaultservice apache2 reload

Restart MailScanner:

/etc/init.d/mailscanner restart

Point your browser to http://hostname_used login with admin user and password and start working. You can now use the interface to add users and process messages, etc.

If you can't login use:

baruwa-admin createsuperuser

to create an initial admin user.

Setup Instructions for Baruwa (on the web insterface):

• Log into Baruwa as admin and navigate to:
  
  • --> Settings --> Accounts --> Create Account
• Once you've created the user account, you'll get two new tabs on that page, Profile Settings and Associated Addresses.
• Fill out Profile Settings choosing Domain Admin and set a low score of 6 and a high score of 9 and check scan email.
• Click the + sign under Associated Addresses and enter a domains for which the user is the admin of eg. domain.com. domain.com will show up under Associated Addresses.
• Click on domain.com and you'll be taken into Domain Information, where you'll be able to setup SMTP delivery information. Go ahead and add the receiving smtp server, or the ip of the receiving smtp server. Select enable and if you use a non-standard port, set it, otherwise use 25. Once you've done that, you can click on the test button next to the pencil, to see if your receiving server will accept the connection.

Now, you can log out as admin, and log in as the user you just setup and mails should start flowing.

The relay_recipients, relay_domains and transports settings in /etc/postfix/main.cf will use the entries you've provided in Baruwa. Therefore, no hash file is required.
The associated queries in the mysql cf files will pull the result in the proper format and feed it to postfix for use.

However, if you'd like to use a hash for any of your config files, use the following as an example of how to setup /etc/postfix/main.cf:

relay_recipient_maps = hash:/etc/postfix/relay_recipients

Of course, you would have to create the hash file(s), populate it and postmap it for postfix to use.

Note: If you do end up using hash for relay_recipients for specific domains, you'll have to remove that domain from /etc/postfix/access. All other domain users can still be verified using look_ahead

Also, if you need to do mx lookups, you'll have to edit /etc/postfix/mysql-transports.cf query to look like:

concat('smtp:', mail_hosts.address, ':', port) 'transport'The [ and ] were removed to allow MX lookups.

Install and configure SPF:

aptitude install postfix-policyd-spf-perl

Then we edit /etc/postfix/master.cf

vi /etc/postfix/master.cf

and add the following stanza at the end:

policy unix - n n - - spawn user=nobody argv=/usr/sbin/postfix-policyd-spf-perl

Then restart Postfix:

/etc/init.d/postfix restart

Install and Configure FuzzyOcr:

FuzzyOCR has some prerequisites like ocrad and gocr that we can install like this:

apt-get install fuzzyocr3 netpbm gifsicle libungif-bin gocr ocrad libstring-approx-perl libmldbm-sync-perl libdigest-md5-perl libdbd-mysql-perl imagemagick tesseract-ocr

So FuzzyOCR is now installed, now we need to configure it. FuzzyOCR's configuration file is /etc/spamassassin/FuzzyOcr.cf.
In that file almost everything is commented out. We open that file now and make some modifications:

vi /etc/spamassassin/FuzzyOcr.cf

Uncomment the following lines:

focr_global_wordlist /etc/spamassassin/FuzzyOcr.wordsfocr_preprocessor_file /etc/spamassassin/FuzzyOcr.prepsfocr_scanset_file /etc/spamassassin/FuzzyOcr.scansets<focr_enable_image_hashing 3focr_digest_db /etc/spamassassin/FuzzyOcr.hashdbfocr_db_hash /etc/spamassassin/FuzzyOcr.dbfocr_db_safe /etc/spamassassin/FuzzyOcr.safe.dbfocr_bin_helper convert, tesseract

Comment out the path:

#focr_path_bin /usr/local/netpbm/bin:/usr/local/bin:/usr/bin

We will be storing the image hashes in a mysql database to improve on performance such that images that we have already scanned do not get scanned again as OCR is a resource intense activity.
Create MySQL Database:

The sql script creates the database for fuzzyocr:

vi FuzzyOcr.mysql

Add:

CREATE DATABASE FuzzyOcr;use FuzzyOcr;
CREATE TABLE `Hash` ( `key` varchar(255) NOT NULL default '', `basic` varchar(64) NOT NULL default '', `fname` varchar(160) NOT NULL default '', `ctype` varchar(64) NOT NULL default '', `ftype` smallint(5) unsigned NOT NULL default '0', `match` int(10) unsigned NOT NULL default '0', `input` int(10) unsigned NOT NULL default '0', `check` int(10) unsigned NOT NULL default '0', `score` decimal(8,3) NOT NULL default '0.000', `dinfo` text NOT NULL, PRIMARY KEY (`key`) );
CREATE TABLE `Safe` ( `key` varchar(255) NOT NULL default '', `basic` varchar(64) NOT NULL default '', `fname` varchar(160) NOT NULL default '', `ctype` varchar(64) NOT NULL default '', `ftype` smallint(5) unsigned NOT NULL default '0', `match` int(10) unsigned NOT NULL default '0', `input` int(10) unsigned NOT NULL default '0', `check` int(10) unsigned NOT NULL default '0', `score` decimal(8,3) NOT NULL default '0.000', `dinfo` text NOT NULL, PRIMARY KEY (`key`) );
GRANT SELECT,UPDATE,INSERT,DELETE ON FuzzyOcr.* TO fuzzyocr@localhost IDENTIFIED BY 'fuzzyocr';

Create the DB:

mysql -p < FuzzyOcr.mysql

Edit the configuration file:

vi /etc/spamassassin/FuzzyOcr.cf

Enable the following lines:

focr_mysql_db FuzzyOcrfocr_mysql_hash Hashfocr_mysql_safe Safefocr_mysql_user fuzzyocrfocr_mysql_pass fuzzyocrfocr_mysql_host localhostfocr_mysql_port 3306focr_mysql_socket /var/run/mysqld/mysqld.sock

Setup FuzzyOcr Database Cleaner script:

vi /usr/sbin/fuzzy-cleanmysql

Add:

#!/usr/bin/perl
#Script to clean out mysql tables of data. Default is to leave data in Safe for 1 day and Hash for 10 days.
#Fuzzyocr-cleanmysql
use Getopt::Long;
use DBI;
use MLDBM qw(DB_File Storable);
my %Files = (
db_hash => '/var/lib/fuzzyocr/FuzzyOcr.db',
db_safe => '/var/lib/fuzzyocr/FuzzyOcr.safe.db',
);
use DBI;
$database = "FuzzyOcr";
$hostname = "localhost";
$socket = "/var/run/mysqld/mysqld.sock";
$port = "3306";
$username = "fuzzyocr";
$password = 'password';
# defaults
my $cfgfile = "/etc/spamassassin/FuzzyOcr.cf";
my %App;
my %age;
$age{'age'} = 10*24; # 10 days
$age{'hash'} = $age{'age'};
$age{'safe'} = 0;
my $help = 0;
my $verbose = 0;
GetOptions( \%age,
'age=i',
'config=s' => \$cfgfile,
'hash=i',
'help' => \$help,
'safe=i',
'verbose' => \$verbose,
);
if ($help) {
print "Usage: fuzzy-cleanmysql [Options]\n";
print "\n";
print "Available options:\n";
print "--age=i Global age in hours to keep in db\n";
print "--config=s Specify location of FuzzyOcr.cf\n";
print " Default: /etc/spamassassin/FuzzyOcr.cf\n";
print "--hash=i Number of hours old to keep in Hash db\n";
print "--safe=i Number of hours old to keep in Safe db\n";
print "--verbose Show more informations\n";
print "\n";
exit 1;
}
# Convert hours to seconds
$age{'age'} *= 60 * 60;
$age{'hash'} *= 60 * 60;
$age{'safe'} *= 60 * 60;
$age{'safe'} = $age{'safe'} ? $age{'safe'} : $age{'age'};
# Read custom paths from FuzzyOcr.cf
my $app_path = q(/usr/local/netpbm/bin:/usr/local/bin:/usr/bin);
open CONFIG, "< $cfgfile" or warn "Can't read configuration file, using defaults...\n";
while () {
chomp;
if ($_ =~ m/^focr_bin_(\w+) (.+)/) {
$App{$1} = $2;
printf "Found custom path \"$2\" for application \"$1\"\n" if $verbose;
}
if ($_ =~ m/^focr_path_bin (.+)/) {
$app_path = $1;
printf "Found new path: \"$1\"\n" if $verbose;
}
if ($_ =~ m/^focr_enable_image_hashing (\d)/) {
$App{hashing_type} = $1;
printf "Found DB Hashing\n" if ($verbose and $1 == 2);
printf "Found MySQL Hashing\n" if ($verbose and $1 == 3);
}
if ($_ =~ m/^focr_mysql_(\w+) (.+)/) {
$MySQL{$1} = $2;
printf "Found MySQL option $1 => '$2'\n" if $verbose;
}
if ($_ =~ m/^focr_threshold_max_hash (.+)/) {
$App{max_hash} = $1;
printf "Updated Thresold{max_hash} = $1\n" if $verbose;
}
}
close CONFIG;
# make shure we have this threshold set
$App{max_hash} = 5 unless defined $App{max_hash};
# search path for bin_util unless already specified in configuration file
foreach my $app (@bin_utils) {
next if defined $App{$app};
foreach my $d (split(':',$app_path)) {
if (-x "$d/$app") {
$App{$app} = "$d/$app";
last;
}
}
}
sub get_ddb {
my %dopts = ( AutoCommit => 1 );
my $dsn = "DBI:mysql:database=$database";
if (defined $socket) {
$dsn .= ";mysql_socket=$socket";
} else {
$dsn .= ";host=$hostname";
$dns .= ";port=$port" unless $port == 3306;
}
printf "Connecting to: $dsn\n" if $verbose;
return DBI->connect($dsn, $username, $password,\%dopts) or die("Could not connect!");
}
if ($App{hashing_type} == 3) {
my $ddb = get_ddb();
if ($ddb) {
my $sql;
foreach my $ff (sort keys %Files) {
$ff =~ s/db_//;
$sqlbase = "FROM $MySQL{$ff} WHERE $MySQL{$ff}.\`check\` < ?";
my $timestamp = time;
$timestamp = $timestamp - $age{$ff};
$sql = "DELETE $sqlbase";
if ( $verbose ) {
printf "Delete from Table $MySQL{$ff}\n";
print "$sql, $timestamp\n";
print "Timestamp is ", scalar(localtime($timestamp)), "\n";
print "That's $age{$ff} seconds earlier than now.\n";
print "\n";
}
$ddb->do($sql,undef,$timestamp);
}
$ddb->disconnect;
}
}

Make the script executable:

chmod +x /usr/sbin/fuzzy-cleanmysql

And add it to cron

vi /etc/crontab

with:

@weekly /usr/sbin/fuzzy-cleanmysql &> /dev/null #FuzzyOcr DB cleaner

Filtering PDF, XLS and Phishing Spam with ClamAV (Sanesecurity Signatures):
There is currently a lot of spam where the spam "information" is attached as .pdf or .xls files, sometimes also hidden inside a .zip file. While these spam mails are not easy to catch with e.g. SpamAssassin or a Bayes filter, the ClamAV virus scanner can catch them easily when it is fed with the correct signatures as ClamAV is built to scan mail attachments.
Install it with:

aptitude install clamav-unofficial-sigs

Edit it's configuration file:

vi /usr/share/clamav-unofficial-sigs/conf.d/00-clamav-unofficial-sigs.conf

Change Path to clamd.pid:

clamd_pid="/var/run/clamav/clamd.pid"

Reload after update:

reload_dbs="yes"

Work Directory:

work_dir="/var/lib/clamav"

And once you're done with the configuration, set the following to yes:

user_configuration_complete="yes"

Now we run the update script to check if the download works:

clamav-unofficial-sigs

Add it to cron:

00 04 * * * root /usr/sbin/clamav-unofficial-sigs &> /dev/null

Grey listing with Postgrey:

apt-get install postgrey libnet-rblclient-perl

Enable it in postfix:

postconf -e "grey_policy = check_policy_service inet:127.0.0.1:10023"

Reload postfix configuration:

sudo /etc/init.d/postfix reload

You may need to modify the whitelisting of certain domains or users. To do so, use the files in /etc/postgrey/.

The default delay is 300 seconds (5 minutes), to change this value edit /etc/default/postgrey. Add --delay=N to the POSTGREY_OPTS:

POSTGREY_OPTS="--inet=10023 --delay=60"

KAM:

Create the script:

vi /etc/cron.daily/kam.sh

Add the following content:

#!/bin/bash

 # Original version modified by Andrew MacLachlan (andrew@gdcon.net)

 # Added additional MailScanner restarts on inital restart failure

 # Made script run silently for normal (successful) operation

 # Increased UPDATEMAXDELAY to 900 from 600

 # Insert a random delay up to this value, to spread virus updates round

 # the clock. 1800 seconds = 30 minutes.

 # Set this to 0 to disable it.

 UPDATEMAXDELAY=0

 if [ -f /opt/MailScanner/var/MailScanner ] ; then

 . /opt/MailScanner/var/MailScanner

 fi

 export UPDATEMAXDELAY

 if [ "x$UPDATEMAXDELAY" = "x0" ]; then

 :

 else

 logger -p mail.info -t KAM.cf.sh Delaying cron job up to $UPDATEMAXDELAY seconds

 perl -e "sleep int(rand($UPDATEMAXDELAY));"

 fi

 # JKF Fetch KAM.cf

 #echo Fetching KAM.cf...

 cd /etc/mail/spamassassin

 rm -f KAM.cf

 wget -O KAM.cf http://www.peregrinehw.com/downloads/SpamAssassin/contrib/KAM.cf > /dev/null 2>&1

 if [ "$?" = "0" ]; then

 #echo It completed and fetched something

 if ( tail -10 KAM.cf | grep -q '^#.*EOF' ); then

 # echo It succeeded so make a backup

 cp -f KAM.cf KAM.cf.backup

 else

 echo ERROR: Could not find EOF marker

 cp -f KAM.cf.backup KAM.cf

 fi

 else

 echo It failed to complete properly

 cp -f KAM.cf.backup KAM.cf

 fi

 #echo Reloading MailScanner and SpamAssassin configuration rules

 /etc/init.d/mailscanner reload > /dev/null 2>&1

 if [ $? != 0 ] ; then

 echo "MailScanner reload failed - Retrying..."

 /etc/init.d/mailscanner force-reload

 if [ $? = 0 ] ; then

 echo "MailScanner reload succeeded."

 else

 echo "Stopping MailScanner..."

 /etc/init.d/mailscanner stop

 echo "Waiting for a minute..."

 perl -e "sleep 60;"

 echo "Attemping to start MailScanner..."

 /etc/init.d/mailscanner start

 fi 

 fi 

And make it executable:

chmod +x /etc/cron.daily/kam.sh

ScamNailer:

Create the script:

vi /usr/sbin/update_scamnailer

Note: Content is in https://docs.google.com/viewer?a=v&pid=explorer&chrome=true&srcid=0B9cN15Q3pKnwNmU2NTY2ZDQtZTdiNS00ZjM4LWI0MDktM2ZhNGYzOTY0NTZk&authkey=CKOFkqIJ&hl=en

and make it executable:

chmod +x /usr/sbin/update_scamnailer

Add it to cron:

@daily /usr/sbin/update_scamnailer &> /dev/null #Update Scamnailer

Firewalling the SpamSnake with Firehol:

Firehol is a stateful iptables packet filtering firewall configurator. It is abstracted, extensible, easy and powerful. It can handle any kind of firewall, but most importantly, it gives you the means to configure it, the same way you think of it.

Install Firehol

apt-get install firehol

Firehol Settings:

vi /etc/default/firehol

and change the following:

START_FIREHOL=YESvi /etc/firehol/firehol.conf

and add the following:

server_zabbix_ports="tcp/10050 udp/10050"client_zabbix_ports="default 10050"
version 5 # Accept all client traffic on any interface interface any internet protection strong server "icmp ping ICMP ssh http https telnet webmin dns dcc snmp smtp smtps submission zabbix mysql" accept client all accept

Restart firehol:

service fireholl restart

And that's it