Mail Server on Ubuntu 18.04 Part 3

Welcome to Mail Server on Ubuntu 18.04 Part 3. This is the third part of a series of blog posts. In this post we will be adding anti-virus and anti-spam along with some other tools to stop spam and viruses getting through to our mailserver.

Mail Server on Ubuntu 18.04 Part 1
Mail Server on Ubuntu 18.04 Part 2

  • Emails will be checked with anti-virus service ClamAV
  • Emails will be checked with anti-spam filters from Spamassassin
  • Grey listing of incoming mail servers with Postgrey

Installing the software

Let the anti-spamming begin. We need to install Amavis-new, ClamAV, Spamassassin and Postgrey along with some tools for dealing with compressed files.

sudo apt install amavisd-new arj cabextract clamav clamav-daemon lhasa \
         libdbi-perl libdbd-mysql-perl liblz4-tool lrzip lzop nomarch \
         p7zip-full postgrey ripole rpm spamassassin unrar

Amavis

Unfortunately, it is a necessary task that we should scan all incoming and outgoing email for viruses and all incoming email for possible spam content. We do that with a service called Amavis. This plugs into Postfix and accepts mail before it is delivered to the users mailbox.

Configuring Amavis

The extra compression modules are not normally installed by default as some are supplied on a less then free license. The choice is yours whether you install them or not.

If you do add them then you will need to turn them on by editing the file Amavis configuration file 50-user. By adding changes to this file will mean these settings override those of the earlier files and makes it simpler to upgrade.

We can now add a bunch of lines to the 50-user file. When configuring Amavis this will be the only file we update. It is loaded last and therefore over-rides any duplicate settings. It is a Perl file, you should pay attention to what you are typing and check in has the correct syntax for Perl.

sudo nano /etc/amavis/conf.d/50-user
$unrar      = ['rar', 'unrar']; #disabled (non-free, no security support)
 
# Anti-Virus code
@bypass_virus_checks_maps = (
   \%bypass_virus_checks, \@bypass_virus_checks_acl, \$bypass_virus_checks_re);
 
$final_virus_destiny = D_PASS;
 
# Anti-Spam checking
@bypass_spam_checks_maps = (
   \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re);
 
$sa_spam_subject_tag = '[**SPAM**] ';
$final_spam_destiny  = D_PASS;
 
@lookup_sql_dsn = (
    ['DBI:mysql:database=postfix;host=127.0.0.1;port=3306',
     'mail',
     'MYSQLPasswd01!']);
 
$sql_select_policy = 'SELECT domain FROM domain WHERE CONCAT("@",domain) IN (%k)';
 
# Add these two lines while testing and debugging, then comment them out
$log_level = 3;
@whitelist_sender_acl = [];

Explaining those settings

The unrar line turns on scanning for, erm umm, rar files by using the non-GPL libraries. When you restart Amavis look in the mail log to see if there are any other missing modules and then look in 01-debian to see what compression programs Amavis is looking for. But remember to add changes to 50-user.

Take note that the data value for @lookup_sql_dsn contains the password for the MySQL user called mail. You will need to change it to whatever password you are using. Also the permissions on the file 50-user are world readable. This is not good, as it contains a password change the permissions so only root can read the file.

sudo chmod 640 /etc/amavis/conf.d/50-user

If the output you get from running the hostname command is the fully qualified domain name for your server, you do not need to update the file 05-node_id. It is better to fix the return from hostname then to update the amavis-new file. See Mail Server on Ubuntu 18.04 Part 1 for details of how to do that.

hostname --fqdn

I like to see that the spam detector is working so I get Spamassassin to update the subject line for any emails the scanner believes are spam. This is purely cosmetic, it simply changes the string that is added to the subject line emails it believes are spam. Since I already have a rule in my email client looking for the string “[**SPAM**] that is what I change it to.

The variable $final_spam_destiny is used to determine the final outcome of an email that is believed to be spam. Since one man’s spam is another man’s valuable message. We shall let the user decide, It will be marked as suspected spam but allow it to be delivered.

The last variable for @lookup_sql_dsn, is required because Amavis tries to find out whether an email is incoming (sent from the Internet to your domains) or outgoing (sent from your system to the internet) by looking at the @acl_local_domains setting. You need to tell Amavis where to check if a certain domain is one of your destination domains. The reason is that you usually don’t want to scan your outgoing emails. Imagine that a mail shot by your own marketing bullshit team is marked as [**SPAM**] by the company email servers on the way out.

While debugging you can also set some values to turn on logging.

Restart Amavis

Restart amavis and take a look at the log file to check for any missing compression modules.

sudo systemctl restart amavis.service

Get Amavis talking to Postfix

We should setup the interface configuration for Postfix to talk to Amavis. Here we will use the command line interface because it is a little simpler than opening a editor.

sudo postconf -e content_filter=amavisfeed:[127.0.0.1]:10024
sudo postconf -e receive_override_options=no_address_mappings

Add the following long set of lines to the end of the master.cf file. These tell Postfix to connect to Amavis and how to do just that.

sudo nano /etc/postfix/master.cf
amavisfeed unix -      -       n     -       2  smtp
    -o smtp_data_done_timeout=1200
    -o smtp_send_xforward_command=yes
    -o disable_dns_lookups=yes
    -o max_use=20
 
127.0.0.1:10025 inet n    -       n       -       -     smtpd
    -o content_filter=
    -o smtpd_delay_reject=no
    -o smtpd_client_restrictions=permit_mynetworks,reject
    -o smtpd_helo_restrictions=
    -o smtpd_sender_restrictions=
    -o smtpd_recipient_restrictions=permit_mynetworks,reject
    -o smtpd_data_restrictions=reject_unauth_pipelining
    -o smtpd_end_of_data_restrictions=
    -o smtpd_restriction_classes=
    -o mynetworks=127.0.0.0/8
    -o smtpd_error_sleep_time=0
    -o smtpd_soft_error_limit=1001
    -o smtpd_hard_error_limit=1000
    -o smtpd_client_connection_count_limit=0
    -o smtpd_client_connection_rate_limit=0
    -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks,no_milters
    -o local_header_rewrite_clients=

Search the master.cf for pickup. If the line is commented out, we need to uncomment it.

pickup    unix  n       -       y       60      1       pickup

Do not forget to reload the Postfix files now you have updated them.

sudo systemctl restart postfix

Testing Amavis

Now is a good time to test services are running for postfix and amavis. Let us start off by checking the ports are open. We are checking that posts 10024 and 10025 are open.

netstat -atu | grep -P 1002[45]
tcp        0      0 localhost:10024         0.0.0.0:*               LISTEN     
tcp        0      0 localhost:10025         0.0.0.0:*               LISTEN     
tcp6       0      0 localhost:10023         [::]:*                  LISTEN     
tcp6       0      0 localhost:10024         [::]:*                  LISTEN
lsof -i | grep 1002[45]
master    1148     root  112u  IPv4  20827      0t0  TCP localhost:10025 (LISTEN)
/usr/sbin 5057   amavis    7u  IPv4 130165      0t0  TCP localhost:10024 (LISTEN)
/usr/sbin 5057   amavis    8u  IPv6 130166      0t0  TCP localhost:10024 (LISTEN)
/usr/sbin 5062   amavis    7u  IPv4 130165      0t0  TCP localhost:10024 (LISTEN)
/usr/sbin 5062   amavis    8u  IPv6 130166      0t0  TCP localhost:10024 (LISTEN)
/usr/sbin 5063   amavis    7u  IPv4 130165      0t0  TCP localhost:10024 (LISTEN)
/usr/sbin 5063   amavis    8u  IPv6 130166      0t0  TCP localhost:10024 (LISTEN)

If your output looks similar to that shown above all is looking good. Shall we try and connect to the service with telnet. After entering the telnet command wait until the output stops and the enter the ehlo line. Then to exit type quit.

telnet localhost 10024
Trying ::1...
Connected to localhost.
Escape character is '^]'.
220 [::1] ESMTP amavisd-new service ready
ehlo localhost                   -= type this line in
250-[::1]
250-VRFY
250-PIPELINING
250-SIZE
250-ENHANCEDSTATUSCODES
250-8BITMIME
250-SMTPUTF8
250-DSN
250 XFORWARD NAME ADDR PORT PROTO HELO IDENT SOURCE
quit                           -= type this line in
221 2.0.0 [::1] amavisd-new closing transmission channel
Connection closed by foreign host.

Okay, now the same thing again on port 10025.

telnet localhost 10025
Trying ::1...
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
220 mail.dragon.lab ESMTP Postfix (Ubuntu)
ehlo localhost                   -= type this line in
250-mail.dragon.lab
250-PIPELINING
250-SIZE 10240000
250-VRFY
250-ETRN
250-STARTTLS
250-ENHANCEDSTATUSCODES
250-8BITMIME
250-DSN
250 SMTPUTF8
quit                           -= type this line in
221 2.0.0 Bye

If your output looks about the same as this then you are good to go.

Configuring Clamav

ClamAV is already configured in the file “/etc/amavis/conf.d/15-av_scanners”, that is the clamav configuration is not commented out, and we added the code to start any scanner to /etc/amavis/conf.d/50-user above.

So that the user who is running clamav can “talk” to the Amavis service we need to add it to the amavis group and visa versa.

sudo adduser clamav amavis
sudo adduser amavis clamav
sudo systemctl restart clamav-daemon.service
 
grep -P 'clamav|amavis' /etc/group

The output from the grep above should look similar, and shows Amavis is a member of the clamav group and vice versa. The numbers may be different on your system.

clamav:x:120:amavis
amavis:x:122:clamav

Update virus databases

Because new viruses are being created everyday we should make sure the clamav databases are up to date by running the freshclam script.

sudo freshclam
ClamAV update process started at Fri Nov 13 15:43:28 2009
main.cvd is up to date (version: 51, sigs: 545035, f-level: 42, builder: sven)
daily.cld is up to date (version: 10022, sigs: 105525, f-level: 44, builder: ccordes

If you see a warning in your log files saying the freshclam.log is locked by another process.

ERROR: /var/log/clamav/freshclam.log is locked by another process
ERROR: Problem with internal logger (UpdateLogFile = /var/log/clamav/freshclam.log).

It means that the freshclam daemon is already running, so nothing to worry about now. To check it has already run look for freshclam on your syslog.

grep freshclam /var/log/syslog

Configuring Clamav and Freshclam

now we can reconfigure the automatic updates an other configuration details for fresh clam.

sudo dpkg-reconfigure clamav-freshclam

As your server will be connected 24/7 to the Internet use the freshclam daemon option for updating to clamav. These are either the defaults or those I choose to use:

  • Please choose the method for virus database updates. daemon
  • Please select the closest local mirror site. United Kingdom
  • HTTP proxy information (leave blank for none):
  • Number of freshclam updates per day: 24
  • Should clamd be notified after updates? Yes
  • Do you want to enable support for Google Safe Browsing? Yes
  • Do you want to download the bytecode database? Yes
  • Private Mirror : (I left this blank)
  • Do you want to enable log rotation? Yes

There are a a few options that need to be setup for clamav to function correctly. ClamAV has sensible defaults. If we run the reconfigure script we can pick those up and clamav will run. You can always rerun the reconfiguration script later to make changes.

sudo dpkg-reconfigure clamav-daemon

There are a large number of questions to answer here, Look out for the options below, I changed them. For all of the remaining inputs, we can use the default value.

  • Do you want to use the system logger? Yes
  • Log file for clamav-daemon (enter none to disable): /var/log/clamav/clamav.log
Do not change the value of the Unix socket from /var/run/clamav/clamd.ctl. It has to match up with the Amavis settings

These settings are all in /etc/clamav/clamd.conf. You can edit it directly if you like.

Now that keeps the virus-DB up to date but not the engine. To keep the engine up to date use apt to install updates.

Testing the AV scanners are Working

To testing that clamav is working will be simple once you get a virus you can email around. Do not panic the test virus will not do anything harmful, in fact it won’t do anything at all. Go to the following link and have a read

https://www.eicar.org/anti_virus_test_file.htm

Copy the string of 68 characters and save them to a file, say test.txt. Now attach test.txt to an email and send it. You SHOULD see some lines in the mail.log that indicate that the file was infected :). Also try out the same text file but packed in an archive, tar, zip or compressed file .gzip, tgz zip. Look in the log file and see that you can see some thing similar to the text below in your log files. if shows that clamav is scanning your emails and finding the Eicar test virus signature.

Blocked INFECTED (Eicar-Test-Signature)

The people that use my mailserver are, actually sensible, not your typical end users. 🙂 Their machines/PC’s are also not on the same network as mine. 🙂 Therefore I do not quarantine mails that are marked as infected with a virus. Infected emails have their subject’s updated to clearly show the mail is infected. This means I also PASS infected mail on to their mail box for them to deal with. In a company this would be a very very bad idea as your end users will not care about viruses as they will not be staying late to clear up the mess.

Configuring Spamassassin

To turn on Spamassassin we need to edit the Spamassassin file in /etc/default.

sudo nano /etc/default/spamassassin

Change the two lines for ENABLE and CRON to a number larger than zero (0).

ENABLED=1
CRON=1

Save the changes and restart the Spamassassin service.

sudo nano /etc/cron.daily/spamassassin
CRON=1
sudo systemctl restart spamassassin.service

Update Spam definitions

The script that updates the rules for Spamassassin is called “sa-update” and is normally run via a daily crontab job when the CRON flag is set to 1. We can run the script now to get everything up to date.

sudo /etc/cron.daily/spamassassin

Checking Spamassassin Is Called

There is a test spam string which spamassassin looks for to test for spam emails. Copy the string from this file, /usr/share/doc/spamassassin/examples/sample-spam.txt, into an email and send it to yourself it should get marked as [**SPAM**]. If it does not get marked you made a mistake somewhere. Mine works 🙂 If you use evolution for your testing it may have gone into the junk folder.

You can also download the Generic Test for Unsolicited Bulk Email test file from here.

Postgrey

There are a number of ways to stop spam from reaching your inbox, we have already setup Amavis to call clamav, to do some anti-virus checking, and to call Spamassassin for anti-spam filtering. Grey-listing is not designed to replace this but to work with it, in fact it works before the emails are uploaded to your server. Postgrey works by providing another hurdle for spam to get past before hitting your inbox. It is yet another tool in your arsenal against the endless barrage of pointless emails you all receive. Grey-listing is very simple and requires very little CPU or processing time. It looks at the senders name, recipients name and IP, if they have not already sent an email in the last 35 days, it politely says to them “Please try again later”. This “Please try later” is in a format that all mail servers should understand and comply with.

For any well setup mail server this is not a problem and the mail will be redelivered later as requested. But for spammers that want to get as many emails out there as possible and they never bother to “try again later”. End of that spam email 🙂

On my live mail server Postgrey is one of the most effective barriers to spam. Since they are never uploaded to my live server it can get on with better things.

Configuring Postgrey

We need to tell postfix how to talk to Postgrey. Postgrey defaults to port 10023, we can see that in the file below:

sudo nano /etc/default/postgrey
POSTGREY_OPTS="--inet=10023"

But that does not work out-of-the-box 🙁 change the POSTGREY_OPTS line to read:

POSTGREY_OPTS="--inet=127.0.0.1:10023"

The default minimum time that Postgrey will force the sender to wait is 300 seconds (5 minutes). This can be changed if you feel that way by adding the delay=300 to the POSTGREY_OPTS line. (You probably guessed that if you change the 300 the delay time will change 😕

The default minimum wait time for postgrey is 300 seconds, this can be changed by appending a parameter and value to the POSTGREY_OPTS line.

POSTGREY_OPTS="--inet=10023 --delay=300"

We can also increase the ‘remember me’ value from the default 30 days by adding max-age=30 to, I’ll let you work out where. 🙂 If set to 30 days a client will not be grey listed again unless they send no more mails for 30 days. take a look at man postgrey pages there is also an auto white list option.

POSTGREY_OPTS="--inet=127.0.0.1:10023 --delay=60 --max-age=30"

If you made any changes to the Postgrey default file you will need to restart Postgrey.

sudo systemctl restart postgrey.service

Make sure it is running, in case you missed the bit above.

sudo ps aux | grep postgrey
postgrey  6217  0.1  0.5  70728 21580 ?        Ss   11:55   0:00 postgrey --pidfile=/var/run/postgrey/postgrey.pid --daemonize --inet=127.0.0.1:10023 --delay=303 --max-age=36

Whitelist Your Own Domain

We do not want to delay our own internal mail that would just cause more work for our own mail server. Postgrey comes with two lists a blacklist and you can work out the other. We want to add our own domain to the whitelist. We can do that by creating a new file called whitelist.local. This file will not get over written when Postgrey is updated.

sudo nano /etc/postgrey/whitelist.local

Add your domain or domains here one per line, you can even use regular expressions. Again the format is detailed in man postgrey.

dragon.lab

Remember to restart the Postgrey service.

sudo systemctl reload postgrey.service

Postfix Talking to Postgrey

Nothing will happen until we tell Postfix to talk to Postgrey. Therefore add the following line to your /etc/postfix/main.cf at the end of the smtpd_recipient_restrictions definition. Make sure you use a comma and a space (, ) if it is on one line or just a newline if the restrictions are on separate line. Also make sure the port number is correct 10023 unless you changed it to something else.

sudo nano /etc/postfix/main.cf
smtpd_recipient_restrictions =
        ...
        check_policy_service inet:127.0.0.1:10023
sudo systemctl reload postfix.service

There are also some white lists in /etc/postgrey you can use and amend if you feel lucky.

When it is working you will see messages in your mail.log that show. Recipient address rejected: Greylisted,.

This would be a good backup point!

That is the end of Mail Server on Ubuntu 18.04 Part 3. We have now got a pretty good mailserver setup. It still needs some work before it goes live the most of the hard work is done.

In Mail Server on Ubuntu 18.04 Part 4 we will tighten up the restrictions used in Postfix to stop spam where spammers incorrectly setup their own mailservers. Maybe they should do it right in the first place 🙂

Leave a Reply

Your email address will not be published. Required fields are marked *