Mail Server on Ubuntu 16.04 Part 4

Welcome to Mail Server on Ubuntu 16.04 Part 4. This is the forth part of a series of blog posts. We will be adding some extra checking to postfix. This will allow you to effectively blacklist some domains and email addresses. We will also be adding checks for properly formatted domain names and checking the IP address against their DNS lookup.

Mail Server on Ubuntu 16.04 Part 1
Mail Server on Ubuntu 16.04 Part 2
Mail Server on Ubuntu 16.04 Part 3

Postfix Rules And Restrictions

We will start to strengthen our defenses against spam by adding some new options to main.cf.
smtpd_delay_reject = yes This will stop Postfix from rejecting the message straight away and allows for better logging of sender and recipient information which we want. This sounds a little strange at first but it is needed to do HELO checks.

sudo postconf -e "smtpd_delay_reject = yes"

disable_vrfy_command All the Postfix documentation says is “This stops some techniques used to harvest email addresses.” This seems like a very good thing so i am suprised it is off by default.

sudo postconf -e "disable_vrfy_command = yes"

If you have tried Sending an Email from telnet then you will know that after connecting to the server you should send either HELO domain or EHLO domain.

Postfix by default allows emails that do not have a correctly formatted HELO/EHLO. We can turn this on as most mail clients, if not all, now follow the protocols correctly. Any clients not following the RFC will most likely have formatting problems elsewhere and cause your server more work. Many spammers set up their bots to miss this step off stop the spam here and you save yourself from spam.

sudo postconf -e "smtpd_helo_required = yes"

When Postfix is contacted by someone sending it an email it will go through a number of steps or rules. With each rule some restrictions are checked. Something to remember is that any of the restrictions can be used in any rule. So although it seems logical to put the restriction reject_unknown_client in the rule smtpd_client_restrictions. They could equally go in the smtpd_data_restrictions rule. Some of the rules that are available for a Mail Server on Ubuntu 16.04 are:

  • smtpd_client_restrictions
  • smtpd_helo_restrictions
  • smtpd_sender_restrictions
  • smtpd_relay_restrictions
  • smtpd_recipient_restrictions
  • smtpd_data_restrictions

Some of the restrictions that be be applied are listed below along with the client side information that they will be looking at.

Restrictions Information
check_client_access maptype:mapname Client IP or hostname
reject_unknown_client_hostname
check_helo_access maptype:mapname HELO hostname
reject_invalid_helo_hostname
check_sender_access maptype:mapname MAIL FROM address
reject_unknown_sender_domain
reject_non_fqdn_sender
smtpd_relay_restrictions RCPT TO address
permit_mynetworks
permit_sasl_authenticated
reject_unauth_destination
check_recipient_access maptype:mapname RCPT TO address
reject_unknown_recipient_domain
reject_non_fqdn_recipient
reject_unauth_destination

For each Postfix rule we can see the suggested restrictions to apply. Take a look at the Postfix official documentation find the rule and then find the meaning for each restriction in the text. For each rule I have listed a suggestion of what you might need. Find any existing rules and comment them out. Then paste in the new restrictions at the end of your main.cnf. Do not worry about the lines that end in hash:filename. I’ll explain those once we get the main.cnf sorted out.

sudo nano /etc/postfix/main.cf

smtpd_client_restrictions

The following restrictions are used to verify and validate the client hostname or client IP address passed to us.

smtpd_client_restrictions =
   reject_unknown_client_hostname
   check_client_access    hash:/etc/postfix/client_access

smtpd_helo_restrictions

These restrictions apply to the data send by the client with the HELO/EHLO command.

smtpd_helo_restrictions =
   reject_unknown_helo_hostname
   reject_non_fqdn_helo_hostname 
   reject_invalid_helo_hostname
   check_helo_access      hash:/etc/postfix/helo_access

smtpd_sender_restrictions

These restrictions apply to the data send by the client with the MAIL FROM command. These restrictions will have a value set for them. I usually delete those and replace them with these at the end of the file so all the restriction definitions are together.

smtpd_sender_restrictions =
   permit_mynetworks
   reject_unknown_sender_domain
   reject_non_fqdn_sender
   check_sender_access    hash:/etc/postfix/sender_access

smtpd_relay_restrictions

Access restrictions for relay control are performed before smtpd_recipient_restrictions. We can therefore stop our postfix mail server from being an open relay here rather than putting them in smtpd_recipient_restrictions. This allows rules for spam blocking and relay permissions, helping to remove any errors.

smtpd_relay_restrictions =
   permit_mynetworks
   permit_sasl_authenticated
   reject_unauth_destination

smtpd_recipient_restrictions

These restrictions apply to the data send by the client with the RCPT TO command. These restrictions already exist in the main.cf. I usually delete those and replace them with these at the end of the file so all the restriction definitions are together. 🙂

smtpd_recipient_restrictions =
   reject_unauth_pipelining
   reject_non_fqdn_recipient
   reject_unknown_recipient_domain
   check_recipient_access hash:/etc/postfix/recipient_access
   check_policy_service inet:127.0.0.1:10023

Creating hash:filename Access Map Files

Access map files in Postfix are just another way for Postfix to lookup data. In these the client information is the key and the action is the value. the value will be an action or return code and an optional short message.

Restrictions Key Data
check_client_access IP & network address, hostnames, parent domains. To match against the client IP or result of reverse an IP address lookup.
check_helo_access hostnames, parent domains. To match against host information from the HELO command
check_sender_access Email addresses, domains and local parts. To match against the address supplied in the MAIL FROM command.
check_recipient_access Email addresses, domains and local parts. To match against the address supplied in the RCPT TO command.

Actions

OK
Accept the address etc. that matches the pattern.
DUNNO
Act as if the lookup key was not found and exit this check. This stops Postfix from continuing to test further substrings in this of the lookup key.
REJECT Optional short message.
Reject the address etc. that matches the pattern. Reply with the numeric value from $access_map_reject_code and output the optional message or a generic message in not provided.

check_client_access

An example /etc/postfix/client_access.in file. Create this file and save it.

sudo nano /etc/postfix/client_access.in
# Restricts which clients this system accepts SMTP connections from.
# It uses data from the initial connection to this server.
 
# allow my domain
dragon.lab                      OK
# even sub domains, note the leading full stop
.dragon.lab                     OK
# Allow IP range10.0.0.0/8
10                             OK
 
# Reject by parts of a domain
# Using a TLD, this is probably over the top for most mailservers
cn                              REJECT Someone from China, probably a spammer.
ru                              REJECT Someone from Russia, probably a spammer.
 
# Ban by domain or sub-domain for prolific spammers
myfax.com                         REJECT Getting a lot of spam from myfaxcom.
# Even sub domains, note the leading full stop
.myfax.com                        REJECT Getting a lot of spam from xxx.myfax.com.
 
# Reject IP addresses 192.168.0.0/24
192.168.0                       REJECT Your IP address is not allowed here
# 172.16/16
172.16                          REJECT You cannot be 172.16.x.x

check_helo_access

An example /etc/postfix/helo_access.in file. Create this file and save it.

sudo nano /etc/postfix/helo_access.in
# This file is checked at the ehlo stage, it allows my servers in
# and then starts to reject servers that pretend to be me
# i.e A list of TLDs, probably over zealous for businesses
# After that follows a list of domains and sub-domains
# we do not want to talk to.
 
# Do not put your own domain or IP in here it can be spoofed by spammers.
 
# Reject by parts of a domain
# Using a TLD, this is probably over the top for most mailservers.
cn                              REJECT Someone from China, probably a spammer.
ru                              REJECT Someone from Russia, probably a spammer.
 
# Ban by domain or sub-domain for prolific spammers
example.com                     REJECT Nothing should come from here
.example.com                    REJECT Nothing should come from here
 
# Reject IP addresses 192.168.0.0/24
192.168.0                       REJECT Your IP address is not allowed here
# 172.16/16
172.16                          REJECT You cannot be 172.16.x.x

check_sender_access

An example /etc/postfix/sender_access.in file. Create this file and save it.

sudo nano /etc/postfix/sender_access.in
# This file uses data from MAIL TO: command. It will do reverse IP 
# lookups to get the hostname if necessary.
# Do not white list your own domain as spammers often spoof that.
 
# Allow IP range10.0.0.0/8
10                              OK
 
# Reject by parts of a domain
# Using a TLD, this is probably over the top for most mailservers.
cn                              REJECT Someone from China, probably a spammer.
ru                              REJECT Someone from Russia, probably a spammer.
 
# Full and part email addresses as well as domains and sub domains
spammer@
myfax.com                       REJECT Scumbag spammer.
bigfoot.com                     REJECT Source of spam.   
user@example.com                REJECT we do not want email from this guy

check_recipient_access

An example /etc/postfix/recipient_access.in file. Create this file and save it.

sudo nano /etc/postfix/recipient_access.in
# This file uses information from the RCPT TO: command. It will use DNS 
# lookups to find the IP and reverse lookups to find the hostname if necessary.
 
# Here you can list old email addresses that are on spam list
# We can list the whole email or shorten them to just the part in front of the @
from_spammer@dragon.lab         REJECT Email address passed on to spammers  
from_spammer@                   REJECT Email address passed on to spammers  
from_playdotcom@                REJECT Play.com tried to cover up they were hacked

Make sure the in file can only be read by root.

sudo chown root:root /etc/postfix/*in
sudo chmod 600 /etc/postfix/*in

Postfix cannot use the .in files directly they need to be converted into db files. We do that with the postmap command as shown below but that gets tired very quickly. We also need to reload postfix whenever they are updated.

sudo postmap /etc/postfix/client_access.in file

Building in Files to db Files

Each time you make a change to your in files you will need to rebuild them into db files and the reload postfix. This gets tiresome quickly, so I have included a makefile that will do the hard work for you. Create and save the makefile in the /etc/postfix directory. When you make changes to the in files just make /etc/postfix your current working directory and run the command below. It will automatically rebuild any db files where the in file has changes or the db file does not exist. You can install make with the package build-essentials This is a make file the indented line have to start with a TAB character or the recipe will not run correctly.

sudo nano /etc/postfix/makefile
# This makefile will build *.in files into *.db files.
# When changes are made to the *.in files or the *.db 
# files do not exist.
# Use the target clean to remove all *.db files
 
objects := $(patsubst %.in,%.db,$(wildcard *.in))
 
all: $(objects)
 
%.db: %.in
	postmap $^
	@mv $^.db $@
	service postfix reload
 
.PHONY: clean
clean:
	rm *.db
cd /etc/postfix
sudo make
This would be a good backup point!
You can always restore back to this point if you mess up on the way.

Sender Policy Framework Records

Up until now we have only been concerned with stopping unwanted mail hitting our mailserver. We can help make sure our real email gets delivered and be identified as genuine by other mail servers simply by publishing a SPF record in order to reduce the abuse of our domain in email headers.

We can do that by adding a TXT record to the DNS for your domain. This will most likely be controlled by the company who you registered your domain with.

In its simplest form you will need

TXT  "v=spf1 mx -all"

Which is saying:

  • v=spf1 this is a SPF record
  • mx All the A records for all the MX records for domain are tested in order of MX priority. If the client IP is found among them, this mechanism matches.
  • Fail,
  • all This mechanism always matches. It usually goes at the end of the SPF record.

Overall this SPF record is saying: Allow emails from my domain only if it was delivered from one of my MX servers, prohibit all others. This only works if all yor mailservers, MX, are listed in your DNS records.

If you email for your domain is only sent by this mail server you can use ‘-‘ rather than ‘~’ in front f the all. The meaning the the leading character is explained clearly on the SPF official website. There are loads of other options to express the SPF record, but take the time to look at the full SPF record syntax.

You can check that the DNS record is in place with the following command:

dig dragon.lab txt

In the text that is returned you should see a line similar to the following

;; ANSWER SECTION:
dragon.lab		86400	IN	TXT	"v=spf1 mx -all"

Checking SPF Record

We can also make Postfix check the validity of in-coming email. We need to install the software. From my reasurch there are two policy services on written in Perl the other in Python. The Python version has more functionality, so we will use that.

sudo apt-get install postfix-policyd-spf-python

We can then tell Postfix how to talk to this new policy service by adding a new restriction to the end of the smtpd_recipient_restrictions rule. As we are using smtpd_relay_restrictions for relay permissions we can put the restriction anywhere as this involves a DNS lookup it will be “expensive” so we put it at the end.

sudo nano /etc/postfix/main.cf

With the new policy service added your smtpd_recipient_restrictions rule should look something like this.

smtpd_recipient_restrictions =
   reject_unauth_pipelining
   reject_non_fqdn_recipient
   reject_unknown_recipient_domain
   check_recipient_access hash:/etc/postfix/recipient_access
   check_policy_service inet:127.0.0.1:10023
   check_policy_service unix:private/policy-spf

Add a new line to the end of main.cf, note that the option has a mixture of hyphens and underscores.

policy-spf_time_limit = 3600s

Final change to let Postfix what to do , we edit the master.cf file.

sudo nano /etc/postfix/master.cf

Add the following to the end of the file.

policy-spf  unix  -       n       n       -       -       spawn
     user=nobody argv=/usr/bin/policyd-spf

Restart postfix to take the new policy service into use.

sudo service postfix restart

Testing Your Mailserver

One very important test you should do is test to see if your mailserver will allow anyone to deliver mail anywhere. That is your mail server must NOT be an open rely. If you are an open relay spammers will love you and you will find your mailserver on black lists very quickly.

To perform an open relay test go to http://mxtoolbox.com/diagnostic.aspx and enter the name of your mailserver as seen from the Internet. The test tasks a second or so and you should see a bunch of green ticks. If any are not green fix the problems.

You can also test your domain records here and do a quick check that your SPF record is valid. It does not test that it will do what you want it to do.

Another site with a good test for open relays is http://www.mailradar.com/openrelay/. This does a more thorough set of test on your server. If all is well with your server there is a message in green saying your seerver is not an open relay. Again the rest of the site is worth reading through.

Monitoring Mailserver

Now you have a mailserver and have gone to the effort of stopping spam. It is a good idea to monitor that it is working correctly.

pflogsumm

There is a tool, pflogsumm, which looks at your /var/log/mail.log and pulls out a lot of information.

sudo apt-get install pflogsumm

To get a report out in its simplest form just run:

sudo pflogsumm /var/log/mail.log > pflogsumm.txt

The output can be massive and well worth looking. See the man page for more details.

Awstats

Installing AWstats is a lille more complex so I wrote a post specifically for
Installing AWstats on Ubuntu 16.04 LTS

Well that is the end of Mail Server on Ubuntu 16.04 Part 4.

This would be a good backup point!
You can always restore back to this point if you mess up on the way.

In Mail Server on Ubuntu 16.04 Part 5 we will add Roundcube as a web mail interface so your users can get at their mail from any browser. This will be updated to use an Apache virtualhost and be secure over https.

Munin

Munin is another monitor service. I have written a post Installing Munin on Ubuntu 16.04 running through how to install this. It gives great information about your mail server.

One thought on “Mail Server on Ubuntu 16.04 Part 4

  1. Steve

    Thank you very much for this absolutely excellent guide.
    It’s very rare to find such a howto that doesn’t contain typos and inaccuracies.

    I’ve been administering postsfix on Ubuntu for a few years on 12.04 LTS and eventually had to bite the bullet and build a new system on 16.04. I’ve been putting it off for over a year. Groan… what a pain in the arse!

    Using your faultless guide I was able to do it in a few hours. Everything, yes everything, worked as documented.

    You saved me hours of work. Thank you.

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *