Welcome to Mail Server on Ubuntu 18.04 Part 4. This is the fourth part of a series of blog posts. We will be adding some extra checking to postfix. This will allow you to effectively blacklist some domains and email addresses. We will also be adding checks for properly formatted domain names and checking the IP address against their DNS lookup.
Postfix Rules And Restrictions
We will start to strengthen our defenses against spam by adding some new options to main.cf. For the official meaning take a look at the Postfix documentation. Also take a loot at Postfix SMTP relay and access control so you can make up your own mind what to set.
smtpd_delay_reject = yes This will stop Postfix from rejecting the message straight away and allows for better logging of sender and recipient information which we want. This sounds a little strange at first but it is needed to do HELO checks.
sudo postconf -e "smtpd_delay_reject = yes"
disable_vrfy_command All the Postfix documentation says is “This stops some techniques used to harvest email addresses.” This seems like a very good thing so I am surprised it is off by default.
sudo postconf -e "disable_vrfy_command = yes"
If you have tried Sending an Email from telnet then you will know that after connecting to the server you should send either HELO domain or EHLO domain.
Postfix by default allows emails that do not have a correctly formatted HELO/EHLO. We can turn this on as most mail clients, if not all, now follow the protocols correctly. Any clients not following the RFC will most likely have formatting problems elsewhere and cause your server more work. Many spammers set up their bots to miss this step off stop the spam here and you save yourself from spam.
sudo postconf -e "smtpd_helo_required = yes"
When Postfix is contacted by someone sending it an email it will go through a number of steps or rules. With each rule some restrictions are checked. Something to remember is that any of the restrictions can be used in any rule. So although it seems logical to put the restriction reject_unknown_client in the rule smtpd_client_restrictions. They could equally go in the smtpd_data_restrictions rule. Some of the rules that are available for a Mail Server on Ubuntu 18.04 are:
Some of the restrictions that be be applied are listed below along with the client side information that they will be looking at.
|check_client_access maptype:mapname||Client IP or hostname|
|check_helo_access maptype:mapname||HELO hostname|
|check_sender_access maptype:mapname||MAIL FROM address|
|smtpd_relay_restrictions||RCPT TO address|
|check_recipient_access maptype:mapname||RCPT TO address|
Suggested Postfix restrictions
For each Postfix rule we can see the suggested restrictions to apply. Take a look at the Postfix official documentation find the rule and then find the meaning for each restriction in the text. For each rule I have listed a suggestion of what you might need. Find any existing rules and comment them out. Then paste in the new restrictions at the end of your main.cf. Do not worry about the lines that end in hash:filename. I’ll explain those once we get the main.cf sorted out.
sudo nano /etc/postfix/main.cf
The following restrictions are used to verify and validate the client hostname or client IP address passed to us.
smtpd_client_restrictions = permit_mynetworks reject_unknown_client_hostname check_client_access hash:/etc/postfix/access_client
These restrictions apply to the data sent by the client with the HELO/EHLO command.
smtpd_helo_restrictions = permit_mynetworks reject_non_fqdn_helo_hostname reject_invalid_helo_hostname check_helo_access hash:/etc/postfix/access_helo
The restrictions smtpd_sender_restrictions apply to the data sent by the client with the MAIL FROM command. These restrictions will have a value set for them. I usually delete those and replace them with these at the end of the file so all the restriction definitions are together.
smtpd_sender_restrictions = permit_mynetworks reject_unknown_sender_domain reject_non_fqdn_sender check_sender_access hash:/etc/postfix/access_sender
Access restrictions for relay control are performed before smtpd_recipient_restrictions. We can therefore stop our postfix mail server from being an open relay here rather than putting them in smtpd_recipient_restrictions. This allows rules for relay permissions, setting who can bounce emails off you to others. You should at the very least have these two permit_sasl_authenticated, reject_unauth_destination restrictions or your postfix server will be an open relay and that is not good.
smtpd_relay_restrictions = permit_mynetworks permit_sasl_authenticated reject_unauth_destination
These restrictions apply to the data sent by the client with the RCPT TO command. We should see some restrictions added to this in the main.cf. AswWe have taken some of those restrictions and moved them to other rules I usually just delete the existing rules and replace them with these. This is the restriction that calls Postgrey it takes time to call the Postgrey daemon and get a reply back compared to the built in rules. That is why it is last.
smtpd_recipient_restrictions = reject_unauth_pipelining reject_non_fqdn_recipient reject_unknown_recipient_domain check_recipient_access hash:/etc/postfix/access_recipient check_policy_service inet:127.0.0.1:10023
Save and exit your new copy of main.cf.
After updating the restrictions in main we need to comment out those used in master.cf. The values in master.cf override those in main.cf. I find it simpler to have them listed just once on main.cf and they are the same for submission/smtpd and smtps/smtpd.
Adjust the lines in master.cf as shown below:
submission inet n - y - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes -o smtpd_reject_unlisted_recipient=no # -o smtpd_client_restrictions=$mua_client_restrictions # -o smtpd_helo_restrictions=$mua_helo_restrictions # -o smtpd_sender_restrictions=$mua_sender_restrictions # -o smtpd_recipient_restrictions=permit_mynetworks,permit_sasl_authenticated,reject # -o smtpd_relay_restrictions=permit_sasl_authenticated,reject -o milter_macro_daemon_name=ORIGINATING smtps inet n - y - - smtpd -o syslog_name=postfix/smtps -o smtpd_tls_wrappermode=yes -o smtpd_sasl_auth_enable=yes -o smtpd_reject_unlisted_recipient=no # -o smtpd_client_restrictions=$mua_client_restrictions # -o smtpd_helo_restrictions=$mua_helo_restrictions # -o smtpd_sender_restrictions=$mua_sender_restrictions # -o smtpd_recipient_restrictions= # -o smtpd_relay_restrictions=permit_sasl_authenticated,reject -o milter_macro_daemon_name=ORIGINATING
Save and exit your new copy of master.cf.
Creating hash:filename Access Map Files
This is where we set up how postfix looks up data for lines such as check_recipient_access hash:/etc/postfix/access_recipient.
Access map files in Postfix are just another way for Postfix to lookup data. In these the client information is the key and the action is the value. The value will be an action or return code and an optional short message. I tend to name them access_TYPE where type is the type of access this map is looking for. That puts the files altogether when the directory is listed.
|check_client_access||Client IP & network address, hostnames, parent domains.|
|check_helo_access||hostnames, parent domains. To match against hostname or parent domain information from the HELO command|
|check_sender_access||Email addresses, domains and local parts. The MAIL FROM address, domain, parent domains, or localpart@|
|check_recipient_access||Email addresses, domains and local parts. To match against the address supplied in the RCPT TO command.|
The actions in the maps file can be OK, DUNNO or REJECT.
Accept the address etc. that matches the pattern.
Act as if the lookup key was not found and exit this check. This stops Postfix from continuing to test further substrings in this of the lookup key.
REJECT Optional short message.
Reject the address etc. that matches the pattern. Reply with the numeric value from $access_map_reject_code and output the optional message or a generic message in not provided.
An example /etc/postfix/access_client.in file. Create this file and save it.
sudo nano /etc/postfix/access_client.in
# Restricts which clients this system accepts SMTP connections from. # It uses data from the initial connection to this server. # allow my domain dragon.lab OK # even sub domains, note the leading full stop .dragon.lab OK # Allow IP range10.0.0.0/8, this is the CIDR for the test network 10 OK # Reject by parts of a domain # Using a TLD, this is probably over the top for most mailservers cn REJECT Someone from China, probably a spammer. ru REJECT Someone from Russia, probably a spammer. # Ban by domain or sub-domain for prolific spammers myfax.com REJECT Getting a lot of spam from myfaxcom. # Even sub domains, note the leading full stop .myfax.com REJECT Getting a lot of spam from xxx.myfax.com. # Reject IP addresses 192.168.0.0/24 as they cannot connect directly to the internet 192.168.0 REJECT Your IP address is not allowed here # 172.16/16 Same as for 192.168.0 172.16 REJECT You cannot be 172.16.x.x
An example /etc/postfix/access_helo.in file. Create this file and save it.
sudo nano /etc/postfix/access_helo.in
# This file is checked at the ehlo stage, it allows my servers in # and then starts to reject servers that pretend to be me # i.e A list of TLDs, probably over zealous for businesses # After that follows a list of domains and sub-domains # we do not want to talk to. # Do not put your own domain or IP in here it can be spoofed by spammers. # Reject by parts of a domain # Using a TLD, this is probably over the top for most mailservers. cn REJECT Someone from China, probably a spammer. ru REJECT Someone from Russia, probably a spammer. # Ban by domain or sub-domain for prolific spammers example.com REJECT Nothing should come from here .example.com REJECT Nothing should come from here # Reject IP addresses 192.168.0.0/24 192.168.0 REJECT Your IP address is not allowed here # 172.16/16 172.16 REJECT You cannot be 172.16.x.x
An example /etc/postfix/access_sender.in file. Create this file and save it.
sudo nano /etc/postfix/access_sender.in
# This file uses data from MAIL TO: command. It will do reverse IP # lookups to get the hostname if necessary. # Do not white list your own domain as spammers often spoof that. # Allow IP range 10.0.0.0/8 10 OK # Reject by parts of a domain # Using a TLD, this is probably over the top for most mailservers. cn REJECT Someone from China, probably a spammer. ru REJECT Someone from Russia, probably a spammer. # Full and part email addresses as well as domains and sub domains spammer@ myfax.com REJECT Scumbag spammer. bigfoot.com REJECT Source of spam. firstname.lastname@example.org REJECT we do not want email from this guy
An example /etc/postfix/access_recipient.in file. Create this file and save it.
sudo nano /etc/postfix/access_recipient.in
# This file uses information from the RCPT TO: command. It will use DNS # lookups to find the IP and reverse lookups to find the hostname if necessary. # Here you can list old email addresses that are on spam list # We can list the whole email or shorten them to just the part in front of the @ email@example.com REJECT Email address passed on to spammers from_spammer@ REJECT Email address passed on to spammers from_playdotcom@ REJECT Play.com tried to cover up they were hacked
Make sure the in file can only be read by root.
sudo chown root:root /etc/postfix/*in sudo chmod 600 /etc/postfix/*in
Postfix cannot use the .in files directly they need to be converted into db files. We do that with the postmap command as shown below but that gets tired very quickly. We also need to reload Postfix whenever they are updated.
sudo postmap /etc/postfix/access_client.in
Building in Files to db Files
Each time you make a change to your ‘*.in’ files you will need to rebuild them into db files and the reload postfix. This gets tiresome quickly. I have included a makefile that will do the hard work for you. Create and save the makefile in the /etc/postfix directory. If you have not seen a make file before, you are in for a treat :-). Make is well documented.
sudo nano /etc/postfix/makefile
# This makefile will build *.in files into *.db files. # When changes are made to the *.in files or the *.db # files do not exist. # Use the target clean to remove all *.db files objects := $(patsubst %.in,%.db,$(wildcard *.in)) all: $(objects) %.db: %.in postmap $^ @mv $^.db $@ service postfix reload .PHONY: clean clean: rm *.db
What make does
in case you have never seen or used make before. When you make changes to the ‘*.in’ files, make sure your current directory is /etc/postfix and run the command below.
cd /etc/postfix sudo make
Make will look though the recipe %.db: %.in.
- It will check to see if there is a corresponding .db file for each .in file. If not is will create it using the commands in the recipe.
- If the *.in file has been changed since the *.db was created, make will rebuild it using the commands in the recipe.
- If the commands were run in 1. or 2. then the service postfix will be reloaded.
If you run the command below it will remove all of the .db files
cd /etc/postfix sudo make clean
You can install make with the package build-essentials.
Note: As this is a make file ALL the indented lines MUST start with a TAB character or the recipe will not run correctly.
This would be a good backup point! You can always restore back to this point if you mess up on the way.
Sender Policy Framework Records
Up until now we have only been concerned with stopping unwanted mail hitting our mailserver. We can help make sure our real email gets delivered and be identified as genuine by other mail servers simply by publishing a SPF record in order to reduce the abuse of our domain in email headers.
We can do that by adding a TXT record to the DNS for your domain. This will most likely be controlled by the company who you registered your domain with.
In its simplest form you will need
TXT "v=spf1 mx -all"
Which is saying:
- v=spf1 this is a SPF record
- mx All the A records for all the MX records for domain are tested in order of MX priority. If the client IP is found among them, this mechanism matches.
- – Fail,
- all This mechanism always matches. It usually goes at the end of the SPF record.
Overall this SPF record is saying: Allow emails from my domain only if it was delivered from one of my MX servers, prohibit all others. This only works if all your mailservers, MX, are listed in your DNS records.
If all email for your domain is only sent by this mail server you can use ‘-‘ rather than ‘~’ in front of the all. The meaning of the leading character is explained clearly on the SPF official website. There are loads of other options to express the SPF record, so please take the time to look at the full SPF record syntax.
You can check that the DNS record is in place with the following command:
dig dragon.lab txt
In the text that is returned you should see a line similar to the following
;; ANSWER SECTION: dragon.lab 86400 IN TXT "v=spf1 mx -all"
Checking SPF Record
We can also make Postfix check the validity of in-coming email. We need to install the software. From my research there are two policy services for postfix one written in Perl the other in Python. The Python version has more functionality, so we will use that.
sudo apt install postfix-policyd-spf-python
We can then tell Postfix how to talk to this new policy service by adding a new restriction to the end of the smtpd_recipient_restrictions rule. As we are using smtpd_relay_restrictions for relay permissions we can put the restriction anywhere as this involves a DNS lookup it will be “expensive” so we put it at the end.
sudo nano /etc/postfix/main.cf
With the new policy service added your smtpd_recipient_restrictions rule should look something like this.
smtpd_recipient_restrictions = reject_unauth_pipelining reject_non_fqdn_recipient reject_unknown_recipient_domain check_recipient_access hash:/etc/postfix/access_recipient check_policy_service inet:127.0.0.1:10023 check_policy_service unix:private/policy-spf
Add a new line to the end of main.cf, note that the option has a mixture of hyphens and underscores.
policy-spf_time_limit = 3600s
Final change to let Postfix what to do , we edit the master.cf file.
sudo nano /etc/postfix/master.cf
Add the following to the end of the file.
# This block adds SPF checking policy-spf unix - n n - - spawn user=nobody argv=/usr/bin/policyd-spf
Restart postfix to take the new policy service into use.
sudo service postfix restart
Testing Your Mailserver
One very important test you should do is test to see if your mailserver will allow anyone to deliver mail anywhere. That is your mail server must NOT be an open relay. If you are an open relay spammers will love you and you will find your mailserver on black lists very quickly.
To perform an open relay test go to http://mxtoolbox.com/diagnostic.aspx and enter the name of your mailserver as seen from the Internet. The test tasks a second or so and you should see a bunch of green ticks. If any are not green fix the problems.
You can also test your domain records here and do a quick check that your SPF record is valid. It does not test that it will do what you want it to do.
Another site with a test for open relays is https://www.unlocktheinbox.com/openrelaytest/. If all is well with your server there is a message in saying your server is not an open relay. Again the rest of the site is worth reading through.
Well that is the end of Mail Server on Ubuntu 18.04 Part 4.
This would be a good backup point! You can always restore back to this point if you mess up on the way.
In Mail Server on Ubuntu 18.04 Part 5 we will add Roundcube as a web mail interface so your users can get at their mail from any browser. This will be updated to use an Apache virtualhost and be secure over HTTPS. We will then talk briefly about monitoring.