Welcome to Mail Server on Ubuntu 18.04 Part 2. This is the second part of a series of blog posts. The mailserver will use Postfix, Dovecot and Amavis. To see Mail Server on Ubuntu 18.04 Part1 follow the link.

In Mail Server on Ubuntu 18.04 Part 2 we will turn on the ability for any user to send and receive emails once they are known to the system via a MySQL database. This is called using virtual users. We will also turn on the ability to use encrypted connections with TLS.

Installing Some Software

Make sure everything is up to date with the latest patches and security updates before continuing with Mail Server on Ubuntu 18.04 Part 2.

sudo apt update
sudo apt dist-upgrade

To install the required software we need packages for MySQL and how MySQL talks to both Postfix and Dovecot and the lmtp extension for Dovecot. This will also pull in loads of other dependent packages.

sudo apt install postfix-mysql dovecot-mysql dovecot-lmtpd mysql-server 

Postfix Configuration

We can now make some changes to the Postfix configuration files ready for Postfix to start looking at our new domain, users and alias database.

Configuring Postfix

To control the configuration for postfix we will need to edit two files main.cf and master.cf.

sudo nano /etc/postfix/main.cf

Make sure the following session cache databases are not commented out. smtpd_use_tls in now obsolete in favour of smtpd_tls_security_level and so it can be commented out.

#smtpd_use_tls = yes
smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache

Amend the mydestination variable to only include the generic and localhost and domain values. It should look like this

mydestination = localhost.dragon.lab, localhost

I am assuming, oops, you do not use IPV6, because only about 25% of all internet sites use IPV6. Make this change if you are currently not using IPV6. If you do not know then most likely you are not using IPV6. Making this change will stop your mailserver from trying to connect using IPV6 and adding loads of warnings in your mail log file.

inet_protocols = ipv4

There are some new lines you will want, just add them to the end of the main.cf file.

# Postfix/TLS does not use the OpenSSL default of 300s, but a longer time of 3600sec 
# (=1 hour). RFC 2246 recommends a maximum of 24 hours. 
smtpd_tls_session_cache_timeout = 3600s

# We will let the client end use STARTTLS if they want, i.e they _may_ use it.
smtpd_tls_security_level = may

Add a new line to enable Dovecot’s LMTP

virtual_transport = lmtp:unix:private/dovecot-lmtp

Importantly we should not allow our mailserver from using the insecure SSLv2 and SSLv3 protocols. Check that they are not being used. The command below will show the values which Postfix is using.

sudo postconf -f smtpd_tls_mandatory_protocols

You are looking for the following useless there have been more protocols broken for found to have backdoors, so Mrs May can read your emails. This line shows that all protocols are allowed except SSLv2 and SSLv3 which is what we want.

smtpd_tls_mandatory_protocols = !SSLv2, !SSLv3

If your output does not show these add the line to the end of main.cf as shown below:

# This will default to !SSLv2 !SSLv3, therefore allowing TLSV1.1, TLSv1.2
smtpd_tls_mandatory_protocols = !SSLv2, !SSLv3

Stop forcing TLS authentication

sudo postconf -e smtpd_tls_auth_only=no

Save the changes and exit.

We can now edit the master.cf file.

sudo nano /etc/postfix/master.cf

Look for the following lines, they should all be commented out. The submission entries allow the use of port 587. We have set smtpd_tls_security_level=may so any one using this port may encrypt their connection if they wish to. Its up to the client end.

#submission inet n       -       y       -       -       smtpd
#  -o syslog_name=postfix/submission
#  -o smtpd_tls_security_level=encrypt
#  -o smtpd_sasl_auth_enable=yes
#  -o smtpd_tls_auth_only=yes
#  -o smtpd_reject_unlisted_recipient=no
#  -o smtpd_client_restrictions=$mua_client_restrictions
#  -o smtpd_helo_restrictions=$mua_helo_restrictions
#  -o smtpd_sender_restrictions=$mua_sender_restrictions
#  -o smtpd_recipient_restrictions=
#  -o smtpd_relay_restrictions=permit_sasl_authenticated,reject
#  -o milter_macro_daemon_name=ORIGINATING   

We want them to look like these lines. Other than simply uncommenting the lines smtpd_tls_security_level changes from encrypt to may.

submission inet n       -       y       -       -       smtpd
  -o syslog_name=postfix/submission
  -o smtpd_tls_security_level=may
  -o smtpd_sasl_auth_enable=yes
#  -o smtpd_tls_auth_only=yes
  -o smtpd_reject_unlisted_recipient=no
#  -o smtpd_client_restrictions=$mua_client_restrictions
#  -o smtpd_helo_restrictions=$mua_helo_restrictions
#  -o smtpd_sender_restrictions=$mua_sender_restrictions
  -o smtpd_recipient_restrictions=permit_mynetworks,permit_sasl_authenticated,reject
  -o smtpd_relay_restrictions=permit_sasl_authenticated,reject
  -o milter_macro_daemon_name=ORIGINATING

Save the changes and exit.

This would be a good backup point!

How Postfix talks to MySQL

We are going to create a bunch of map files that will tell Postfix how to get the information out of a MySQL database. They are fairly simple and I am sure you will work out how they do their mapping without explanation. In each file you will need to change the string harry@dragon.org.uk to show the actual password you use for your MySQL user called mail.

Create Postfix Map Files For MySQL

The Postfix map files are how we tell Postfix to talk to the MySQL database. Each file contains key = value pairs of data telling Postfix how it can connect and send a query to the database and therefore retrieve the data it requires. Once these map files are created we will add them to the Postfix configuration file /etc/postfix/main.cf using the command line tool postconf. It is simpler to use the postfix command line because you do not have to open the main.cf file in an editor. The resulting change will be written to the file and the postfix configuration can then be reloaded.

Mapping valid Domains

This map file is how Postfix will find valid domains within our mailserver. The domain table holds one row for each domain that our mailserver will be accepting mail for and that it will deliver to the end user. With an editor create the domain lookup file with the following details. You will need to change the password from MYSQLPasswd01! to whatever you are using as the password for your MySQL user mail.

sudo nano /etc/postfix/mysql-virtual-mailbox-domains.cf

user = mail
hosts = 127.0.0.1
dbname = postfix
password = MYSQLPasswd01!
query = SELECT domain FROM domain WHERE domain = '%s'

To add the necessary line to the file /etc/postfix/main.cf from the command line:

sudo postconf -e virtual_mailbox_domains=mysql:/etc/postfix/mysql-virtual-mailbox-domains.cf

Mapping Users

As we are intending to use our mailserver to deliver emails to users who are registered on the MySQL database and not just those that have Linux accounts. For Postfix to deliver the emails, each user will be a directory on the file system. We certainly do not want to create a new Linux user account and home directory for all our email users as we did for the quick test in part 1 when we used the user localadmin. What we will do is define a top level directory /var/vmail where all the incoming the emails will be stored. The data held in the virtual tables for the user mappings will be used to construct the lower part of the directory tree, select the domain and email address. For each user we will have:

/var/vmail/vhosts//

The domain and username are derived from their actual email address. For example, using fred@dragon.lab we will see this directory structure.

/var/vmail/vhosts/dragon.lab/fred

Let’s create the map file:

sudo nano /etc/postfix/mysql-virtual-mailbox-maps.cf

user = mail
hosts = 127.0.0.1
dbname = postfix
password = MYSQLPasswd01!
query = SELECT email FROM user WHERE email='%s'

sudo postconf -e virtual_mailbox_maps=mysql:/etc/postfix/mysql-virtual-mailbox-maps.cf

Mapping aliases to Users

This file maps email aliases or forwarders to real email addresses. This means for one username or destination we can have many email addresses.

For example, in a company, all the emails for support@dragon.lab and info@dragon.lab will be delivered to fred@dragon.lab. Harry is the IT administrator and therefore gets mail that is not addressed to anyone in particular. That is Harry will get mail that cannot be delivered anywhere else. Most of which will be spam so he will no doubt work out a better way to deal with it later as he gets more experience :). This is called a catch all email alias.

Using email aliases can also be used as an effective anti-spam system. Whenever you deal with a new company simply create a new email alias from_company@dragon.lab aliased to your own real email address. If you start getting spam on that address, you know they are spamming you or they have sold your address on or worse they have been hacked. In the same way that Play.com were hacked and then they failed to notify their customers in an effort to cover it up. Either way, you know it’s a company you no longer wish to deal with. You can remove the email listing and no more spam from them. If there is no catch all email setup then those spam emails get thrown away by your mail server.

sudo nano /etc/postfix/mysql-virtual-alias-maps.cf

user = mail
hosts = 127.0.0.1
dbname = postfix
password = MYSQLPasswd01!
query = SELECT destination FROM alias WHERE source = '%s'

This final mapping table is used as part of the virtual_alias_maps mapping and saves you having to map every user back to themselves. Why would you need to do that? You ask. See below, it is to do with the way that catch all addresses work.

sudo nano /etc/postfix/mysql-virtual-alias-maps-self.cf

user = mail
hosts = 127.0.0.1
dbname = postfix
password = MYSQLPasswd01!
query = SELECT email FROM user WHERE email = '%s'

The following command line should appear all on one line with no spaces after the ‘=’.

sudo postconf -e virtual_alias_maps=mysql:/etc/postfix/mysql-virtual-alias-maps.cf,mysql:/etc/postfix/mysql-virtual-alias-maps-self.cf

As these lookup table files all contain passwords they should not be readable to the world. We can do that by restricting the group permissions and removing all world read and write permissions.

sudo chgrp postfix /etc/postfix/mysql-*.cf
sudo chmod 640 /etc/postfix/mysql-*.cf

-rw-r----- 1 root postfix 124 2018-10-12 16:46 /etc/postfix/mysql-virtual-alias-maps.cf
-rw-r----- 1 root postfix 116 2018-10-12 16:46 /etc/postfix/mysql-virtual-alias-maps-self.cf
-rw-r----- 1 root postfix 135 2018-10-12 16:45 /etc/postfix/mysql-virtual-mailbox-domains.cf
-rw-r----- 1 root postfix 114 2083-10-12 16:46 /etc/postfix/mysql-virtual-mailbox-maps.cf

Aliases are the mechanism that allows mail to be delivered in a very flexible way with a very simple setup. That is the users get their mail and you as the Administrator, do not have too much extra work. 😉

To take the changes into use, restart Postfix.

sudo systemctl restart postfix

Configuring MySQL

Now we can start to config the MySQL database on Mail Server on Ubuntu 18.04 Part 2. We can start off by making MySQL a little more secure and making our own access a little simpler for this test server.

Simple MySQL Security Measures

Run the script mysql_secure_installation to remove unnecessary users and the test database. But you already did that, didn’t you?

sudo mysql_secure_installation

This will ask a number of questions that will help strengthen the security of your MySQL database. The first question will be asking for a MySQL root password, not sure why as accessing the root user for Mysql by password authentication is turned off by default now. Use sudo mysql to get access to the mysql prompt.

MySQL Password, just make one up and remember it! I will use MYSQLPasswd01!.

• VALIDATE PASSWORD PLUGIN: Y
• Level of password validation: 2 (up to you how secure you want it)
• Enter password for user root: MYSQLPasswd01!
• Re-enter new password: MYSQLPasswd01!
• Do you wish to continue with the password provided?:
• Remove anonymous users?: y
• Disallow root login remotely?: y
• Remove test database and access to it?: y
• Reload privilege tables now?: y

Make logging in simpler

We can make logging into MySQL with users other than root a little simpler by creating a .my.cnf file in the users HOME directory. This example assumes the local Linux user also has a Mysql user already created.

cd $HOME
nano .my.cnf

[client]
host = localhost
user = root
password = MYSQLPasswd01!

The file contains a password so make sure you change the permissions of that file so only you can read and write to it.

chmod 600 .my.cnf

Now all you have to do in order to log into a MySQL prompt is type mysql on the command line and the contents of the .my.cnf file will be read in and used. This is not a good idea on production servers. But while we are learning how to setup our server in a test area it saves time.

Configuring MySQL

We will edit the system MySQL mysqld.cnf file to check on access and get some logging to happen.

sudo nano /etc/mysql/mysql.conf.d/mysqld.cnf

The following should already be set. We want to bind to 127.0.0.1, not localhost. This stops connections from remote machines, which means that we have a security advantage in that only users already logged in to mailserver can access the MySQL database.

bind-address = 127.0.0.1

I find it is helpful to see what the MySQL DB is doing especially in the early stages. Use these options to start the logging of any queries or requests that are made to MySQL. these settings are probably commented out.

general_log_file = /var/log/mysql/mysql.log
general_log = 1

When all is running correctly comment those out again, as the logging slows MySQL down. Remember to turn these off on your production server when it goes live.

Restart the MySQL daemon.

sudo systemctl restart mysql.service

Monitor the log files

To monitor what is going on behind the scenes with Postfix and MySQL we can watch the tail of the log files and syslog file. Open a new terminal and use the following command to output appended data as the file grows. I leave this running all the time, it helps with debugging. Try it out, you will see what I mean. You can also run a tail on each file in its own terminal.

sudo tail -f /var/log/syslog /var/log/mysql/mysql.log /var/log/mysql/error.log /var/log/mail.log

Creating the Postfix Database

To create a MySQL database is very simple from the command line. We will be calling our database ‘postfix‘ as it is for Postfix and I like to keep things simple. 🙂 We need to login as the user root. If you setup the .my.cnf there is no need to use the -u root and -p command line switches. Create the database with the following:

sudo mysqladmin create postfix

If you look at the output in the log files you will see a line saying “create database `postfix`”.
You can also check by logging into mysql and running the show databases command. If all went well you will see a line with postfix.

sudo mysql

mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| mysql              |
| performance_schema |
| postfix            | =-- This is the one we are looking for
| sys                |
+--------------------+
5 rows in set (0.00 sec)

Creating our MySQL User

We will be needing a user with restricted permissions that can access the tables for lookups. This user will only need to SELECT data they will never be INSERTing or UPDATEing data.

sudo mysql

Enter the following SQL command when you see the mysql> prompt. Change the password to something complex on the live server it will only be used internally meaning you will rarely have to type it in yourself. Do not have a space in the password it makes adding the password to scripts harder. This command tells MySQL to create a new user called mail. That user will have a password of ‘MYSQLPasswd01!’ and can only use the SELECT clause on the database postfix.

Look at MySQL Users

GRANT SELECT ON postfix.* TO mail@localhost IDENTIFIED BY 'MYSQLPasswd01!';

This will create a new mysql user called mail who has SELECT access only to our database. To get these new permissions noticed by MySQL you need to flush them. So before you exit the mysql prompt use the following command.

flush privileges;

We can check that the user was created with the following query.

SELECT host, User, authentication_string FROM mysql.user where User = 'mail';

+-----------+------------------+-------------------------------------------+-----------------------+
| host      | user             | authentication_string                     | plugin                |
+-----------+------------------+-------------------------------------------+-----------------------+
| localhost | mail             | *EDF23DTHISISDENCRYPTEDEBCFFD45BB0151849B | mysql_native_password |
+-----------+------------------+-------------------------------------------+-----------------------+

While we are looking at the user access table, run the following query to see all the current users who can access our database.

SELECT host, User, authentication_string FROM mysql.user;

+-----------+------------------+-------------------------------------------+-----------------------+
| host      | user             | authentication_string                     | plugin                |
+-----------+------------------+-------------------------------------------+-----------------------+
| localhost | root             |                                           | auth_socket           |
| localhost | mysql.session    | *THISISNOTAVALIDPASSWORDTHATCANBEUSEDHERE | mysql_native_password |
| localhost | mysql.sys        | *THISISNOTAVALIDPASSWORDTHATCANBEUSEDHERE | mysql_native_password |
| localhost | debian-sys-maint | *98C82CDD7SECRETPASSWORDCCCA5EA21B97E1599 | mysql_native_password |
| localhost | mail             | *EDF23DTHISISDENCRYPTEDEBCFFD45BB0151849B | mysql_native_password |
+-----------+------------------+-------------------------------------------+-----------------------+

You will set at for the root user the plugin is set to auth_socket whereas the other users are all set to mysql_native_password. This is why you cannot login to Mysql with root using a password. If you want to be able to login to mysql with root and use a password rather then using sudo then the command below will sort that out. Remember to change ‘password’ to something more secure.

ALTER USER 'root'@'localhost' IDENTIFIED WITH mysql_native_password BY 'password';

Creating the tables

We will be using three tables to interface between Postfix and MySQL. The way I create them is to add the SQL commands to a text file and then load them in via the mysql command. This saves a lot of typing. It also means that when you are testing and trying out different options it is easy to delete all the data and start again. You just load the text file again. 🙂

Create reusable script file

USE postfix;

DROP TABLE IF EXISTS user;
DROP TABLE IF EXISTS alias;
DROP TABLE IF EXISTS domain;

CREATE TABLE domain (
   domain VARCHAR(50) NOT NULL,
   PRIMARY KEY (domain)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;


CREATE TABLE user (
   email VARCHAR(255) NOT NULL,
   domain VARCHAR(50) NOT NULL,
   password VARCHAR(110) NOT NULL,
   PRIMARY KEY (email),
   KEY (domain),
   FOREIGN KEY (domain) REFERENCES domain(domain)
           ON UPDATE NO ACTION ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8;


CREATE TABLE alias (
   id int(11) unsigned NOT NULL AUTO_INCREMENT,
   source varchar(255) NOT NULL,
   destination varchar(255) NOT NULL,
   PRIMARY KEY (id),
   KEY source_idx (source),
   KEY dest_idx (destination)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

The user table holds as you probably guessed from its name, data about the actual users of our email server. That is their email address and password.
The domain table is the simplest and is really just a list of domains our mailserver will process emails for.
The alias table is used to lookup an email address and forward them to the real user address.

Create a blank file called create_postfix_mysql_tables.sql, add the text above and then save it. You can then import create_postfix_mysql_tables.sql to refresh the table structure.

sudo mysql postfix < create_postfix_mysql_tables.sql

You should see the commands displayed in the tail from the log file if you have that running. If not you can see check the tables have now been created, by using the following commands:

sudo mysql postfix

mysql> show tables;
+-------------------+
| Tables_in_postfix |
+-------------------+
| alias             |
| domain            |
| user              |
+-------------------+
3 rows in set (0.00 sec)

If you got this far and it is all working, therefore this would be a good backup point!

Creating Some Test Users

To test those MySQL tables and the interface between Postfix and MySQL we will add some test data to the database. We can then test this is working by using the postmap command to check postfix can read the correct data.

Quick aside about passwords

The passwords for the users are encrypted with SHA-512, hence $6$. The salt is created on the fly with SUBSTRING(SHA(RAND()), -16) which will generate a random 16 character salt. This salt is actually added to the encrypted password so it can be used when checking passwords. The CONCAT part just adds the string '$6$' to the beginning so everyone knows this is a SHA-512 password.

If we use our usual 'MYSQLPasswd01! as the password to be encrypted and we will use 'Fred was sitting' as the salt so it is easy to see in the resulting string. Now run the command below in a mysql terminal we _should_ always get the same encrypted string back. If you replace the 'Fred was sitting' with SUBSTRING(SHA(RAND()), -16) you should get a different salt each time. and therefore a different encryption string.

sudo mysql postfix

SELECT ENCRYPT('MYSQLPasswd01!', CONCAT('$6$', 'Fred was sitting'));

Note: The string 'Fred was sitting' is after $6$ and between the 2nd and 3rd '$' in the string.

$6$Fred was sitting$oBRBC6DpIpc9S.CmjRaT7pwJCqB/n1PvZhE3/bCvstyShOB2caHkW84Drrtc0ZSk6Pc2cUHmikEvl36K3ZEVU/

Hopefully that explains that a little :-), clear as mud.

Create a SQL file

If we put the SQL required to build up some test data into a file say insert_test_data.sql. We can load it again when necessary while testing is on going and also add more data if required. This saves typing in the same commands over and over.

USE postfix;

DELETE FROM domain;
DELETE FROM user;
DELETE FROM alias;

INSERT INTO domain (domain) VALUES ('dragon.lab');

INSERT INTO user (email, domain, password) VALUES
    ('fred@dragon.lab', 'dragon.lab',  ENCRYPT('MYSQLPasswd01!', CONCAT('\$6\$', SUBSTRING(SHA(RAND()), -16)))),
    ('harry@dragon.lab', 'dragon.lab', ENCRYPT('MYSQLPasswd01!', CONCAT('\$6\$', SUBSTRING(SHA(RAND()), -16))));

INSERT INTO alias (source, destination) VALUES
-- emails to bert@dragon.lab actually go to Fred
    ('bert@dragon.lab', 'fred@dragon.lab'),
-- and so do emails to support and info
    ('support@dragon.lab', 'fred@dragon.lab'),
    ('info@dragon.lab', 'fred@dragon.lab'),
-- Harry is the IT guy so gets all emails to root and postmaster 
-- along with the catch all email alias, @dragon.lab -> harry@dragon.lab
    ('root', 'harry@dragon.lab'),
    ('postmaster@dragon.lab', 'harry@dragon.lab'),
    ('@dragon.lab', 'harry@dragon.lab');

You can then rebuild your database with test data with the following command.

sudo mysql postfix < insert_test_data.sql

You will see these being INSERTED in the tail of the log file. To check that the data was inserted correctly run the following queries.

sudo mysql postfix

mysql> SELECT * FROM domain;
+-------------+
| domain      |
+-------------+
| dragon.lab |
+-------------+
1 rows in set (0.00 sec)

mysql> SELECT * FROM user;
+------------------+------------+------------------------------------------------------------------------------------------------------------+
| email            | domain     | password                                                                                                   |
+------------------+------------+------------------------------------------------------------------------------------------------------------+
| fred@dragon.lab  | dragon.lab | $6$8a024ade09ec4b2e$70PtLJbZ.S316xiyhSnSECRETPASSWORDSsb68NzFiJPhNTpSZP7dRyDkdRMyZ9cMAw0JODNNTXFtZjP2Eu4.2HPlN8u. |
| harry@dragon.lab | dragon.lab | $6$67b5d001ed54ba85$ANOTERENCRYPETDPASSSWORDMgC677PfrLmB7OtEHiyAaXLmYEGjtpI5DO/ZhcYsA520/tupLt.ij191gGlYmokGCQj6seRJtN7. |
+------------------+------------+------------------------------------------------------------------------------------------------------------+
2 rows in set (0.00 sec)

mysql> SELECT * FROM alias;
+----+----------------------+------------------+
| id | source               | destination      |
+----+----------------------+------------------+
| 1 | root                  | harry@dragon.lab |
| 2 | bert@dragon.lab       | fred@dragon.lab  |
| 3 | support@dragon.lab    | fred@dragon.lab  |
| 4 | info@dragon.lab       | fred@dragon.lab  |
| 5 | postmaster@dragon.lab | harry@dragon.lab |
| 6 | @dragon.lab           | harry@dragon.lab |
+----+----------------------+------------------+
6 rows in set (0.00 sec)

We can now do some very simple tests to check the Postfix - MySQL interface it working. The first test will lookup dragon.lab to see if it is valid domain for our system. So go back to a bash command prompt. If it is working it will return dragon.lab. Also try this with a domain name that is not valid, example.com.

sudo postmap -q dragon.lab mysql:/etc/postfix/mysql-virtual-mailbox-domains.cf

We can lookup a user to see if they are a real user. This test should return fred@dragon.lab as he is valid. If you try with an invalid user mary@dragon.lab and it should return nothing.

sudo postmap -q fred@dragon.lab mysql:/etc/postfix/mysql-virtual-mailbox-maps.cf

This final test checks that aliases/forwarding is working bert@dragon.lab is an alias for fred@dragon.lab. That is all mail sent to bert@dragonlab is forwarded to fred@dragon.lab. So we should see Fred's email address returned as the answer.

sudo postmap -q bert@dragon.lab mysql:/etc/postfix/mysql-virtual-alias-maps.cf

If any of those tests fail you will need to go back and find your typo in the mysql*.cf files or maybe you forgot to restart postfix 🙂

This would be a good backup point!

Dovecot

Installing Dovecot

To get POP3 and IMAP features from your mail server we will need Dovecot. It should have been installed above.

Creating a vmail user

We are using virtual users on this mailserver. That means not all users will have a HOME directory in which we can store incoming emails. So we will store all mail in one directory structure as explained above. Look for a silver text box above.

We will be creating a non privileged user vmail to get access to the email directory structure as it will be more secure. Let's create that user and group now. I am using a user id (UID) and a group ID (GID) of 6004 , although you can use any unused UID and GID. This numeric value is needed again later on, so remember what you used.

sudo groupadd -g 6004 vmail
sudo useradd -g vmail -u 6004 vmail -d /var/vmail -m -s /sbin/nologin

Let’s check the group and user have been created correctly. We can do that by looking in the files passwd and group.

grep vmail /etc/passwd /etc/group

/etc/passwd:vmail:x:6004:6004::/var/vmail:/sbin/nologin
/etc/group:vmail:x:6004:

The top level directory holding the emails will not exist. We will create it now. The system will automaticially generate the remaining directory tree as required. We also need to change the owner and group for the directory tree.

sudo mkdir -p /var/vmail/vhosts/dragon.lab
sudo chown -R vmail:vmail /var/vmail
ls -dl /var/vmail/vhosts

This is the expected output.

drwxr-xr-x 1 vmail vmail 4096 Sep 9 09:52 /var/vmail/vhosts/

Configuring Dovecot

Make a copy of the files we are going to amend before we start. When a mistake is made you still have the original to compare against. 🙂

10-auth.conf

cd /etc/dovecot
sudo mkdir backup
sudo cp -a conf.d/10-auth.conf \
     conf.d/99-mail-stack-delivery.conf \
     dovecot-sql.conf.ext backup

Modify conf.d/10-auth.conf so that it will load auth-sql.conf.ext and not auth-system.conf.ext. The lines you are looking for are at the end of the file.

sudo nano /etc/dovecot/conf.d/10-auth.conf

Two lines to modify here, comment one and uncomment the other.

#!include auth-system.conf.ext
!include auth-sql.conf.ext

dovecot-sql.conf.ext

There are some changes which need to be made in the system file /etc/dovecot/dovecot-sql.conf.ext. Use search/find and amend the lines as necessary.

sudo nano /etc/dovecot/dovecot-sql.conf.ext

Because we used the UID and GID 6004 for vmail above we must use the same value in the user query for the vmail user and group from above see the silver box.

# /etc/dovecot/dovecot-sql.conf.ext
driver = mysql
# There is a password on this line update it to the one you are using
connect = host=127.0.0.1 dbname=postfix user=mail password=MYSQLPasswd01!
default_pass_scheme = SHA512-CRYPT
password_query = SELECT email as user, password FROM user WHERE email='%u'
user_query = SELECT email as user, 6004 AS uid, 6004 AS gid FROM user WHERE email = '%n@%d' AND domain = '%d'

Update the permissions as dovecot-sql.conf.ext contains a password in the open.

sudo chown root:root /etc/dovecot/dovecot-sql.conf.ext
sudo chmod 600 /etc/dovecot/dovecot-sql.conf.ext

99-mail-stack-delivery.conf

Dovecot reads the configuration files in order with the lowest being read first. Any settings found in a later file will override those already read in. Therefore, the settings in 99-mail-stack-delivery.conf override all other settings. We will put the majority of our changes in this file. It also means that when dovecot gets updated most of our changes will be untouched by new files.

sudo nano /etc/dovecot/conf.d/99-mail-stack-delivery.conf

For each line or group of lines find them with search and then amend them as necessary. If the line cannot be found then simply add it to the end of the file. The file name in comments shows the original file where the setting was found. The file name in comments does not have to be changed.

# /etc/dovecot/dovecot.conf
protocols = imap pop3 lmtp
disable_plaintext_auth = no

# /etc/dovecot/conf.d/10-master.conf
mail_location = maildir:/var/vmail/vhosts/%d/%n

# etc/dovecot/conf.d/10-ssl.conf
service auth {
  # Listener path must match Postfix smtpd_sasl_path
  unix_listener /var/spool/postfix/private/dovecot-auth {
    mode = 0660
    user = postfix
    group = postfix
  }

    unix_listener auth-userdb {
       mode = 0600
       user = vmail
       #group =
    }

    # Auth process is run as this user.
    user = dovecot
}

We need to add some additional configuration lines to 99-mail-stack-delivery.conf. Just add them to the end of the file.

# conf.d/10-mail.conf                         
mail_privileged_group = vmail

# conf.d/auth-sql.conf.ext
passdb {
   driver = sql
   args = /etc/dovecot/dovecot-sql.conf.ext
}

# conf.d/auth-sql.conf.ext
userdb {
   driver = static
   # The home value should be the same as the mail_location
   args = uid=vmail gid=vmail home=/var/vmail/vhosts/%d/%n
}

# conf.d/10-master.conf
service imap-login {
  inet_listener imap {
    port = 143
  }
  inet_listener imaps {
    port = 993
    ssl = yes
  }
}

# conf.d/10-master.conf
service pop3-login {
  inet_listener pop3 {
    port = 110
  }
  inet_listener pop3s {
    port = 995
    ssl = yes
  }
}

# conf.d/10-master.conf
service lmtp {
   unix_listener /var/spool/postfix/private/dovecot-lmtp {
     mode = 0600
     user = postfix
     group = postfix
   }
}

# conf.d/10-master.conf
service auth-worker {
  # Auth worker process is run as root by default, so that it can access
  # /etc/shadow. If this isn't necessary, the user should be changed to
  # $default_internal_user.
  user = vmail
}

# conf.d/10-auth.conf
# Authentication configuration, Login is used by M$ for Outlook express and Windows Mail.
# If your users do not use either of those no need to add the line as plain is default
auth_mechanisms = plain login  

# If you turned off IPV6 above then add this line. "*" listens in all IPv4 
# interfaces, "::" listens in all IPv6 interfaces. defaults to (listen = *, ::)
listen = *

# These last few lines turn on logging while we are debugging, to turn 
# off the verbose logging comment these lines and restart dovecot

log_path = syslog
syslog_facility = mail
auth_verbose = yes
auth_debug = yes
auth_debug_passwords = yes
mail_debug = yes
verbose_ssl = yes

Save your changes and exit.

Restart dovecot to take the changes into use.

sudo systemctl restart dovecot.service

In your log files, you should see some output similar to this

Sep  5 17:57:11 mailserver systemd[1]: Stopping Dovecot IMAP/POP3 email server...
Sep  5 17:57:11 mailserver dovecot: master: Warning: Killed with signal 15 (by pid=1165 uid=0 code=kill)
Sep  5 17:57:11 mailserver systemd[1]: Stopped Dovecot IMAP/POP3 email server.
Sep  5 17:57:11 mailserver systemd[1]: Started Dovecot IMAP/POP3 email server.
Sep  5 17:57:11 mailserver dovecot: master: Dovecot v2.2.33.2 (d6601f4ec) starting up for imap, pop3, lmtp (core dumps disabled)

When you restart dovecot, you may see an error message about a file not readable (yet?). After this one time it should exist and be readable.

Now would be a good time to make a backup of your new server.

Testing Emails Get Sent

For testing out the mailserver we can use OpenSSL or telnet. Using these commands is a little more complex but means you do not need to install the mail utilities. You can also see the output in the log files as you enter the commands. It makes it simpler to debug. See these blog posts:

• Send an email with Telnet
• Test a POP3 mailserver

Pop over to those posts and send and read some emails. You should be able to send and read emails to Fred and Harry. As well as the aliases we set up for postmaster, root and bert. For now use the telnet option when opening a connection via POP3.

Setting Up A GUI Email Client

After the initial testing using the command line which gives you finer control and can see errors as you step through the email sending and receiving process, I like to use a GUI Client. We can set up Thunderbird or Evolution or any other email client. I'm sure you can work this out but here is a very quick and dirty description. Because we setup Postfix to say the user MAY use SSL, we can use an SSL value of None or STARTTLS or even fully encrypted. If you are using Thunderbird and no encryption you will see a big scary message saying your connections will not be encrypted.

I tend to use Evolution as I can set up multiple accounts for both Fred and Harry, this gives me the ability to test out sending and receiving with and without encryption and use POP and IMAP.

As we have not yet turned on TLS connections for now set the SSL or Encryption method to NONE

• Name: Fred
• Email Address: fred@dragon.lab
• Username: fred@dragon.lab
• Password: MYSQLPasswd01!
• POP3:
  • Server: mailserver.dragon.lab
  • Authentication: Normal Password
  • Port: 110
  • SSL: STARTTLS
• IMAP:
  • Server: mailserver.dragon.lab
  • Port: 993 | 143
  • SSL: TLS on dedicated port | STARTTLS
  • Authentication: Normal Password
• SMTP:
  • Server: mailserver.dragon.lab
  • Port: 587
  • SSL: STARTTLS | None
  • Authentication: Plain
• SMTP:
  • Server: mailserver.dragon.lab
  • Port: 465
  • SSL: TLS on Dedicated port
  • Authentication: Plain

We should now be able to send and receive emails for fred@dragon.lab and harry@dragon.lab YAY! Emails sent to bert@dragon.lab will be forwarded to Fred when he logs in and downloads his email. Bert@dragon.lab is an alias for Fred@dragon.lab. Any emails sent to unknown email accounts should be forwarded on to Harry via the forwarder @dragon.lab.

Send some emails to test out the forwarding, just to make sure it is all working as expected.

This would be a good backup point! Changing the encryption protocols in Evolution many times can screw it up.

Turn on TLS

Now we have a Mail Server on Ubuntu 18.04 we can add the ability to use encrypted connections. This will make sure the guys working for GCHQ and NSA have something to do. 🙂

SSL Certificates

To turn on encrypted connections we need to edit three files and generate some certificates. In this post I will use SSL certificates from Let's Encrypt, but not explain how to create them :). The lets Encrypt documentation is rather good and the process of getting SSL certificates from them is straight forward once you have read the documentation. Go and take a look.

As this is a test server you may want to generate your own self-signed SSL certificate, I have a post for that. That post creates .crt and .key files that can be used without any conversion needed. Which ever method you chose when you have your certificate come back.

Postfix Changes

We can start by amending the post fix config files, main.cf and master.cf.

sudo nano /etc/postfix/main.cf

Add the certificate and key files to the entries. They should point to the files for the key and certificate. I will be using letsencrypt certificates.
Add these to the end of the file. Check that the options are not already enabled in your own file. These lines add the certificate key and crt files. The line for smtpd_tls_loglevel adds some logging, which is always useful when setting up a new system. This can be commented out when you go live.

# For TLS
smtpd_tls_cert_file = /etc/letsencrypt/live/mailserver.dragon.lab/fullchain.pem
smtpd_tls_key_file =  /etc/letsencrypt/live/mailserver.dragon.lab/privkey.pem
smtpd_tls_loglevel = 3
# This can be medium or high 
smtpd_tls_mandatory_ciphers = medium    

We can uncomment some lines from master.cf, these will be found just below the submission lines we uncommented before. These lines enable SMTP on port 465. The submission lines use port 587. Your choice.

sudo nano /etc/postfix/master.cf

smtps     inet  n       -       y       -       -       smtpd
  -o syslog_name=postfix/smtps
  -o smtpd_tls_wrappermode=yes
  -o smtpd_sasl_auth_enable=yes
#  -o smtpd_tls_auth_only=yes
  -o smtpd_reject_unlisted_recipient=no
#  -o smtpd_client_restrictions=$mua_client_restrictions
#  -o smtpd_helo_restrictions=$mua_helo_restrictions
#  -o smtpd_sender_restrictions=$mua_sender_restrictions
  -o smtpd_recipient_restrictions=permit_mynetworks,permit_sasl_authenticated,reject
  -o smtpd_relay_restrictions=permit_sasl_authenticated,reject
  -o milter_macro_daemon_name=ORIGINATING

For more information about the Postfix Configuration Parameters see this page.

Dovecot Changes

Now to update dovecot, add the full path and filenames to the certificate options. The two other lines turn off SSLv3 and other broken protocols. This will stop the POODLE hacks.

sudo nano /etc/dovecot/conf.d/99-mail-stack-delivery.conf

ssl_cert = </etc/letsencrypt/live/mailserver.dragon.lab/fullchain.pem
ssl_key = </etc/letsencrypt/live/mailserver.dragon.lab/privkey.pem
ssl_cipher_list = ALL:!LOW:!aNULL:!ADH:!eNULL:!EXP:!SSLv3:RC4+RSA:+HIGH:+MEDIUM
ssl_protocols = !SSLv3

Restart Postfix and Dovecot as we have updated their configurations. To test this out tweak the setting in your email client, see the settings above or look at one of my posts about accessing Emails with telnet or openssl.

sudo systemctl restart dovecot
sudo systemctl restart postfix

Testing SSL

We can either use a GUI client like Evolution of use the command line. Using the command line you can watch the log files as you proceed through each stop. This makes it easier to debug. Here are a couple of posts that take you through the process.

• Send an email over openssl
• Testing IMAP mailservers

Force The Use of Encryption

If you are setting up a mailserver for a business you will want to force the use of encrypted connections. this can be achieved with a few small changes.

sudo postconf -e smtpd_tls_auth_only=yes

sudo nano /etc/postfix/master.cf

In the submission block change this setting.

smtpd_tls_security_level=encrypt

And the changes for pop and IMAP stop the use of plain text logins.

sudo nano /etc/dovecot/conf.d/99-mail-stack-delivery.conf

disable_plaintext_auth = yes

sudo systemctl restart dovecot
sudo systemctl restart postfix

Give that all a test you should now only be able to send and receive emails when using STARTTLS or TLS on a dedicated port.

All should be working for our Mail Server on Ubuntu 18.04 Part 2. Therefore, now would be a good time to make a full backup of your new server, because you can then roll back to here if things break later on.

In Part 3

In Mail Server on Ubuntu 18.04 Part 3 of this series we will be adding anti-virus and anti-spam software and a tool postgrey. This will greylist emails and will cut down the work our server has to do.