Test Networking Lab Ubuntu On 18.04

Overview

I wanted to create a test networking lab Ubuntu on 18.04 to try out configurations without having to hack my real LAN about. I decided to use VirtualBox virtual machines as the software is freely available and works with Linux very nicely. For the network in the test networking lab Ubuntu on 18.04 we will need to separate from our own normal LAN. VirtualBox provides this in of the box. We simply set the network interfaces to use “Internal Network”. This way any gust VMs on the host within the internal network can only see each other.

The test networking lab Ubuntu on 18.04 will need a router to connect to the outside networks in the same way as you have an ADSL modem/router to connect to the internet. You will also need some servers and desktops so you can experiment with different configurations. Form the point of view of our our test networking lab Ubuntu on 18.04, our real LAN will be the internet.

Network Topology for the lab

A quick pictorial overview of the test networking lab Ubuntu on 18.04.

During this post I talk about Local LAN and real LAN this is your LAN as it was before you added the test networking lab Ubuntu On 18.04. Anything your side of your router to the internet.

Networking within the test lab

We need to create some test machines inside the test networking lab Ubuntu on 18.04. The test machines I use are an Ubuntu 18.04 Minimal Server Installation. Then I will add MATE for a desktop machine. I always add openssh-server so I can access the new machine via a terminal and therefore copy paste while I’m setting it up. It also allows for a larger terminal rather than the default 80×23 🙂 In fact, I have a PXE boot server and preseed configurations setup to create these installations. It takes 7-12 minutes to do the automated install. Just long enough to make a coffee.

VirtualBox settings for lab-router

So back to creating our test networking lab Ubuntu on 18.04. The settings needed for the Virtualbox guest for the lab-router are not much different than for a machine running on your LAN. The only difference will be the networking. We will need two NIC’s an external, which connects to our real LAN to the guest and an internal which connects to all of the machines in the test lab. The eth0 in the configurations below are from the virtualbox host machine. It does get a little confusing. 🙂

Virtualbox settings for the external NIC

This is the WAN connection from the point of view of our lab-router. It will end up with an IP address of 192.168.0.20.

Virtualbox settings for the internal NIC

This is the LAN connection from the point of view of our lab-router. It will end up with an IP address of 10.1.200.1.

The Lab Router

So get yourself a minimal Ubuntu 18.04 server installation. The lab router will require two network cards to be configured. The first will be automatically setup when you install the server edition of Ubuntu. In my case it was called enp0s3. This uses the new naming convention taken in part from the position of the NIC on the pci bus.

lab-router network configuration

Now to setup the initial network interface configuration for lab-router The address can be any IP address that is free on your real LAN. I will use 192.168.0.20.

The gateway for this NIC will most probably be your real LAN gateway as are the DNS addresses setup on this interface.

Using the older NIC naming schema second NIC would to be called eth1. With the new naming scheme for NIC, we need to find out where on the pci bus it is sitting. To find the name the system has decided to call eth1 we can use

dmesg | grep eth

[    1.515269] e1000 0000:00:03.0 eth0: (PCI:33MHz:32-bit) 08:01:02:03:04:05
[    1.515276] e1000 0000:00:03.0 eth0: Intel(R) PRO/1000 Network Connection
[    1.941639] e1000 0000:00:08.0 eth1: (PCI:33MHz:32-bit) 08:01:02:03:04:06
[    1.941649] e1000 0000:00:08.0 eth1: Intel(R) PRO/1000 Network Connection
[    1.944338] e1000 0000:00:03.0 enp0s3: renamed from eth0
[    1.960369] e1000 0000:00:08.0 enp0s8: renamed from eth1

You can see from the output above that the two interfaces on my virtual machine are now renamed to enp0s3 and enp0s8. On real hardware the names will be different, I get en01 on an MSI motherboard go figure.

We can also see a list of all NICs that are physically connected to the virtual machine even if they are not configured.

cat /proc/net/dev

 face |bytes    packets errs drop fifo frame compressed multicast|bytes ...
enp0s3: 136806606   90946    0    0    0     0          0               ...
    lo:    3600      28    0    0    0     0          0                 ...
enp0s8:       0       0    0    0    0     0          0                 ...

External NIC

Name enp0s3
IP Address 192.168.0.20
Netmask 255.255.255.0
Gateway 192.168.0.1 <= This is the gateway on your real LAN
DNS server 192.168.0.2 & 192.168.0.3 <= This is the DNS on your LAN

Internal NIC

Name enp0s8
IP Address 10.1.200.1
Netmask 255.255.255.0

The second NIC is internal as it is the one that connects the machines that are in our test network lab. This information can be used to update the netplan YAML file to configure the network. I do wish “they” would stop playing around with the config files. More information can be found here although there are examples they tend to quickly diverge into some pretty strange setups. What was wrong with the interfaces file anyway? 🙂

sudo nano /etc/netplan/01-netcfg.yaml

network:
   version: 2
   renderer: networkd
   ethernets:
      enp0s3:
         dhcp4: no
         addresses: [192.168.0.20/24]
         gateway4: 192.168.0.1
         nameservers:
             addresses: [192.168.0.2, 10.1.200.3]
             search: [dragon.lab]
      enp0s8:
         dhcp4: no
         addresses: [10.1.200.1/24]

We can now generate the settings this will test validate the yaml file without making it live, but it will be at the next reboot. If all went well or when you have fixed the typos you can apply the settings and they will become live now.

sudo netplan generate
# If no errors are reported then continue.
sudo netplan apply
ifconfig

enp0s3: flags=4163  mtu 1500
        inet 192.168.0.20  netmask 255.255.255.0  broadcast 192.168.0.255
        inet6 fe80::a00:27ff:fe89:1b45  prefixlen 64  scopeid 0x20
        ether 08:00:27:89:1b:45  txqueuelen 1000  (Ethernet)
        RX packets 137246  bytes 206627470 (206.6 MB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 5918  bytes 471176 (471.1 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

enp0s8: flags=4163  mtu 1500
        inet 10.1.200.1  netmask 255.255.255.0  broadcast 10.1.200.255
        inet6 fe80::a00:27ff:fe42:a1b7  prefixlen 64  scopeid 0x20
        ether 08:00:27:42:a1:b7  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 54  bytes 8044 (8.0 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

Dnsmasq on lab-router

You will want to install some form of DNS and DHCP server on to the lab-router unless you plan on having one somewhere else within your own test networking lab Ubuntu on 18.04. See my post for setting up dnsmasq. HOWTO Setup Dnsmasq as a DNS and DHCP server/. You can easily turn on and off the DHCP for Dnsmasq by commenting out the dhcp-range options in the configuration file and restarting the dnsmasq daemon. See at the end of this post for some scripts.

Here is a quick config file for Dnsmasq. You will need to rename it to test_lab_dnsmasq_setting.tgz and then unzip it. The two files are simple configs for Dnsmasq to get you started.

The Desktop and Server Machines

These machines are used to simulate servers and workstations or desktop computers inside the lab. They will not communicate directly with the local LAN only other lab machines and lab-router. They can also be installed outside of the lab environment and then the network changed as follows:

VirtualBox Network Settings

This machine only needs one NIC and it is on the Virtualbox Internal Network with a name of lab1. Which is the same internal network you used for lab-router. See below for an example of a network configuration using a dynamic IP address. The commented out config is for a static IP address and assumes lab-router is the DNS server in the test networking lab Ubuntu on 18.04.

lab-desktop1

sudo nano /etc/netplan/01-netcfg.yaml

network:
   version: 2
   renderer: networkd
   ethernets:
      enp0s3:
         dhcp4: yes
#  To use a static IP address comment the above settings and uncomment these ones below.
#      enp0s3:
#         dhcp4: no
#         addresses: [10.1.200.20/24]
#         gateway4: 10.1.200.1
#         nameservers:
#             addresses: [192.168.0.2, 10.1.200.3]
#             search: [dragon.lab]

Do not forget to change the VirtualBox settings on the network card to Internal Networking, and when you start this machine up again you may not be able to access anything outside of the test lab. If you do not have a DNS running on your lab-router that will be true.

The Firewall on lab-router

This is the part that makes your test lab usable. It will isolate the test lab from your real LAN. The lab-router will be the gateway in and out of the test network just like your router that is connected to the internet is for your own LAN.

Stop The Firewall

Lets start with a little script to turn off all rules and flush everything for iptables. You may think that is a strange thing to start with. Why I do it this way round is if when we make a mistake or a typo with the IPTABLE rules we have an easy way to turn off the firewall and regain access again. 🙂

sudo nano /root/bin/fw.stop

#!/bin/bash

echo "Stopping firewall and allowing everyone..."
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
iptables -t filter -F
iptables -t raw -F
iptables -t mangle -F
iptables -t nat -F
iptables -F
iptables -X

Change the permissions so only root can execute it or in fact see the file.

sudo chmod 700 /root/bin/fw.stop
sudo chown root:root /root/bin/fw.stop

Now any time we need to remove all the rules you have added to iptable we can simply simply execute the script after logging in at the console, i.e. the VirtualBox window.

sudo /root/bin/fw.stop

We can see the table rules in our firewall with either iptables-save or iptables -S.

iptables -S

-P INPUT ACCEPT
-P FORWARD ACCEPT
-P OUTPUT ACCEPT

From the above you can see all the chains are open that is they are ACCEPTing connections/packets. It is as if we have no firewall the windows are wide open. 🙂

Allow Port Forwarding

Since the lab machines will need to talk to the outside world, and the internet, we will need to turn on port forwarding. This is done by configuring iptables/sysctrl to allow port forwarding. To turn this on we have some choices, to do it until the new reboot use the following.

sudo echo "1" > /proc/sys/net/ipv4/ip_forward

You can use sysctl to perform the same function but this will, also, not survive through a reboot.

sudo sysctl net.ipv4.ip_forward=1

Or you can edit the .conf file as shown below which will continue to work over a reboot.

sudo nano /etc/sysctl.conf

# Uncomment the next line to enable packet forwarding for IPv4
net.ipv4.ip_forward=1

Adding Some Rules

We need to add three rules one for the loopback interface and the other two for the external interface, enp0s3, so that packets get SNATed to the outside and we can accept returning packets to established connections and anything on the local loopback interface.

sudo iptables -A INPUT -i lo -j ACCEPT
sudo iptables -A INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
sudo iptables -t nat -A POSTROUTING -o enp0s3 -j SNAT --to-source 192.168.0.20

With this rule connections from the lab will reach the internet and should work. 🙂

sudo su -c 'iptables-save > /etc/iptables.rules'

It should look something like this:

-P INPUT ACCEPT
-P FORWARD ACCEPT
-P OUTPUT ACCEPT
-A INPUT -i lo -j ACCEPT
-A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT

These rules will not survive a reboot though. One way to load the rules at boot time is to use this handy utility iptables-persistent.

sudo apt-get update
sudo apt-get install iptables-persistent

To save the rules that we want to be loaded at boot time either of the commands below, they both give the same end result.

sudo iptables-save > /etc/iptables/rules.v4

sudo netfilter-persistent save

Whatever rules are in /etc/iptables/rules.v4 will be reloaded at boot time.

The above, iptables-save, should be done whenever you want to update the rules loaded at boot time. That is, try out the new rules from the command line. When you are happy with them save them to the /etc/iptables/rules.v4 file.

At this point we should be able to ping or dig sites like bbc.co.uk or cnn.com and machines on your own local LAN remember to use the fully qualified names for your LAN. As these are considered part of the internet when looking from within our test networking lab Ubuntu on 18.04. You should also be able to ssh to lab-router.

The network is totally open. All ports are open with no connections are being forwarded to any servers such as mail or web servers.

Lock Down All the Ports

I find that once I get to this stage I want to be able to add and change the IPTABLE rules so I put them into a script. This is because the order of the rules is significant. Lets create a file called fw.rules_common.

#!/bin/bash
#
# If you change this file and want them to be used when the machine reboots 
# remember to rebuild the default rules loaded when the network comes up,
# /etc/iptables.rules
# Use the command below after testing to regenerate that file:
# iptables-save > /etc/iptables.rules
#
#
# Get the firewall into a known clean state, that is empty and fully open.
echo "Turn off the firewall and remove all existing rules"
/root/bin/fw.stop

echo "Enable rules to block stuff"

# Allow all connections to lo (loopback)
# ======================================
iptables -A INPUT -i lo -j ACCEPT

# Allow replies to established connections
# ========================================
iptables -A INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT

# Sort out NATing for lab machines
# ================================
iptables -t nat -A POSTROUTING -o enp0s3 -j SNAT --to-source 192.168.0.20

# ================================
# These lines should be at the very end of the script.
# REJECT any remaining packets
# ===========================
# This must be LAST, it blocks all remaining new INPUTs on enp0s3
#iptables -A INPUT -i enp0s3 -j DROP

# OR set the default action for INPUT to be DROP
# This has the same effect. This rule can go anywhere,that is it's position
# does not matter, it can go first, in the middle or last.
iptables -P INPUT DROP

Remember to change the permissions as we did above for the fw.stop script.

If you run bin/fw.stop and then this script it will first remove all the rules and then put the new ones back ready for testing. At the end of this post is full listing with some examples you can build on it also has a few extra wrinkles to help debug your rules.

The script above has a rule at the end which DROPs all remaining packets. I DROP them you can also REJECT them. There are many many posts advocating both methods. You can read up on that and then choose which way to go. Both REJECT and DROP have principally the same effect they block or turn off the port. So at this point it does not really matter. Whether to REJECT or DROP is discussed at length and and sometimes with religious fanaticism. It is your choice. 🙂

After running the two scripts above you will NOT be able to ssh from your LAN into the lab-router. ALL ports are closed. Due to the last rule or the INPUT default.

It is usual when configuring a firewall to block everything and then only allow in the stuff you want. This means unless you have made a rule to allow something, it is not allowed in. In all the example anything going OUT is allowed. This makes it simpler to start with they can be added later.

/root/bin/fw.rules_common

We will run these two commands each time we add rules to the firewall. If we just add the rules from the command line they will end up in the wrong order. Try it and see 🙂

Allow SSH to lab-router again

We need to be able to ssh into lab-router so lets get that set of rules going first. This rule will allow new incoming packets from the IP range 192.168.0.0-192.168.0.255 on port 22. To put that another way we can ssh from our real LAN (192.168.0.0/24) into lab-router. If after running the script /root/bin/fw.rules_common you run the following command with sudo it will appear to have no effect. This is because we have append (-A) the new rule to the end of the existing rules. Before the ssh rule the last rule was to REJECT all INPUT on enp0s3. enp0s3 is the test lab WAN connection.

iptables -A INPUT -i enp0s3 -s 192.168.0.0/24 -p tcp --dport 22 -j ACCEPT

To get it working for real this time. Add the new line above the comment “These lines should be at the very end of the script.“. Run the script again and now you can create new ssh connections to lab-router.

We could also lock down ssh connections to lab-router so only one IP address could connect. For example, you have two machines on your LAN 192.168.0.5 & 192.168.0.6 and you only want 192.168.0.6 to be able to connect. Change the -s 192.1268.0.0/24 from the range to a single IP address like so.

iptables -A INPUT -i enp0s3 -s 192.168.0.6 -p tcp --dport 22 -j ACCEPT

Remember to rerun the script. Then try and open a new connection from both machines. Remember existing connections will no longer work.

We can now allow and drop/reject connections to lab-router as we like. Also limiting access to a range of IP addresses or just a single address. But we can only access lab-router. If we have a mailserver or webserver or wanted to ssh to a lab-server1 we could not do that directly.

Simple Port Forwarding from the lab-router

In this example of simple port forwarding we will be allowing users to ssh to both lab-router and lab-server1. To access lab-router we will use port 22 and to access lab-server1 we will start off using on port 2222 and forward that to port 22 on lab-server1.

Since you can only redirect/forward once we will need to use a different port number along with the usual port 22 for ssh. We will use port 2222, it is easy to remember. 🙂

We need to open port 22 as we did above, so no change there. As PREROUTE rules are performed before any INPUT rules we do not need an INPUT rule to ACCEPT port 2222. We will need to ACCEPT port 22 when FORWARDing it to 10.1.200.10. The two rules we need are:

    # Change packets on port 2222 so they go to lab-server1 on port 22.
iptables -t nat -A PREROUTING -p tcp --dport 2222 -i enp0s3 -j DNAT --to 10.1.200.10:22
    # Allow packets on port 22 to go from the WAN interface to the LAN interface.
    # Remember the packet has already been mangled so port 2222
    # is no longer port 2222 it is port 22.
iptables -A FORWARD -i enps03 -s enp0s8 -p tcp --dport 22 -j ACCEPT

Now that is fully explained, clear as mud :).

Rerun the rules script to check that out. From a machine on your LAN you should be able to ssh to lab-router as you did before and also get to lab-server1 with the following. Note: You will have to tell ssh which port you want to use with -p 2222.

ssh -p 2222 username@lab-router

The command above should take you to lab-server1 even though you specify on the command line lab-router. Remember the port forwarding for port 22. If you are not convinced, checkout the name of the host you end up on with hostname.

To make sure the rules survive a reboot save them to a file as we did above and they will be loaded when the network is brought up. Or you can run the script manually to put them back after you log in.

sudo iptables-save > /etc/iptables/rules.v4

You can add additional rules for say, 80 & 443, (HTTP & HTTPS) and they could be forwarded a lab-webserver. How about for a lab-mailserver, use ports 25,110,143,587,993,995.

Save everything and make a backup of your lab-router and then you can start trying out other usecases.

Bandwidth Monitoring Tool

I like to install this little command line tool to see the two NIC’s working away on the router.

sudo apt-get install bwm-ng

Then just run it with bwm-ng on the command line. Press ‘h’ once it is running, you will work it out. 🙂

Other simple tools you might like to look at in no particular order are: iptraf, iotop, ifstat, iftop, netstat.

Now you have a very basic lab-router setup and ready for you to start playing around with networks and remote access.

Useful Links

Netfilter, Using iptables
Netfilter, Source NAT
Ubuntu iptables Howto

Script to Set Basic Firewall Configuration

This script, fw.rules_common, has examples for other iptable rules you might like you use as a basis for your own. Download it and then rename to remove the .txt. Also make it executable. Take a good look at it before running it. There are be a number of variables at the top you will need to change to suit your environment.

To have a script to turn on all dnsmasq DNS and DHCP functions you could use the following. Save them to a suitable file name in /root/bin and then they will be ready for next time.

#!/bin/bash

# ==========================
# Turn on dhcp
# ==========================
echo "Turn on the DHCP server in dnsmasq"
sed -e 's/^#\(dhcp-range=10.1.200.50.*\)/\1/' -i /etc/dnsmasq.conf
service dnsmasq restart

To turn off dnsmasq DNS functions you could use:

#!/bin/bash

# ==========================
# Turn on dhcp
# ==========================
echo "Turn on the DHCP server in dnsmasq"
sed -e 's/^\(dhcp-range=10.1.200.50.*\)/#\1/' -i /etc/dnsmasq.conf
service dnsmasq restart

To stop dnsmasq simply stop the service

sudo service dnsmasq stop

Beware Here Be Musings

I enjoy using Linux, I have to be paid to use Microsoft Windows.