If you want to see loads of stats for your web site try Installing AWstats on Ubuntu 18.04. You can get the number of visitors, Which OS or Browser they used.
Install
Installing AWstats on Ubuntu 18.04 LTS and some additional packages necessary for the geo and net IP location stats.
sudo apt install awstats libgeo-ip-perl libgeo-ipfree-perl
Configure AWstats
This post assumes your domain is example.com and the network is 192.168.0.0/24.
sudo cp /etc/awstats/awstats.conf /etc/awstats/awstats.example.com.conf
These are the options I changed, to get some results. There are loads of options to configure read through the file looking at the comments. You can then customize Awstats to your needs.
LogFormat=1 SiteDomain="example.com" HostAliases="localhost 127.0.0.1 example.com" AllowFullYearView=3 AllowAccessFromWebToFollowingIPAddresses="127.0.0.1 192.168.0.0-192.168.0.255" LoadPlugin="tooltips" LoadPlugin="graphgooglechartapi" LoadPlugin="geoipfree"
Also, we need to update the following in /etc/awstats/awstats.conf.local. This .local file and the domain specific file will not get over written when awstats gets updated
SiteDomain="example.com" HostAliases="localhost 127.0.0.1 example.com"
If you are looking to add AWstats monitoring to your Postfix mail server take a look at this it has all the necessary information regarding the settings for the awstats.domain.conf file. Awstats documentation for setting it up for Postfix.
Remove Any Existing Data
By the time you get here you could find that the crontab job has already run. This would have failed and created an empty database. It fails because www-data does not have access to /var/log/apache directory or the access.log within. (I would have thought the Ubuntu maintainers would have noticed that). You will not be able to add any initial data as it will have been marked as processed. We therefore, need to stop the crontab job, delete and existing data, run the update and put the crontab job back again. Which all sounds a bit complex but takes a few commands 🙂
The first two steps are move the cronfile out of the way. We will need it later so do not delete it. This will stop the updates from running and the emails saying the the update failed. The ndelete the existing data base.
sudo mv /etc/cron.d/awstats /root
sudo rm /var/lib/awstats/*
Update Permissions
The permissions are wrong on the Apache2 access.log file for awstats to collect data. They need to be accessible by the user that runs the cronjob, see below, and the user that runs the web service. On Ubuntu this will be www-data. We need to change the group on both the access.log file and the directory it is in.
sudo chgrp www-data /var/log/apache2 /var/log/apache2/*log /var/log/apache2/access.log
sudo chmod 755 /var/log/apache2
sudo chmod 644 /var/log/apache2/*
Awstats can now access the log file to gather its data. This will only work until the log file is rotated. 🙂 To fix that we need to amend the config for the Apache2 log files.
sudo nano /etc/logrotate.d/apache2
Find the line that starts with the key word create and amend the group permission as shown below
notifempty create 640 root www-data sharedscripts
Now when the log file is rotated and then recreate it will be readable by www-data.
Automatic Statistics Updates
To maintain and gather the stats on a frequent and regular basis you will need a cron job to run the awstats update script. If you remember we moved it out of the way. Time to move it back again now that the permissions are sorted out.
sudo nano /root/awstats
The cron job below will run every 10 minutes, adjust the time to your needs. if you have a busy site and want up to date stats configure it so it runs more often.
MAILTO=root */10 * * * * www-data [ -x /usr/share/awstats/tools/update.sh ] && /usr/share/awstats/tools/update.sh # Generate static reports: 10 03 * * * www-data [ -x /usr/share/awstats/tools/buildstatic.sh ] && /usr/share/awstats/tools/buildstatic.sh
sudo mv /root/awstats /etc/cron.d
Time to grab a quick coffee while we wait for the up to 10 minutes to pass and the awstats job to run. 🙂
You can check that the database has been created /updated by looking in
sudo ls -l /var/lib/awstats/
-rw-rw-r-- 1 www-data www-data 8953 Sep 20 11:40 awstats092018.example.com.txt -rw-rw-r-- 1 www-data www-data 7651 Sep 20 11:40 awstats092018.txt -rw-rw-r-- 1 www-data www-data 60 Sep 20 11:36 dnscachelastupdate.example.com.hash -rw-rw-r-- 1 www-data www-data 60 Sep 20 11:40 dnscachelastupdate.hash
The date/time stamp on the top two files will change each time the database is updated.
Configure Apache2
To access awstats from a browser, this assumes you are using Apache2. update the default virtual host with the following, to tell Apache where to find the awstats start script.
sudo nano /etc/apache2/sites-available/000-default.conf
Add the following lines to the end of the file before the closing </VirtualHost> tag.
Note: All the posts I’ve seen fail to use the ‘+’ in front of the ExecCGI option which will break your Apache config and it will not restart. The RequireAny lines stop access to Awstats data from outside of the LAN. If you want to restrict it further see later on.
Alias /awstatsclasses "/usr/share/awstats/lib/"
Alias /awstats-icon/ "/usr/share/awstats/icon/"
Alias /awstatscss "/usr/share/doc/awstats/examples/css"
ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/
ScriptAlias /awstats/ /usr/lib/cgi-bin/
Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch
Require ip 192.168.0.0/24
Require ip 127.0.0.1
Require ip 192.168.0.0/24
Require ip 127.0.0.1
Make sure that the CGI module is enabled and restart your Apache server.
sudo a2enmod cgi
sudo systemctl restart apache2
You should now be able to browse to https://example.com/cgi-bin/awstats.pl and get the main awstats page.
Stop Unauthorized Users
As the above setup stands anyone on your LAN, can now see your web server stats. Probably not a good idea or something you want. We can let Awstats protect itself with the rather well named parameter AllowAccessFromWebToFollowingIPAddresses or we can add a restriction forcing the user to enter a password.
To allow awstats to protect itself edit the config file and add a list of IP addresses or ranges see the comment above the option of the syntax.
sudo nano /etc/awstats/awstats.example.com.conf
The fist is the same as we did in the apache setup. The second line only allows access from the webserver localhost and 192.168.0.66. You get the idea.
AllowAccessFromWebToFollowingIPAddresses="127.0.0.1 192.168.0.0-192.168.0.255" AllowAccessFromWebToFollowingIPAddresses="127.0.0.1 192.168.0.66"
Logging Data From Old logs
If you want to log the data in old log files that have been rotated out of the way you can do that using a command line similar to the below. This may take some time depending on the number and size of your log files and the files need to be run in chronological order. If you ran the manual update above you will need to delete the database. Anyway, it was worth while as that is generally very quick and you can see some data being displayed. Also, it is worth moving the cron file for awstats to stop that from running.
The database files are where ever you pointed the option DirData to, it defaults to /var/lib/awstats. The first command moves cronfile out of the way disabling it.
sudo mv /etc/cron.d/awstats /root
sudo rm /var/lib/awstats/*
Now we can run the log import or update from the command line. My Apache logs are compressed and called access.log, access.log.1 or access.log.n.gz. Giving the following command line to import them all. Depending on the size and number of files this may put a heavy load on the server, so I also run the command through nice and it takes long enough to get a coffee.
Note: the command should all be on one line.
sudo nice /usr/lib/cgi-bin/awstats.pl -update -config="example.com" -LogFile="/usr/share/awstats/tools/logresolvemerge.pl /var/log/apache2/access.log* |"
We need to reset the permissions on the awstats database. If we do not then the next time the update script runs it will fail.
sudo chown -R www-data:www-data /var/lib/awstats
sudo chmod 755 /var/log/apache2 /var/lib/awstats
sudo chmod 644 /var/log/apache2/* /var/lib/awstats/*
Remember to put the cron file back again.
sudo mv /root/awstats /etc/cron.d
Check the web page to see the extra history has been added. So that is the end of the post Installing AWstats on Ubuntu 18.04. I hope it works for you. Happy gathering stats. 🙂