Monitoring (Server)

Tip

This page covers all aspects regarding monitoring on the server level. For informations about the monitoring of individual websites, see Monitoring (Website).

Availability (external)

We closely monitor all aspects of your server. According to your service level, our on call organisation will take appropriate actions if required.

Availability (internal)

Monit, nginx and PHP FPM (if installed) status pages are available at http://localhost:2813/:

  • http://localhost:2813/monit/: Monit service manager displaying status of all locally monitored processes

  • http://localhost:2813/nginx/: nginx stub status output

  • http://localhost:2813/fpm-<poolname>/: PHP FPM per pool status page

Tip

this status vhost is running on localhost only. Expose port 2813 through SSH to access locally: ssh <hostname> -L 2813:localhost:2813

Tip

The monit status can also be checked in the terminal with monit-status as devop user (see Generic Admin User).

Reboot

A automatic reboot is initiated to solve certain high usage scenarios:

  • 5 minute average load higher than CPU count * 10 for 5 minutes

  • memory usage higher than 95% for 5 minutes

Tip

always make sure that any required services will be up and running automatically

Utilization

Netdata

Netdata is a real-time, interactive web dashboard collecting data every second. Metrics are saved in memory and kept for 1 hour only. You can reach its webinterface at http://localhost:19999.

To connect from your local computer, either forward port 19999 through SSH (ssh <hostname> -L 19999:localhost:19999), or add a reverse proxy website forwarding requests to http://localhost:19999

Warning

when using the reverse proxy method, make sure to enable HTTPS and password protection

collectd

System statistics are collected every 10 seconds by collectd and written to RRD files in /var/lib/collectd. For performance reasons, we don’t create graphs by default, therefore you have to download and render them with a tool of your choice by yourself. Please select a rendering-tool from list of frontends within the collectd wiki. We recommend collectd-web.

For Debian-based Linux Distributions

Installation:

sudo apt-get install librrds-perl libjson-perl libhtml-parser-perl
git clone https://github.com/httpdss/collectd-web.git
echo 'datadir: "/tmp/rrd"' | sudo tee /etc/collectd/collection.conf

Fetch data and render graphs:

rsync -avz <server>:/var/lib/collectd/rrd/ /tmp/rrd/
cd /path/to/collectd-web
python runserver.py

Then open collectd-web at http://127.0.0.1:8888/.

collectd-web with Docker

A Docker image is also available.

rsync -avz <server>:/var/lib/collectd/rrd/ /tmp/rrd/
docker run -p 8888:80 --volume /tmp:/tmp -it registry.gitlab.com/opsone_ch/docker-collectd-web:latest

Then open collectd-web at http://127.0.0.1:8888/.