There are some times when a Docker container doesn’t get restarted after it stops and that will cause some loss. Here you will learn to monitor and restart a Docker container automatically when it stops abruptly or crashes, by using the tool Monit.

Introduction

Monit is a small open source monitoring utility tool for Unix and Linux. Monit helps to monitor any processes like a web server, any database, any daemon on a server, or any other application. So Monit basically checks those services continuously in different intervals on a Linux or Unix server and makes sure they are running and if those services are not running then it will attempt to restart those services.

So today we will be going to learn how we can monitor Docker containers so that Monit can restart them when they crash and get stopped. In the case of Docker, we will be creating some scripts for containers that use the “docker top” command to monitor them. There are those phases here. 1^st phase to install Monit on the server (ubuntu) and 2^nd configuring Monit to restart the crashed containers.

Install Monit on Linux server

Step 1: Install Monit

$ sudo apt-get update

$ sudo apt-get install monit

Step 2: Start Monit Daemon

$ sudo monit

Step3. Configure Monit Web Service. (Default port 2812)

$ sudo nano /etc/monit/monitrc

add below lines to this file /etc/monit/monitrc:

set httpd port 2812

allow admin:monit # use user 'admin' with password 'monit'

1st line will set the port number 2812 for the Monit web server.

2nd line will set a user and password to access the Monit dashboard/web portal at 2812.

Step4. Reload the Monit to get the changes.

$ sudo monit -t 

$ sudo monit reload

sudo monit -t , this command will check the syntax.

Step5. Access Monit on the dashboard

Now we can access the Monit on the dashboard on port 2812 or you can check the Monit status also by this command.

$ sudo monit status

Setup Monit to restart Docker containers automatically

Step 6: Prepare a Monitor Script for Containers

So the main concept here is to monitor the container with the help of the “docker top” command. We have to create a /etc/monit/scripts/check_container_<container-name>.sh file for the container we want to restart by Monit which is going to be this command in it.

#! /bin/bash
docker top "<container-name>"
exit $?

Replace <container-name> in both file’s name and the in the file itself with the container you want to monitor. For example, in a container named nginx the respective file name would be etc/monit/scripts/check_container_nginx.sh, and the command in it would be like docker top “nginx”.

Step 7: Make The Script Executable

$ sudo chmod +x /etc/monit/scripts/check_container_<container-name>.sh

Step 7: Create a Monit File For Actions

Create a file named check_container_<container-name> in /etc/monit/conf.d which has below content:

CHECK PROGRAM <container-name> WITH PATH /etc/monit/scripts/check_container_<container-name>.sh
  START PROGRAM = "/usr/bin/docker start <container-name>"
  STOP PROGRAM = "/usr/bin/docker stop <container-name>"
  IF status != 0 FOR 3 CYCLES THEN RESTART
  IF 2 RESTARTS WITHIN 5 CYCLES THEN UNMONITOR

Then check that in /etc/monit/monitrc this line “include /etc/monit/conf.d/*” is included and not commented out, and then restart the Monit

$ sudo systemctl restart monit