In this post, I’m going to be talking about how to automatically control programs on Linux. The programs in question are likely to be essential to the operation of your website, so you need them to be available whenever your website is running. You may also want them to automatically start if you reset your server. To understand this problem a little better, I will discuss how programs are generally managed on Linux, before talking about some modern solutions, such as Supervisor.
On Unix-based operating systems, init is the first process started by the kernel during startup. It is the ancestor of all other processes, and is usually assigned the first process identifier (PID1). It is a daemon, which means that it is a program that runs in the background and isn’t associated with any particular human user. The main aim of init is to get the computer ready to use during boot up. The traditional design of init is synchronous and blocking. This means that init only starts one process at a time, and it only starts the next process once the current process has fully initialised. This is fine for a boot sequences, as low level programs may form a chain of dependencies, so one program cannot start until the previous program is ready.
With modern computers, however, there can be lots of independent processes to start, and the order in which they start doesn’t matter too much. Also, there may be more than one core available, so more that one process can be started at once. In this case, init can be slow and inefficient.
This is where asynchronous or event-driven process management comes in. This means that multiple processes can be triggered at the same time and the process manager doesn’t wait around for the program to finish loading. Instead the process reports back to the process manager once it has finished loading. With an event-driven architecture, processes are also created on demand, rather than waiting for the process manager to check on them. An example might be a USB drive that needs to be mounted. Instead of waiting for the process manager to check if there are any new USB drives, inserting a USB drive triggers an event, which requests the drive mounting process to start.
For Redhat Linux and related distributions, systemd was the default init system. On Ubuntu, Upstart was one of the first asynchronous, event-driven process managers. Upstart was used for many years, but slowly shortcomings were identified, and newer programs were introduced, such as the Runit suite written for Ruby. Runit had better log handling, provided a clear overview of process statuses and managed dependencies, i.e. when groups of processes need to be started in sequence.
So finally, we get to Supervisor, written for Python. Supervisor is a relatively simple, yet powerful process manager. It has lots of the advantages of Runit, such as a single place for overseeing lots of processes and grouping processes into hierarchies, for managing programs depending on each other. It also has a relatively simple syntax, and can allow users to control the state of key programs, without giving them full shell access. Supervisor also works well with processes that have master and worker processes, such Gunicorn. It even features a basic web interface for viewing and controlling individual processes.
As an addendum, modern versions of Ubuntu come with a program called systemd. This is an improved version of upstart, that provides a more robust way of making middleware available. If you are on a newer platform, you might want to take a look at it, as it is a native Ubuntu component, so you can install 1 less thing in your stack.