Run at Startup (without rc.local)
A question that comes up quite frequently in Pi Wars circles is “how do I make
my robot program run at start-up?” Someone asked me that directly at the Pi Wars
Miniconference - and I gave the same answer I always do: “Use systemd
”.
In itself, that not a very useful answer, because systemd
is a large and
varied project with lots of tools and functionality. It’s just a bit too
complex to describe over voice without context. However, with the help of
examples, it really is quite straightforward to make a simple systemd
service.
There’s also all kinds of bells and whistles you can try out too if you’re
feeling adventurous!
If you just want a quick fix, then either jump to the “tl; dr” section below, or head over to the page on raspberrypi.org. But if you want to know more about what you’re doing, read on!
Not rc.local
Many of the tutorials you will see online mention rc.local
as a way to run
things at boot. This is a practice which I really wish would end!
rc.local
is an archaic way to run one-shot things at boot-time. Even before
systemd
was around, there were much better ways to run “services” than
rc.local
.
From my perspective, there’s several major problems with rc.local (all relevant in the context of a Pi Wars robot):
- If something you run in
rc.local
gets stuck, then your system will hang during boot - By default, things run from
rc.local
can’t output anywhere; so you can’t see any errors or other output - There’s no “lifetime management”.
rc.local
runs once and once only.- If you want to run it again, you have to reboot.
- If you want to stop something which was started by
rc.local
, you need to find its process manually andkill
it.
systemd
Nowadays, for better or for worse (better, in my opinion), your Linux-based
system will be running systemd
unless you’ve explicitly avoided it.
systemd
is the “system daemon”. It’s not a single thing - it’s a whole
collection of programs which handle services, networking, logging, containers
and other things. It got a lot of bad press from people who preferred the “old”
way of doing things, but that’s an argument you can go and find somewhere else
on the internet.
For the purposes of starting programs at boot, we only care about one of its concepts: services.
systemd
service
A
systemd.service
is simply a program or operation which should be run under the control of
systemd
. Why do it under systemd
?
- It can track dependencies, and make sure that e.g. your program only runs once network or GUI is available
- It can run your program as any user you like, with any environment you like
- It can restart your program if it crashes or exists
- If you need multiple programs to run,
systemd
can handle the ordering and restart them all if one of them fails. - It can capture the output of your program into its logging system
- It’s easy to stop, start, and restart your program
All a service is to systemd
is a .service
file, which should be placed in
/etc/systemd/system
1. A very basic systemd
service would look like this:
[Unit]
Description=My first systemd service
[Service]
ExecStart=/usr/bin/echo "Hello systemd world!"
The easiest “gotcha!” in writing systemd
services is that the environment which
they run in is by default quite “empty” - for example $PATH
isn’t set, which
means you have to use the full path to your command.
If you save that as /etc/systemd/system/hello.service
then you can start it
with the systemctl
command, which is how we control systemd
:
$ sudo systemctl start hello.service
systemd
will run it, then you can check the output with journalctl
, which
stores all the system’s log messages (-e
jumps to the end of the log):
$ journalctl -e
[lots of log messages]
Nov 24 17:02:15 eva systemd[919]: Started My first systemd service.
Nov 24 17:02:15 eva echo[23731]: Hello systemd world!
Nov 24 17:02:15 eva systemd[919]: hello.service: Succeeded.
If you wanted to run this service on boot, then we need to add a new section
telling systemd
how to “Install” this service:
[Unit]
Description=My first systemd service
[Service]
ExecStart=/usr/bin/echo "Hello systemd world!"
[Install]
WantedBy=multi-user.target
If you modify a service file, you need to tell systemd
to reload it (normally
it loads them all during system boot, and so won’t notice your changes until you
reboot) with systemctl daemon-reload
; then, we can enable
our service to
make it run at boot:
$ sudo systemctl daemon-reload
$ sudo systemctl enable hello.service
Created symlink /etc/systemd/system/multi-user.target.wants/hello.service → /etc/systemd/system/hello.service.
The multi-user.target
tells systemd
that our service should be considered
part of the system when it’s running in “multi-user” mode. This name comes from
the old days of computing, but it basically means “normal operation”.
Now, if you reboot, you can look in the journal for the service getting run.
(-b0
means “only show messages from this boot, -b-1
would be from the
previous boot):
$ journalctl -b0
... lots of log messages
Nov 24 17:25:01 eva systemd[919]: Started My first systemd service.
Nov 24 17:25:01 eva echo[23731]: Hello systemd world!
Nov 24 17:25:01 eva systemd[919]: hello.service: Succeeded.
... lots of log messages
We can actually do better, and only show the messages from our service with
--unit=
:
$ journalctl -b0 --unit=hello.service
Nov 24 17:25:01 eva systemd[919]: Started My first systemd service.
Nov 24 17:25:01 eva echo[23731]: Hello systemd world!
Nov 24 17:25:01 eva systemd[919]: hello.service: Succeeded.
Stopping and restarting
The simple service we just wrote runs a single command which exits immediately.
However, the program for running a robot should continue running forever.
systemd
has classifications for these different Type
s of services.
For one whose program keeps running, we should use Type=exec
. systemd
will
consider an exec
service running once it has successfully started the program,
and ended if the program exits.
To test this out, let’s create a very simple python
program which just prints
a number every second:
#!/usr/bin/python3
import time
i = 0
while True:
print(str(i))
i +=1
time.sleep(1.0)
And a systemd
service to run it. There’s one little detail here which is
important: the -u
option passed to python3
makes sure that the output is
unbuffered, and so shows up in the journal immediately. As before, full paths
are needed:
[Unit]
Description=Python counter
[Service]
Type=exec
ExecStart=/usr/bin/python3 -u /home/pi/counter.py
Now, if we start the service we will see its output in the journal:
$ sudo systemctl start counter.service
$ sleep 5
$ sudo journalctl -b0 --unit=counter.service
Nov 24 22:35:43 eva systemd[919]: Starting Python counter...
Nov 24 22:35:43 eva systemd[919]: Started Python counter.
Nov 24 22:35:44 eva python3[18030]: 0
Nov 24 22:35:45 eva python3[18030]: 1
Nov 24 22:35:46 eva python3[18030]: 2
Nov 24 22:35:47 eva python3[18030]: 3
Nov 24 22:35:48 eva python3[18030]: 4
We can stop the program with sudo systemctl stop counter.service
and restart
(stop
then start
again) it with sudo systemctl restart counter.service
- no
more looking for PIDs to kill!
$ sudo systemctl restart counter.service
$ sleep 3
$ sudo systemctl stop counter.service
$ sudo journalctl -b0 --unit=counter.service
tl;dr
The too-long, didn’t read summary is:
-
Create a
/etc/systemd/system/robot.service
file, containing:[Unit] Description=Robot [Service] Type=exec ExecStart=/usr/bin/python3 -u /home/pi/my-robot-script.py [Install] WantedBy=multi-user.target
-
Make it run at boot with
sudo systemctl enable robot.service
-
Start/stop/restart it with
sudo systemctl {stop,start,restart} robot.service
-
Prevent it from running at boot with
sudo systemctl disable robot.service
-
View the output with
sudo journalctl --unit=robot.service
Other useful options
systemd
has literally hundreds of options for controlling how services are
run, all documented in great detail in the various manual pages.
Here are some of the options which I think are most useful for something like a Pi Wars robot:
Automatic restarting
If your robot code crashes (or even if it exits normally), you might want it to start again automatically without having to reboot.
systemd
has some useful settings for this, which should go in the [Service]
section:
[Service]
Restart=always
RestartSec=5s
Restart=always
: Whenever the program exits, start it againRestart=on-failure
: If the program exits with a non-zero exit code, then restart it (e.g.exit(1)
-> restart,exit(0)
-> don’t restart). This will also restart if your program crashes.RestartSec=5s
: You can add a delay beforesystemd
restarts the program. This can be helpful to stop an infinite very fast loop if your program crashes every time it starts!
Set-up or clean-up scripts
As well as the ExecStart=
command for running the program, you can add
ExecStartPre=
and ExecStartPost=
which can be used to execute commands
before or after the main program is started.
This might do something like delete a file or create a directory before starting the main script.
However, this cannot be used to run other programs which are part of the
robot. Any programs run from ExecStartPre
/ExecStartPost
will be terminated by
systemd
before the next service is run.
If you have multiple programs which form part of your robot, see the next section on dependencies.
Dependencies between services
On Mini Mouse, I have two programs making up the robot:
yapidh
is a C program driving the stepper motorsbot
is the main robot program
bot
can’t work properly unless yapidh
is running first, systemd
lets you
describe this kind of dependency in the unit files.
There’s several different ways to describe dependencies, each being subtly different. If you’re planning to make use of any of them, I strongly recommend you read the unit man page.
- You can say that one service should be started
Before=
orAfter=
another service
systemd
uses these to decide which order to start services in, but only if they were already being started!
- You can say that one service
Requires=
or isRequiredBy=
another one
- If a service
Requires=
another, then when the service is started, the one(s) specified inRequires=
will be started too. - If any of the services in
Requires=
is explicitlystop
ped, then this one will be stopped, too. RequiredBy=
is the same, but in the opposite direction.
- You can say a service
Wants=
or isWantedBy=
another one
Wants=
is likeRequires=
but weaker. If one of the “wanted” services fails, it won’t cause theWant
-er to fail.
PartOf=
will make this service restart when the one it’sPartOf
is restarted.
For Mini Mouse, I make both components have Restart=always
, and add a
Requires=
and PartOf=
dependency from bot
to yapidh
. That gives the
desired behaviour:
yapidh
is always started whenbot
is started- If either program crashes or exits, it will be restarted
- If
yapidh
is restarted, thenbot
is restarted first
Also note that yapidh
doesn’t need to be enable
-d, because it gets started
by bot
’s Requires=yapidh.service
.
Here are the .service
files:
bot.service
[Unit]
Description=Bot service
Requires=yapidh.service
After=yapidh.service
PartOf=yapidh.service
[Service]
Type=exec
ExecStart=/home/pi/go/src/github.com/usedbytes/mini_mouse/mini_mouse-bot/bot
Restart=always
RestartSec=10s
[Install]
WantedBy=multi-user.target
yapidh.service
[Unit]
Description=Yapidh service
[Service]
Type=exec
Nice=-20
CPUSchedulingPolicy=rr
CPUSchedulingPriority=99
ExecStart=/home/pi/Programming/yapidh/yapidh
If I force yapidh
to crash, we can see the following sequence in the logs
after running sudo systemctl start bot
:
Nov 24 22:19:46 eva systemd[919]: Starting Yapidh service...
Nov 24 22:19:46 eva systemd[919]: Started Yapidh service.
Nov 24 22:19:46 eva systemd[919]: Starting Bot service...
Nov 24 22:19:46 eva systemd[919]: Started Bot service.
Nov 24 22:19:49 eva systemd[919]: yapidh.service: Main process exited, code=exited, status=1/FAILURE
Nov 24 22:19:49 eva systemd[919]: yapidh.service: Failed with result 'exit-code'.
Nov 24 22:19:59 eva systemd[919]: yapidh.service: Service RestartSec=10s expired, scheduling restart.
Nov 24 22:19:59 eva systemd[919]: yapidh.service: Scheduled restart job, restart counter is at 1.
Nov 24 22:19:59 eva systemd[919]: Stopping Bot service...
Nov 24 22:19:59 eva systemd[919]: bot.service: Main process exited, code=killed, status=15/TERM
Nov 24 22:19:59 eva systemd[919]: bot.service: Succeeded.
Nov 24 22:19:59 eva systemd[919]: Stopped Bot service.
Nov 24 22:19:59 eva systemd[919]: Stopped Yapidh service.
Nov 24 22:19:59 eva systemd[919]: Starting Yapidh service...
Nov 24 22:19:59 eva systemd[919]: Started Yapidh service.
Nov 24 22:19:59 eva systemd[919]: Starting Bot service...
Nov 24 22:19:59 eva systemd[919]: Started Bot service.
We see that bot
forces yapidh
to start. Then yapidh
fails, and 10 seconds
later is restarted - which triggers a restart of bot
too.
Further Reading
There’s plenty more that you can learn about systemd - and the best way (although a little dry) is to read the documentation:
- systemd.unit
- systemd.service
- systemd.exec
- systemd.timer
- systemd.mount
- systemd.automount
- And all the others…
-
systemd
stores services in a number of places./lib/systemd
is usually used by the distribution, and/etc/systemd
for things added by the end user or system administrator (you!) ↩︎