1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
|
<!doctype html>
<html lang="en">
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="author" content="aki">
<meta name="tags" content="">
<meta name="published-on" content="2024-01-26T00:36:45+01:00">
<link rel="icon" type="image/png" href="favicon.png">
<link rel="stylesheet" href="style.css">
<title>My Server Keeps Turning Off!</title>
<header>
<nav><a href="https://ignore.pl">ignore.pl</a></nav>
<time>26 January 2024</time>
<h1>My Server Keeps Turning Off!</h1>
</header>
<article>
<p>I have a number of personal computers, servers, laptops, microcomputers, PCs and other devices that I work with. Most
of the services are published to the Internet via server that hosts this blog. Quite some time ago, I dug out some
parts, put them together and had a build server running on a tabletop. Recently I moved it into a random full tower case
that I found on top of a wardrobe. To do it, I had to turn it off. And then it hit me.</p>
<img src="my_server_keeps_turning_off-1.jpg" alt="photo of a piece of junk">
<p>Why is this piece of junk running 24 hours a day and 7 days a week?
<p>I'm no longer running builds with it. I plan to in near future but not yet. The only service that I run on it is
<a href="https://git.ignore.pl/mirror/">a git mirror</a>. Once a day, it pulls all of my repositories from all of the
hosting services that I use. Thanks to its distributed nature git takes care of "backups" even with a decent redundancy.
Still, there are cases in which you may lose it. For example when it isn't used that often by anyone in your near
circle. I don't have friends so, here I am.
<p>The server runs 24/7 and the significant service is active for how much? Let's see...
<pre>
Jan 23 00:00:15 mayohiga systemd[1]: Starting Git Repositories Mirror...
Jan 23 00:01:39 mayohiga systemd[1]: Finished Git Repositories Mirror.
Jan 24 23:10:33 mayohiga systemd[1]: Starting Git Repositories Mirror...
Jan 24 23:12:05 mayohiga systemd[1]: Finished Git Repositories Mirror.
Jan 25 01:18:09 mayohiga systemd[1]: Starting Git Repositories Mirror...
Jan 25 01:19:34 mayohiga systemd[1]: Finished Git Repositories Mirror.
</pre>
<p>A minute and a half per day.
<p>Well, that's a waste. To be honest it could be even shorter but I didn't bother introducing any parallelism and so it
pulls about 70 repositories sequentially.
<p>Right away I configured the server to react to <a href="https://en.wikipedia.org/wiki/Wake-on-LAN">Wake on LAN</a>
magic packets. Each odd or so day I manually ran a script that was waking up the server, running the mirror, and then
shutting everything down.
<p>Luckily no failures happened during this time.
<p>None at all.
<p>The internet-facing server never broke when running a regular apt upgrade at 4:00 AM on the 31st of December. It
didn't get stuck and crash on man-db update somehow after removing old initramfs image and before running fresh
mkinitcpio. And even if this actually happened, OVH's KVM console running emergency boot would obviously load right away
so that I wouldn't fall asleep while it was still loading at 5:00 AM.
<p>That didn't happen.
<p>Anyway. From time to time I was running a script that triggered the update. At some point I configured better timer
unit that started mirrors soon after boot, but I still had to wait for it to turn off the server manually. Of course, in
the long run using my usual workstation for running scheduled (or manual) wake ups wouldn't make sense because it uses
more power than the target server, it's on only when I'm doing something, and I'm not cron or a timer.
<p>It had to be something smaller. My first idea was to use a Raspberry Pi that would run all the time maybe providing
some additional services but in the end I chose something even smaller. An ESP8266 I had laying around.
<p>This thing at 5 V pulls at minimum around 9 mA in deep sleep (in which it spends most of the time) and about 126 mA
when using WiFi in a full extent (e.g., for scanning). In comparison, based on purely theoretical estimation, the server
was slightly above 2 kWh per day. Of course, ESP turns on the server, so it also consumes power. Still, it stays up for
a very limited period. Even assuming an hour of uptime, it's at least about 24 times lower.</p>
<img src="my_server_keeps_turning_off-2.png" alt="coal power plant">
<p>In a grand scheme of carbon dioxide emissions it doesn't really change much. The cut to payments isn't really that
significant either - maybe up to $250 USD in local currency per year. I wanted this for different reasons. As always, I
simply decided to act on a hopefully decent idea, for fun. Second reason is to learn and look at everything else I do
with the newly earned perspective (even if it's not yet matured). This especially affects how I will look at queueing
builds for this box.
<p>ESP runs <a href="https://git.ignore.pl/nodemcu-wakeup/">a simple set of Lua scripts</a> that periodically connects
to network, syncs clock with SNTP, check if it's time for waking the server up based on a simple configuration, and then
either goes back to deep sleep for at most 50 minutes or waits at most 5 minutes and broadcasts WoL packets. The sad
part is that server's motherboard does not have USB ports with constant voltage, so I connected the little guy somewhere
else. For a moment I was considering buying pure ESP8266 not as NodeMCU development board but it was time to wrap up the
project.
<p>After ESP was ready, I realized I can and should shutdown the server once all scheduled jobs are done with a systemd
unit. This realization came to me only after all work was done. Better late than never. I wrote
<a href="https://git.ignore.pl/systemd-ready-for-shutdown/">a target for shutdowns</a> and meanwhile I fixed
<a href="https://git.ignore.pl/systemd-dns-ready/">a race for DNS readiness</a>. I connected all of these units with
a couple of simple drop-in configs, for example <i>/etc/systemd/system/ready-for-shutdown.target.d/10-mirror.conf</i>:
<pre>
[Unit]
After=mirror.service
[Install]
WantedBy=mirror.service
</pre>
<p>Now, instead of worrying about not running the mirror manually, I patiently wait each evening to watch a
spectacle:</p>
<video controls>
<source src="my_server_keeps_turning_off-3.webm" type="video/webm">
<p>A short video showing boot of the server</p>
</video>
</article>
<script src="https://stats.ignore.pl/track.js"></script>
|
