1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
|
<!doctype html>
<html lang="en">
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="author" content="aki">
<meta name="tags" content="make, makefile, upload, update">
<meta name="published-on" content="2024-01-23T21:26:54+01:00">
<link rel="icon" type="image/png" href="favicon.png">
<link rel="stylesheet" href="style.css">
<title>Make to Act Only on Modified Files From Prerequisites</title>
<header>
<nav><a href="https://ignore.pl">ignore.pl</a></nav>
<time>23 January 2024</time>
<h1>Make to Act Only on Modified Files From Prerequisites</h1>
</header>
<article>
<p>Make is very clear on how it tracks changes. There are output files and input files. If input is newer than the
output, things need building. It may not be <a href="https://gittup.org/tup/make_vs_tup.html">fast</a> and it does not
lack layers upon layers (e.g., <a href="https://www.gnu.org/software/make/manual/html_node/Archive-Members.html">Archive
Members</a>), but its essence is rather clear. With:
<pre>
%.b: %.a
cat $< >$@
hello.b:
</pre>
<p>We can read that whenever <i>hello.a</i> is modified a new <i>hello.b</i> will be generated from its content.
<p>Today let's talk about situation in which we have multiple prerequisite files. A sample rule that permits to
concatenate multiple <i>.a</i> files can be written in a makefile as follows:
<pre>
%.b: %.a
cat $^ >$@
hello.b: hi.a hey.a
</pre>
<p>We can already make two significant observations. The first point of interest is common and usually, I hope,
consciously ignored. Usually described as "an implied dependency on <i>hello.a</i>". To be precise, what is implied here
is not the dependency itself, but the entire recipe - name of the output and one of the inputs. We simply add two more
dependencies on top of something that is not really visible but matches. If we were to change the name of the output to
<i>greetings.b</i>, nothing would generate.
<p>Second thing to note here is that the output does not allow updating. On each run it must regenerate the entire
<i>hello.b</i>.
<p>To tweak the first behaviour we can remove prerequisites from the rule:
<pre>
%.b:
cat $^ >$@
greetings.b: hello.a hi.a hey.a
</pre>
<p>This changes rule deduction behaviour a bit since there is no longer a visible input file extension that we can use
for deduction. The rule is chosen solely with the output pattern. Of course, we also need to specify the previously
implied input file that matched output name based on the previous rule.
<p>In order to tweak the rule to allow partial regeneration (or simply: updating), we must change context and use
different <a href="https://www.gnu.org/software/make/manual/html_node/Automatic-Variables.html">automatic variable</a>:
<code>$?</code>. For now, let's try with just the new variable. The new makefile will look like this:
<pre>
%.b:
cat $? >$@
greetings.b: hello.a hi.a hey.a
</pre>
<p>Context problem is visible only after we perform some subsequent builds:
<pre>
$ make
cat hello.a hi.a hey.a >hello.b
$ cat hello.b
hello
hi
hey
$ touch hi.a
$ make
cat hi.a >hello.b
$ cat hello.b
hi
</pre>
<p>First run is just fine, like predicted the problem shows after it and thanks to <b>make</b> printing commands it is
also nicely visible. Using redirection that appends to the end of a file would simply write another <code>hi</code> at
the end. The problem is that we are operating on plain text files and simply append to them. The same situation would
occur with other similar formats (e.g., tarballs).
<p>The problem is the object does not support updating that's equivalent to the first creation. That's not always the
case:
<pre>
%.b:
ar r $@ $?
greetings.b: hello.a hi.a hey.a
</pre>
<p><b>ar</b> supports such operation. Consequent runs will replace content of the <i>greetings.b</i> archive as files
are modified. If a new file is added to the dependencies, it will only append this new files assuming nothing else was
modified.</p>
<img src="make_to_act_only_on_modified_files_from_prerequisites-1.png" alt="todo list drawing">
<p>What if we do not have an output file? Recently I was playing around with a new idea and I used NodeMCU as a platform
to implement it on. It's small. Its filesystem and set of services is very simple, so usual things like <b>rsync</b>
were not an option. To reduce the time spent on uploading files I wanted to send only ones that were updated, but they
were sent to another device, so there was no target file to compare the dates with.
<p>Usual approach to similar problems is usually just ignoring it and making a phony target instead. Sometimes even with
a file present we choose to do so, for example installation targets often completely omit output specification.
<p>Well, if we don't have a file, we simply need to create one:
<pre>
upload: .marker
.marker: a.lua b.lua c.lua
nodemcu-uploader -p $(PORT) upload $?
@touch $@
.PHONY: upload
</pre>
<p>Here, a special file <i>.marker</i> is silently created or updated on each upload. Action is performed silently
thanks to <code>@</code> at the beginning of the command. The idea is to avoid clutter in logs. The additional
<i>upload</i> target is there to provide a more human recognizable name. Marker filename is not put into a variable to
reduce the complexity of the example.
<pre>
$ make
nodemcu-uploader -p /dev/ttyUSB0 upload a.lua b.lua c.lua
$ touch a.lua
nodecmu-uploader -p /dev/ttyUSB0 upload a.lua
$ make clean upload
rm -f .marker
nodemcu-uploader -p /dev/ttyUSB0 upload a.lua b.lua c.lua
</pre>
<p>Of course, state of the files in the device is not tracked and different devices are not recognized. Still, this
method has a rather good balance between the overall complexity and how often each branch of its logic is used.
<p>This sounds like a good opportunity to use an archive member target and pretend we put files into the marker file,
but it will not work as expected, because it will attempt to rebuild (and thus upload) each individual target every
single time. Shame.
</article>
<script src="https://stats.ignore.pl/track.js"></script>
|
