diff options
Diffstat (limited to 'make_to_act_only_on_modified_files_from_prerequisites.html')
| -rw-r--r-- | make_to_act_only_on_modified_files_from_prerequisites.html | 130 |
1 files changed, 130 insertions, 0 deletions
diff --git a/make_to_act_only_on_modified_files_from_prerequisites.html b/make_to_act_only_on_modified_files_from_prerequisites.html new file mode 100644 index 0000000..657fc70 --- /dev/null +++ b/make_to_act_only_on_modified_files_from_prerequisites.html @@ -0,0 +1,130 @@ +<!doctype html> +<html lang="en"> +<meta charset="utf-8"> +<meta name="viewport" content="width=device-width, initial-scale=1"> +<meta name="author" content="aki"> +<meta name="tags" content="make, makefile, upload, update"> +<meta name="published-on" content="2024-01-23T21:26:54+01:00"> +<link rel="icon" type="image/png" href="favicon.png"> +<link rel="stylesheet" href="style.css"> + +<title>Make to Act Only on Modified Files From Prerequisites</title> + +<header> +<nav><a href="https://ignore.pl">ignore.pl</a></nav> +<time>23 January 2024</time> +<h1>Make to Act Only on Modified Files From Prerequisites</h1> +</header> + +<article> +<p>Make is very clear on how it tracks changes. There are output files and input files. If input is newer than the +output, things need building. It may not be <a href="https://gittup.org/tup/make_vs_tup.html">fast</a> and it does not +lack layers upon layers (e.g., <a href="https://www.gnu.org/software/make/manual/html_node/Archive-Members.html">Archive +Members</a>), but its essence is rather clear. With: +<pre> +%.b: %.a + cat $< >$@ + +hello.b: +</pre> +<p>We can read that whenever <i>hello.a</i> is modified a new <i>hello.b</i> will be generated from its content. +<p>Today let's talk about situation in which we have multiple prerequisite files. A sample rule that permits to +concatenate multiple <i>.a</i> files can be written in a makefile as follows: +<pre> +%.b: %.a + cat $^ >$@ + +hello.b: hi.a hey.a +</pre> +<p>We can already make two significant observations. The first point of interest is common and usually, I hope, +consciously ignored. Usually described as "an implied dependency on <i>hello.a</i>". To be precise, what is implied here +is not the dependency itself, but the entire recipe - name of the output and one of the inputs. We simply add two more +dependencies on top of something that is not really visible but matches. If we were to change the name of the output to +<i>greetings.b</i>, nothing would generate. +<p>Second thing to note here is that the output does not allow updating. On each run it must regenerate the entire +<i>hello.b</i>. +<p>To tweak the first behaviour we can remove prerequisites from the rule: +<pre> +%.b: + cat $^ >$@ + +greetings.b: hello.a hi.a hey.a +</pre> +<p>This changes rule deduction behaviour a bit since there is no longer a visible input file extension that we can use +for deduction. The rule is chosen solely with the output pattern. Of course, we also need to specify the previously +implied input file that matched output name based on the previous rule. +<p>In order to tweak the rule to allow partial regeneration (or simply: updating), we must change context and use +different <a href="https://www.gnu.org/software/make/manual/html_node/Automatic-Variables.html">automatic variable</a>: +<code>$?</code>. For now, let's try with just the new variable. The new makefile will look like this: +<pre> +%.b: + cat $? >$@ + +greetings.b: hello.a hi.a hey.a +</pre> +<p>Context problem is visible only after we perform some subsequent builds: +<pre> +$ make +cat hello.a hi.a hey.a >hello.b +$ cat hello.b +hello +hi +hey +$ touch hi.a +$ make +cat hi.a >hello.b +$ cat hello.b +hi +</pre> +<p>First run is just fine, like predicted the problem shows after it and thanks to <b>make</b> printing commands it is +also nicely visible. Using redirection that appends to the end of a file would simply write another <code>hi</code> at +the end. The problem is that we are operating on plain text files and simply append to them. The same situation would +occur with other similar formats (e.g., tarballs). +<p>The problem is the object does not support updating that's equivalent to the first creation. That's not always the +case: +<pre> +%.b: + ar r $@ $? + +greetings.b: hello.a hi.a hey.a +</pre> +<p><b>ar</b> supports such operation. Consequent runs will replace content of the <i>greetings.b</i> archive as files +are modified. If a new file is added to the dependencies, it will only append this new files assuming nothing else was +modified.</p> +<img src="make_to_act_only_on_modified_files_from_prerequisites-1.png" alt="todo list drawing"> +<p>What if we do not have an output file? Recently I was playing around with a new idea and I used NodeMCU as a platform +to implement it on. It's small. Its filesystem and set of services is very simple, so usual things like <b>rsync</b> +were not an option. To reduce the time spent on uploading files I wanted to send only ones that were updated, but they +were sent to another device, so there was no target file to compare the dates with. +<p>Usual approach to similar problems is usually just ignoring it and making a phony target instead. Sometimes even with +a file present we choose to do so, for example installation targets often completely omit output specification. +<p>Well, if we don't have a file, we simply need to create one: +<pre> +upload: .marker + +.marker: a.lua b.lua c.lua + nodemcu-uploader -p $(PORT) upload $? + @touch $@ + +.PHONY: upload +</pre> +<p>Here, a special file <i>.marker</i> is silently created or updated on each upload. Action is performed silently +thanks to <code>@</code> at the beginning of the command. The idea is to avoid clutter in logs. The additional +<i>upload</i> target is there to provide a more human recognizable name. Marker filename is not put into a variable to +reduce the complexity of the example. +<pre> +$ make +nodemcu-uploader -p /dev/ttyUSB0 upload a.lua b.lua c.lua +$ touch a.lua +nodecmu-uploader -p /dev/ttyUSB0 upload a.lua +$ make clean upload +rm -f .marker +nodemcu-uploader -p /dev/ttyUSB0 upload a.lua b.lua c.lua +</pre> +<p>Of course, state of the files in the device is not tracked and different devices are not recognized. Still, this +method has a rather good balance between the overall complexity and how often each branch of its logic is used. +<p>This sounds like a good opportunity to use an archive member target and pretend we put files into the marker file, +but it will not work as expected, because it will attempt to rebuild (and thus upload) each individual target every +single time. Shame. +</article> +<script src="https://stats.ignore.pl/track.js"></script> |