1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
|
<!doctype html>
<html lang="en">
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="author" content="aki">
<meta name="tags" content="lua, module, make, install, target, installation">
<meta name="published-on" content="2023-12-29T21:45:00+01:00">
<link rel="icon" type="image/png" href="favicon.png">
<link rel="stylesheet" href="style.css">
<style>
.comment { color: #2552d7; }
.explanation {
height: fit-content;
border-collapse: collapse;
}
.explanation td {
height: 100%;
padding: 0;
}
.explanation td > div {
height: 100%;
display: flex;
box-sizing: border-box;
flex-direction: column;
}
.explanation td > div > * { flex-grow: 1; }
.explanation td > *, .explanation td > div > * { margin-top: 0; }
.explanation td > *:last-child, .explanation td > div > *:last-child { margin-bottom: 0; }
.explanation td.description { padding: 0.6em 1em; }
.explanation pre { margin-right: inherit; }
</style>
<title>How to Write Install Targets for Lua Modules in Makefiles</title>
<header>
<nav><a href="https://ignore.pl">ignore.pl</a></nav>
<time>29 December 2023</time>
<h1>How to Write Install Targets for Lua Modules in Makefiles</h1>
</header>
<article>
<p>You can skip the full context and go right to the <a href="#proposition">explanation of the proposed makefile</a>.
<p>"Installation" can mean two things: installing packages using package manager (e.g., <b>pacman</b>(8), <b>apt</b>(8))
or copying files from one spot to another. Of course, package managers may copy files during the installation process,
but in the latter case I mean <em>only</em> copying and maybe creating destination directories.</p>
<img src="how_to_write_install_targets_for_lua_modules_in_makefiles-1.png" alt="simplified pigeon installation process">
<p>The usual encounter goes like: download some sources, build them, run checks, then run as is or <em>install</em> to
<code>/usr/local</code> to make it available system-wide and then run.
<p>With <b>make</b>(1) it's like:
<pre>
$ make
$ make test <span class="comment"># or: check</span>
# make install
</pre>
<p>We might see many different installation targets, but if there's any, <code>install</code> should be one of them.
These targets conventionally use several variables to change their behaviour. Among those we can see:
<pre>
DESTDIR=
PREFIX=/usr/local
BINDIR=$(PREFIX)/bin
LIBDIR=$(PREFIX)/lib
INCLUDEDIR=$(PREFIX)/include
DATADIR=$(PREFIX)/share
MANDIR=$(DATADIR)/man
MAN1DIR=$(MANDIR)/man1
</pre>
<p>When we run <code>make PREFIX=/usr install</code>, the <code>PREFIX</code> will change to <code>/usr</code> and so
will all downstream directories. Note that nothing uses <code>DESTDIR</code> yet. <code>DESTDIR</code> is used for
staging the files with their intended structure before they make it to the real target location. This is commonly used
by package maintainers along with <b>fakeroot</b>(1) or similar method.
<p>For more details about these paths consult
<a href="https://refspecs.linuxfoundation.org/FHS_3.0/index.html">Filesystem Hierarchy Standard</a>, <b>hier</b>(7),
<b>file‑hierarchy</b>(7) and system specific documentation. For more information about names of the variables see
<a href="https://www.gnu.org/software/make/manual/html_node/Standard-Targets.html">Standard Targets for Users</a> and
<a href="https://www.gnu.org/software/make/manual/html_node/Directory-Variables.html">Variables for Installation
Directories</a> from GNU Make documentation.
<p>Now consider a simple target definition:
<pre>
install: all
install -m644 -Dt $(DESTDIR)$(INCLUDEDIR) include/*.h
install -m755 -Dt $(DESTDIR)$(LIBDIR) libuseful.so.1.0.0
ln -s $(DESTDIR)$(LIBDIR)/libuseful.so.1.0.0 $(DESTDIR)$(LIBDIR)/libuseful.so.1
ln -s $(DESTDIR)$(LIBDIR)/libuseful.so.1 $(DESTDIR)$(LIBDIR)/libuseful.so
</pre>
<p>Note the <code>$(DESTDIR)</code> use.
<p>First we copy all header files then the shared library. Here I use <b>install</b>(1), but a combination of
<b>mkdir</b>(1), <b>cp</b>(1) and <b>chmod</b>(1) would create an equivalent result. I also make some specific-version
symbolic links for the library. Variables to hold filenames and the version could be used. They are omitted to limit
scope of the example.
<p>What about Lua?
<p>Even with standards, conventions, recommendations makefiles in the wild have a lot of variation. For Lua there are no
conventions around, so we will see even more variation among the projects. I want to respect this variation when
proposing an install target for a Lua project. Let's take a look.
<p>I selected a set of projects available in arch package repository, took a look upstream, took a look at PKGBUILDs,
searched some more through Github. From that I identified four groups of interest:
<ul>
<li>Projects that leave the entire installation process to the user (e.g., LPeg or any one-file Lua module).
<li>Projects that have makefile with <code>install</code> target or equivalent (we're interested in these).
<li>
Projects that have rockspec files and rely on Luarocks builtin installation handling or equivalent (e.g., Penlight,
Busted).
<li>Projects that have other (Luke, CMake) or hybrid installation definitions (LuaRocks+sth) (e.g., luaposix, luasdl2).
</ul>
<p>Some projects had both makefiles and rockspecs (using builtin build type). I did not try to dig out why, but I guess
it was either: migration to rockspec, developer preference, or similar. Another scenario are projects that did not have
their own rockspec got one written by a package maintainer for LuaRocks (since it's required) or distro-specific package
repository (due to preference). These kind of rockspecs could get merged to the upstream project.
<p>From projects with makefiles I selected:
<ul>
<li>
<a href="https://github.com/alerque/cldr-lua/blob/master/Makefile">cldr-lua</a>, shamelessly uses luarocks in
<code>install</code> target.
<li>
<a href="https://github.com/mascarenhas/cosmo/blob/master/Makefile">cosmo</a>, <code>LUA_DIR</code> and
<code>LUA_LIBDIR</code>.
<li>
<a href="https://github.com/luarocks/luarocks/blob/master/GNUmakefile">LuaRocks</a>, <code>luadir</code>, which is
inline with their recommendation.
<li>
<a href="https://github.com/lunarmodules/luaexpat/blob/master/Makefile">luaexpat</a>, <code>LUA_V</code> with
<code>LUA_LDIR</code> and <code>LUA_CDIR</code>.
<li>
<a href="https://github.com/LuaLanes/lanes/blob/master/Makefile">lanes</a>, <code>LUA_SHAREDIR</code> and
<code>LUA_LIBDIR</code>
<li>
<a href="https://github.com/lgi-devs/lgi/blob/master/lgi/Makefile">lgi/lgi</a>, same names with addition of
<code>LUA_VERSION</code>
<li><a href="https://github.com/hoelzro/lua-repl/blob/main/Makefile">lua-repl</a>, :)
<li>
<a href="https://github.com/libmpack/libmpack-lua/blob/master/Makefile">libmpack-lua</a>, rather unique
<code>LUA_CMOD_INSTALLDIR</code>.
<li>
<a href="https://github.com/Lua-cURL/Lua-cURLv3/blob/master/Makefile">Lua-cURLv3</a>, <code>LUA_VERSION</code> with
<code>LUA_LMOD</code> and <code>LUA_CMOD</code>
<li>
<a href="https://github.com/lunarmodules/luasocket/blob/master/src/makefile">luasocket</a>, <code>LUAV</code> with
<code>LDIR</code> and <code>CDIR</code>.
<li>
<a href="https://github.com/o-lim/luasystem/blob/master/src/Makefile">luasystem</a>, same but
<code>LUA_VERSION</code>.
<li>
<a href="https://github.com/hoelzro/lua-term/blob/master/Makefile">lua-term</a>, <code>LUA_VER</code> with
<code>LUA_SHARE</code> and <code>LUA_LIBDIR</code>
<li>
<a href="https://framagit.org/fperrad/lua-TestMore/-/blob/master/Makefile">lua-TestMore</a>, <code>LUAVER</code>
with <code>LIBDIR</code> and nothing for shared libraries
</ul>
<p>How are these makefiles used when their parent project is packaged for different OS distributions? Well, each
distribution has its own packaging processes. Let's take a quick look at two.</p>
<img src="how_to_write_install_targets_for_lua_modules_in_makefiles-2.png" alt="luarocks illusionary logo">
<p>Debian/Ubuntu in case of Lua packages will not use them. They have <b>dh</b>(1) and <b>dh_lua</b>(1) which together
take care of the packaging process. We could try to take a look at the helper scripts, but at the time of writing they
didn't appear helpful for this article.
<p>Arch Linux and its PKGBUILDs for lua packages prefer to use LuaRocks to support package making. In some cases if a
makefile from an upstream project was close enough to a conventional makefile, it could and was used. However, at least
one package maintainer later on judged that it's simply easier to use LuaRocks for everything and even recommended
writing rockspecs if not present upstream. Despite that there's a number of packages that simply copy things around
since the upstream projects did not have makefiles, install targets or rockspecs.
<p>I think this is enough information to make decisions. I also looked at some CMakeFiles and pkg-configs.
<p>Let's set goals. Our makefiles and <code>install</code> targets must be:
<ul>
<li>Simple; they must be readable and understandable, so they can be used and trusted.
<li>Generic; they must be able to work under slightly (or more) different path conventions of different distributions.
<li>
Targeted; they must build and install our software for only one version of Lua but they need to support choosing
what that version it is (there was an interesting pattern in Arch PKGBUILDs where none of the multi-version builds
were used). Switching between versions must be simple.
<li>Consistent; they must be similar to other makefiles, Lua and beyond, we have a decent set of samples to attempt it.
<li>
No duplication; you know me, they must have exactly one set of unique variables that affect all targets, i.a., not
split between build and install stages. Of course, this might backfire in cross-compilation or another similar
context.
</ul>
<h2 id="proposition">Proposition</h2>
<p>There's a shorter version better suited for closer inspection and copying <a href="#makefile">below</a>.
<table class="explanation">
<!-- This table is the ugliest HTML thing I did in a while. -->
<tr><td><div><pre>
LUA_VERSION=<u>5.4</u>
</pre></div>
<td class="description">
<p>The first or second spot seems fitting for the version. Before build definitions, too.
<p>I couldn't decide. Full name seems the best. I was thinking about <code>LUAVERSION</code> quite a lot since I was
using it for my projects. In the end underscore is consistent with Lua's <code>LUA_PATH</code> variables. Shorter
variants like <code>LUAV</code> seem to create an impression of a library or utility name (e.g., luvit/luv,
guidanoli/luav).
<p>Change your default value from <code><u>5.4</u></code> as necessary.
<tr><td><div><pre>
PREFIX=/usr/local
<td class="description">
<p>The first spot rival. It lost because it wouldn't need to appear before build definitions.
<p>It defines the base path for everything else. Do not mistake or mix it with <code>DESTDIR</code>. The default value
rarely changes, to understand it better consult sources listed somewhere above.
<tr><td><div><pre>
BINDIR=$(PREFIX)/bin
LIBDIR=$(PREFIX)/lib
DATADIR=$(PREFIX)/share
MANDIR=$(DATADIR)/man
MAN1DIR=$(MANDIR)/man1
</pre></div>
<td class="description">
<p>Not-Lua-specific install directories. Like before I skipped some since there's only a low chance of using them.
<p>We want these for consistency and to allow users do more granular path injection, e.g., <code>LIBDIR</code> with a
platform triplet.
<tr><td><div><pre>
LUA_CMOD=$(LIBDIR)/lua/$(LUA_VERSION)
LUA_LMOD=$(DATADIR)/lua/$(LUA_VERSION)
</pre></div>
<td class="description">
<p>These are the target locations respectively for shared library and script Lua modules.
<p>These names slowly grow in me every time I review the article. They read nicely: "<code>C</code> language, so
library, <code>MOD</code>-ules for <code>LUA</code>" and similarly "pure <code>L</code>ua" or "<code>L</code>ua, so
pure". These are used mainly by pkg-config files (e.g., Arch Linux, homebrew, chromebrew) and sometimes in projects.
<p>Another candidate for pure modules was <code>LUADIR</code> which is the most popular and recommended by LuaRocks but
it is conflicting with the same name pointing at Lua include files (e.g., LPeg uses it like this). A candidate for
library modules <code>LUA_LIBDIR</code> has the exact same problem (this one is even worse, because it reads as
"<code>LIBDIR</code> for <code>LUA</code>).
<p>One concern I still think about is the broken parallel between <code>LUA_LMOD</code>/<code>LUA_CMOD</code> and
<code>LUA_PATH</code>/<code>LUA_CPATH</code>.
<tr><td><div><pre>
LUA_CFLAGS=\
`pkg-config \
--cflags \
lua$(LUA_VERSION)`
LUA_LDLIBS=\
`pkg-config \
--libs \
lua$(LUA_VERSION)`
</pre></div>
<td class="description">
<p>If you need to compile some modules you will need these. I'm lazy and prefer to use <b>pkg-config</b>(1). It nicely
cooperates with the version variable allowing to meet the simple version switching goal.
<p>Now, since <b>pkg-config</b> is used rather than just writing things by hand, a question appears. More than one
actually, since a simple <code>CFLAGS</code> creates multiple questions about customization (some are answered by e.g.,
<code>+=</code>), defaults, optimization. However, I don't think this is the time to discuss them.
<p>Here, I only wanted to point out that <code>LUA_VERSION</code> can and most likely should be reused when getting Lua
versioned include directories.
<p>Lines are broken for the sake of the column width, readability of this page and to save me from tweaking the style
more than I need, since I can get lost in an endless loop of moving elements by one pixel in random directions.
<tr><td><div><pre>
export LUA_PATH=<u>\</u>
<u>$(PWD)/src/?.lua;\</u>
<u>$(PWD)/src/?/init.lua;;</u>
export LUA_CPATH=<u>;;</u>
</pre></div>
<td class="description">
<p>Occasionally, we may need to execute some Lua code as part of our makefile and use some of our code.
<p><code>export</code> is not yet part of POSIX, so workaround it with shell or <b>env</b>(1).
<p><code>LUA_CPATH</code> is an almost no-op for the sake of appearance.
<tr><td><div><pre>
LUA=lua
</pre></div>
<td class="description">
<p>Traditionally executed commands have their own variables (btw. they are also called macros). This adds just another
opportunity to modify them since make already runs in some environment that can be tweaked. Still, it can be useful for
e.g., injecting <code>LUA_PATH</code> mentioned above.
</table>
<p>I had to break the two column view for the last part, because it would look comically. Installation target itself:
<pre>
install: all
install -m755 -Dt $(DESTDIR)$(BINDIR) wrap
install -m644 -Dt $(DESTDIR)$(MAN1DIR) wrap.1
install -m755 -Dt $(DESTDIR)$(LUA_CMOD) useful.so
install -m644 -Dt $(DESTDIR)$(LUA_LMOD)/wrap wrap/*.lua
</pre>
<p>This assumes that:
<ul>
<li>Build produces an executable <code>wrap</code> and shared library Lua module <code>useful.so</code> in the same
directory makefile resides in.
<li>There's a <code>wrap/</code> subdirectory that contains pure Lua modules that require installation.
<li>There's <code>wrap.1</code> manpage in the makefile directory.
</ul>
<p>I adjusted two of my unfinished projects to use this convention, so that you can see a makefile without this kind of
assumptions. See <a href="https://git.ignore.pl/activity/tree/Makefile">activity</a> and
<a href="https://git.ignore.pl/lua-srcinfo/tree/Makefile">srcinfo</a>.
<p>As an alternative to direct use of <b>install</b>(1) and to increase control over commands executed with make see
<a href="https://www.gnu.org/software/make/manual/html_node/Command-Variables.html">Variables for Specifying
Commands</a>.
<p>Finally, the short version of the makefile:
<pre id="makefile">
LUA_VERSION=5.4
PREFIX=/usr/local
BINDIR=$(PREFIX)/bin
LIBDIR=$(PREFIX)/lib
DATADIR=$(PREFIX)/share
MANDIR=$(DATADIR)/man
MAN1DIR=$(MANDIR)/man1
LUA_CMOD=$(LIBDIR)/lua/$(LUA_VERSION)
LUA_LMOD=$(DATADIR)/lua/$(LUA_VERSION)
LUA_CFLAGS=`pkg-config --cflags lua$(LUA_VERSION)`
LUA_LDLIBS=`pkg-config --libs lua$(LUA_VERSION)`
CFLAGS=$(LUA_CFLAGS)
LDLIBS=$(LUA_LDLIBS)
all:
touch wrap useful.so
install: all
install -m755 -Dt $(DESTDIR)$(BINDIR) wrap
install -m644 -Dt $(DESTDIR)$(MAN1DIR) wrap.1
install -m755 -Dt $(DESTDIR)$(LUA_CMOD) useful.so
install -m644 -Dt $(DESTDIR)$(LUA_LMOD)/wrap wrap/*.lua
</pre>
<p>I still think I would prefer them without the underscores, but then <code>LUALMOD</code> would be one letter away
from <code>LUAMOD</code> which would be annoying to deal with since it sounds like it would hold a name of a module.
<p>What do you think? What do you use yourself (other than LuaRocks)? Do you like the two column code view thing? Until
I create some proper feedback channel, just feel free to <a href="mailto:please@ignore.pl">send me an email</a>!
</article>
<script src="https://stats.ignore.pl/track.js"></script>
|
