diff options
Diffstat (limited to 'how_to_organize_your_lua_project.html')
| -rw-r--r-- | how_to_organize_your_lua_project.html | 239 |
1 files changed, 239 insertions, 0 deletions
diff --git a/how_to_organize_your_lua_project.html b/how_to_organize_your_lua_project.html new file mode 100644 index 0000000..013f712 --- /dev/null +++ b/how_to_organize_your_lua_project.html @@ -0,0 +1,239 @@ +<!doctype html> +<html lang="en"> +<meta charset="utf-8"> +<meta name="viewport" content="width=device-width, initial-scale=1"> +<meta name="author" content="aki"> +<meta name="tags" content="tutorial, structuring, organizing, lua, project, howto"> +<link rel="icon" type="image/png" href="cylo.png"> +<link rel="stylesheet" href="style.css"> + +<title>How To Organize Your Lua Project</title> + +<nav><p><a href="https://ignore.pl">ignore.pl</a></p></nav> + +<article> +<h1>How To Organize Your Lua Project</h1> +<p class="subtitle">Published on 2021-01-07 15:45:00+01:00 +<p>From time to time I hear complaints about how Lua handles modules. Here and there I see and even answer myself +questions regarding <code>require</code> and adjusting the paths in <code>package</code> to allow some desired +behaviour, with the most prominent issue of relative imports that always work. +<p>Before we hop into the explanation of how to organize files in your Lua projects, let's talk about default importing +mechanism in Lua: <a href="https://www.lua.org/manual/5.4/manual.html#pdf-require"><code>require</code></a>.</p> + +<img src="how_to_organize_your_lua_project-1.png" alt="lua hierarchy"> + +<h2>How <code>require</code> handles paths</h2> +<p>Both <code>package</code> and <code>require</code> are surprisingly interesting tools. At first glance they are +simple. When you look into them, they are still understandable while gaining some complexity that doesn't reach +unnecessary extremes. They are elegant. +<p>They use a mechanism to find desired files called path resolution or usually simply <strong>path</strong>. The main +component of is a sequence of patterns that may become a pathname, e.g.: +<pre> +/usr/lib/lua/?.lua;/usr/lib/lua/?/init.lua +</pre> +<p>What does it tell us? First off, <code>?</code> is going to be replaced by the argument that was provided to the +<code>require</code>. All dots will be replaced by an appropriate path separator so that: <code>a.b.c</code> will +become <code>a/b/c</code> in *nix systems. So, for call like <code>require "a.b.c"</code>, out path will look +like this: +<pre> +/usr/lib/lua/a/b/c.lua;/usr/lib/lua/a/b/c/init.lua +</pre> +<p>Now, each of these paths are tried and the first one that actually exists in the system will be used. If none of them +match an existing file, the import fails. Simple as that. +<p>The path that is used in resolution is set in <code>package.path</code>. You can modify it in Lua, but it is +intrusive and may depend on a single entry point. Generally, if you plan to release your project as a module for people +to use, I encourage you to avoid modifying anything global. And that's global. Anyway, <code>package.path</code> doesn't +appear out of nowhere - it is populated by one of: +<ol> + <li>Environmental variable <code>LUA_PATH_x_x</code>, where <code>x_x</code> is version such as <code>5_4</code> + <li>Environmental variable <code>LUA_PATH</code> + <li>Default as defined in <a href="https://www.lua.org/source/5.4/luaconf.h.html#LUA_PATH_DEFAULT">luaconf.h</a> +</ol> +<p>Interestingly, if two separators <code>;;</code> show up in the environmental variable path, they will be replaced by +the default path. Meaning <code>/path/to/project/?.lua;;</code> works as prepending your custom path to the default one. +<p>Of course, there is way more to it than just this i.a.: requiring modules written in C, searchers or preloads. +However, in our case this knowledge will suffice. +<p>If you are curious how exactly path is loaded be sure to check out +<a href="https://www.lua.org/source/5.4/loadlib.c.html#setpath"><code>setpath</code></a>. + +<h2>Endgame</h2> +<p>To prepare for development, we need to know where we are heading. First step is to consider the execution +environment. Of course, this and packaging are journeys on their own, so let's just look at two common examples: an +application that uses some framework that uses Lua (e.g. a game made in <a href="https://love2d.org/">LÖVE</a>) and a +standalone module for others to use. +<p>In the first case, it's the duty of the framework to configure the path properly and inform you through the +documentation about it. Paths in LÖVE use their own file hierarchy that is managed by <code>love.filesystem</code> and +by default contains both the game's source (directory or the mounted <code>.love</code> archive) and the save directory. +This means that the structure in your source files is directly reflected in the calls to <code>require</code>, so that +<code>require "module.submodule"</code> will always try <code>game/module/submodule.lua</code>, no matter how you run +the game. This case usually doesn't involve any additional environment configuration for the development stage. +<p>In the second case, your project will end up in an already configured environment and will need to fit in. The +installation of the package usually involves copying your files to the directory that is already included in the path, +so that no further configuration is needed for the execution (at least regarding the path). You can assume, that the +successful installation will make your modules available in the way you want them. +<p>This doesn't happen in the development stage, when you rarely install your package, and most certainly you don't +install it each time you want to test it. This means, that you need to adjust the path so that your modules appear in it +as if they were installed in the system. The principle of minimizing the intrusiveness remains, so the best option is to +use the environmental variables to prepare for development. If you run your application or any tool in such environment, +then Lua will have access to your modules no matter where it is run. Additionally, it will be consistent with the +target environment and won't need any additional hacks. + +<h2>Development environment</h2> +<p>All this talk comes down to: set <code>LUA_PATH</code> in your development environment so that it includes your +project files even if they are not installed in system. A simple approach is to source following in each session: +<pre> +export PROJECT=/path/to/project +export LUA_PATH="$PROJECT/?.lua;$PROJECT/?/init.lua;;" +</pre> +<p>Note the double semicolon that will get replaced by default path, so that other modules that are already installed +are also available. +<p>Let's try it out: +<pre> +$ source env.sh +$ find . +./env.sh +./modulea/submodule.lua +./modulea/init.lua +./moduleb.lua +$ cd modulea +$ lua +Lua 5.4.2 +> require "moduleb" +table: 0x561e4b72fb20 +> require "modulea" +table: 0x561e4b73aa40 +> require "modulea.submodule" +table: 0x561e4b743030 +</pre> +<p>As you can see, despite being in the subdirectory, you can still use modules with their fully qualified names that +will remain the same once you install the package. Note, that you could <code>require "init"</code> or <code>require +"submodule"</code> in this case, but I strongly recommend against it. Remain specific, follow the rules and pretend that +you use an installed package from an unknown working directory. <strong>Don't</strong> depend on current working +directory as it is not always the same. Using full names that consider the path setup <strong>guarantees +results</strong>.</p> + +<img src="how_to_organize_your_lua_project-2.png" alt="a random whale"> + +<h2>Organizing your files</h2> +<p>Finally, this is what we're waiting for. Assume you have a directory that is a parent of all of your project files. +We'll call it a project <strong>root</strong>. Usually, this is also root directory for your version control system, be +it git or anything else, and for other tools such as building systems or even entire IDEs. +<p>Because it is such a central place to the project, I usually just go ahead and prepend it to <code>LUA_PATH</code> in +the very same way as in the section above: +<pre> +export PROJECT=/path/to/project +export LUA_PATH="$PROJECT/?.lua;$PROJECT/?/init.lua;;" +</pre> +<p>Just like previously, any Lua file that will be descendent of the root will be accessible to us through +<code>require</code>. But what is that <i>init.lua</i>? +<p>It's there to create a way to improve hierarchical structure of your project - to allow splitting bigger modules into +smaller parts (or even submodules that could be included on their own), so that the module doesn't grow into a single +millions-lines-long file. <strong>In simpler words</strong>: you can create a directory named after module and put +<i>init.lua</i> file there and it will act just like a sole <i>module.lua</i> in root. +<p>You could also create a directory named after module and <i>module.lua</i> file in root at the same time, but this +way you will have two entries per module in the root instead of just one. +<p>Additionally, you can then put any module-related files into that directory. You can also use <i>init.lua</i> as +a simple wrapper that calls <code>require</code> for each of its submodules and returns a table with them. +<p>Consider a verbose example: +<pre> +$ find . +./conf.lua +./env.sh +./main.lua +./persistence/init.lua +./persistence/tests.lua +./version.lua +./wave/init.lua +./wave/sawtooth.lua +./wave/sine.lua +./wave/square.lua +$ cat wave/init.lua +-- This is a wrapper example. +return { + sawtooth = require "wave.sawtooth", + sine = require "wave.sine", + square = require "wave.square", +} +$ cat persistence/init.lua +-- This is a normal module example. +return {} +$ cat persistance/tests.lua +-- This is a script that tests an example module. +local p = require "persistence" +assert(type(p) == "table") +$ cat main.lua +-- This is an example main of love application. +local persistence = require "persistence" +local wave = require "wave" +$ cat version.lua +-- This is an example module that acts as version string of the application. +return "1.0.0" +</pre> +<p>Now, this is a mash-up of everything we've discussed. Despite it pretending to be LÖVE application it has +<i>env.sh</i>. Why? The reason is simple: the <i>persistence</i> and <i>wave</i> modules are not meant to be distributed +alone, and they won't ever appear in path of any other environment than LÖVE's. But LÖVE is not the only execution +environment in here: <i>persistence/tests.lua</i> is also meant to be executed. Possibly alone through Lua interpreter. +To allow it <i>env.sh</i> is present and used. +<p>Let's have another example of a simple module meant for installation: +<pre> +$ find . +./env.sh +./hello/Class.lua +./hello/init.lua +./hello/tests.lua +./hello/version.lua +./LICENSE +./Makefile +./README +$ cat Makefile +PREFIX?=/usr/local/lib/lua/5.4 +all: + @echo Nothing to be done +install: + cp -r hello $(PREFIX) +uninstall: + rm -fd $(PREFIX)/hello/* $(PREFIX)/hello +</pre> +<p>As you can see <i>Makefile</i> in this example has targets for installation and removal of the package. The structure +again is simple. Root works as part of the resolution path and so our module is placed in it's own directory named after +it. +<p>The last example is a project of a single file module: +<pre> +$ find . +./env.sh +./LICENSE +./object.lua +./README +</pre> +<p>Yes, it's that simple. +<p>Now, having <i>env.sh</i> in every single project might get bothersome, so I usually use a shell function for +managing them, similarly to what Python's venv does or LuaRocks' env. Speaking of, +<a href="https://luarocks.org/">LuaRocks</a> is yet another interesting story to be told. + +<h2>Summary</h2> +<ul> + <li>Consider the execution environment when preparing the development environment. + <li>Contain your project. Avoid changing globals if the modules may be used by someone else in an unknown + environment. + <li>Never depend on the current working directory being something specific. + <li>While developing include your project's root in <code>LUA_PATH</code> to influence <code>require</code>'s + behaviour. + <li>Split modules into parts and use <i>init.lua</i> which is available by default to enclose the module into a + single directory. +</ul> + +<h2>Alternatives</h2> +<p>This is just one of the ways to handle structuring your Lua project. It's based on simple rules but has broad usage. +One tempting alternative is this little snippet: +<pre> +local parents = (...):match "(.-)[^%.]+$" +require(parents .. "sibling") +</pre> +<p>Another already mentioned alternatives is adjusting <code>package.path</code> directly in Lua. However, I decided to +skip it due to it's intrusiveness. +<p>All in all, Lua is extremely customizable and adjustable. I would be surprised if these three would be the only ways +to organize projects in Lua. + +</article> + +<script src="https://stats.ignore.pl/track.js"></script> |