From 8e1f3c9ebc0ccd132e3836f3d198415a15932877 Mon Sep 17 00:00:00 2001 From: Aki Date: Sun, 25 Jul 2021 19:46:25 +0200 Subject: Renamed guides to include "How To" in their names --- how_to_organize_your_lua_project.html | 239 ++++++++++++++++++++++++++++++++++ 1 file changed, 239 insertions(+) create mode 100644 how_to_organize_your_lua_project.html (limited to 'how_to_organize_your_lua_project.html') diff --git a/how_to_organize_your_lua_project.html b/how_to_organize_your_lua_project.html new file mode 100644 index 0000000..013f712 --- /dev/null +++ b/how_to_organize_your_lua_project.html @@ -0,0 +1,239 @@ + + + + + + + + + +How To Organize Your Lua Project + + + +
+

How To Organize Your Lua Project

+

Published on 2021-01-07 15:45:00+01:00 +

From time to time I hear complaints about how Lua handles modules. Here and there I see and even answer myself +questions regarding require and adjusting the paths in package to allow some desired +behaviour, with the most prominent issue of relative imports that always work. +

Before we hop into the explanation of how to organize files in your Lua projects, let's talk about default importing +mechanism in Lua: require.

+ +lua hierarchy + +

How require handles paths

+

Both package and require are surprisingly interesting tools. At first glance they are +simple. When you look into them, they are still understandable while gaining some complexity that doesn't reach +unnecessary extremes. They are elegant. +

They use a mechanism to find desired files called path resolution or usually simply path. The main +component of is a sequence of patterns that may become a pathname, e.g.: +

+/usr/lib/lua/?.lua;/usr/lib/lua/?/init.lua
+
+

What does it tell us? First off, ? is going to be replaced by the argument that was provided to the +require. All dots will be replaced by an appropriate path separator so that: a.b.c will +become a/b/c in *nix systems. So, for call like require "a.b.c", out path will look +like this: +

+/usr/lib/lua/a/b/c.lua;/usr/lib/lua/a/b/c/init.lua
+
+

Now, each of these paths are tried and the first one that actually exists in the system will be used. If none of them +match an existing file, the import fails. Simple as that. +

The path that is used in resolution is set in package.path. You can modify it in Lua, but it is +intrusive and may depend on a single entry point. Generally, if you plan to release your project as a module for people +to use, I encourage you to avoid modifying anything global. And that's global. Anyway, package.path doesn't +appear out of nowhere - it is populated by one of: +

    +
  1. Environmental variable LUA_PATH_x_x, where x_x is version such as 5_4 +
  2. Environmental variable LUA_PATH +
  3. Default as defined in luaconf.h +
+

Interestingly, if two separators ;; show up in the environmental variable path, they will be replaced by +the default path. Meaning /path/to/project/?.lua;; works as prepending your custom path to the default one. +

Of course, there is way more to it than just this i.a.: requiring modules written in C, searchers or preloads. +However, in our case this knowledge will suffice. +

If you are curious how exactly path is loaded be sure to check out +setpath. + +

Endgame

+

To prepare for development, we need to know where we are heading. First step is to consider the execution +environment. Of course, this and packaging are journeys on their own, so let's just look at two common examples: an +application that uses some framework that uses Lua (e.g. a game made in LÖVE) and a +standalone module for others to use. +

In the first case, it's the duty of the framework to configure the path properly and inform you through the +documentation about it. Paths in LÖVE use their own file hierarchy that is managed by love.filesystem and +by default contains both the game's source (directory or the mounted .love archive) and the save directory. +This means that the structure in your source files is directly reflected in the calls to require, so that +require "module.submodule" will always try game/module/submodule.lua, no matter how you run +the game. This case usually doesn't involve any additional environment configuration for the development stage. +

In the second case, your project will end up in an already configured environment and will need to fit in. The +installation of the package usually involves copying your files to the directory that is already included in the path, +so that no further configuration is needed for the execution (at least regarding the path). You can assume, that the +successful installation will make your modules available in the way you want them. +

This doesn't happen in the development stage, when you rarely install your package, and most certainly you don't +install it each time you want to test it. This means, that you need to adjust the path so that your modules appear in it +as if they were installed in the system. The principle of minimizing the intrusiveness remains, so the best option is to +use the environmental variables to prepare for development. If you run your application or any tool in such environment, +then Lua will have access to your modules no matter where it is run. Additionally, it will be consistent with the +target environment and won't need any additional hacks. + +

Development environment

+

All this talk comes down to: set LUA_PATH in your development environment so that it includes your +project files even if they are not installed in system. A simple approach is to source following in each session: +

+export PROJECT=/path/to/project
+export LUA_PATH="$PROJECT/?.lua;$PROJECT/?/init.lua;;"
+
+

Note the double semicolon that will get replaced by default path, so that other modules that are already installed +are also available. +

Let's try it out: +

+$ source env.sh
+$ find .
+./env.sh
+./modulea/submodule.lua
+./modulea/init.lua
+./moduleb.lua
+$ cd modulea
+$ lua
+Lua 5.4.2
+> require "moduleb"
+table: 0x561e4b72fb20
+> require "modulea"
+table: 0x561e4b73aa40
+> require "modulea.submodule"
+table: 0x561e4b743030
+
+

As you can see, despite being in the subdirectory, you can still use modules with their fully qualified names that +will remain the same once you install the package. Note, that you could require "init" or require +"submodule" in this case, but I strongly recommend against it. Remain specific, follow the rules and pretend that +you use an installed package from an unknown working directory. Don't depend on current working +directory as it is not always the same. Using full names that consider the path setup guarantees +results.

+ +a random whale + +

Organizing your files

+

Finally, this is what we're waiting for. Assume you have a directory that is a parent of all of your project files. +We'll call it a project root. Usually, this is also root directory for your version control system, be +it git or anything else, and for other tools such as building systems or even entire IDEs. +

Because it is such a central place to the project, I usually just go ahead and prepend it to LUA_PATH in +the very same way as in the section above: +

+export PROJECT=/path/to/project
+export LUA_PATH="$PROJECT/?.lua;$PROJECT/?/init.lua;;"
+
+

Just like previously, any Lua file that will be descendent of the root will be accessible to us through +require. But what is that init.lua? +

It's there to create a way to improve hierarchical structure of your project - to allow splitting bigger modules into +smaller parts (or even submodules that could be included on their own), so that the module doesn't grow into a single +millions-lines-long file. In simpler words: you can create a directory named after module and put +init.lua file there and it will act just like a sole module.lua in root. +

You could also create a directory named after module and module.lua file in root at the same time, but this +way you will have two entries per module in the root instead of just one. +

Additionally, you can then put any module-related files into that directory. You can also use init.lua as +a simple wrapper that calls require for each of its submodules and returns a table with them. +

Consider a verbose example: +

+$ find .
+./conf.lua
+./env.sh
+./main.lua
+./persistence/init.lua
+./persistence/tests.lua
+./version.lua
+./wave/init.lua
+./wave/sawtooth.lua
+./wave/sine.lua
+./wave/square.lua
+$ cat wave/init.lua
+-- This is a wrapper example.
+return {
+	sawtooth = require "wave.sawtooth",
+	sine = require "wave.sine",
+	square = require "wave.square",
+}
+$ cat persistence/init.lua
+-- This is a normal module example.
+return {}
+$ cat persistance/tests.lua
+-- This is a script that tests an example module.
+local p = require "persistence"
+assert(type(p) == "table")
+$ cat main.lua
+-- This is an example main of love application.
+local persistence = require "persistence"
+local wave = require "wave"
+$ cat version.lua
+-- This is an example module that acts as version string of the application.
+return "1.0.0"
+
+

Now, this is a mash-up of everything we've discussed. Despite it pretending to be LÖVE application it has +env.sh. Why? The reason is simple: the persistence and wave modules are not meant to be distributed +alone, and they won't ever appear in path of any other environment than LÖVE's. But LÖVE is not the only execution +environment in here: persistence/tests.lua is also meant to be executed. Possibly alone through Lua interpreter. +To allow it env.sh is present and used. +

Let's have another example of a simple module meant for installation: +

+$ find .
+./env.sh
+./hello/Class.lua
+./hello/init.lua
+./hello/tests.lua
+./hello/version.lua
+./LICENSE
+./Makefile
+./README
+$ cat Makefile
+PREFIX?=/usr/local/lib/lua/5.4
+all:
+	@echo Nothing to be done
+install:
+	cp -r hello $(PREFIX)
+uninstall:
+	rm -fd $(PREFIX)/hello/* $(PREFIX)/hello
+
+

As you can see Makefile in this example has targets for installation and removal of the package. The structure +again is simple. Root works as part of the resolution path and so our module is placed in it's own directory named after +it. +

The last example is a project of a single file module: +

+$ find .
+./env.sh
+./LICENSE
+./object.lua
+./README
+
+

Yes, it's that simple. +

Now, having env.sh in every single project might get bothersome, so I usually use a shell function for +managing them, similarly to what Python's venv does or LuaRocks' env. Speaking of, +LuaRocks is yet another interesting story to be told. + +

Summary

+ + +

Alternatives

+

This is just one of the ways to handle structuring your Lua project. It's based on simple rules but has broad usage. +One tempting alternative is this little snippet: +

+local parents = (...):match "(.-)[^%.]+$"
+require(parents .. "sibling")
+
+

Another already mentioned alternatives is adjusting package.path directly in Lua. However, I decided to +skip it due to it's intrusiveness. +

All in all, Lua is extremely customizable and adjustable. I would be surprised if these three would be the only ways +to organize projects in Lua. + +

+ + -- cgit v1.1