From 8e1f3c9ebc0ccd132e3836f3d198415a15932877 Mon Sep 17 00:00:00 2001 From: Aki Date: Sun, 25 Jul 2021 19:46:25 +0200 Subject: Renamed guides to include "How To" in their names --- archiving_with_posix_utilities-1.png | Bin 1792 -> 0 bytes archiving_with_posix_utilities-2.png | Bin 2317 -> 0 bytes archiving_with_posix_utilities-3.png | Bin 1470 -> 0 bytes archiving_with_posix_utilities.html | 238 -------------------- flashing_lolin_nodemcu_v3-1.png | Bin 2757 -> 0 bytes flashing_lolin_nodemcu_v3.html | 46 ---- how_to_archive_with_posix_tar_cpio_and_pax-1.png | Bin 0 -> 1792 bytes how_to_archive_with_posix_tar_cpio_and_pax-2.png | Bin 0 -> 2317 bytes how_to_archive_with_posix_tar_cpio_and_pax-3.png | Bin 0 -> 1470 bytes how_to_archive_with_posix_tar_cpio_and_pax.html | 238 ++++++++++++++++++++ ...ate_templates_with_shell_cat_and_envsubst-1.png | Bin 0 -> 1159 bytes ...eate_templates_with_shell_cat_and_envsubst.html | 72 +++++++ how_to_flash_lolin_nodemcu_v3-1.png | Bin 0 -> 2757 bytes how_to_flash_lolin_nodemcu_v3.html | 46 ++++ how_to_organize_your_lua_project-1.png | Bin 0 -> 3346 bytes how_to_organize_your_lua_project-2.png | Bin 0 -> 2192 bytes how_to_organize_your_lua_project.html | 239 +++++++++++++++++++++ index.html | 12 +- organizing_your_lua_project-1.png | Bin 3346 -> 0 bytes organizing_your_lua_project-2.png | Bin 2192 -> 0 bytes organizing_your_lua_project.html | 239 --------------------- ...id_templating_with_shell_cat_and_envsubst-1.png | Bin 1159 -> 0 bytes stupid_templating_with_shell_cat_and_envsubst.html | 72 ------- 23 files changed, 603 insertions(+), 599 deletions(-) delete mode 100644 archiving_with_posix_utilities-1.png delete mode 100644 archiving_with_posix_utilities-2.png delete mode 100644 archiving_with_posix_utilities-3.png delete mode 100644 archiving_with_posix_utilities.html delete mode 100644 flashing_lolin_nodemcu_v3-1.png delete mode 100644 flashing_lolin_nodemcu_v3.html create mode 100644 how_to_archive_with_posix_tar_cpio_and_pax-1.png create mode 100644 how_to_archive_with_posix_tar_cpio_and_pax-2.png create mode 100644 how_to_archive_with_posix_tar_cpio_and_pax-3.png create mode 100644 how_to_archive_with_posix_tar_cpio_and_pax.html create mode 100644 how_to_create_templates_with_shell_cat_and_envsubst-1.png create mode 100644 how_to_create_templates_with_shell_cat_and_envsubst.html create mode 100644 how_to_flash_lolin_nodemcu_v3-1.png create mode 100644 how_to_flash_lolin_nodemcu_v3.html create mode 100644 how_to_organize_your_lua_project-1.png create mode 100644 how_to_organize_your_lua_project-2.png create mode 100644 how_to_organize_your_lua_project.html delete mode 100644 organizing_your_lua_project-1.png delete mode 100644 organizing_your_lua_project-2.png delete mode 100644 organizing_your_lua_project.html delete mode 100644 stupid_templating_with_shell_cat_and_envsubst-1.png delete mode 100644 stupid_templating_with_shell_cat_and_envsubst.html diff --git a/archiving_with_posix_utilities-1.png b/archiving_with_posix_utilities-1.png deleted file mode 100644 index 68b100a..0000000 Binary files a/archiving_with_posix_utilities-1.png and /dev/null differ diff --git a/archiving_with_posix_utilities-2.png b/archiving_with_posix_utilities-2.png deleted file mode 100644 index 2f31089..0000000 Binary files a/archiving_with_posix_utilities-2.png and /dev/null differ diff --git a/archiving_with_posix_utilities-3.png b/archiving_with_posix_utilities-3.png deleted file mode 100644 index 4a43e34..0000000 Binary files a/archiving_with_posix_utilities-3.png and /dev/null differ diff --git a/archiving_with_posix_utilities.html b/archiving_with_posix_utilities.html deleted file mode 100644 index 17ce7bc..0000000 --- a/archiving_with_posix_utilities.html +++ /dev/null @@ -1,238 +0,0 @@ - - - - - - - - - -Archiving With POSIX Utilities - - - -
-

Archiving With POSIX Utilities

-

Published on 2020-07-22 22:30:00+02:00 -

The usual answer is tar. As you may see I intentionally linked to the -GNU Tar. If you are a *BSD user then you use some other implementation. Both of them follow and extend POSIX'es standard -for tar utility. Or so you would think. -

Right now there is no POSIX tar utility. It has been marked as legacy -already in 1997 and disappeared from the -standard soon after. It's place took a behemoth called -pax. The name gets even funnier when -you consider the rationale and the size of this thing. But pax didn't came from just tar. There was one more influencer -in here called cpio. You may know this one -if you ever tinkered with RPM packages or initramfs. -

In other words we have three utilities on today's table: tar, cpio and pax. According to -Debian's popularity contest the frequency of each being installed is in -the exact same order, with tar being at 8th place overall, cpio at 52nd, and pax at 6089th. I can't just talk about the -least popular one, so I'll explain shortly how to use each of them in your usual Linux distribution while keeping in -mind what POSIX had to tell us back in the day. - -

tar

-

Like I've already mentioned tarballs are the most popular. Not only that, they are commonly described as the easiest -to use, although the interface is something that you can find jokes about. All operations on tarballs are handled via -single tar utility.

-box -

Let's go through three basic operations: create an archive, list out the content, and extract it. Tar expects to have -first argument to match this regular expression: [rxtuc][vwfblmo]*. The first part is function, -and the second is a modifier. I'll focus only on those necessary to accomplish before-mentioned tasks. -

To create an archive you:

-
-$ tar cf ../archive.tar a_file a_directory
-
-

This will create an archive that will be located in parent directory of current working directory, and will contain -a_file and recursively a_directory. Let's map every part of the command for clarity:

-
-
tar
Call tar -
c
Create an archive -
f
Use first argument after cf as the path to the archive -
../archive.tar
Path to the archive (without f it would be treated as another file to - include in the archive) -
a_file a_directory
Files to include in the archives -
-

Now that you have an archive, you can see it's content:

-
-$ tar tf ../archive.tar
-a_file
-a_directory/
-a_directory/another_file
-
-

As you have probably guessed t function is used to write the names of files that are in the archive. -f works exactly the same way: first argument after tf is meant to point to the archive file. -

To extract everything from the archive you:

-
-$ tar xf ../archive.tar
-
-

Or add more arguments to extract selected files:

-
-$ tar xf ../archive.tar a_file
-
-

This one will extract only a_file from the archive. -

That's pretty much it about tar. The are two more functions: r that adds new file to existing archive, -and u that first tries to update the file in archive if it exists and if it doesn't then it adds it. Note, -that the usual compression options are not available in POSIX, they are an extension. - -

cpio

-

Heading off from the usual routes we encounter cpio. It's a more frequent sight than pax, but it still is quite niche -compared to tar's omnipresence. Frankly, I like this one the most because of the way it handles input of file lists. -Sadly, this also makes it slightly bothersome to use. -

Now, now, cpio operates in three modes: copy-out, copy-in and pass-through. Our goals are -still the same: to create an archive, list files inside, and extract it somewhere else and for that we'll only need the -first two modes. -

To create an archive, use the copy-out mode, as in: copy to the standard output:

-
-$ find a_file a_directory | cpio -o >../archive.cpio
-
-

This instant you probably noticed that cpio doesn't accept files as arguments. In copy-out mode it expects list of -files in standard input, and it will return the formatted archive through standard output. See a somehow step-by-step -explanation:

-
-
find a_file a_directory |
List files, directories and their content from arguments and pipe the - output to the next command -
cpio
Call cpio (duh!) -
-o
Use copy-out mode -
>../archive.cpio
Redirect standard output of cpio to a file -
-

You now have an archive file called archive.cpio in parent directory. To see its content type in:

-
-$ cpio -it <../archive.cpio
-a_file
-a_directory
-a_directory/another_file
-1 block
-
-

Nice! What's left is extraction. You do it with copy-in mode like this:

-
-$ cpio -i <../archive.cpio
-1 block
-
-

Huh? What's that? Listing files and extracting both use copy-in mode? That's right. Like "copy-out" means "copy to -standard output", "copy-in" can be understood as "copy from standard input". The t option prohibits any -files to be written or created by cpio, nonetheless archive is read from standard input and then translated to list of -files in standard output. Some extended implementations let you use t directly as sole option and imply the -copy-in mode. -

You can also use patterns when extracting to select files:

-
-$ cpio -i a_file <../archive.cpio
-1 block
-
-

You can copy nested files if you use d option:

-
-$ cpio -id a_directory/another_file <../archive.cpio
-1 block
-
-

This option tells cpio that it's allowed to create directories whenever it is necessary.

-pass-through -

Bonus! Pass-through mode can be used to copy files listed in standard input to specified directory. It doesn't create -an archive at all.

-
-$ ls ../destination
-$ ls
-a_directory  a_file
-$ find a_file a_directory | cpio -p ../destination
-0 blocks
-$ ls ../destination
-a_directory  a_file
-
- -

pax

-

Finally, at the destination! This one lives up to the name of this post as it's still part of POSIX. The fun part is -that you probably don't even have it installed, but don't worry, I didn't have it until like two days ago. It truly -feels like a compromise forced on you and your siblings by your parents. Jokes aside, I actually started to like it, -bulky but kind of cute. -

Anyway, let's see what this coffee machine can do for us; same goals as previously. This will be confusing, because -this utility is a compromise, and so it supports both usage styles: tar-like and cpio-like. -

To create an archive you can use either:

- -
-$ pax -wf ../archive.pax a_directory a_file
-$ find a_file a_directory | pax -wd >../archive.pax
-$ find a_file a_directory | pax -wdf ../archive.pax
-
- -

They are equivalent. You can mix the style as much as you want, as long as it doesn't become mess it's quite handy. -As for what option does what:

- -
-
-w
Indicates that pax will act in write mode (tar's c and cpio's -o) -
f ../archive.pax
Argument after f is the path to the archive; note that it behaves - slightly different compared to tar, it always takes next argument instead of first path that appears after flags. It - means you can't put any options between -f and the path. -
a_directory a_file -
find a_file a_directory |
Both of these accomplish the same goal of letting know pax - what files should be in archive. They are mutually exclusive! If there is at least one argument pointing to a file, - then standard input is not supposed to be read. -
d
This one is used to prevent recursively adding files that are in a directory, so that the - behaviour is the same as in cpio: -
-$ find a_file a_directory | pax -wvf ../archive.pax
-a_directory
-a_directory/another_file
-a_directory/another_file
-a_file
-pax: ustar vol 1, 4 files, 0 bytes read, 10240 bytes written.
-$ find a_directory a_file | pax -wvdf ../archive.pax
-a_directory
-a_directory/another_file
-a_file
-pax: ustar vol 1, 3 files, 0 bytes read, 10240 bytes written.
-
-
- -

The v option is used to increase verbosity of the "error" output. You can find similar functionality in -most of command line utilities, including tar and cpio. -

To list files that are in archive you can also use both styles:

-
-$ pax <../archive.pax
-a_directory
-a_directory/another_file
-a_file
-$ pax -f ../archive.pax
-a_directory
-a_directory/another_file
-a_file
-
-

Yes, that's the default behaviour of pax and you don't need to specify any argument (in case of cpio-like style). -Sweet, isn't it? -

To extract the archive use one of:

-
-$ pax -r <../archive.pax
-$ pax -rf ../archive.pax
-
-

For selecting files to extract use the usual patterns:

-
-$ pax -r a_file -f ../archive.pax
-$ pax -r a_directory/another_file <../archive.pax
-
-

That's all of the most basic use case. There's more, for instance pax supports mode similar to the pass-through mode -we already know from the cpio. But there is something more important to mention about pax. It's supposed to easily -support various different formats. -

POSIX tells that pax should support: pax, cpio and ustar formats. I installed GNU pax and it seems to support: ar, -bcpio, cpio, sv4cpio, sc4crc, tar and ustar. The default format for my installation is ustar as you have probably -noticed in verbose output in one of the examples above. Pax format is extension for ustar, that's most likely the reason -it's usually omitted. -

You can select format with -x option, for supported formats please refer to your manual. Also note that -explicitly specifying format should be only needed when writing an archive. When reading pax can identify archive's -format efficiently:

-
-$ find a_file a_directory | cpio -o >../archive.cpio
-$ pax -vf ../archive.cpio
--rw-rw-r--  1 ignore   ignore    0 Jul 22 22:30 a_file
-drwxrwxr-x  2 ignore   ignore    0 Jul 22 22:30 a_directory
--rw-rw-r--  1 ignore   ignore    0 Jul 22 22:30 a_directory/another_file
-pax: bcpio vol 1, 3 files, 512 bytes read, 0 bytes written.
-
- -

Final thoughts

-

Now then, it's time to finally wrap it all up. There is nothing left to say but remember to always check your manual, -all of those utilities have various implementations that are compliant to POSIX in various degrees. Don't be naive and -don't get tricked by them. I find pax the most reliable of them as its "novelty" and the interface that was quite -"modern" from the start resulted in decently compliant implementations. Moreover, it includes nice things one may know -from both cpio and tar. Find a moment to check it out! -

Let's pretend that ar doesn't exist. -Thank you.

-boo! -
- diff --git a/flashing_lolin_nodemcu_v3-1.png b/flashing_lolin_nodemcu_v3-1.png deleted file mode 100644 index daceb97..0000000 Binary files a/flashing_lolin_nodemcu_v3-1.png and /dev/null differ diff --git a/flashing_lolin_nodemcu_v3.html b/flashing_lolin_nodemcu_v3.html deleted file mode 100644 index e535f74..0000000 --- a/flashing_lolin_nodemcu_v3.html +++ /dev/null @@ -1,46 +0,0 @@ - - - - - - - - -Flashing LOLin NodeMCU v3 - - - -
-

Flashing LOLin NodeMCU v3

-

Published on 2020-06-29 17:58:00+02:00 -

Republishing old content. This is from when I bought and flashed my first NodeMCU clone (perhaps "loose -implementation"?) of NodeMCU v3 called LOLin v3. -

This little board is using CH304G TTL to USB converter, so if you happen to work on Windows you must make sure you -have the driver installed beforehand. Mentioned chip is quite common, and I've seen it on various Arduino, so there is -a high chance that you already have it. -

Without further notes, here is the specification:

- -
Baud rateFlash modeFlash sizeFlash frequency -
9600 QIO 4MiB 40MHz -
-

You will need a combined firmware binary. If you are compiling it by yourself, I suggest using one of the development -branches. If you don't have environment set up there are online building services available e.g. -NodeMCU custom builds. -

To write the firmware use e.g. esptool:

-
-$ esptool.py --port /dev/ttyUSB0 --baud 9600 erase_flash
-$ esptool.py --port /dev/ttyUSB0 --baud 9600 write_flash \
-  --flash_mode qio --flash_size 4MB --flash_freq 40m \
-  0x00000 combined-firmware-file.bin
-
-

To test it you can connect to it with e.g. screen:

-
-$ screen /dev/ttyUSB0 115200
-
-

Or use your favourite terminal emulator:

-
-$ st -l /dev/ttyUSB0 115200
-
-nodemcu drawing -
- diff --git a/how_to_archive_with_posix_tar_cpio_and_pax-1.png b/how_to_archive_with_posix_tar_cpio_and_pax-1.png new file mode 100644 index 0000000..68b100a Binary files /dev/null and b/how_to_archive_with_posix_tar_cpio_and_pax-1.png differ diff --git a/how_to_archive_with_posix_tar_cpio_and_pax-2.png b/how_to_archive_with_posix_tar_cpio_and_pax-2.png new file mode 100644 index 0000000..2f31089 Binary files /dev/null and b/how_to_archive_with_posix_tar_cpio_and_pax-2.png differ diff --git a/how_to_archive_with_posix_tar_cpio_and_pax-3.png b/how_to_archive_with_posix_tar_cpio_and_pax-3.png new file mode 100644 index 0000000..4a43e34 Binary files /dev/null and b/how_to_archive_with_posix_tar_cpio_and_pax-3.png differ diff --git a/how_to_archive_with_posix_tar_cpio_and_pax.html b/how_to_archive_with_posix_tar_cpio_and_pax.html new file mode 100644 index 0000000..998afb7 --- /dev/null +++ b/how_to_archive_with_posix_tar_cpio_and_pax.html @@ -0,0 +1,238 @@ + + + + + + + + + +How To Archive With POSIX tar, cpio and pax + + + +
+

How To Archive With POSIX tar, cpio and pax

+

Published on 2020-07-22 22:30:00+02:00 +

The usual answer to archive anything is tar. As you may see I +intentionally linked to the GNU Tar. If you are a *BSD user then you use some other implementation. Both of them follow +and extend POSIX'es standard for tar utility. Or so you would think. +

Right now there is no POSIX tar utility. It has been marked as legacy +already in 1997 and disappeared from the +standard soon after. It's place took a behemoth called +pax. The name gets even funnier when +you consider the rationale and the size of this thing. But pax didn't came from just tar. There was one more influencer +in here called cpio. You may know this one +if you ever tinkered with RPM packages or initramfs. +

In other words we have three utilities on today's table: tar, cpio and pax. According to +Debian's popularity contest the frequency of each being installed is in +the exact same order, with tar being at 8th place overall, cpio at 52nd, and pax at 6089th. I can't just talk about the +least popular one, so I'll explain shortly how to use each of them in your usual Linux distribution while keeping in +mind what POSIX had to tell us back in the day. + +

tar

+

Like I've already mentioned tarballs are the most popular. Not only that, they are commonly described as the easiest +to use, although the interface is something that you can find jokes about. All operations on tarballs are handled via +single tar utility.

+box +

Let's go through three basic operations: create an archive, list out the content, and extract it. Tar expects to have +first argument to match this regular expression: [rxtuc][vwfblmo]*. The first part is function, +and the second is a modifier. I'll focus only on those necessary to accomplish before-mentioned tasks. +

To create an archive you:

+
+$ tar cf ../archive.tar a_file a_directory
+
+

This will create an archive that will be located in parent directory of current working directory, and will contain +a_file and recursively a_directory. Let's map every part of the command for clarity:

+
+
tar
Call tar +
c
Create an archive +
f
Use first argument after cf as the path to the archive +
../archive.tar
Path to the archive (without f it would be treated as another file to + include in the archive) +
a_file a_directory
Files to include in the archives +
+

Now that you have an archive, you can see it's content:

+
+$ tar tf ../archive.tar
+a_file
+a_directory/
+a_directory/another_file
+
+

As you have probably guessed t function is used to write the names of files that are in the archive. +f works exactly the same way: first argument after tf is meant to point to the archive file. +

To extract everything from the archive you:

+
+$ tar xf ../archive.tar
+
+

Or add more arguments to extract selected files:

+
+$ tar xf ../archive.tar a_file
+
+

This one will extract only a_file from the archive. +

That's pretty much it about tar. The are two more functions: r that adds new file to existing archive, +and u that first tries to update the file in archive if it exists and if it doesn't then it adds it. Note, +that the usual compression options are not available in POSIX, they are an extension. + +

cpio

+

Heading off from the usual routes we encounter cpio. It's a more frequent sight than pax, but it still is quite niche +compared to tar's omnipresence. Frankly, I like this one the most because of the way it handles input of file lists. +Sadly, this also makes it slightly bothersome to use. +

Now, now, cpio operates in three modes: copy-out, copy-in and pass-through. Our goals are +still the same: to create an archive, list files inside, and extract it somewhere else and for that we'll only need the +first two modes. +

To create an archive, use the copy-out mode, as in: copy to the standard output:

+
+$ find a_file a_directory | cpio -o >../archive.cpio
+
+

This instant you probably noticed that cpio doesn't accept files as arguments. In copy-out mode it expects list of +files in standard input, and it will return the formatted archive through standard output. See a somehow step-by-step +explanation:

+
+
find a_file a_directory |
List files, directories and their content from arguments and pipe the + output to the next command +
cpio
Call cpio (duh!) +
-o
Use copy-out mode +
>../archive.cpio
Redirect standard output of cpio to a file +
+

You now have an archive file called archive.cpio in parent directory. To see its content type in:

+
+$ cpio -it <../archive.cpio
+a_file
+a_directory
+a_directory/another_file
+1 block
+
+

Nice! What's left is extraction. You do it with copy-in mode like this:

+
+$ cpio -i <../archive.cpio
+1 block
+
+

Huh? What's that? Listing files and extracting both use copy-in mode? That's right. Like "copy-out" means "copy to +standard output", "copy-in" can be understood as "copy from standard input". The t option prohibits any +files to be written or created by cpio, nonetheless archive is read from standard input and then translated to list of +files in standard output. Some extended implementations let you use t directly as sole option and imply the +copy-in mode. +

You can also use patterns when extracting to select files:

+
+$ cpio -i a_file <../archive.cpio
+1 block
+
+

You can copy nested files if you use d option:

+
+$ cpio -id a_directory/another_file <../archive.cpio
+1 block
+
+

This option tells cpio that it's allowed to create directories whenever it is necessary.

+pass-through +

Bonus! Pass-through mode can be used to copy files listed in standard input to specified directory. It doesn't create +an archive at all.

+
+$ ls ../destination
+$ ls
+a_directory  a_file
+$ find a_file a_directory | cpio -p ../destination
+0 blocks
+$ ls ../destination
+a_directory  a_file
+
+ +

pax

+

Finally, at the destination! This one lives up to the name of this post as it's still part of POSIX. The fun part is +that you probably don't even have it installed, but don't worry, I didn't have it until like two days ago. It truly +feels like a compromise forced on you and your siblings by your parents. Jokes aside, I actually started to like it, +bulky but kind of cute. +

Anyway, let's see what this coffee machine can do for us; same goals as previously. This will be confusing, because +this utility is a compromise, and so it supports both usage styles: tar-like and cpio-like. +

To create an archive you can use either:

+ +
+$ pax -wf ../archive.pax a_directory a_file
+$ find a_file a_directory | pax -wd >../archive.pax
+$ find a_file a_directory | pax -wdf ../archive.pax
+
+ +

They are equivalent. You can mix the style as much as you want, as long as it doesn't become mess it's quite handy. +As for what option does what:

+ +
+
-w
Indicates that pax will act in write mode (tar's c and cpio's -o) +
f ../archive.pax
Argument after f is the path to the archive; note that it behaves + slightly different compared to tar, it always takes next argument instead of first path that appears after flags. It + means you can't put any options between -f and the path. +
a_directory a_file +
find a_file a_directory |
Both of these accomplish the same goal of letting know pax + what files should be in archive. They are mutually exclusive! If there is at least one argument pointing to a file, + then standard input is not supposed to be read. +
d
This one is used to prevent recursively adding files that are in a directory, so that the + behaviour is the same as in cpio: +
+$ find a_file a_directory | pax -wvf ../archive.pax
+a_directory
+a_directory/another_file
+a_directory/another_file
+a_file
+pax: ustar vol 1, 4 files, 0 bytes read, 10240 bytes written.
+$ find a_directory a_file | pax -wvdf ../archive.pax
+a_directory
+a_directory/another_file
+a_file
+pax: ustar vol 1, 3 files, 0 bytes read, 10240 bytes written.
+
+
+ +

The v option is used to increase verbosity of the "error" output. You can find similar functionality in +most of command line utilities, including tar and cpio. +

To list files that are in archive you can also use both styles:

+
+$ pax <../archive.pax
+a_directory
+a_directory/another_file
+a_file
+$ pax -f ../archive.pax
+a_directory
+a_directory/another_file
+a_file
+
+

Yes, that's the default behaviour of pax and you don't need to specify any argument (in case of cpio-like style). +Sweet, isn't it? +

To extract the archive use one of:

+
+$ pax -r <../archive.pax
+$ pax -rf ../archive.pax
+
+

For selecting files to extract use the usual patterns:

+
+$ pax -r a_file -f ../archive.pax
+$ pax -r a_directory/another_file <../archive.pax
+
+

That's all of the most basic use case. There's more, for instance pax supports mode similar to the pass-through mode +we already know from the cpio. But there is something more important to mention about pax. It's supposed to easily +support various different formats. +

POSIX tells that pax should support: pax, cpio and ustar formats. I installed GNU pax and it seems to support: ar, +bcpio, cpio, sv4cpio, sc4crc, tar and ustar. The default format for my installation is ustar as you have probably +noticed in verbose output in one of the examples above. Pax format is extension for ustar, that's most likely the reason +it's usually omitted. +

You can select format with -x option, for supported formats please refer to your manual. Also note that +explicitly specifying format should be only needed when writing an archive. When reading pax can identify archive's +format efficiently:

+
+$ find a_file a_directory | cpio -o >../archive.cpio
+$ pax -vf ../archive.cpio
+-rw-rw-r--  1 ignore   ignore    0 Jul 22 22:30 a_file
+drwxrwxr-x  2 ignore   ignore    0 Jul 22 22:30 a_directory
+-rw-rw-r--  1 ignore   ignore    0 Jul 22 22:30 a_directory/another_file
+pax: bcpio vol 1, 3 files, 512 bytes read, 0 bytes written.
+
+ +

Final thoughts

+

Now then, it's time to finally wrap it all up. There is nothing left to say but remember to always check your manual, +all of those utilities have various implementations that are compliant to POSIX in various degrees. Don't be naive and +don't get tricked by them. I find pax the most reliable of them as its "novelty" and the interface that was quite +"modern" from the start resulted in decently compliant implementations. Moreover, it includes nice things one may know +from both cpio and tar. Find a moment to check it out! +

Let's pretend that ar doesn't exist. +Thank you.

+boo! +
+ diff --git a/how_to_create_templates_with_shell_cat_and_envsubst-1.png b/how_to_create_templates_with_shell_cat_and_envsubst-1.png new file mode 100644 index 0000000..d390eac Binary files /dev/null and b/how_to_create_templates_with_shell_cat_and_envsubst-1.png differ diff --git a/how_to_create_templates_with_shell_cat_and_envsubst.html b/how_to_create_templates_with_shell_cat_and_envsubst.html new file mode 100644 index 0000000..99a9c7c --- /dev/null +++ b/how_to_create_templates_with_shell_cat_and_envsubst.html @@ -0,0 +1,72 @@ + + + + + + + + + +How To Create Templates With Shell, cat and envsubst + + + +
+

How To Create Templates With Shell, cat and envsubst

+

Published on 2020-07-14 20:26:00+02:00 +

Now something trivial and fun. Creating templates of documents or configurations in shell. There are two reasons why +I considered doing that instead of using some more verbose utilities. First off, availability - I have POSIX compliant +shell pretty much everywhere I go. It's a big part of my usual environment and because of that I'm used to it. That's +the second reason - frequency I use it with. It's good to mention that it's one of more verbose utilities out there. +

Let's start with plain shell. I actually use this method to serve the blog posts via fcgiwrap. I use it with a single +file, but it can accept external files. However those files won't have their content expanded in any way. If you keep it +in a one script file then it's possible. Basically, I'm talking about cat:

+
+#!/bin/sh
+cat /dev/fd/3 $@ /dev/fd/4 3<<BEFORE 4<<AFTER
+<!doctype html>
+<html lang="en">
+BEFORE
+<script src=""></script>
+AFTER
+
+actual cat +

What you see here is a combination of +heredoc and plain +redirection. It's done like this to avoid calling cat more than once. The script simply concatenates BEFORE +heredoc with any arguments passed to the script, and finally with AFTER heredoc. The output is similar to what you see +in source of this page. The template is trimmed for the sake of the example. +

Now, now. Heredoc can easily expand variables but if we were to cat before.inl $@ after.inl it wouldn't +work. For that I use +envsubst. +Consider file called template.conf: +

+server {
+	listen 80;
+	server_name $DOMAIN$ALIASES;
+	root /srv/http/$DOMAIN/public;
+}
+
+

It could be wrapped in cat and heredoc, but for me it's not desired. Let's say I want my configuration templates to +be exactly that not executable scripts. There are three different solutions that I've heard: eval, +sed and envsubst. Eval can lead to dangerous situations and minor problems with +whitespace. Sed is what I believe is the most typical and straight-forward solution. As for the last one, +it goes like this:

+
+$ export DOMAIN=example.tld
+$ export ALIASES=' www.example.tld'
+$ envsubst '$DOMAIN$ALIASES' <template.conf
+server {
+	listen 80;
+	server_name example.tld www.example.tld;
+	root /srv/http/example.tld/public;
+}
+
+

First, you set the variables (if they are not yet there). Then call the envsubst with a single variable +and redirect the template file into it. This single variable is called SHELL-FORMAT. It restricts the variables +that will be substituted in the input. It's completely optional, but if you have some $server_name you +don't want to substitute, then it's kind of useful. It actually doesn't have any format, you just need to reference the +variables in it: '$DOMAIN$ALIASES', '$DOMAIN $ALIASES', and '$DOMAIN,$ALIASES' +all work the same. +

+ diff --git a/how_to_flash_lolin_nodemcu_v3-1.png b/how_to_flash_lolin_nodemcu_v3-1.png new file mode 100644 index 0000000..daceb97 Binary files /dev/null and b/how_to_flash_lolin_nodemcu_v3-1.png differ diff --git a/how_to_flash_lolin_nodemcu_v3.html b/how_to_flash_lolin_nodemcu_v3.html new file mode 100644 index 0000000..39760d0 --- /dev/null +++ b/how_to_flash_lolin_nodemcu_v3.html @@ -0,0 +1,46 @@ + + + + + + + + +How To Flash LOLin NodeMCU v3 + + + +
+

How To Flash LOLin NodeMCU v3

+

Published on 2020-06-29 17:58:00+02:00 +

Republishing old content. This is from when I bought and flashed my first NodeMCU clone (perhaps "loose +implementation"?) of NodeMCU v3 called LOLin v3. +

This little board is using CH304G TTL to USB converter, so if you happen to work on Windows you must make sure you +have the driver installed beforehand. Mentioned chip is quite common, and I've seen it on various Arduino, so there is +a high chance that you already have it. +

Without further notes, here is the specification:

+ +
Baud rateFlash modeFlash sizeFlash frequency +
9600 QIO 4MiB 40MHz +
+

You will need a combined firmware binary. If you are compiling it by yourself, I suggest using one of the development +branches. If you don't have environment set up there are online building services available e.g. +NodeMCU custom builds. +

To write the firmware use e.g. esptool:

+
+$ esptool.py --port /dev/ttyUSB0 --baud 9600 erase_flash
+$ esptool.py --port /dev/ttyUSB0 --baud 9600 write_flash \
+  --flash_mode qio --flash_size 4MB --flash_freq 40m \
+  0x00000 combined-firmware-file.bin
+
+

To test it you can connect to it with e.g. screen:

+
+$ screen /dev/ttyUSB0 115200
+
+

Or use your favourite terminal emulator:

+
+$ st -l /dev/ttyUSB0 115200
+
+nodemcu drawing +
+ diff --git a/how_to_organize_your_lua_project-1.png b/how_to_organize_your_lua_project-1.png new file mode 100644 index 0000000..aa71459 Binary files /dev/null and b/how_to_organize_your_lua_project-1.png differ diff --git a/how_to_organize_your_lua_project-2.png b/how_to_organize_your_lua_project-2.png new file mode 100644 index 0000000..0917288 Binary files /dev/null and b/how_to_organize_your_lua_project-2.png differ diff --git a/how_to_organize_your_lua_project.html b/how_to_organize_your_lua_project.html new file mode 100644 index 0000000..013f712 --- /dev/null +++ b/how_to_organize_your_lua_project.html @@ -0,0 +1,239 @@ + + + + + + + + + +How To Organize Your Lua Project + + + +
+

How To Organize Your Lua Project

+

Published on 2021-01-07 15:45:00+01:00 +

From time to time I hear complaints about how Lua handles modules. Here and there I see and even answer myself +questions regarding require and adjusting the paths in package to allow some desired +behaviour, with the most prominent issue of relative imports that always work. +

Before we hop into the explanation of how to organize files in your Lua projects, let's talk about default importing +mechanism in Lua: require.

+ +lua hierarchy + +

How require handles paths

+

Both package and require are surprisingly interesting tools. At first glance they are +simple. When you look into them, they are still understandable while gaining some complexity that doesn't reach +unnecessary extremes. They are elegant. +

They use a mechanism to find desired files called path resolution or usually simply path. The main +component of is a sequence of patterns that may become a pathname, e.g.: +

+/usr/lib/lua/?.lua;/usr/lib/lua/?/init.lua
+
+

What does it tell us? First off, ? is going to be replaced by the argument that was provided to the +require. All dots will be replaced by an appropriate path separator so that: a.b.c will +become a/b/c in *nix systems. So, for call like require "a.b.c", out path will look +like this: +

+/usr/lib/lua/a/b/c.lua;/usr/lib/lua/a/b/c/init.lua
+
+

Now, each of these paths are tried and the first one that actually exists in the system will be used. If none of them +match an existing file, the import fails. Simple as that. +

The path that is used in resolution is set in package.path. You can modify it in Lua, but it is +intrusive and may depend on a single entry point. Generally, if you plan to release your project as a module for people +to use, I encourage you to avoid modifying anything global. And that's global. Anyway, package.path doesn't +appear out of nowhere - it is populated by one of: +

    +
  1. Environmental variable LUA_PATH_x_x, where x_x is version such as 5_4 +
  2. Environmental variable LUA_PATH +
  3. Default as defined in luaconf.h +
+

Interestingly, if two separators ;; show up in the environmental variable path, they will be replaced by +the default path. Meaning /path/to/project/?.lua;; works as prepending your custom path to the default one. +

Of course, there is way more to it than just this i.a.: requiring modules written in C, searchers or preloads. +However, in our case this knowledge will suffice. +

If you are curious how exactly path is loaded be sure to check out +setpath. + +

Endgame

+

To prepare for development, we need to know where we are heading. First step is to consider the execution +environment. Of course, this and packaging are journeys on their own, so let's just look at two common examples: an +application that uses some framework that uses Lua (e.g. a game made in LÖVE) and a +standalone module for others to use. +

In the first case, it's the duty of the framework to configure the path properly and inform you through the +documentation about it. Paths in LÖVE use their own file hierarchy that is managed by love.filesystem and +by default contains both the game's source (directory or the mounted .love archive) and the save directory. +This means that the structure in your source files is directly reflected in the calls to require, so that +require "module.submodule" will always try game/module/submodule.lua, no matter how you run +the game. This case usually doesn't involve any additional environment configuration for the development stage. +

In the second case, your project will end up in an already configured environment and will need to fit in. The +installation of the package usually involves copying your files to the directory that is already included in the path, +so that no further configuration is needed for the execution (at least regarding the path). You can assume, that the +successful installation will make your modules available in the way you want them. +

This doesn't happen in the development stage, when you rarely install your package, and most certainly you don't +install it each time you want to test it. This means, that you need to adjust the path so that your modules appear in it +as if they were installed in the system. The principle of minimizing the intrusiveness remains, so the best option is to +use the environmental variables to prepare for development. If you run your application or any tool in such environment, +then Lua will have access to your modules no matter where it is run. Additionally, it will be consistent with the +target environment and won't need any additional hacks. + +

Development environment

+

All this talk comes down to: set LUA_PATH in your development environment so that it includes your +project files even if they are not installed in system. A simple approach is to source following in each session: +

+export PROJECT=/path/to/project
+export LUA_PATH="$PROJECT/?.lua;$PROJECT/?/init.lua;;"
+
+

Note the double semicolon that will get replaced by default path, so that other modules that are already installed +are also available. +

Let's try it out: +

+$ source env.sh
+$ find .
+./env.sh
+./modulea/submodule.lua
+./modulea/init.lua
+./moduleb.lua
+$ cd modulea
+$ lua
+Lua 5.4.2
+> require "moduleb"
+table: 0x561e4b72fb20
+> require "modulea"
+table: 0x561e4b73aa40
+> require "modulea.submodule"
+table: 0x561e4b743030
+
+

As you can see, despite being in the subdirectory, you can still use modules with their fully qualified names that +will remain the same once you install the package. Note, that you could require "init" or require +"submodule" in this case, but I strongly recommend against it. Remain specific, follow the rules and pretend that +you use an installed package from an unknown working directory. Don't depend on current working +directory as it is not always the same. Using full names that consider the path setup guarantees +results.

+ +a random whale + +

Organizing your files

+

Finally, this is what we're waiting for. Assume you have a directory that is a parent of all of your project files. +We'll call it a project root. Usually, this is also root directory for your version control system, be +it git or anything else, and for other tools such as building systems or even entire IDEs. +

Because it is such a central place to the project, I usually just go ahead and prepend it to LUA_PATH in +the very same way as in the section above: +

+export PROJECT=/path/to/project
+export LUA_PATH="$PROJECT/?.lua;$PROJECT/?/init.lua;;"
+
+

Just like previously, any Lua file that will be descendent of the root will be accessible to us through +require. But what is that init.lua? +

It's there to create a way to improve hierarchical structure of your project - to allow splitting bigger modules into +smaller parts (or even submodules that could be included on their own), so that the module doesn't grow into a single +millions-lines-long file. In simpler words: you can create a directory named after module and put +init.lua file there and it will act just like a sole module.lua in root. +

You could also create a directory named after module and module.lua file in root at the same time, but this +way you will have two entries per module in the root instead of just one. +

Additionally, you can then put any module-related files into that directory. You can also use init.lua as +a simple wrapper that calls require for each of its submodules and returns a table with them. +

Consider a verbose example: +

+$ find .
+./conf.lua
+./env.sh
+./main.lua
+./persistence/init.lua
+./persistence/tests.lua
+./version.lua
+./wave/init.lua
+./wave/sawtooth.lua
+./wave/sine.lua
+./wave/square.lua
+$ cat wave/init.lua
+-- This is a wrapper example.
+return {
+	sawtooth = require "wave.sawtooth",
+	sine = require "wave.sine",
+	square = require "wave.square",
+}
+$ cat persistence/init.lua
+-- This is a normal module example.
+return {}
+$ cat persistance/tests.lua
+-- This is a script that tests an example module.
+local p = require "persistence"
+assert(type(p) == "table")
+$ cat main.lua
+-- This is an example main of love application.
+local persistence = require "persistence"
+local wave = require "wave"
+$ cat version.lua
+-- This is an example module that acts as version string of the application.
+return "1.0.0"
+
+

Now, this is a mash-up of everything we've discussed. Despite it pretending to be LÖVE application it has +env.sh. Why? The reason is simple: the persistence and wave modules are not meant to be distributed +alone, and they won't ever appear in path of any other environment than LÖVE's. But LÖVE is not the only execution +environment in here: persistence/tests.lua is also meant to be executed. Possibly alone through Lua interpreter. +To allow it env.sh is present and used. +

Let's have another example of a simple module meant for installation: +

+$ find .
+./env.sh
+./hello/Class.lua
+./hello/init.lua
+./hello/tests.lua
+./hello/version.lua
+./LICENSE
+./Makefile
+./README
+$ cat Makefile
+PREFIX?=/usr/local/lib/lua/5.4
+all:
+	@echo Nothing to be done
+install:
+	cp -r hello $(PREFIX)
+uninstall:
+	rm -fd $(PREFIX)/hello/* $(PREFIX)/hello
+
+

As you can see Makefile in this example has targets for installation and removal of the package. The structure +again is simple. Root works as part of the resolution path and so our module is placed in it's own directory named after +it. +

The last example is a project of a single file module: +

+$ find .
+./env.sh
+./LICENSE
+./object.lua
+./README
+
+

Yes, it's that simple. +

Now, having env.sh in every single project might get bothersome, so I usually use a shell function for +managing them, similarly to what Python's venv does or LuaRocks' env. Speaking of, +LuaRocks is yet another interesting story to be told. + +

Summary

+ + +

Alternatives

+

This is just one of the ways to handle structuring your Lua project. It's based on simple rules but has broad usage. +One tempting alternative is this little snippet: +

+local parents = (...):match "(.-)[^%.]+$"
+require(parents .. "sibling")
+
+

Another already mentioned alternatives is adjusting package.path directly in Lua. However, I decided to +skip it due to it's intrusiveness. +

All in all, Lua is extremely customizable and adjustable. I would be surprised if these three would be the only ways +to organize projects in Lua. + +

+ + diff --git a/index.html b/index.html index 5a9b5e1..2612ae4 100644 --- a/index.html +++ b/index.html @@ -40,14 +40,15 @@ completely discard the concept of a keyboard.
  • Miscellaneous Informative
  • Rants @@ -68,6 +69,9 @@ completely discard the concept of a keyboard.

    News

    +Renamed most of the guides to include "How To" in their name to make them more visible. I'm thinking about doing another +round of website restructurization. +

    Published Deconstructing Web Browsers, summary of now-removed Plumbing Your Own Browser and Integrating Browser into Your Environment.

    diff --git a/organizing_your_lua_project-1.png b/organizing_your_lua_project-1.png deleted file mode 100644 index aa71459..0000000 Binary files a/organizing_your_lua_project-1.png and /dev/null differ diff --git a/organizing_your_lua_project-2.png b/organizing_your_lua_project-2.png deleted file mode 100644 index 0917288..0000000 Binary files a/organizing_your_lua_project-2.png and /dev/null differ diff --git a/organizing_your_lua_project.html b/organizing_your_lua_project.html deleted file mode 100644 index d6bae4a..0000000 --- a/organizing_your_lua_project.html +++ /dev/null @@ -1,239 +0,0 @@ - - - - - - - - - -Organizing Your Lua Project - -

    - -
    -

    Organizing Your Lua Project

    -

    Published on 2021-01-07 15:45:00+01:00 -

    From time to time I hear complaints about how Lua handles modules. Here and there I see and even answer myself -questions regarding require and adjusting the paths in package to allow some desired -behaviour, with the most prominent issue of relative imports that always work. -

    Before we hop into the explanation of how to organize files in your Lua projects, let's talk about default importing -mechanism in Lua: require.

    - -lua hierarchy - -

    How require handles paths

    -

    Both package and require are surprisingly interesting tools. At first glance they are -simple. When you look into them, they are still understandable while gaining some complexity that doesn't reach -unnecessary extremes. They are elegant. -

    They use a mechanism to find desired files called path resolution or usually simply path. The main -component of is a sequence of patterns that may become a pathname, e.g.: -

    -/usr/lib/lua/?.lua;/usr/lib/lua/?/init.lua
-
    -

    What does it tell us? First off, ? is going to be replaced by the argument that was provided to the -require. All dots will be replaced by an appropriate path separator so that: a.b.c will -become a/b/c in *nix systems. So, for call like require "a.b.c", out path will look -like this: -

    -/usr/lib/lua/a/b/c.lua;/usr/lib/lua/a/b/c/init.lua
-
    -

    Now, each of these paths are tried and the first one that actually exists in the system will be used. If none of them -match an existing file, the import fails. Simple as that. -

    The path that is used in resolution is set in package.path. You can modify it in Lua, but it is -intrusive and may depend on a single entry point. Generally, if you plan to release your project as a module for people -to use, I encourage you to avoid modifying anything global. And that's global. Anyway, package.path doesn't -appear out of nowhere - it is populated by one of: -

      -
    1. Environmental variable LUA_PATH_x_x, where x_x is version such as 5_4 -
    2. Environmental variable LUA_PATH -
    3. Default as defined in luaconf.h -
    -

    Interestingly, if two separators ;; show up in the environmental variable path, they will be replaced by -the default path. Meaning /path/to/project/?.lua;; works as prepending your custom path to the default one. -

    Of course, there is way more to it than just this i.a.: requiring modules written in C, searchers or preloads. -However, in our case this knowledge will suffice. -

    If you are curious how exactly path is loaded be sure to check out -setpath. - -

    Endgame

    -

    To prepare for development, we need to know where we are heading. First step is to consider the execution -environment. Of course, this and packaging are journeys on their own, so let's just look at two common examples: an -application that uses some framework that uses Lua (e.g. a game made in LÖVE) and a -standalone module for others to use. -

    In the first case, it's the duty of the framework to configure the path properly and inform you through the -documentation about it. Paths in LÖVE use their own file hierarchy that is managed by love.filesystem and -by default contains both the game's source (directory or the mounted .love archive) and the save directory. -This means that the structure in your source files is directly reflected in the calls to require, so that -require "module.submodule" will always try game/module/submodule.lua, no matter how you run -the game. This case usually doesn't involve any additional environment configuration for the development stage. -

    In the second case, your project will end up in an already configured environment and will need to fit in. The -installation of the package usually involves copying your files to the directory that is already included in the path, -so that no further configuration is needed for the execution (at least regarding the path). You can assume, that the -successful installation will make your modules available in the way you want them. -

    This doesn't happen in the development stage, when you rarely install your package, and most certainly you don't -install it each time you want to test it. This means, that you need to adjust the path so that your modules appear in it -as if they were installed in the system. The principle of minimizing the intrusiveness remains, so the best option is to -use the environmental variables to prepare for development. If you run your application or any tool in such environment, -then Lua will have access to your modules no matter where it is run. Additionally, it will be consistent with the -target environment and won't need any additional hacks. - -

    Development environment

    -

    All this talk comes down to: set LUA_PATH in your development environment so that it includes your -project files even if they are not installed in system. A simple approach is to source following in each session: -

    -export PROJECT=/path/to/project
-export LUA_PATH="$PROJECT/?.lua;$PROJECT/?/init.lua;;"
-
    -

    Note the double semicolon that will get replaced by default path, so that other modules that are already installed -are also available. -

    Let's try it out: -

    -$ source env.sh
-$ find .
-./env.sh
-./modulea/submodule.lua
-./modulea/init.lua
-./moduleb.lua
-$ cd modulea
-$ lua
-Lua 5.4.2
-> require "moduleb"
-table: 0x561e4b72fb20
-> require "modulea"
-table: 0x561e4b73aa40
-> require "modulea.submodule"
-table: 0x561e4b743030
-
    -

    As you can see, despite being in the subdirectory, you can still use modules with their fully qualified names that -will remain the same once you install the package. Note, that you could require "init" or require -"submodule" in this case, but I strongly recommend against it. Remain specific, follow the rules and pretend that -you use an installed package from an unknown working directory. Don't depend on current working -directory as it is not always the same. Using full names that consider the path setup guarantees -results.

    - -a random whale - -

    Organizing your files

    -

    Finally, this is what we're waiting for. Assume you have a directory that is a parent of all of your project files. -We'll call it a project root. Usually, this is also root directory for your version control system, be -it git or anything else, and for other tools such as building systems or even entire IDEs. -

    Because it is such a central place to the project, I usually just go ahead and prepend it to LUA_PATH in -the very same way as in the section above: -

    -export PROJECT=/path/to/project
-export LUA_PATH="$PROJECT/?.lua;$PROJECT/?/init.lua;;"
-
    -

    Just like previously, any Lua file that will be descendent of the root will be accessible to us through -require. But what is that init.lua? -

    It's there to create a way to improve hierarchical structure of your project - to allow splitting bigger modules into -smaller parts (or even submodules that could be included on their own), so that the module doesn't grow into a single -millions-lines-long file. In simpler words: you can create a directory named after module and put -init.lua file there and it will act just like a sole module.lua in root. -

    You could also create a directory named after module and module.lua file in root at the same time, but this -way you will have two entries per module in the root instead of just one. -

    Additionally, you can then put any module-related files into that directory. You can also use init.lua as -a simple wrapper that calls require for each of its submodules and returns a table with them. -

    Consider a verbose example: -

    -$ find .
-./conf.lua
-./env.sh
-./main.lua
-./persistence/init.lua
-./persistence/tests.lua
-./version.lua
-./wave/init.lua
-./wave/sawtooth.lua
-./wave/sine.lua
-./wave/square.lua
-$ cat wave/init.lua
--- This is a wrapper example.
-return {
-	sawtooth = require "wave.sawtooth",
-	sine = require "wave.sine",
-	square = require "wave.square",
-}
-$ cat persistence/init.lua
--- This is a normal module example.
-return {}
-$ cat persistance/tests.lua
--- This is a script that tests an example module.
-local p = require "persistence"
-assert(type(p) == "table")
-$ cat main.lua
--- This is an example main of love application.
-local persistence = require "persistence"
-local wave = require "wave"
-$ cat version.lua
--- This is an example module that acts as version string of the application.
-return "1.0.0"
-
    -

    Now, this is a mash-up of everything we've discussed. Despite it pretending to be LÖVE application it has -env.sh. Why? The reason is simple: the persistence and wave modules are not meant to be distributed -alone, and they won't ever appear in path of any other environment than LÖVE's. But LÖVE is not the only execution -environment in here: persistence/tests.lua is also meant to be executed. Possibly alone through Lua interpreter. -To allow it env.sh is present and used. -

    Let's have another example of a simple module meant for installation: -

    -$ find .
-./env.sh
-./hello/Class.lua
-./hello/init.lua
-./hello/tests.lua
-./hello/version.lua
-./LICENSE
-./Makefile
-./README
-$ cat Makefile
-PREFIX?=/usr/local/lib/lua/5.4
-all:
-	@echo Nothing to be done
-install:
-	cp -r hello $(PREFIX)
-uninstall:
-	rm -fd $(PREFIX)/hello/* $(PREFIX)/hello
-
    -

    As you can see Makefile in this example has targets for installation and removal of the package. The structure -again is simple. Root works as part of the resolution path and so our module is placed in it's own directory named after -it. -

    The last example is a project of a single file module: -

    -$ find .
-./env.sh
-./LICENSE
-./object.lua
-./README
-
    -

    Yes, it's that simple. -

    Now, having env.sh in every single project might get bothersome, so I usually use a shell function for -managing them, similarly to what Python's venv does or LuaRocks' env. Speaking of, -LuaRocks is yet another interesting story to be told. - -

    Summary

    -
      -
    • Consider the execution environment when preparing the development environment. -
    • Contain your project. Avoid changing globals if the modules may be used by someone else in an unknown - environment. -
    • Never depend on the current working directory being something specific. -
    • While developing include your project's root in LUA_PATH to influence require's - behaviour. -
    • Split modules into parts and use init.lua which is available by default to enclose the module into a - single directory. -
    - -

    Alternatives

    -

    This is just one of the ways to handle structuring your Lua project. It's based on simple rules but has broad usage. -One tempting alternative is this little snippet: -

    -local parents = (...):match "(.-)[^%.]+$"
-require(parents .. "sibling")
-
    -

    Another already mentioned alternatives is adjusting package.path directly in Lua. However, I decided to -skip it due to it's intrusiveness. -

    All in all, Lua is extremely customizable and adjustable. I would be surprised if these three would be the only ways -to organize projects in Lua. - -

    - - diff --git a/stupid_templating_with_shell_cat_and_envsubst-1.png b/stupid_templating_with_shell_cat_and_envsubst-1.png deleted file mode 100644 index d390eac..0000000 Binary files a/stupid_templating_with_shell_cat_and_envsubst-1.png and /dev/null differ diff --git a/stupid_templating_with_shell_cat_and_envsubst.html b/stupid_templating_with_shell_cat_and_envsubst.html deleted file mode 100644 index 7a84554..0000000 --- a/stupid_templating_with_shell_cat_and_envsubst.html +++ /dev/null @@ -1,72 +0,0 @@ - - - - - - - - - -Stupid Templating With Shell, cat and envsubst - - - -
    -

    Stupid Templating With Shell, cat and envsubst

    -

    Published on 2020-07-14 20:26:00+02:00 -

    Now something trivial and fun. Creating templates of documents or configurations in shell. There are two reasons why -I considered doing that instead of using some more verbose utilities. First off, availability - I have POSIX compliant -shell pretty much everywhere I go. It's a big part of my usual environment and because of that I'm used to it. That's -the second reason - frequency I use it with. It's good to mention that it's one of more verbose utilities out there. -

    Let's start with plain shell. I actually use this method to serve the blog posts via fcgiwrap. I use it with a single -file, but it can accept external files. However those files won't have their content expanded in any way. If you keep it -in a one script file then it's possible. Basically, I'm talking about cat:

    -
    -#!/bin/sh
-cat /dev/fd/3 $@ /dev/fd/4 3<<BEFORE 4<<AFTER
-<!doctype html>
-<html lang="en">
-BEFORE
-<script src=""></script>
-AFTER
-
    -actual cat -

    What you see here is a combination of -heredoc and plain -redirection. It's done like this to avoid calling cat more than once. The script simply concatenates BEFORE -heredoc with any arguments passed to the script, and finally with AFTER heredoc. The output is similar to what you see -in source of this page. The template is trimmed for the sake of the example. -

    Now, now. Heredoc can easily expand variables but if we were to cat before.inl $@ after.inl it wouldn't -work. For that I use -envsubst. -Consider file called template.conf: -

    -server {
-	listen 80;
-	server_name $DOMAIN$ALIASES;
-	root /srv/http/$DOMAIN/public;
-}
-
    -

    It could be wrapped in cat and heredoc, but for me it's not desired. Let's say I want my configuration templates to -be exactly that not executable scripts. There are three different solutions that I've heard: eval, -sed and envsubst. Eval can lead to dangerous situations and minor problems with -whitespace. Sed is what I believe is the most typical and straight-forward solution. As for the last one, -it goes like this:

    -
    -$ export DOMAIN=example.tld
-$ export ALIASES=' www.example.tld'
-$ envsubst '$DOMAIN$ALIASES' <template.conf
-server {
-	listen 80;
-	server_name example.tld www.example.tld;
-	root /srv/http/example.tld/public;
-}
-
    -

    First, you set the variables (if they are not yet there). Then call the envsubst with a single variable -and redirect the template file into it. This single variable is called SHELL-FORMAT. It restricts the variables -that will be substituted in the input. It's completely optional, but if you have some $server_name you -don't want to substitute, then it's kind of useful. It actually doesn't have any format, you just need to reference the -variables in it: '$DOMAIN$ALIASES', '$DOMAIN $ALIASES', and '$DOMAIN,$ALIASES' -all work the same. -

    - -- cgit v1.1