diff options
| author | Aki <please@ignore.pl> | 2021-08-14 20:03:58 +0200 |
|---|---|---|
| committer | Aki <please@ignore.pl> | 2021-08-14 20:03:58 +0200 |
| commit | 330f5572cee1a50658b6944dc43a304c7b4aa07c (patch) | |
| tree | 699ea31757d1d8a97bfdc7e69e01257f17ff4dae | |
| parent | b4bb22f06c128fcd925bf9014a03ca19da0e10ff (diff) | |
| download | ignore.pl-330f5572cee1a50658b6944dc43a304c7b4aa07c.zip ignore.pl-330f5572cee1a50658b6944dc43a304c7b4aa07c.tar.gz ignore.pl-330f5572cee1a50658b6944dc43a304c7b4aa07c.tar.bz2 | |
Published compress in POSIX guide
| -rw-r--r-- | how_to_compress_files_in_posix-1.png | bin | 0 -> 1765 bytes | |||
| -rw-r--r-- | how_to_compress_files_in_posix-2.png | bin | 0 -> 1810 bytes | |||
| -rw-r--r-- | how_to_compress_files_in_posix.html | 117 | ||||
| -rw-r--r-- | index.html | 3 |
4 files changed, 120 insertions, 0 deletions
diff --git a/how_to_compress_files_in_posix-1.png b/how_to_compress_files_in_posix-1.png Binary files differnew file mode 100644 index 0000000..5cbf959 --- /dev/null +++ b/how_to_compress_files_in_posix-1.png diff --git a/how_to_compress_files_in_posix-2.png b/how_to_compress_files_in_posix-2.png Binary files differnew file mode 100644 index 0000000..e3d21c8 --- /dev/null +++ b/how_to_compress_files_in_posix-2.png diff --git a/how_to_compress_files_in_posix.html b/how_to_compress_files_in_posix.html new file mode 100644 index 0000000..2ef743f --- /dev/null +++ b/how_to_compress_files_in_posix.html @@ -0,0 +1,117 @@ +<!doctype html> +<html lang="en"> +<meta charset="utf-8"> +<meta name="viewport" content="width=device-width, initial-scale=1"> +<meta name="author" content="aki"> +<meta name="tags" content="POSIX, compression, archiving, tutorial, guide, howto"> +<link rel="icon" type="image/png" href="cylo.png"> +<link rel="stylesheet" href="style.css"> + +<title>How to Compress Files in POSIX</title> + +<nav><p><a href="https://ignore.pl">ignore.pl</a></p></nav> + +<article> +<h1>How to Compress Files in POSIX</h1> +<p class="subtitle">Published on 2021-08-14 19:48:00+02:00 +<p>This was quite an amusing one to read about. On one hand, the results kind of surprised me, but then, on the other... +What exactly did I expect? +<p>Anyway! How does one compress files in a POSIX-compliant system? +<p>By the power of the Hinchliffe's rule, I say: you don't. <i>Wait, what kind of tutorial is this</i>?</p> + + +<h2>The standard way</h2> +<img src="how_to_compress_files_in_posix-1.png" alt="the way"> +<p>POSIX defines three utilities for compression but let's focus on two of them: +<a href="https://pubs.opengroup.org/onlinepubs/9699919799/utilities/compress.html">compress</a> and +<a href="https://pubs.opengroup.org/onlinepubs/9699919799/utilities/uncompress.html">uncompress</a>. They have quite +descriptive names and are incredibly simple to use. Just give them names of the files to process and it will do the +work. Result will be stored in a <code>*.Z</code> file: + +<pre> +$ ls +archive.tar +$ <mark>compress archive.tar</mark> +$ ls +archive.tar.Z +$ <mark>uncompress archive.tar.Z</mark> +$ ls +archive.tar +</pre> + +<p>By default, the input file is replaced by the output. This can be avoided with <code>-c</code> option that redirects +the output to the standard output: + +<pre> +$ compress <mark>-c</mark> archive.tar <mark>>archive.tar.Z</mark> +$ uncompress <mark>-c</mark> archive.tar.Z <mark>>archive.tar.bak</mark> +$ ls +archive.tar archive.tar.bak archive.tar.Z +</pre> + +<p>And of course standard input can be used as well with the special filename: <code>-</code>. + +<p>So far it all looks good. That's because we're discussing here an imaginary implementation of these utilities the way +they are described by the POSIX standard. However, that's not the real world. + + +<h2>The actual way</h2> +<p>If you are just like me and come from more Linux background, then prepare for disappointment. If you are coming from +BSD, then I have great news for you: you can stop here, because your system actually implements the standard.</p> + +<img src="how_to_compress_files_in_posix-2.png" alt="the other way"> + +<p>Instead of <b>compress</b> most Linux distributions come with <b>gzip</b>(1). Usually, it is the +<a href="https://www.gnu.org/software/gzip/">GNU Gzip</a>. The reason for that is of course legal work and patenting +issues. The full reasoning is covered in <a href="https://www.gnu.org/philosophy/gif.html">No GIF Files</a>. However, +this is all in past because the LZW patents already expired. +<p>Let's put the story and reasons aside. What we have is an inability to conform to a standard due to legal reasons and +this inability became a standard. And so in Linux systems you will end up using <b>gzip</b> or <b>xz</b>(1), or +<b>bzip2</b>(1), or really anything else: + +<pre> +$ ls +archive.tar +$ <mark>gzip archive.tar</mark> +$ ls +archive.tar.gz +$ <mark>gzip -d archive.tar</mark> +$ ls +archive.tar +</pre> + +<p>You can replace <code>gzip</code> with any of the mentioned utilities - they have very similar interfaces. Not only +that, they are also partially compatible with the interface for <b>compress</b> defined by the POSIX standard. Each of +them has an additional <i>un</i> command (e.g., <code>gunzip</code>, <code>unxz</code>) that can be used instead of +<code>-d</code> option. If you feel adventurous you could try symlinking them (especially <b>gzip</b> since it +implements LZ77). +<p>This brings a question regarding formats compatibility, but it's a comparison big enough to have its own article. +<p>In the end, if you want to use POSIX <b>compress</b> in GNU/Linux - you don't. Unless... + + +<h2>The other way</h2> +<p>Unless you use <a href="https://github.com/vapier/ncompress">ncompress</a> which has both <b>compress</b>(1) and +<b>uncompress</b>(1). Even more, it inherits directly from the original implementation. But there is one thing you need +to know about it. +<p>It's bad. Yes, a detailed comparison of compression algorithms is yet another huge and interesting topic, but this +particular case really can be summed up in: it's bad. It's OK with text. At least, it implements the POSIX standard and +is most likely available in your distribution's repository. +<p>Here are results of compression of an arbitrary tarball with majority of source code and some resources, all done +with default options: + +<table> +<tr><td>Source<td>22M +<tr><td><b>bzip2</b><td>5.4M +<tr><td><b>gzip</b><td>5.9M +<tr><td><b>xz</b><td>2.6M +<tr><td><b>ncompress</b><td>9.1M +</table> + +<p>In other words, it's not bad, but it's staying behind (more) modern programs. This also could be an additional reason +for why it is not used or even installed by default in most Linux distributions. I didn't check BSD's implementation, +but I expect rather good results. +<p>The main takeaway from this article is that if you plan to write anything that is portable across POSIX-compliant or +semi-compliant systems, then you need to give compressing slightly more attention. + +</article> +<script src="https://stats.ignore.pl/track.js"></script>
\ No newline at end of file @@ -47,6 +47,7 @@ completely discard the concept of a keyboard. <li><a href="dear_imgui_and_love.html">Dear ImGui and LÖVE,</a> <li><a href="how_to_archive_with_posix_ar.html">How to Archive With POSIX ar</a> <li><a href="how_to_archive_with_posix_tar_cpio_and_pax.html">How to Archive With POSIX tar, cpio and pax</a> + <li><a href="how_to_compress_files_in_posix.html">How to Compress Files in POSIX</a> <li><a href="how_to_create_templates_with_shell_cat_and_envsubst.html">How to Create Templates With Shell, cat and envsubst</a> <li><a href="how_to_flash_lolin_nodemcu_v3.html">How to Flash LOLin NodeMCU v3</a> <li><a href="how_to_organize_your_lua_project.html">How to Organize Your Lua Project</a> @@ -66,6 +67,8 @@ completely discard the concept of a keyboard. <section id="news"> <h2>Short Updates</h2> <p><strong><time>2021-08-14</time></strong> +Published <a href="how_to_compress_files_in_posix.html">How to Compress Files in POSIX</a>. +<p><strong><time>2021-08-14</time></strong> Minor updates to tutorials including a new paragraph in <a href="how_to_organize_your_lua_project.html">Lua Project</a>. <p><strong><time>2021-08-11</time></strong> Minor updates to <a href="how_to_archive_with_posix_tar_cpio_and_pax.html">How to Archive With POSIX tar, cpio and pax</a>. |
