Containers
One compression, several wrappers
A container is the packaging around a compressed stream: a magic number so tools recognize it, a header with metadata, a checksum at the end. The compression inside doesn't change. gzip and zlib carry the exact same deflate data, they just wrap it differently.
So here a container is an option of its format, never a format of its own:
compress("deflate", data); // raw deflate, RFC 1951
compress("deflate", data, { container: "zlib" }); // RFC 1950
compress("deflate", data, { container: "gzip" }); // RFC 1952
The first container is the default. Asking for gzip as a format gets you an error that names deflate and the container, so nobody has to guess.
Every container
| Format | Container | Magic | Header | Check |
|---|---|---|---|---|
| deflate | raw | none | none | none |
| deflate | zlib | none, two header bytes with a check | method, window, level hint | Adler-32 |
| deflate | gzip | 1f 8b 08 | flags, mtime, OS, optional name and comment | CRC-32 and size |
| bzip2 | bzip2 | BZh and the block size digit | per block magic and CRC | CRC-32/BZIP2 per block and per stream |
| lzma | alone | none, the properties byte is usually 5d | properties, dictionary size, size | none |
| lzma | xz | fd 37 7a 58 5a 00 | stream flags, block headers with filters | CRC-32, CRC-64 or SHA-256 per block, CRC-32 everywhere else |
| zstd | zstd | 28 b5 2f fd | window, dictionary id, content size | XXH64, low 32 bits |
| brotli | brotli | none | window bits | none |
| lz4 | frame | 04 22 4d 18 | flags, block size, content size | XXH32 of the header, blocks and content |
| lz4 | legacy | 02 21 4c 18 | none | none |
| lzw | compress | 1f 9d | maximum code width, block mode | none |
What each one adds
gzip
It stores an optional file name and comment, a modification time and the OS. compress writes name and mtime when you pass them and OS 255, unknown, so the bytes don't depend on the machine. decompress reads every member, as gunzip does.
zlib
Two header bytes and an Adler-32. The second byte carries a level hint, which decompress reports as fastest, fast, default or best. A stream that needs a preset dictionary is refused with UnsupportedError.
xz
The elaborate one. Blocks, each with its own filter chain and check, an index of the blocks and a footer that points back at it. This package reads LZMA2 with the delta and x86 filters before it, and every check type it knows how to compute. It writes one block with LZMA2 and the check you pick, CRC-64 by default.
.lzma
alone here. It's what LZMA Utils wrote before xz: thirteen header bytes and the raw range coded stream. No checksum, so a damaged file decodes into garbage instead of an error. xz --format=lzma still writes it.
LZ4 frame
It checks itself three ways: a byte of XXH32 closes the header, and the frame can carry XXH32 after every block and after the content. compress writes the content checksum, like the lz4 command.
LZ4 legacy
It's the old lz4 -l format. Blocks of up to 8 MiB, no checksum, no end mark. Linux still boots kernels packed in it.