Zstandard

What Linux packages, kernels and half of HTTP use now. LZ77 again, with the matches coded by FSE, a table driven cousin of arithmetic coding, and the literals by Huffman. The decoder here reads all of RFC 8878: raw, RLE and compressed blocks, predefined, RLE, compressed and repeated tables, the three repeat offsets, the frame content size and the checksum.

decompress("zstd", Buffer.from("28b52ffd0468a500007068656c6c6f20776f726c642120680100e0994a31ba782b", "hex"));
// { bytes: "hello world! hello world! hello world! hello world!",
//   details: { window: 8388608, checksum: true, frames: 1 } }

That stream came out of zstd 1.5.7. 33 bytes for 51, and four of them are the checksum.

Read only

compress("zstd", …) throws zstd: this package reads Zstandard but does not write it. A zstd writer worth using is a long project, and zstd is one package install away on every system that has a terminal. Reading is the part a blob from somewhere else needs.

Frames

Frames in a row decompress into one output, the way zstd -d reads them, and details.frames counts them. Skippable frames, the ones that start 5x 2a 4d 18, get skipped. The checksum is the low 32 bits of XXH64 over the frame's output, checked when the frame header says it's there. A frame that needs a dictionary is an UnsupportedError naming the dictionary ID, since a dictionary is a file nobody handed over.