LZ4 (compression algorithm)

LZ4 is a lossless data compression algorithm optimized for fast compression and decompression. It belongs to the LZ77 family of byte-oriented compression schemes and offers a trade-off between speed and ratio: its compression ratio is typically worse than the similar LZO algorithm, which is in turn worse than algorithms such as DEFLATE, but its compression speed is similar to LZO and several times faster than DEFLATE, and its decompression is significantly faster than LZO. Unlike many common compressors, LZ4 uses only a dictionary-matching stage and does not add an entropy coding stage such as Huffman coding. Data is written as a series of sequences, each beginning with a one-byte token split into two 4-bit fields that give the number of literal bytes to copy and the length of the match to copy from already decoded output, followed by the literals and an offset back into the output buffer. The reference implementation in C is by Yann Collet under a BSD license, and LZ4 is used by Apache Hadoop, natively in the Linux kernel since version 3.11, and in several ZFS implementations.