LZ78 Compression in Low Main Memory Space

We present an LZ78 compression tool that compressed a text of length n over alphabet sigma, whose output is z integers, using only O(z lg sigma) bits of main memory. The algorithms read the input text from disk in a single pass, and write the compressed output to disk. The text can also be decompressed within the same main memory usage, which is unprecedented too. The algorithms are based on hashing and, under some simplifying assumptions, run in O(n) expected time.

code: https://github.com/rcanovas/Low-LZ78