LZ4 compressor

This library provides access to two compression methods, both of which generate a valid LZ4 stream:

  1. Fast scan (LZ4):
    • Low memory footprint (~16 KB)
    • Very fast (fast scan with skipping heuristics in case the input looks incompressible)
    • Reasonable compression ratio (depending on the redundancy of the input)
  2. High compression (LZ4 HC):
    • Medium memory footprint (~ 256 KB)
    • Rather slow (~10 times slower than LZ4)

A good compression ratio (depending on the size and the redundancy of the input).

The streams produced by these two compression algorithms use the same compression format; they are very fast to decompress and can be decompressed by the same decompressor instance.

How to do it…

LZ4 comes bundled with Hadoop. Check for the shared object(so) file, ...

Get HBase High Performance Cookbook now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial