15 lines
855 B
Plaintext
15 lines
855 B
Plaintext
Blosc is a high performance compressor optimized for binary data. It
|
|
has been designed to transmit data to the processor cache faster than
|
|
the traditional, non-compressed, direct memory fetch approach via a
|
|
memcpy() OS call. Blosc is the first compressor (that I'm aware of)
|
|
that is meant not only to reduce the size of large datasets on-disk or
|
|
in-memory, but also to accelerate memory-bound computations.
|
|
|
|
It uses the blocking technique so as to reduce activity in the memory
|
|
bus as much as possible. In short, this technique works by dividing
|
|
datasets in blocks that are small enough to fit in caches of modern
|
|
processors and perform compression / decompression there. It also
|
|
leverages, if available, SIMD instructions (SSE2, AVX2) and
|
|
multi-threading capabilities of CPUs, in order to accelerate the
|
|
compression / decompression process to a maximum.
|