mirror of https://github.com/apache/cassandra
96 lines
4.9 KiB
ReStructuredText
96 lines
4.9 KiB
ReStructuredText
.. Licensed to the Apache Software Foundation (ASF) under one
|
|
.. or more contributor license agreements. See the NOTICE file
|
|
.. distributed with this work for additional information
|
|
.. regarding copyright ownership. The ASF licenses this file
|
|
.. to you under the Apache License, Version 2.0 (the
|
|
.. "License"); you may not use this file except in compliance
|
|
.. with the License. You may obtain a copy of the License at
|
|
..
|
|
.. http://www.apache.org/licenses/LICENSE-2.0
|
|
..
|
|
.. Unless required by applicable law or agreed to in writing, software
|
|
.. distributed under the License is distributed on an "AS IS" BASIS,
|
|
.. WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
.. See the License for the specific language governing permissions and
|
|
.. limitations under the License.
|
|
|
|
.. highlight:: none
|
|
|
|
Compression
|
|
-----------
|
|
|
|
Cassandra offers operators the ability to configure compression on a per-table basis. Compression reduces the size of
|
|
data on disk by compressing the SSTable in user-configurable compression ``chunk_length_in_kb``. Because Cassandra
|
|
SSTables are immutable, the CPU cost of compressing is only necessary when the SSTable is written - subsequent updates
|
|
to data will land in different SSTables, so Cassandra will not need to decompress, overwrite, and recompress data when
|
|
UPDATE commands are issued. On reads, Cassandra will locate the relevant compressed chunks on disk, decompress the full
|
|
chunk, and then proceed with the remainder of the read path (merging data from disks and memtables, read repair, and so
|
|
on).
|
|
|
|
Configuring Compression
|
|
^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
|
Compression is configured on a per-table basis as an optional argument to ``CREATE TABLE`` or ``ALTER TABLE``. By
|
|
default, three options are relevant:
|
|
|
|
- ``class`` specifies the compression class - Cassandra provides four classes (``LZ4Compressor``,
|
|
``SnappyCompressor``, ``DeflateCompressor`` and ``ZstdCompressor``). The default is ``LZ4Compressor``.
|
|
- ``chunk_length_in_kb`` specifies the number of kilobytes of data per compression chunk. The default is 64KB.
|
|
- ``crc_check_chance`` determines how likely Cassandra is to verify the checksum on each compression chunk during
|
|
reads. The default is 1.0.
|
|
- ``compression_level`` is only applicable for ``ZstdCompressor`` and accepts values between ``-131072`` and ``2``.
|
|
|
|
Users can set compression using the following syntax:
|
|
|
|
::
|
|
|
|
CREATE TABLE keyspace.table (id int PRIMARY KEY) WITH compression = {'class': 'LZ4Compressor'};
|
|
|
|
Or
|
|
|
|
::
|
|
|
|
ALTER TABLE keyspace.table WITH compression = {'class': 'SnappyCompressor', 'chunk_length_in_kb': 128, 'crc_check_chance': 0.5};
|
|
|
|
Once enabled, compression can be disabled with ``ALTER TABLE`` setting ``enabled`` to ``false``:
|
|
|
|
::
|
|
|
|
ALTER TABLE keyspace.table WITH compression = {'enabled':'false'};
|
|
|
|
Operators should be aware, however, that changing compression is not immediate. The data is compressed when the SSTable
|
|
is written, and as SSTables are immutable, the compression will not be modified until the table is compacted. Upon
|
|
issuing a change to the compression options via ``ALTER TABLE``, the existing SSTables will not be modified until they
|
|
are compacted - if an operator needs compression changes to take effect immediately, the operator can trigger an SSTable
|
|
rewrite using ``nodetool scrub`` or ``nodetool upgradesstables -a``, both of which will rebuild the SSTables on disk,
|
|
re-compressing the data in the process.
|
|
|
|
Benefits and Uses
|
|
^^^^^^^^^^^^^^^^^
|
|
|
|
Compression's primary benefit is that it reduces the amount of data written to disk. Not only does the reduced size save
|
|
in storage requirements, it often increases read and write throughput, as the CPU overhead of compressing data is faster
|
|
than the time it would take to read or write the larger volume of uncompressed data from disk.
|
|
|
|
Compression is most useful in tables comprised of many rows, where the rows are similar in nature. Tables containing
|
|
similar text columns (such as repeated JSON blobs) often compress very well.
|
|
|
|
Operational Impact
|
|
^^^^^^^^^^^^^^^^^^
|
|
|
|
- Compression metadata is stored off-heap and scales with data on disk. This often requires 1-3GB of off-heap RAM per
|
|
terabyte of data on disk, though the exact usage varies with ``chunk_length_in_kb`` and compression ratios.
|
|
|
|
- Streaming operations involve compressing and decompressing data on compressed tables - in some code paths (such as
|
|
non-vnode bootstrap), the CPU overhead of compression can be a limiting factor.
|
|
|
|
- The compression path checksums data to ensure correctness - while the traditional Cassandra read path does not have a
|
|
way to ensure correctness of data on disk, compressed tables allow the user to set ``crc_check_chance`` (a float from
|
|
0.0 to 1.0) to allow Cassandra to probabilistically validate chunks on read to verify bits on disk are not corrupt.
|
|
|
|
Advanced Use
|
|
^^^^^^^^^^^^
|
|
|
|
Advanced users can provide their own compression class by implementing the interface at
|
|
``org.apache.cassandra.io.compress.ICompressor``.
|