This is unnecessary code as the current implementation of CEP-54 is not implemeting / exercising these features.
patch by Stefan Miklosovic; reviewed by Yifan Cai for CASSANDRA-21154
This change adds support for using O_DIRECT (direct I/O) when reading SSTables
during compaction. Direct I/O bypasses the OS page cache, which can be beneficial
for compaction workloads where data is typically read once and not accessed again
soon after.
Key changes:
- Add DiskAccessMode.direct option for scan operations
- Introduce DirectThreadLocalByteBufferHolder and DirectThreadLocalReadAheadBuffer
for aligned buffer management required by O_DIRECT
- Add ByteBufferHolder interface to abstract buffer management
- Modify FileHandle to support building direct I/O readers via toBuilder()
- Add startup check to verify Direct IO support on configured data directories
- Enable read-ahead for uncompressed data in Direct CompressedChunkReader
- Move direct IO support logic into FileHandle (from SSTableReader)
- Add comprehensive tests for direct I/O chunk readers and buffer holders
patch by Sam Lightfoot; reviewed by Ariel Weisberg and Maxwell Guo for CASSANDRA-19987
This commit adds a new configuration, data_disk_usage_keyspace_wide_protection_enabled, which ensures that if any node which replicates a given keyspace is full, all writes to that keyspace are blocked.
patch by Isaac Reath; reviewed by Stefan Miklosovic, Paulo Motta for CASSANDRA-20124
Closes#4547
It is also possible to override them via nodetool if necessary.
This patch also fixes the computation of sampling ratio to not lose the precision.
patch by Stefan Miklosovic; reviewed by Jyothsna Konisha, Yifan Cai for CASSANDRA-21078
For GC like ShenandoahGC/ZGC etc., there are GC events that do not have STW pauses (Concurrent phases).
Operator might find it reasonable to use lower thresholds for events require STW pauses and higher thresholds for concurrent phases.
gc_concurrent_phase_log_threshold and gc_concurrent_phase_warn_threshold configuration options are introduced in cassandra.yaml
Patch by Yuqi Yan; reviewed by Dmitry Konstantinov, Stefan Miklosovic for CASSANDRA-20980
Co-authored-by: Stefan Miklosovic
As per CASSANDRA-20045, we want to prevent full repair against
disk full scenarios. Current protection exists only for incremental
repair. This change updates the config name to not be
incremental repair specific, using the Replace annotation.
patch by Himanshu Jindal; reviewed by David Capwell, Jaydeepkumar Chovatia for CASSANDRA-20045
This also allows overriding complex settings as a JSON value and adds documentation about these overrides to conf/jvm-server.options
patch by Paulo Motta; reviewed by Stefan Miklosovic, David Capwell for CASSANDRA-20749
The SimpleProgressLog had a number of problems:
1. It polled for progress with no attempt to determine whether progress could realistically be made, so:
- as the number of pending transactions grew, the proportion of useful work dropped (as many would be unable to make progress without earlier transactions completing)
- each transaction in the chain could recover only on average 1/2 poll interval behind the last transaction to complete
2. It requested full transaction state from every replica on each attempt
3. It maintained a lot of in-memory state
4. Polling happened en-masse, allowing for little per-transaction control
We also separately maintained fairly expensive per-command listener state that negatively affected our command loading and caching.
The new DefaultProgressLog makes use of several new features: LocalListeners, RemoteListeners, Timers and Await messages.
- LocalListeners provide a memory-efficient collection for managing each CommandStore<E2><80><99>s transaction listeners, with dedicated record keeping for inter-transaction relationships.
- RemoteListeners provide a mechanism for request/response pairs that may be separated by longer than the normal Cassandra message timeout, and require minimal state on sender and recipient. This permits replicas to cheaply update their local state machine as soon as distributed information becomes available.
The DefaultProgressLog tracks each transaction with separate timers to handle per-transaction scheduling, backoff etc, and a succinct state machine. To reduce overhead correspondence is preferentially limited to a handful of replicas, and limited to the home shard where appropriate.
patch by Benedict; reviewed by Ariel Weisberg for CASSANDRA-19870
G1 can rely on `-XX:G1NewSizePercent=50` to always floor the young generation's size to 50% of the heap. (In production this can be raised to 66% for optimal performance.)
Fixing the young generation size with `-Xmn` is only applicable to CMS, so move it to the appropriate section.
When using G1, we can automatically set `-XX:ParallelGCThreads` and `-XX:ConcGCThreads` to the number of system cpu cores.
The auto-generated heap size is now half the server's physical RAM, capped at 16G for CMS and 31G for G1.
Increase MaxTenuringThreshold from 1 to 2, as evidence has shown it has no cost (over values of zero or one), but can have significant benefits in keeping objects in the young generation. While, values above 2 don't have any noticeable benefit.
Always set CASSANDRA_HEAPDUMP_DIR, default it to $CASSANDRA_LOG_DIR, to avoid hprof filling up unexpected disk volumes. Assumption here is that the logs directory is large enough to handle these dumps, and/or operators are monitoring these directories more than other randon/unknown directories.
patch by Mick Semb Wever; reviewed by Jon Haddad for CASSANDRA-20296
The presence of an extra space the `:` in `key: value` pairs
makes the default `conf/cassandra.yaml` file included in
this repository unparseable and non-functional for some downstream tooling.
patch by Daniel Lenski; reviewed by Stefan Miklosovic, Brandon Williams for CASSANDRA-20359
Deprecate IEndpointSnitch entirely, to be replaced with new interfaces:
* Locator for endpoint -> location mapping
* InitialLocationProvider to supply the DC & rack for registration
* NodeProximity for sorting endpoints and replicas at query time
For migration/upgrade/deprecation, nodes can still be configured with
an IEndpointSnitch implementation via endpoint_snitch in config, but
we hide this with a facade and only present the new interfaces.
Patch by Sam Tunnicliffe and Marcus Eriksson; reviewed by Sam
Tunnicliffe and Marcus Eriksson for CASSANDRA-19488
Co-authored-by: Marcus Eriksson <marcuse@apache.org>
Co-authored-by: Sam Tunnicliffe <samt@apache.org>
patch by Zhongxiang Zheng; reviewed by Stefan Miklosovic, Maulin Vasavada, Cheng Wang, Jordan West for CASSANDRA-11695
Co-authored-by: Stefan Miklosovic <smiklosovic@apache.org>
Co-authored-by: Sam Tunnicliffe <samt@apache.org>
configuration section for native_transport_max_auth_threads with default value of 4 was added to cassandra.yaml
as part of this merge-up as this section was forgotten to be added in the original patch of CASSANDRA-17812
configuration section for native_transport_max_auth_threads with default value of 4 was added to cassandra.yaml
as part of this merge-up as this section was forgotten to be added in the original patch of CASSANDRA-17812
This backport differs from CASSANDRA-17812 in such a way that by default the number
of auth request threads is set to 0. That will route all requests to request executor as before this change.
The patch in 5.0 and later sets the default number of auth request threads to 4.
patch by Josh McKenzie; reviewed by Chris Lohfink for CASSANDRA-20057
Co-authored-by: Stefan Miklosovic <smiklosovic@apache.org>
Batchlog endpoint strategy was previously only random placements on other racks. Options now are random_remote, prefer_local, dynamic_remote, and dynamic.
patch by Shayne Hunsaker; reviewed by Mick Semb Wever, Brandon Williams for CASSANDRA-18120
Stefan Miklosovic