This commit adds a new configuration, data_disk_usage_keyspace_wide_protection_enabled, which ensures that if any node which replicates a given keyspace is full, all writes to that keyspace are blocked.
patch by Isaac Reath; reviewed by Stefan Miklosovic, Paulo Motta for CASSANDRA-20124
Closes#4547
It is also possible to override them via nodetool if necessary.
This patch also fixes the computation of sampling ratio to not lose the precision.
patch by Stefan Miklosovic; reviewed by Jyothsna Konisha, Yifan Cai for CASSANDRA-21078
For GC like ShenandoahGC/ZGC etc., there are GC events that do not have STW pauses (Concurrent phases).
Operator might find it reasonable to use lower thresholds for events require STW pauses and higher thresholds for concurrent phases.
gc_concurrent_phase_log_threshold and gc_concurrent_phase_warn_threshold configuration options are introduced in cassandra.yaml
Patch by Yuqi Yan; reviewed by Dmitry Konstantinov, Stefan Miklosovic for CASSANDRA-20980
Co-authored-by: Stefan Miklosovic
As per CASSANDRA-20045, we want to prevent full repair against
disk full scenarios. Current protection exists only for incremental
repair. This change updates the config name to not be
incremental repair specific, using the Replace annotation.
patch by Himanshu Jindal; reviewed by David Capwell, Jaydeepkumar Chovatia for CASSANDRA-20045
This also allows overriding complex settings as a JSON value and adds documentation about these overrides to conf/jvm-server.options
patch by Paulo Motta; reviewed by Stefan Miklosovic, David Capwell for CASSANDRA-20749
The SimpleProgressLog had a number of problems:
1. It polled for progress with no attempt to determine whether progress could realistically be made, so:
- as the number of pending transactions grew, the proportion of useful work dropped (as many would be unable to make progress without earlier transactions completing)
- each transaction in the chain could recover only on average 1/2 poll interval behind the last transaction to complete
2. It requested full transaction state from every replica on each attempt
3. It maintained a lot of in-memory state
4. Polling happened en-masse, allowing for little per-transaction control
We also separately maintained fairly expensive per-command listener state that negatively affected our command loading and caching.
The new DefaultProgressLog makes use of several new features: LocalListeners, RemoteListeners, Timers and Await messages.
- LocalListeners provide a memory-efficient collection for managing each CommandStore<E2><80><99>s transaction listeners, with dedicated record keeping for inter-transaction relationships.
- RemoteListeners provide a mechanism for request/response pairs that may be separated by longer than the normal Cassandra message timeout, and require minimal state on sender and recipient. This permits replicas to cheaply update their local state machine as soon as distributed information becomes available.
The DefaultProgressLog tracks each transaction with separate timers to handle per-transaction scheduling, backoff etc, and a succinct state machine. To reduce overhead correspondence is preferentially limited to a handful of replicas, and limited to the home shard where appropriate.
patch by Benedict; reviewed by Ariel Weisberg for CASSANDRA-19870
G1 can rely on `-XX:G1NewSizePercent=50` to always floor the young generation's size to 50% of the heap. (In production this can be raised to 66% for optimal performance.)
Fixing the young generation size with `-Xmn` is only applicable to CMS, so move it to the appropriate section.
When using G1, we can automatically set `-XX:ParallelGCThreads` and `-XX:ConcGCThreads` to the number of system cpu cores.
The auto-generated heap size is now half the server's physical RAM, capped at 16G for CMS and 31G for G1.
Increase MaxTenuringThreshold from 1 to 2, as evidence has shown it has no cost (over values of zero or one), but can have significant benefits in keeping objects in the young generation. While, values above 2 don't have any noticeable benefit.
Always set CASSANDRA_HEAPDUMP_DIR, default it to $CASSANDRA_LOG_DIR, to avoid hprof filling up unexpected disk volumes. Assumption here is that the logs directory is large enough to handle these dumps, and/or operators are monitoring these directories more than other randon/unknown directories.
patch by Mick Semb Wever; reviewed by Jon Haddad for CASSANDRA-20296
The presence of an extra space the `:` in `key: value` pairs
makes the default `conf/cassandra.yaml` file included in
this repository unparseable and non-functional for some downstream tooling.
patch by Daniel Lenski; reviewed by Stefan Miklosovic, Brandon Williams for CASSANDRA-20359
Deprecate IEndpointSnitch entirely, to be replaced with new interfaces:
* Locator for endpoint -> location mapping
* InitialLocationProvider to supply the DC & rack for registration
* NodeProximity for sorting endpoints and replicas at query time
For migration/upgrade/deprecation, nodes can still be configured with
an IEndpointSnitch implementation via endpoint_snitch in config, but
we hide this with a facade and only present the new interfaces.
Patch by Sam Tunnicliffe and Marcus Eriksson; reviewed by Sam
Tunnicliffe and Marcus Eriksson for CASSANDRA-19488
Co-authored-by: Marcus Eriksson <marcuse@apache.org>
Co-authored-by: Sam Tunnicliffe <samt@apache.org>
patch by Zhongxiang Zheng; reviewed by Stefan Miklosovic, Maulin Vasavada, Cheng Wang, Jordan West for CASSANDRA-11695
Co-authored-by: Stefan Miklosovic <smiklosovic@apache.org>
Co-authored-by: Sam Tunnicliffe <samt@apache.org>
configuration section for native_transport_max_auth_threads with default value of 4 was added to cassandra.yaml
as part of this merge-up as this section was forgotten to be added in the original patch of CASSANDRA-17812
configuration section for native_transport_max_auth_threads with default value of 4 was added to cassandra.yaml
as part of this merge-up as this section was forgotten to be added in the original patch of CASSANDRA-17812
This backport differs from CASSANDRA-17812 in such a way that by default the number
of auth request threads is set to 0. That will route all requests to request executor as before this change.
The patch in 5.0 and later sets the default number of auth request threads to 4.
patch by Josh McKenzie; reviewed by Chris Lohfink for CASSANDRA-20057
Co-authored-by: Stefan Miklosovic <smiklosovic@apache.org>
Batchlog endpoint strategy was previously only random placements on other racks. Options now are random_remote, prefer_local, dynamic_remote, and dynamic.
patch by Shayne Hunsaker; reviewed by Mick Semb Wever, Brandon Williams for CASSANDRA-18120
Uses ParameterizedClass for IAuthorizer, INetworkAuthorizer, and
IRoleManager implementations enabling those to be configurable from
parameters specified directly in cassandra.yaml.
Opportunistically makes consistent the initialization and setting of
all auth-related implementations and removes code duplication.
Notes:
* IInternodeAuthenticator implementations are expected to be set as
default directly in DatabaseDescriptor instead of done via
AuthConfig.applyAuth(). This is assumed in tests and client logic.
For instance ReconnectableSnitchHelperTest fetches this authenticator
before calling daemonInitialize(). Also, BulkLoader fetches this
directly when creating an outbound connection.
* Changing this behavior in BulkLoader causes a cascade of changes.
First, one would need to add AuthConfig.applyAuth() directly in
clientInitialize() and then would need to enable all the additional
classes that this change causes. Long term this might be justified
as if we're depending on the auth logic to be properly set. However,
in the context of this change it was decided to postpone further
changes.
* Standardize auth-related configurations by using a short-form with
the class name instead of the full qualified class name containing
the package name.
patch by Tiago Alves; reviewed by Mick Semb Wever, Stefan Miklosovic for CASSANDRA-19946
nickbar01234