Also Add:
- txn_graph and txn_graph_all virtual tables for debugging transaction dependency graphs
- txn_cache virtual table to debug accord cache contents
Also Fix:
- NPE in AccordDebugKeyspace.TxnOpsTable
- AccordCoordinatorMetrics latency metrics should use Timer not Histogram
- OnDiskIndex.readLast
Also Improve:
- Catchup should run after progress log is started
- Cancelling AccordTask should early-terminate range scanning
- Improve RangeTxnScanner description
- Add ballot to txn tables
- Add NodeId to logging output
patch by Benedict; reviewed by Alex Petrov for CASSANDRA-21306
With negotiated authentication (CEP-50), nodes may be configured with multiple authenticators. Prior to this change,
a number of areas in the code assumed that there was a single configured authenticator and contained logic that
switched depending on the authenticator type. This logic won't work when multiple authenticators can be configured.
This change eliminates most calls to DataDescriptor.getAuthenticator(), by enabling individual authenticators to
declare the role attributes they support, requiring callers to specify the type of authenticator they're looking
for, and directly returning whether the node can enforce authn or not rather than inferring it by the presence of
an authenticator.
Testing done: Unit tests for auth and config packages; d-tests for auth-related functionality (e.g. ColumnMasks).
patch by Joel Shepherd; reviewed by Stefan Miklosovic, Andy Tolbert for CASSANDRA-20834
Includes bug fixes and features:
- Improved observability in AutoRepair (CASSANDRA-20581)
- Stop repair scheduler if two major versions detected (CASSANDRA-20048)
- Safeguard Full repair against disk protection (CASSANDRA-20045)
- Stop AutoRepair monitoring thread upon shutdown (CASSANDRA-20623)
- Fix race condition in auto-repair scheduler (CASSANDRA-20265)
- Minimum repair task duration setting (CASSANDRA-20160)
- Preview_repaired auto-repair type (CASSANDRA-20046)
- Gate auto-repair behind cassandra.autorepair.enable JVM property
- Add cassandra.autorepair.check_min_version to gate minimum version enforcement
- Prevent auto-repair from running if any node is below 5.0.7
- Make system_distributed auto-repair schema conditional on feature being enabled
- Add user-friendly errors for disabled auto-repair and schema incompatibility
patch by Paulo Motta; reviewed by Andy Tolbert, Jaydeepkumar Chovatia for CASSANDRA-21138
Co-Authored-By: Andy Tolbert <andy_tolbert@apple.com>
Co-Authored-By: Chris Lohfink <clohfink@netflix.com>
Co-Authored-By: Francisco Guerrero <frankgh@apache.org>
Co-Authored-By: Himanshu Jindal <himanshj@amazon.com>
Co-Authored-By: Jaydeepkumar Chovatia <jchovati@uber.com>
Co-Authored-By: Kristijonas Zalys <kzalys@uber.com>
Co-Authored-By: jaydeepkumar1984 <chovatia.jaydeep@gmail.com>
this diff was supposed to be included in 5f9de8ca13
Rate limit password changes
Introduces property cassandra.role_password_update_interval_in_ms, with default value of 5000.
Logging added when passwords on roles are changed or are rate-limited.
patch by Mick Semb Wever; reviewed by Doug Rohrer, Jeremiah Jordan for CASSANDRA-21202
Add an implementation of isExpired to ReusableLivenessInfo, before the change the method implementation was interited from an interface and always returned false, so livenessInfo().supersedes(...) did not work properly.
patch by Dmitry Konstantinov; reviewed by Branimir Lambov, Nitsan Wakart for CASSANDRA-21152
Best effort attempt to get tableId from CFS, fallback to null which, still allows snapshot to be loaded
End result is that nodetool clearsnapshot and listsnapshot function for such snapshots on restart.
patch by Matt Byrd; reviewed by Stefan Miklosovic, Brandon Williams and Francisco Guerrero for CASSANDRA-21173
Rewrites ANN search query execution logic to more efficiently merge graph search results using
similarity score-ordered (descending) iterators to merge segments efficiently. Allows for reduced memory
consumption during queries, reduced impact of overwrites and tombstones, selective re-querying of
minimally necessary graphs, and reduced shuffling of PrimaryKey objects.
patch by Michael Marshall; reviewed by Caleb Rackliffe and Michael Semb Wever for CASSANDRA-20086
Only inflight MSOs which affect the local node need to be
applied to determine the settled local ranges
Patch by Sam Tunnicliffe; reviewed by Marcus Eriksson for
CASSANDRA-21144
Introduces property cassandra.role_password_update_interval_in_ms, with default value of 5000.
Logging added when passwords on roles are changed or are rate-limited.
patch by Mick Semb Wever; reviewed by Doug Rohrer, Jeremiah Jordan for CASSANDRA-21202
* Expand affected ranges to include removals
* Don't create waiters for replica groups of future ranges not yet in placements
Patch by Sam Tunnicliffe; reviewed by Marcus Eriksson for
CASSANDRA-21142
Make MemoryUtil.clean() throw for slices/duplicates that cannot be cleaned and fix mmap cleanup
Parameterise main compaction tests for cursor/iterator and buffered/direct combinations
patch by Sam Lightfoot; reviewed by Ariel Weisberg, Dmitry Konstantinov for CASSANDRA-21147
This is a partial solution to IllegalStateException thrown by VectorPostings. It works by using a single shard at L0 when a vector index is present. As noted in the jira ticket, there are edge cases that may still produce errors, notably the case where there are multiple data directories.
The key trade offs here are related to the time complexity for search. Since graph search is log(n), and searching m graphs is m * log(n), we see better search performance by building bigger graphs which is essentially log(m * n). We could pre-shard, which comes at a cost of increased search time complexity.
patch by Michael Marshall,Dmitry Konstantinov; reviewed by Caleb Rackliffe,Dmitry Konstantinov,Michael Semb Wever for CASSANDRA-19661
Co-authored-by: Michael Marshall <mmarshall@apache.org>
Co-authored-by: Dmitry Konstantinov <netudima@gmail.com>
Introduce the ability to override compaction strategy for specific keyspaces
and tables at startup via two new system properties:
- cassandra.override_compaction.entities: comma-separated list of keyspaces
and keyspace.table pairs (e.g. "ks1,ks2.tbl1,ks3.tbl2")
- cassandra.override_compaction.params: JSON compaction parameters to apply
Patch by Paulo Motta; Reviewed by Jaydeepkumar Chovatia for CASSANDRA-21169
Add test container memory events printing to check if Linux OOM killer was active
Fix thread dump printing on junit test timeout
Process.pid() API is available since JDK9
patch by Dmitry Konstantinov; reviewed by Michael Semb Wever for CASSANDRA-21172
Too many threads can lead to OOM, so new Thread is replaced with an executor
spin wait is added to HintsServiceTest.testPageSeek to avoid flaky NPE (HintsStore is populated in an async way)
patch by Dmitry Konstantinov; reviewed by Brandon Williams for CASSANDRA-21166
The runtime overflow guard test (testLargeIndexSummary) required ~2 GiB of
off-heap memory to trigger the Integer.MAX_VALUE boundary in
IndexSummaryBuilder.maybeAddEntry. SafeMemoryWriter's 50% buffer growth
strategy caused ~5 GiB peak memory during reallocation (old + new buffer),
exceeding CI medium node limits (3.5-5 GB).
Extract the hardcoded Integer.MAX_VALUE threshold in IndexSummaryBuilder
into a @VisibleForTesting field (maxEntriesSize), allowing the test to
exercise the same guard logic with a 2 MB threshold instead. Rename both
large summary tests to reflect their distinct code paths: runtime truncation
vs constructor-initiated downsampling.
patch by Sam Lightfoot; reviewed by Brandon Williams,Dmitry Konstantinov,Michael Semb Wever for CASSANDRA-20599
The two deletes use the same CQL timestamp, so compaction should merge
their overlapping range tombstones into a single open/close pair (no
boundary markers). However, DeletionTime also includes local_delete_time
(wall-clock seconds), which may differ between deletes if a second
boundary is crossed during the flush. Differing local_delete_time causes
the merger to treat them as distinct deletions, producing boundary
markers instead. Using executeInternalWithNowInSec with a fixed nowInSec
ensures both deletes get identical DeletionTime values.
patch by Sam Lightfoot; reviewed by Dmitry Konstantinov,Brandon Williams for CASSANDRA-21163
This is unnecessary code as the current implementation of CEP-54 is not implemeting / exercising these features.
patch by Stefan Miklosovic; reviewed by Yifan Cai for CASSANDRA-21154
This change adds support for using O_DIRECT (direct I/O) when reading SSTables
during compaction. Direct I/O bypasses the OS page cache, which can be beneficial
for compaction workloads where data is typically read once and not accessed again
soon after.
Key changes:
- Add DiskAccessMode.direct option for scan operations
- Introduce DirectThreadLocalByteBufferHolder and DirectThreadLocalReadAheadBuffer
for aligned buffer management required by O_DIRECT
- Add ByteBufferHolder interface to abstract buffer management
- Modify FileHandle to support building direct I/O readers via toBuilder()
- Add startup check to verify Direct IO support on configured data directories
- Enable read-ahead for uncompressed data in Direct CompressedChunkReader
- Move direct IO support logic into FileHandle (from SSTableReader)
- Add comprehensive tests for direct I/O chunk readers and buffer holders
patch by Sam Lightfoot; reviewed by Ariel Weisberg and Maxwell Guo for CASSANDRA-19987
Add microbench-test target for quick test run
Unify declaration and add assertions on target types
Fix dirname usages for macos
Parameter for docker_timeout
Add more jmh class ignores (that are broken)
Add disableResume() and better host debug to Jenkinsfile
patch by Mick Semb Wever; reviewed by Dmitry Konstantinov for CASSANDRA-18873
Legacy Cassandra 3.x sstables use an old bloom filter format that is
incompatible with zero-copy streaming in Cassandra 5.0+. This patch
automatically detects sstables with the old bloom filter format
(pre-4.0) and disables zero-copy streaming for them, allowing legacy
sstables to be loaded via sstableloader without requiring manual flags.
The fix adds a version check in CassandraOutgoingFile.computeShouldStreamEntireSSTables()
that calls descriptor.version.hasOldBfFormat() to detect legacy sstables.
patch by Paulo Motta; reviewed by Stefan Miklosovic for CASSANDRA-21092
This commit adds a new configuration, data_disk_usage_keyspace_wide_protection_enabled, which ensures that if any node which replicates a given keyspace is full, all writes to that keyspace are blocked.
patch by Isaac Reath; reviewed by Stefan Miklosovic, Paulo Motta for CASSANDRA-20124
Closes#4547
Based on the official documentation here (1), AsyncProfiler works best with these kernel parameters.
Not all CI environments have them set. That would fail the tests which run the startup check
if kernel parameters are not like that.
It might be empirically observed that some AsyncProfiler functionality works even without these
kernel parameters, it might be investigated in follow-up work if we should not relax these conditions a bit.
(1) https://github.com/async-profiler/async-profiler/blob/master/docs/GettingStarted.md#before-profiling
It is also possible to override them via nodetool if necessary.
This patch also fixes the computation of sampling ratio to not lose the precision.
patch by Stefan Miklosovic; reviewed by Jyothsna Konisha, Yifan Cai for CASSANDRA-21078
Adds a compaction implementation utilizing new fixed allocation SSTable reader/writer implementations, and other purpose built code, leading to improved efficiencies.
patch by Nitsan Wakart; reviewed by Branimir Lambov, Dmitry Konstantinov for CASSANDRA-20918
By default the expiry time is calculated on each peer independently. It can be
made to converge by disabling gossip quarantine using the configuration setting
gossip_quarantine_disabled or via a hotprop on GossiperMBean.
Patch by Sam Tunnicliffe; reviewed by Marcus Eriksson for CASSANDRA-21035
Alan Wang