Commit Graph

9280 Commits

Author SHA1 Message Date
Sunil Ramchandra Pawar a84abe7ba1 Extend nodetool verify to (optionally) validate SAI files
patch by Sunil Ramchandra Pawar; reviewed by Caleb Rackliffe and David Capwell for CASSANDRA-20949
2025-12-12 13:22:06 -06:00
Stefan Miklosovic a14406022d
Merge branch 'cassandra-5.0' into trunk 2025-12-11 11:24:51 +01:00
Stefan Miklosovic f894b8440d
Merge branch 'cassandra-4.1' into cassandra-5.0 2025-12-11 11:10:58 +01:00
Stefan Miklosovic efa0ead445
Merge branch 'cassandra-4.0' into cassandra-4.1 2025-12-11 11:02:05 +01:00
Mikołaj Diakowski 4c33f1f2d7
Restrict BytesType compatibility to scalar types only
Converting collections or UDTs to raw bytes is nonsensical - it
allows reading raw serialized bytes which have no meaningful
interpretation.

patch by Mikołaj Diakowski; reviewed by Stefan Miklosovic, Brandon Williams for CASSANDRA-20982
2025-12-11 10:53:29 +01:00
Stefan Miklosovic f7fab8a60a
Merge branch 'cassandra-5.0' into trunk 2025-12-10 12:41:51 +01:00
Stefan Miklosovic bbbf70a9a2
Merge branch 'cassandra-4.1' into cassandra-5.0 2025-12-10 12:39:43 +01:00
ireath c79f176685
Fix disk usage guardrail disablement when failure threshold is reached
patch by Isaac Reath; reviewed by Jyothsna Konisa, Stefan Miklosovic for CASSANDRA-21057
2025-12-10 12:24:22 +01:00
Benedict Elliott Smith 13a3ae9297 Fix:
- DurabilityQueue/ShardScheduler deadlock
 - MemtableCleanerThread.Cleanup assumes Boolean parameter is non-null, which is invalid if an exception has been thrown
 - AccordDurableOnFlush may be invoked while Accord is starting up, so should use AccordService.unsafeInstance
 - AccordCache shrink without lock regression
 - Cleanup system_accord compaction leftovers before starting up
 - system_accord_debug.txn order
 - system_accord_debug.txn_blocked_by order
 - system_accord_debug.shard_epochs order
Improve:
 - Set DefaultProgressLog.setMode(Catchup) during Catchup
 - IdentityAccumulators only need to readLast, not readAll
 - Limit number of static segments we compact at once to sstable
 - If too many static segments on startup, wait for them to be compacted

patch by Benedict; reviewed by Alex Petrov for CASSANDRA-21053
2025-12-09 14:01:38 +00:00
Yifan Cai 21382e6874 Fix CompressionDictionary being closed while still in use
Patch by Yifan Cai; Reviewed by Stefan Miklosovic for CASSANDRA-21047
2025-12-08 14:37:13 -08:00
David Capwell c08666567d When updating a multi cell collection element, if the update is rejected then the shared Row.Builder is not freed causing all future mutations to be rejected
patch by David Capwell; reviewed by Bernardo Botella Corbi, Caleb Rackliffe, Dmitry Konstantinov for CASSANDRA-21055
2025-12-05 16:14:40 -08:00
jkonisa c4e1e2c8a8 CASSANDRA-21046 Schema annotations escape validation on CREATE and ALTER DDL statements
Patch by Jyothsna Konisa; Reviewed by Sam Tunnicliffe, Yifan Cai, Stefan Miklosovic for CASSANDRA-21046
2025-12-05 12:00:39 -08:00
Marcus Eriksson 2b5124d3f6 Merge branch 'cassandra-5.0' into trunk 2025-12-05 12:59:23 +01:00
Marcus Eriksson d17e6d1c6a Merge branch 'cassandra-4.1' into cassandra-5.0 2025-12-05 12:57:49 +01:00
Marcus Eriksson cbf974f2f0 Merge branch 'cassandra-4.0' into cassandra-4.1 2025-12-05 12:57:31 +01:00
Marcus Eriksson 76a7e43613 ArrayIndexOutOfBoundsException with repaired data tracking and counters
Patch by marcuse; reviewed by Sam Tunnicliffe for CASSANDRA-20871
2025-12-05 12:55:59 +01:00
Dmitry Konstantinov 241f478ce6 Fix flaky ThreadLocalMeterTest by disabling background triggering of ThreadLocalMeter#tickAll within the test case
Patch by Dmitry Konstantinov; reviewed by Brandon Williams, Stefan Miklosovic for CASSANDRA-21056
2025-12-03 16:55:53 +00:00
Valery Baranov 7ed0dc33d0
Add a test for checking permissions after grant authorize
patch by Valery Baranov; reviewed by Stefan Miklosovic, Brandon Williams for CASSANDRA-21051
2025-12-03 16:17:15 +01:00
Marcus Eriksson b5f8ac2604 Update system schema tables with new distributed keyspace on upgrade
Patch by marcuse; reviewed by Sam Tunnicliffe for CASSANDRA-20872
2025-12-03 14:28:16 +01:00
Marcus Eriksson 7802743460 Fix issue when running cms reconfiguration with paxos repair disabled
Patch by marcuse; reviewed by Sam Tunnicliffe for CASSANDRA-20869
2025-12-03 14:27:21 +01:00
Yifan Cai 0212713750 Fix flaky test CompressionDictionarySchedulerTest.testScheduleSSTableBasedTrainingWithSSTables
Patch by Yifan Cai; Reviewed by Dmitry Konstantinov for CASSANDRA-21054
2025-12-02 16:13:55 -08:00
Alan Wang fd2986b8bc
Added additional parameter to JVM shutdown to allow for logs to be properly shutdown
patch by Alan Wang; reviewed by Abe Ratnofsky, Caleb Rackliffe for CASSANDRA-20978
2025-12-01 19:32:51 +01:00
Marcus Eriksson e1e39b074a Refactor the way we check if a transformation is allowed to be committed during upgrades
Patch by marcuse and Sam Tunnicliffe; reviewed by Sam Tunnicliffe for CASSANDRA-21043

Co-authored-by: Sam Tunnicliffe <samt@apache.org>
2025-12-01 10:39:25 +01:00
Marcus Eriksson 4ed54ecb2a Improve isGossipOnlyMember and location lookup performance
Patch by marcuse; reviewed by Sam Tunnicliffe for CASSANDRA-21039
2025-12-01 10:39:07 +01:00
Sam Tunnicliffe f2406b1070 Merge branch 'cassandra-5.0' into trunk 2025-11-28 11:12:04 +00:00
Sam Tunnicliffe f8f456f0f6 Fix default max interval calculation in FailureDetector
Patch by Sam Tunnicliffe; reviewed by Maxim Muzafarov and Stefan Miklosovic
for CASSANDRA-21025
2025-11-28 09:28:57 +00:00
ireath ce541b6ddf LegacyStateListener should call onRemove for gossip state listeners
Patch by Isaac Reath; reviewed by Sam Tunnicliffe and Paulo Motta for
CASSANDRA-21033
2025-11-28 07:46:45 +00:00
Benedict Elliott Smith 4b0df8ad6d ninja: fix IndexBuildsAfterStreamingTest regression due to now correctly capturing all uncaught exceptions 2025-11-27 18:23:14 +00:00
Benedict Elliott Smith 4defbb9b1c Make TopologyException a checked exception to ensure they are handled carefully, as they may occur at surprising times
Also Fix:
 - Restore MaxDecidedRX on replay
 - When catchup_on_start_exit_on_failure == false, should startup on any kind of failure, not only timeout
 - lazy vtable LIMIT clause regression
 - DurabilityService.onEpochRetired
 - Command.validate when uniqueHlc differs
 - Avoid unsafe publication of AccordExecutor to scheduledFastTasks
 - AccordCache hitRate metric names
 - use long for return type of DurationSpec.toNanoseconds
 - Repair without all replicas should not request all Accord replicas participate
 - ExecuteAtSerializer
 - SyncPoints should be coordinated in an epoch that contains the ranges
Also Improve:
 - Split Accord startup into local+distributed, ensure we
 - Add logging to FetchDurableBefore on startup
 - Add randomised testing of AbstractLazyVirtualTable
 - Add validation of lazy virtual table key ordering
 - Don't send requests to faulty replicas
 - shrinkOrEvict large objects without holding lock
 - Accord dtest shutdown

patch by Benedict; reviewed by Alex Petrov for CASSANDRA-21042
2025-11-27 13:12:25 +00:00
Sam Tunnicliffe dbfcd52fd5 Fix schema diffs when comments or security labels are modified
Patch by Sam Tunnicliffe; reviewed by Marcus Eriksson for
CASSANDRA-21045
2025-11-27 11:05:44 +00:00
Sam Tunnicliffe 209288cebe Reject PrepareJoin if tokens already assigned
Patch by Sam Tunnicliffe; reviewed by Marcus Eriksson for CASSANDRA-21006
2025-11-25 17:44:57 +00:00
Sam Tunnicliffe 496959a8f2 Don't update registration status if node state for decommissioned peer is found with the same address
Patch by Sam Tunnicliffe; reviewed by Marcus Eriksson for CASSANDRA-21005
2025-11-25 16:15:10 +00:00
Sam Tunnicliffe ec7794f20f Avoid NPE when meta keyspace placements are empty before CMS is initialized
Patch by Sam Tunnicliffe; reviewed by Marcus Eriksson for CASSANDRA-21004
2025-11-25 16:13:42 +00:00
Sam Tunnicliffe 62bd9ab1ae Gossip entries for hibernating non-members don't block truncate
Patch by Sam Tunnicliffe; reviewed by Marcus Eriksson for CASSANDRA-21003
2025-11-25 16:13:05 +00:00
Sam Tunnicliffe 856db2afa0 Retry without time limit calculates wait time correctly
Patch by Sam Tunnicliffe; reviewed by Marcus Eriksson for CASSANDRA-21002
2025-11-25 16:10:36 +00:00
Sam Tunnicliffe 44f6a32718 AlterSchemaStatement with no effect locally isn't submitted to CMS
Patch by Sam Tunnicliffe; reviewed by Marcus Eriksson for CASSANDRA-21001
2025-11-25 16:09:34 +00:00
Dmitry Konstantinov 909ed8b949 Merge branch 'cassandra-5.0' into trunk
* cassandra-5.0:
  Fix cleanup of old incremental repair sessions in case of owned token range changes or a table deleting
2025-11-22 11:49:07 +00:00
Dmitry Konstantinov 1a12cbb349 Merge branch 'cassandra-4.1' into cassandra-5.0
* cassandra-4.1:
  Fix cleanup of old incremental repair sessions in case of owned token range changes or a table deleting
2025-11-22 11:35:37 +00:00
Dmitry Konstantinov 7f592803a4 Merge branch 'cassandra-4.0' into cassandra-4.1
* cassandra-4.0:
  Fix cleanup of old incremental repair sessions in case of owned token range changes or a table deleting
2025-11-22 11:23:21 +00:00
Dmitry Konstantinov 8d2c11ed2e Fix cleanup of old incremental repair sessions in case of owned token range changes or a table deleting
Patch by Dmitry Konstantinov; reviewed by Marcus Eriksson, Jaydeepkumar Chovatia for CASSANDRA-20877
2025-11-22 11:17:35 +00:00
Sam Tunnicliffe f05b27502f Improve CMS initialization
* Better handling of DOWN unupgraded nodes
* Test that aborted CMS initialization cleans up state properly
* Clean up orphaned PreInitialize entries in the log on bounce
* Unconditionally reset initiator during abort
* Ensure that metadata log entries aren't exchanged before CMS
  initialization is complete
* Precalculate common serialization version, excluding non-upgraded and
  LEFT nodes
* Decide if metadata-impacting upgrade is in progress using min common
  version
* Add metadata identifier to nodetool cms output

Patch by Sam Tunnicliffe and Marcus Eriksson; reviewed by Sam
Tunnicliffe and Marcus Eriksson for CASSANDRA-21036

Co-authored-by: Marcus Eriksson <marcuse@apache.org>
Co-authored-by: Sam Tunnicliffe <samt@apache.org>
2025-11-21 16:40:24 +00:00
jkonisa b11633be4f CASSANDRA-20943 Introducing comments and security labels for schema elements 2025-11-19 11:32:50 -08:00
Stefan Miklosovic dc89b8c802
Extend nodetool tablestats for dictionary memory usage
patch by Stefan Miklosovic; reviewed by Yifan Cai for CASSANDRA-20940
2025-11-19 19:12:26 +01:00
Yuqi Yan e621d8a6c1 Introduce separate GCInspector thresholds for concurrent GC events.
For GC like ShenandoahGC/ZGC etc., there are GC events that do not have STW pauses (Concurrent phases).
Operator might find it reasonable to use lower thresholds for events require STW pauses and higher thresholds for concurrent phases.
gc_concurrent_phase_log_threshold and gc_concurrent_phase_warn_threshold configuration options are introduced in cassandra.yaml

Patch by Yuqi Yan; reviewed by Dmitry Konstantinov, Stefan Miklosovic for CASSANDRA-20980
Co-authored-by: Stefan Miklosovic
2025-11-16 21:29:00 +00:00
Dmitry Konstantinov 6b57acc7f4 Reduce contention in MemtableAllocator.allocate
- Estimate memory required to allocate for applying a partition update into a memtable and allocate this memory in one shot, then use it as a request-local SLAB
- Reduce contention by switching MemtableAllocator.SubAllocator#owns from updates via AtomicLongFieldUpdater to LongAdder usage. MemtableAllocator.SubAllocator#acquired(..) method updates "owns" value but does not use the updated result.
- Reduce contention by replacing of CAS loop in MemtablePool.SubPool#tryAllocate with allocatedUpdater.addAndGet(this, size)

Patch by Dmitry Konstantinov; reviewed by Michael Semb Wever for CASSANDRA-20226
2025-11-15 15:33:02 +00:00
Dmitry Konstantinov 4cf684b464 Merge branch 'cassandra-5.0' into trunk
* cassandra-5.0:
  Fix memory leak in BufferPoolAllocator when a capacity needs to be extended
2025-11-14 17:53:46 +00:00
Dmitry Konstantinov 2fec119f09 Merge branch 'cassandra-4.1' into cassandra-5.0
* cassandra-4.1:
  Fix memory leak in BufferPoolAllocator when a capacity needs to be extended
2025-11-14 17:46:13 +00:00
Dmitry Konstantinov ed7d5840b7 Merge branch 'cassandra-4.0' into cassandra-4.1
* cassandra-4.0:
  Fix memory leak in BufferPoolAllocator when a capacity needs to be extended
2025-11-14 17:34:10 +00:00
Dmitry Konstantinov 3f079b2345 Fix memory leak in BufferPoolAllocator when a capacity needs to be extended
Before the fix during a capacity extension BufferPoolAllocator returned to BufferPool a sliced ByteBuffer wrapper object instead of the originally allocated one, so the ByteBuffer was not recycled by BufferPool
Adjust BufferPoolAllocatorTest to test the ByteBuf capacity extension with a real BufferPool behavior

Patch by Dmitry Konstantinov; reviewed by Michael Semb Wever for CASSANDRA-20753
2025-11-14 17:11:25 +00:00
Stefan Miklosovic 99be618ec1
Merge branch 'cassandra-5.0' into trunk 2025-11-13 16:46:06 +01:00
shruti-p-s e2f74ed65b
Adding missing configs in system_views.settings to be backward compatible
patch by Sudipta Laha; reviewed by Stefan Miklosovic, Tiago L. Alves, Francisco Guerrero, Yifan Cai for CASSANDRA-20863
2025-11-13 16:42:38 +01:00
Stefan Miklosovic 136e5fd222
Add export, list, import sub-commands for nodetool compressiondictionary
patch by Stefan Miklosovic; reviewed by Yifan Cai for CASSANDRA-20941
2025-11-13 15:38:42 +01:00
Benedict Elliott Smith 494c3c7bef Add AccordExecutorMetrics
Also Introduce:
 - Sharded/LogLinearDecayingHistogram
Also Improve:
 - Do not take a reference to CFK unless relevant
Also Fix:
 - Sharded/LogLinearHistogram
 - ExecuteFlags serialization bug in ReadData

patch by Benedict; reviewed by Alex Petrov for CASSANDRA-21017
2025-11-13 14:36:15 +00:00
David Capwell f43d9d0f7f Add support in the binary protocol to allow transactions to have multiple conditions
patch by David Capwell; reviewed by Benedict Elliott Smith for CASSANDRA-20883
2025-11-12 10:13:41 -08:00
Benedict Elliott Smith f6fd49566f Improve DurabilityQueue:
- All retries are appended to a delay queue so overlaps can be pruned
 - Quorum successes are not retried if there are superseding sync points covering the ranges
 - User-initiated requests are not purged unless the request has timed out or otherwise completed
 - Overlapping requests are queued up against the next to run
Alsp (C*):
 - Catch-up with quorums on restart
 - Manage an ordered set of keys in cache for faster range searches
Also (Accord):
 - Update copy of BTree and import IntervalBTree
 - Fix RedundantStatus WAS_OWNED_OVERRIDE_MASK
 - Add Catchup mechanism to reach parity with a quorum on restart

patch by Benedict; reviewed by Alex Petrov for CASSANDRA-21013
2025-11-12 17:20:25 +00:00
David Capwell 9a534ba06f Merge branch 'cassandra-5.0' into trunk 2025-11-07 11:53:48 -08:00
David Capwell 4091613298 Merge branch 'cassandra-4.1' into cassandra-5.0 2025-11-07 11:52:14 -08:00
David Capwell e4ea61c1cc Merge branch 'cassandra-4.0' into cassandra-4.1 2025-11-07 11:46:57 -08:00
Nikhil Kumawat 61014f2ae7 Leveled Compaction doesn't validate maxBytesForLevel when the table is altered/created
patch by Nikhil Kumawat, Nikhil; reviewed by David Capwell, guo Maxwell for CASSANDRA-20570
2025-11-07 11:44:12 -08:00
Stefan Miklosovic f0cb7f1c1a
Merge branch 'cassandra-5.0' into trunk 2025-11-07 14:09:41 +01:00
Stefan Miklosovic 5ea935d6b8
Merge branch 'cassandra-4.1' into cassandra-5.0 2025-11-07 14:04:21 +01:00
Stefan Miklosovic 7b312d4329
Merge branch 'cassandra-4.0' into cassandra-4.1 2025-11-07 13:57:43 +01:00
Ruslan Fomkin 2d7e99f7b9
Document new table name length limit
CASSANDRA-20389 implemented limit on table name length of 222
characters. This commit updates the documentation to reflect this.

It also adds an assert in CreateTableValidationTest to ensure that the
documented limit matches the actual constant.

patch by Ruslan Fomkin; reviewed by Brad Schoening, Dmitry Konstantinov for CASSANDRA-20914
2025-11-07 13:54:03 +01:00
Benedict Elliott Smith aa5c3aba1d Improve Topology Management
Merge ConfigurationService with TopologyManager to remove cyclic dependency and duplicated work.
Also:
 - Improve visibility of work blocking topology processing
 - Ensure we cannot double-count peers when deciding an epoch's distributed readiness
 - Remove the possibility of distributed stalls, by processing new topologies as soon as a
   contiguous sequence is known locally, regardless of whether anyprior local epoch is ready to
   coordinate or has recorded this fact to peers. The notification of local readiness continues
   to be processed only once all prior epochs are ready, but we can begin using and readying
   later epochs immediately.

patch by Benedict; reviewed by Alex Petrov for CASSANDRA-20998
2025-11-06 14:38:01 +00:00
Stefan Miklosovic 1a96b8803a
Enable CQLSSTableWriter to create SSTables compressed with a dictionary
patch by Stefan Miklosovic; reviewed by Yifan Cai for CASSANDRA-20938
2025-11-04 11:11:14 +01:00
Benedict Elliott Smith c9e31f297c Introduce "hardRemoved" concept for marking nodes permanently offline, to permit recovery from availability loss
Also fix:
 - Paxos should not update minHlc if AccordService is not setup
 - Remove EndpointMapper methods that require non-null, to ensure call-sites handle null case explicitly
 - AccordRepair should specify the Node Ids that must participate, but these should not affect the durability requirements for GC
 - Avoid erroneously marking UniversalOrInvalidated when only known to quorum
 - AccordSyncPropagator should not consult FailureDetector as it may throw UnknownEndpointException
 - Refuse ReplaceSameAddress if accord enabled
 - Reset Accord topologies on restart if no local command stores to ensure we can catch up (as intervening epochs may have otherwise been GC'd)
 - MaybeRecover should not abort if found durable if either: 1) prevProgressToken already witnessed durable; or 2) Home state has requested we recover anyway
 - Invariant in MaybeRecover is too strong now we permit recovering when known to be stable
 - ReadData must respond with InsufficientAndWaiting if not Stable to ensure Durability is inferred correctly by ExecuteTxn
 - Avoid StackOverflowException in NotifyWaitingOn
 - PreAccept should honour the REJECTED flag of any member of the latest topology if the vote would be relied upon for any prior epoch's quorum
 - Resume Accord bootstrap on restart
 - Fix burn test non-determinism
 - Reset Accord topologies on restart if no local command stores to ensure we can catch up (as intervening epochs may have otherwise been GC'd)
 - Exclude stale or removed ids from sync points
 - Terminate a failed DurabilityRequest if a required participant is now removed/hardRemoved
Also improve:
 - Introduce accord_debug_keyspace.{listeners_txn, listeners_local}
 - Filter and record faulty in AbstractCoordination
 - If we are the home shard, and our home progress status is done, we should not wait for the home shard to advance the waiting progress state
 - Introduce CommandStore.tryExecuteListening to try to execute all waiting transactions and their dependencies
 - ExecuteTxn on recovery should not wait, to avoid consuming progress log concurrency slots for transactions that cannot execute
 - HomeState should update its phase based on the command status to handle operator request to retry all states
 - Separate home/wait queue in DefaultProgressLog
 - Additional tracing
 - Do not call trySendMore from prerecordFailure
 - Force full recovery for sync points if we have lost quorum

patch by Benedict; reviewed by Alex Petrov for CASSANDRA-20921
2025-11-03 15:16:27 +00:00
Yifan Cai 307603a4ad Support ZSTD dictionary compression
patch by Yifan Cai; reviewed by Jon Haddad, Stefan Miklosovic for CASSANDRA-17021
2025-10-30 16:18:37 -07:00
Stefan Miklosovic 9142d0c851
Fix ExceptionsTable when stacktrace has zero elements
patch by Stefan Miklosovic; reviewed by Brandon Williams, Dmitry Konstantinov for CASSANDRA-20992
2025-10-30 17:37:17 +01:00
Stefan Miklosovic 80938477ef
ninja: get rid of warning during compilation in AccordCQLTestBase.setComplexWithReferenceOnAnotherColumn test by casting stmt.bindsEncoded() value to Object[] to pass to var-arg method 2025-10-29 17:01:44 +01:00
Benedict Elliott Smith e768418696 Accord: Improve Tracing
- Introduce pattern tracing, that can intercept failed or new coordinations matching various filters
     - Support additional tracing event collection modes (SAMPLE and RING)

    patch by Benedict; reviewed by Alex Petrov for CASSANDRA-20911
2025-10-29 10:16:32 +00:00
Benedict Elliott Smith b712cfb930 RemoteToLocalVirtualKeyspace: supporting access to all nodes' local virtual tables from any node in the cluster
patch by Benedict; reviewed by Alex Petrov for CASSANDRA-20900
2025-10-28 18:51:44 +00:00
Benedict Elliott Smith f7743fe3ab Lazy Virtual Tables
Also Improve:
 - Searchable system_accord_debug.txn
 - Integrate txn_blocked_by deadline and depth filter with execution logic, to ensure we terminate promptly and get a best effort reply

patch by Benedict; reviewed by Aleksey Yeschenko for CASSANDRA-20899
2025-10-28 11:19:37 +00:00
Alex Petrov 3d2b0efc34 Accord: Add Rebootstrap and unsafe Bootstrap
To support recovering a node that has lost some of its local transaction log, introduce rebootstrap and unsafe bootstrap modes, where Accord ensures no responses are produced for transactions the node cannot be certain it had not previously answered.

patch by Benedict and Alex Petrov for CASSANDRA-20908
2025-10-27 14:24:06 +00:00
Mick Semb Wever 74de57713a
Merge branch 'cassandra-5.0' into trunk
* cassandra-5.0:
  ninja-fix – Fix eclipse-warnings error for CASSANDRA-19564
  ReadCommandController should close fast to avoid deadlock when building secondary index
2025-10-27 10:02:07 +01:00
Mick Semb Wever d0c5f6b0ce
Merge branch 'cassandra-4.1' into cassandra-5.0
* cassandra-4.1:
  ninja-fix – Fix eclipse-warnings error for CASSANDRA-19564
  ReadCommandController should close fast to avoid deadlock when building secondary index
2025-10-27 09:52:51 +01:00
Runtian Liu 59574a86a3 ReadCommandController should close fast to avoid deadlock when building secondary index
patch by Runtian Liu; reviewed by Caleb Rackliffe, Jaydeepkumar Chovatia for CASSANDRA-19564
2025-10-26 17:39:39 -07:00
Doug Rohrer d40e4ecc09 Merge branch 'cassandra-5.0' into trunk 2025-10-24 14:40:15 -04:00
Doug Rohrer b4dcef7841 Merge branch 'cassandra-4.1' into cassandra-5.0 2025-10-24 14:26:39 -04:00
Doug Rohrer c988b609b0 Merge branch 'cassandra-4.0' into cassandra-4.1
* cassandra-4.0:
  CASSANDRA-20884 - Move JMX classes to the in-jvm-dtest API project
2025-10-24 14:23:42 -04:00
Doug Rohrer aa0e2f1631 CASSANDRA-20884 - Move JMX classes to the in-jvm-dtest API project
patch by Doug Rohrer; reviewed by Francisco Guerrero and Bernardo Botella Corbi for CASSANDRA-20884
2025-10-24 13:37:08 -04:00
Stefan Miklosovic 035a220d55
Implementation of CEP-55 - Generation of role names
patch by Stefan Miklosovic; reviewed by Bernardo Botella for CASSANDRA-20897
2025-10-23 13:53:56 +02:00
blambov 9cc3c085de Merge branch 'cassandra-5.0' into trunk 2025-10-20 16:30:49 +03:00
blambov 1db6f54925 Fix range queries on early-open BTI files
patch by Branimir Lambov; reviewed by Stefan Miklosovic for CASSANDRA-20976
2025-10-20 16:28:01 +03:00
Marcus Eriksson 0afa31cd5c Merge branch 'cassandra-5.0' into trunk 2025-10-14 13:21:30 +02:00
Marcus Eriksson e721705152 Avoid re-initializing underlying iterator in LazilyInitializedUnfilteredRowIterator after closing
Patch by marcuse; reviewed by Aleksey Yeschenko and Branimir Lambov for CASSANDRA-20972
2025-10-14 13:19:51 +02:00
Caleb Rackliffe 11278547e1 Merge branch 'cassandra-5.0' into trunk
* cassandra-5.0:
  Flush SAI segment builder when current SSTable writer is switched
2025-10-13 15:45:20 -05:00
Caleb Rackliffe 98e7cd5d99 Flush SAI segment builder when current SSTable writer is switched
patch by Pranav Shenoy; reviewed by Caleb Rackliffe and Zhao Yang for CASSANDRA-20752
2025-10-13 15:17:27 -05:00
Aparna Naik 61959e215c Add DDL Guardrail enabling administrators to disallow creation/modification of keyspaces with durable_writes = false
patch by Aparna Naik; reviewed by Caleb Rackliffe and Stefan Miklosovic for CASSANDRA-20913
2025-10-02 15:50:31 -05:00
Dmitry Konstantinov e6a97025bd Optimize Counter, Meter and Histogram metrics using thread local counters
Codahale metrics do not provide the ability to create custom metric implementations, so we have to inherit from Codahale classes.
For better cache locality rate and counter values are extracted to a common thread-local arrays.
Threads death is tracked using 2 approaches: FastThreadLocal.onRemoval callback and phantom references to Thread objects.
Phantom references are used to track aliveness of metric users and reusing of metric IDs.

Patch by Dmitry Konstantinov; reviewed by Benedict Elliott Smith for CASSANDRA-20250
2025-09-28 16:47:50 +01:00
Stefan Miklosovic 608a1b59e2
Merge branch 'cassandra-5.0' into trunk 2025-09-24 10:25:23 +02:00
Stefan Miklosovic b23db408c5
Merge branch 'cassandra-4.1' into cassandra-5.0 2025-09-24 10:22:16 +02:00
Stefan Miklosovic 4a201d4f6f
Merge branch 'cassandra-4.0' into cassandra-4.1 2025-09-24 10:20:21 +02:00
Yifan Cai 90f231ad56
Fixed incorrect error message constant for keyspace name length validation
patch by Yifan Cai; reviewed by Maxwell Guo, Stefan Miklosovic for CASSANDRA-20915
2025-09-24 10:13:04 +02:00
Stefan Miklosovic e42599a094
Expose uncaught exceptions in system_views.uncaught_exceptions table
patch by Stefan Miklosovic; reviewed by Dmitry Konstantinov for CASSANDRA-20858
2025-09-23 13:52:00 +02:00
Stefan Miklosovic 02ef8f8a0d
Merge branch 'cassandra-4.0' into cassandra-4.1 2025-09-19 14:18:29 +02:00
Stefan Miklosovic 818de8e11e
Merge branch 'cassandra-5.0' into trunk 2025-09-19 12:05:59 +02:00
Marko Tsymbaluk b26e33d506
Represent complex settings as JSON on system_views.settings table
The patch for 5.0 is preserving old behavior, it is possible to turn on JSON representation by a system property.
The patch for trunk is by default transforming collections to JSON string but there is the property (same as in 5.0) which has default to be set to true.

patch by Marko Tsymbaluk; reviewed by Paulo Motta, Stefan Miklosovic for CASSANDRA-20827

Co-authored-by: Stefan Miklosovic <smiklosovic@apache.org>
2025-09-19 11:58:23 +02:00
Benedict Elliott Smith 12000406c3 Follow-up to CASSANDRA-20906: Fix Simulator 2025-09-18 14:14:53 +01:00
Benedict Elliott Smith 0b2012826a Split AsyncChain and AsyncResult; normalise AsyncResult with C* Future
patch by Benedict; reviewed by Alex Petrov for CASSANDRA-20906
2025-09-18 12:31:13 +01:00
Benedict Elliott Smith c7de33c597 Strict Serializability Verifier may incorrectly handle blind writes when witnessed before the writing operation ACKs
patch by Benedict; reviewed by Alex Petrov for CASSANDRA-20905
2025-09-17 21:24:37 +01:00
jaydeepkumar1984 bf2c1c124e Improved observability in AutoRepair to report both expected vs. actual repair bytes and expected vs. actual keyspaces
patch by Jaydeepkumar Chovatia; reviewed by Chris Lohfink for CASSANDRA-20581
2025-09-16 20:38:39 -07:00
Ariel Weisberg d7a46b52ef Deterministic Simulator thread ids
Also: Don't ignore self reconcile test

patch by Ariel Weisberg; reviewed by David Capwell for CASSANDRA-20841
2025-09-15 16:44:00 +01:00
Maxwell Guo dddbe1ad87 Execution of CreateTriggerStatement should not rely on external state
patch by Maxwell Guo; reviewed by Sam Tunnicliffe for CASSANDRA-20287
2025-09-15 17:49:05 +08:00
Pranav Shenoy cf806cac1a Support LIKE expressions in filtering queries
patch by Pranav Shenoy; reviewed by Caleb Rackliffe and David Capwell for CASSANDRA-17198
2025-09-11 11:20:20 -05:00
Dmitry Konstantinov 06440e947a Merge branch 'cassandra-5.0' into trunk
* cassandra-5.0:
  Prevent too long table names not fitting file names
2025-09-10 23:39:17 +01:00
Dmitry Konstantinov 7f92e1ac2c Merge branch 'cassandra-4.1' into cassandra-5.0
* cassandra-4.1:
  Prevent too long table names not fitting file names
2025-09-10 23:09:58 +01:00
Dmitry Konstantinov bff43df7db Merge branch 'cassandra-4.0' into cassandra-4.1
* cassandra-4.0:
  Prevent too long table names not fitting file names
2025-09-10 22:59:34 +01:00
Ruslan Fomkin 6c29686ea7 Prevent too long table names not fitting file names
The length of table names was not controlled. This is likely due to
confusion between validation methods with similar names. As result
creating tables with too long names led to the too long file name
exceptions during table creations.

This commit adds a validation of table name lengths to avoid the too
long file name errors. The validation length is based on how the table
name is used to create file/directory names, and needs to be exact to
prevent the too long file name exception, but allow all other table
names, which didn't lead to the too long file name exception. This
length limit is different from the existing name length limit of 48
characters used by common validation functions.

Thus, this commit moves out the length validation from the validation
methods into a separate length validation method, so the errors on
names are more specific. The non-length validation methods combined
into a single method, which checks for empty names and valid characters.

New constants are added for the length limits.

Table name related code are moved into methods in TableMetadata class,
so their semantics are more clear and to allow reuse, e.g., in
asserting the table name length constant.
Tests are added for the long table names and non-alphanumeric names.

Keyspace name validation function is now shared between two classes and
a unit test of it is added.

Patch by Ruslan Fomkin; reviewed by Piotr Kołaczkowski, Dmitry
Konstantinov, Maxwell Guo for CASSANDRA-20389
2025-09-10 22:48:43 +01:00
Marcus Eriksson 55db3181fb Make legacy index rebuilds safe on Gossip -> TCM upgrades
patch by Caleb Rackliffe; reviewed by Marcus Eriksson for CASSANDRA-20887

Co-authored-by: Caleb Rackliffe <calebrackliffe@gmail.com>
Co-authored-by: Marcus Eriksson <marcuse@apache.org>
2025-09-09 13:59:26 -05:00
Benedict Elliott Smith b4f6c7c617 Topology compaction omits image records if minEpoch is ahead of last image
Also Fix:
 - RegisteredCallback should remove itself from callback map when cancelled
 - Do not throw CancellationException when processing requests that have been aborted, as may be caused by a successful meaningful reply that can be overridden
 - system_accord_debug.{executors,coordinations} fail with ClassCastException
 - CommandStore.updateMinHlc eats up CPU as called much too often
 - AccordCache not notifying flushed on shutdown
Also Improve:
 - Support skipping Deps
 - Violation information reported
 - Sort CommandStore shards by id

patch by Benedict; reviewed by Aleksey Yeschenko for CASSANDRA-20896
2025-09-09 11:11:23 +01:00
Himanshu Jindal 965a39166c Stop repair scheduler if two major versions are detected
patch by Himanshu Jindal; reviewed by Jaydeepkumar Chovatia, Andy Tolbert for CASSANDRA-20048
2025-09-08 09:26:31 -07:00
Benedict Elliott Smith b91731b5a3 All pending topology notifications must be propagated to a newly published epoch, not only the pending notification for that epoch
Also Fix:
 - JournalAndTableKeyIterator not merging in consistent order, which can lead to replaying topologies in non-ascending order
 - Invariant failure when reporting topologies if fetchTopologies on startup yields a gap
 - removeRedundantMissing could erroneously remove missing dependencies occurring after the new appliedBeforeIndex
 - Invariant failure for some propagate calls supplying 'wrong' addRoute
 - Disambiguate nextTxnId and nextTxnIdWithDefaultFlags
Also Improve:
 - LocalLog should not use NoSuchElementException for control flow (instead use ConcurrentSkipListMap)

patch by Benedict; reviewed by Aleksey Yeschenko for CASSANDRA-20886

 - JournalAndTableKeyIterator not merging in consistent order
2025-09-04 16:40:42 +01:00
Alex Petrov 0151131e43 Split out Topologies into Partitions
patch by Alex Petrov; reviewed by Benedict for CASSANDRA-20838
2025-09-03 19:04:45 +01:00
Alex Petrov ae673a31cc Startup sequence improvements follow-up
Patch by Alex Petrov; reviewed by Benedict Elliott Smith for CASSANDRA-20822
2025-09-03 16:08:15 +01:00
Benedict Elliott Smith a813a51f7c Improve Observability:
- Track all active Coordinations
 - Refactor Replica/Coordinator metrics and report Coordinator exhausted/preempted/timeout
 - DurabilityQueue metrics and visibility
Also Fix:
 - WaitingState can get cause distributed stall when asked to wait for CanApply if not yet PreCommitted; track separate querying state and advance this to the next achievable state rather than the desired final state
 - Stalled coordinators should not prevent recovery
 - Edge case with fetch unable to make progress when pre-bootstrap and all peers have GC'd
 - Dependency initialisation for sync points across certain ownership changes
 - SyncPoint propagation may not include all of the epochs required on the receiving node for ranges they have lost but not closed, and receiving node does not validate them
 - Stable tracker accounting with LocalExecute
 - Do not prune non-durable APPLIED as must be reported in dependencies until durably applied (so as not to break recovery)
 - Ensure we cannot race with replies when initiating Coordination
 - ProgressLog does not guarantee to clear home or waiting states when erased or invalidated by compaction
 - WaitingState on non-home shard cannot guarantee progress once home shard is Erased
 - WaitingOnSync handles retired ranges incorrectly
Also Improve:
 - Standardise failure accounting, use null to represent single reply timeouts
 - BurnTest record/replay to/from file

patch by Benedict; reviewed by Alex Petrov for CASSANDRA-20878
2025-09-02 18:47:46 +01:00
Stefan Miklosovic 9c93d52ac9
Implement nodetool history
patch by Stefan Miklosovic; reviewed by Maxim Muzafarov for CASSANDRA-20851
2025-09-01 16:18:19 +02:00
Stefan Miklosovic 63fb8741b8
Merge branch 'cassandra-5.0' into trunk 2025-09-01 13:00:36 +02:00
Stefan Miklosovic e45a330f54
Merge branch 'cassandra-4.1' into cassandra-5.0 2025-09-01 12:53:53 +02:00
Stefan Miklosovic c9717b4ed5
Redact security-sensitive information in system_views.settings
patch by Stefan Miklosovic; reviewed by Maxwell Guo, Francisco Guerrero for CASSANDRA-20856
2025-09-01 12:44:43 +02:00
Isaac Reath f41b625e48 Expose Metric for Prepared Statement Cache Size (in bytes)
Patch by Isaac Reath; Reviewed by Paulo Motta and guo Maxwell for
CASSANDRA-20864

Closes #4345
2025-08-28 10:54:00 -04:00
Marcus Eriksson f53c051118 Accord: Minor serialization changes
Patch by marcuse; reviewed by David Capwell and Benedict Elliott Smith for CASSANDRA-20840
2025-08-28 13:31:35 +02:00
David Capwell bf6a82ec82 SSTableLoaderEncryptionOptionsTest#bulkLoaderSuccessfullyStreamsOverSslWithDeprecatedSslStoragePort failing on missing stdout content
patch by David Capwell; reviewed by Caleb Rackliffe for CASSANDRA-20698
2025-08-27 10:32:07 -07:00
Stefan Miklosovic 792e823af0
Merge branch 'cassandra-4.0' into cassandra-4.1 2025-08-27 19:01:31 +02:00
Dmitry Konstantinov 1c5286f4d1
Fix IndexOutOfBoundsException in sstablemetadata tool when a range tombstone is a max clustering value
patch by Dmitry Konstantinov; reviewed by Stefan Miklosovic for CASSANDRA-20855
2025-08-27 18:54:10 +02:00
David Capwell 61b3d5a54b Add support for BEGIN TRANSACTION to allow mutations that touch multiple partitions
patch by David Capwell; reviewed by Ariel Weisberg for CASSANDRA-20857
2025-08-25 16:09:28 -07:00
Himanshu Jindal daef7b5b98 Safeguard Full repair against disk protection
As per CASSANDRA-20045, we want to prevent full repair against
disk full scenarios. Current protection exists only for incremental
repair. This change updates the config name to not be
incremental repair specific, using the Replace annotation.

patch by Himanshu Jindal; reviewed by David Capwell, Jaydeepkumar Chovatia for CASSANDRA-20045
2025-08-22 14:34:15 -07:00
Stefan Miklosovic b7e2b3b9dc
Merge branch 'cassandra-5.0' into trunk 2025-08-20 10:55:52 +02:00
Stefan Miklosovic 74e0794d92
Merge branch 'cassandra-4.1' into cassandra-5.0 2025-08-20 10:53:22 +02:00
Stefan Miklosovic 4824ecc4a5
Merge branch 'cassandra-4.0' into cassandra-4.1 2025-08-20 10:50:20 +02:00
Stefan Miklosovic eb9586dc68
Make secondary index implementations notified about rows in fully expired SSTables in compaction
patch by Stefan Miklosovic; reviewed by Branimir Lambov for CASSANDRA-20829
2025-08-20 10:42:01 +02:00
David Capwell acec78abf4 BEGIN TRANSACTION crashes if a mutation touches multiple rows
patch by David Capwell; reviewed by Ariel Weisberg for CASSANDRA-20844
2025-08-19 18:07:12 -07:00
Dmitry Konstantinov 657d9578f0 Fix version range check in MessagingService.getVersionOrdinal
ninja: remove unused imports in CommandsForRanges to pass checkstyle

Patch by Dmitry Konstantinov; reviewed by Stefan Miklosovic, Caleb Rackliffe for CASSANDRA-20842
2025-08-19 18:25:06 +01:00
Benedict Elliott Smith fdff319739 Accord fixes:
- Fix erroneous call to registerWithDelay with absolute deadline
 - ExecuteSyncPoint should discount votes from unbootstrapped replicas
 - Bootstrap no longer needs to first ensure durability (now ExecuteSyncPoint discounts votes from unbootstrapped replicas)
 - Progress stall with home shard stopping before Stable is propagated to all shards
 - Only upgrade Durability.HasPhase if we have queried all shards
 - Incorrectly inferring Durability in CheckStatus.finish
 - Incorrectly inferring/propagating stable from fast unstable reply; clarify to prevent further mistakes
 - MaxDecidedRX should track separate hlc bounds for filtering non-RX dependencies
 - minGcBefore.hlc() can move backwards across epochs, even if each shard moves forwards; must track minHlc directly
 - slowTimeout init in ExecuteTxn.LocalExecute
 - TxnId.parse for RV
Also improve:
 - Don't query recovery state if fast path durably decided
 - Support deferred partial dep deserialisation
 - Support both orientations of KeyDeps/RangeDeps
 - Support partialDeps as ByteBuffer
Also improve in Cassandra:
 - Deps serialization and Journal skipping
 - Don't inflate when reading from cache for CommandsForRanges

patch by Benedict; reviewed by Alex Petrov for CASSANDRA-20847
2025-08-19 15:20:26 +01:00
Stefan Miklosovic 9d5cef7f8c
Allow custom constraints to be loaded via SPI
patch by Stefan Miklosovic; reviewed by Bernardo Botella, Saranya Krishnakumar for CASSANDRA-20824
2025-08-18 10:05:38 +02:00
Alex Petrov 59b45eb8bf Add max depth to blocked txn vtable
Patch by Alex Petrov; reviewed by Benedict Elliott Smith for CASSANDRA-20839
2025-08-18 09:39:22 +02:00
Maxim Muzafarov ca1dca902c
Fix ShortPaxosSimulationTest and AccordSimulationRunner do not execute from the cli
patch by Maxim Muzafarov; reviewed by Ariel Weisberg for CASSANDRA-20805
2025-08-15 10:46:47 +02:00
Paulo Motta b2037e473f
Allow overriding arbitrary settings via environment variables
This also allows overriding complex settings as a JSON value and adds documentation about these overrides to conf/jvm-server.options

patch by Paulo Motta; reviewed by Stefan Miklosovic, David Capwell for CASSANDRA-20749
2025-08-14 19:00:30 +02:00
Alex Petrov 2de6078289 Improve / simplify topology reporting on startup
Patch by Alex Petrov; reviewed by Benedict Elliott Smith for CASSANDRA-20822
2025-08-14 09:39:22 +02:00
Benedict Elliott Smith 81bc18ad63 Enrich Durability with each phase, so we can both prune unapplied dependencies to mitigate replicas that are behind causing Deps growth,
and more reliably avoid initialising recovery progress log state of transactions that cannot yet make progress
Also fix:
 - Harden AccordExecutor state cleanup to failures
 - Handle SAVING state in AccordCache.tryEvict, as now possible to save for reasons besides eviction so normal to both be in evict queue and saving
 - Infer invalid in MaybeRecover and FetchData
 - MaybeRecover sometimes aborts before home shard knows outcome
 - Epoch sync with VisibilitySyncPoint
 - Retired implies synced
 - Don't interpret force repair as excluding nodes from Accord sync conditions
 - TxnData.without
Also improve:
 - Add Topology.removedNodes
 - If Durability implies we can fetch a status, update the waiting state to fetch it
 - DurableBefore debug table should have searchable txnId

patch by Benedict; reviewed by Alex Petrov for CASSANDRA-20832
2025-08-13 11:10:18 +01:00
Dmitry Konstantinov 7546d9e895 Optimize MessagingService.getVersionOrdinal
The map lookup (with an implicit auto-boxing) can be replaced with a plain arithmetic operation.
A unit test is used to ensure that the assumption about serialization version incrementing for new versions is still correct.
greaterThanOrEqual method is removed as unused.

Patch by Dmitry Konstantinov; reviewed by Stefan Miklosovic for CASSANDRA-20816
2025-08-08 14:20:13 +01:00
Dmitry Konstantinov 19118a6cff Optimize TrieMemtable#getFlushSet
A flushing range is a combination of partial and full shards, for full shards we use write time captured values for count and total size of partition keys, for partial shards we iterate over keys
When we iterate over partial shards we count key bytes only instead of reading them and also try to skip tokens full parsing

Patch by Dmitry Konstantinov; reviewed by Branimir Lambov for CASSANDRA-20760
2025-08-08 13:34:34 +01:00
Stefan Miklosovic 429b2f8be4
Merge branch 'cassandra-5.0' into trunk 2025-08-08 09:16:21 +02:00
Isaac Reath 170bf57eac
Fix ThreadLocalReadAheadBufferTest to avoid FileNotFoundException when -Djava.io.tmpdir set to /tmp
patch by Isaac Reath; reviewed by Paulo Motta, Stefan Miklosovic for CASSANDRA-20819
2025-08-08 09:00:57 +02:00
David Capwell 6470020dc2 Fix org.apache.cassandra.simulator.test.EpochStressTest so it can run again
patch by David Capwell; reviewed by Alex Petrov for CASSANDRA-20812
2025-08-07 11:21:09 -07:00
Benedict Elliott Smith 7b12404a4e Avoid cache modification reentrancy when cancelling loads
Also fix:
 - Catch exceptions thrown by Task.cleanupExclusive to ensure we do not corrupt SequentialExecutor state
 - ConcurrentModificationException with CommandStore.waitingOnSync
 - Exclude promisedHlc that are in the future, even if not equal to Long.MAX_VALUE
 - Filter CFK on load as not expected to be up to date (for later on demand filtering validation)

patch by Benedict; reviewed by Alex Petrov for CASSANDRA-20821
2025-08-07 12:45:05 +01:00
Stefan Miklosovic bb7e706213
Merge branch 'cassandra-5.0' into trunk 2025-08-06 18:37:11 +02:00
Romain Hardouin 3b9669b116
Sort SSTable TOC entries for determinism
patch by Romain Hardouin; reviewed by Stefan Miklosovic, Dmitry Konstantinov for CASSANDRA-20494

Co-authored-by: Stefan Miklosovic <smiklosovic@apache.org>
2025-08-06 18:34:26 +02:00
Benedict Elliott Smith 46bda174d5 Bootstrap must first wait for a quorum to apply the sync point to ensure the DurableBefore Majority condition holds transitively,
since later quorums may include the bootstrapping node which does not participate in the durability of preceding transactions
Also Improve:
 - Apply MaxDecidedRX filtering to CommandsForKey RX dependencies
 - Optimise AbstractRanges.sliceMinimal (burn test hotspot)
 - Don't start shard schedulers on topology changes if not already started
 - Only registerTransitive if waitingOnSync
 - Reserve DurabilityRequest until ShardDurability is started
 - Don't notify progress log if stopped
 - Remove duplicated CFK unmanaged filtering
Also fix:
 - Edge case in notification across bootstrap boundary by resubmitting Unmanaged to be recomputed
 - Serialization of CommandsForKey.TxnInfo.missing() collection when non-identity flag bits are present

patch by Benedict; reviewed by Alex Petrov for CASSANDRA-20811
2025-08-04 08:41:24 +01:00
Benedict Elliott Smith 54b2f6acbc Improve:
- system_accord_debug.txn_update->txn_ops + force_apply, force_update, try_execute
 - system_accord_debug.commands_for_key
 - system_accord_debug.commands_for_key_unmanaged
 - system_accord_debug.redundant_before + locally_durable_to_X_store
 - flush accord caches on shutdown

patch by Benedict; reviewed by Alex Petrov for CASSANDRA-20814
2025-08-03 15:16:00 +01:00
Caleb Rackliffe eb65c0e600 Support manual secondary index selection at the CQL level
patch by Caleb Rackliffe; reviewed by Andres de la Peña for CASSANDRA-18112

Co-authored-by: Caleb Rackliffe <calebrackliffe@gmail.com>
Co-authored-by: Andres de la Peña <a.penya.garcia@gmail.com>
2025-08-01 14:42:27 -05:00
Stefan Miklosovic 82e1c3d1c0
Merge branch 'cassandra-5.0' into trunk 2025-07-31 11:16:37 +02:00
Stefan Miklosovic 345df4e723
Merge branch 'cassandra-4.1' into cassandra-5.0 2025-07-31 11:15:15 +02:00
Dmitry Konstantinov b4210acd3d
Improve CommitLogSegmentReader to skip SyncBlocks correctly in case of CRC errors
patch by Dmitry Konstantinov; reviewed by Stefan Miklosovic for CASSANDRA-20664
2025-07-31 11:09:11 +02:00
David Capwell 6adac494aa When regulars CQL mutations run on Accord use the txn timestamp rather than server timestamp
patch by David Capwell; reviewed by Ariel Weisberg for CASSANDRA-20744
2025-07-30 13:19:41 -07:00
Stefan Miklosovic 5aeaef5f77
Merge branch 'cassandra-5.0' into trunk 2025-07-30 17:23:14 +02:00
Stefan Miklosovic cf83f8b2a4
Merge branch 'cassandra-4.1' into cassandra-5.0 2025-07-30 17:21:45 +02:00
Isaac Reath fcaa1b3e39
Do not crash on first boot with data_disk_usage_max_disk_size set when data directory is not created yet
patch by Isaac Reath; reviewed by Paulo Motta, Stefan Miklosovic for CASSANDRA-20787
2025-07-30 17:19:10 +02:00
David Capwell 8c9627076b When using BEGIN TRANSACTION if a complex mutation exists in the same statement as one that uses a reference, then the complex delete is dropped
patch by David Capwell; reviewed by Caleb Rackliffe for CASSANDRA-20788
2025-07-29 14:18:36 -07:00
Benedict Elliott Smith 4102ea13e5 Fix CFK restore after replay:
- Remove from CFK any unapplied transactions we know cannot apply
 - Force notification of waiting commands in CFK on replay
Also fix:
 - Don't truncateWithOutcome if pre bootstrap or stale
 - Fix RangeDeps.without(RangeDeps)
 - Fix InMemoryCommandStore replay bug with clearing DefaultLocalListeners
 - Ensure SaveStatus and executeAt are updated together to prevent corruption via expunge
Improve:
 - Inform home shard that command is decided if we cannot execute, to avoid recovery contention
 - Don't recover sync points on the fast path
 - Don't calculate recovery info for RX (since we don't use it anymore, so no need to do the work)

patch by Benedict; reviewed by Alex Petrov for CASSANDRA-20797
2025-07-28 13:32:58 +01:00
Stefan Miklosovic 87069e4dec
Merge branch 'cassandra-5.0' into trunk
This commit only contains in-jvm dtest, logically same as in 5.0, which is ignored as it would fail.
It was identified as part of CASSANDRA-19902 review that trunk branch returns STARTING instead of JOINING operation mode.
This discrepancy between 5.x and trunk is meant to be solved under CASSANDRA-20795.
2025-07-28 09:48:18 +02:00
Paulo Motta 07163cfb51
Fix StorageService JMX mbean not available during bootstrap
Patch by Paulo Motta; reviewed by Stefan Miklosovic for CASSANDRA-19902
2025-07-28 09:28:02 +02:00
Maxim Muzafarov 94b251f3ce
Migrate all nodetool commands from airline to picocli
Replace io.airlift:airline dependency with info.picocli:picocli across
all nodetool command implementations. This migration includes:

- Convert 160+ nodetool command classes from airline to picocli
- Add AbstractCommand base class for commands and utility methods
- Add NodetoolCommand top-level class for the cli utility
- Update all command classes to use @Command, @Option, and @Parameters
- Add plain text files to test command help output
- Add mock test classes for improved test coverage and parse validation
- Modify test infrastructure to work with picocli-based commands
- Add new layouts for cli formatting to preserve backwards compatibility

The migration maintains backward compatibility while providing improved
command-line parsing, better help system, and more robust argument
validation through picocli enhanced features.

patch by Maxim Muzafarov; reviewed by Caleb Rackliffe, Dmitry Konstantinov, Stefan Miklosovic for CASSANDRA-17445
2025-07-26 18:19:18 +02:00
Maxim Muzafarov f18d5ad851
ninja: remove unused imports in JournalGCTest and AccordCommandStore to pass checkstyle 2025-07-26 00:45:21 +02:00
David Capwell e48d0dcd2c Journal.TopologyUpdate should not store the local topology as it can be inferred from the global on
patch by David Capwell; reviewed by Benedict Elliott Smith for CASSANDRA-20785
2025-07-24 18:51:24 -07:00
Benedict Elliott Smith 49b99c9319 Fix DurableBefore updates:
- DurableBefore.fullyContainedIn should compare txnId
Also fix:
 - Messages with nullable state should check if they are cancelled after reading the state but before using it
 - Don't snapshot during replay
 - CFK.updateRedundantBefore incorrectly updating mayExecute and sharing byId in some cases
 - Refine GC with global durability bound
 - Fix JournalGCTest

patch by Benedict; reviewed by Alex Petrov for CASSANDRA-20786
2025-07-24 20:54:35 +01:00
Benedict Elliott Smith fbe12e6291 Accord Query Tracing
Also improve:
 - Metric listeners

patch by Benedict; reviewed by David Capwell for CASSANDRA-20773
2025-07-24 20:20:17 +01:00
Ariel Weisberg 88805177d2 Fix AccordWriteInteroperability message validation
Patch by Ariel Weisberg; Reviewed by David Capwell for CASSANDRA-20770
2025-07-24 11:07:22 -04:00
Stefan Miklosovic e8e312b4d1
Merge branch 'cassandra-5.0' into trunk 2025-07-24 10:27:58 +02:00
Stefan Miklosovic 922819dc08
Merge branch 'cassandra-4.1' into cassandra-5.0 2025-07-24 10:25:54 +02:00
Stefan Miklosovic f5f71b7fae
Rework / simplification of nodetool get/setguardrailsconfig commands
patch by Stefan Miklosovic; reviewed by Maxim Muzafarov for CASSANDRA-20778
2025-07-24 10:22:55 +02:00
Alex Petrov 53eb2fdac8 Improve GC testing; fix durability startup order
Patch by Alex Petrov; reviewed by Benedict Elliott Smith for CASSANDRA-20767
2025-07-23 14:59:12 +02:00
Benedict Elliott Smith 54e39a90b0 Limit replay to those records not durably persisted to both command store and data store
Also fix:
 - Limit truncation to TruncatedApplyWithOutcome until data is persisted durably to local store
 - IntervalUpdater not invoking super.close()
 - Do not invoke preRunExclusive without holding lock
 - IntervalUpdater not correctly initialise BranchBuilder.inUse
 - AccordExecutor should notify if more work on unlock of caches
 - Relax paranoid CFK validation during restart
Also improve:
 - Flush logs before System.exit
 - Start/stop progress log explicitly
 - Limit progress log concurrency
 - Clear heavy fields in some messages once processed

patch by Benedict; reviewed by Alex Petrov for CASSANDRA-20780
2025-07-23 12:19:06 +01:00
Alex Petrov 260c9c9334 Add a way to patch accord journal commands via virtual tables
Patch by Alex Petrov; reviewed by Benedict Elliott Smith for CASSANDRA-20765
2025-07-23 11:06:33 +01:00
David Capwell 0574db7bd9 Accord Route Index does not filter based off txn_id range which causes segments which cant contain the data to be queried
patch by David Capwell; reviewed by Benedict Elliott Smith for CASSANDRA-20772
2025-07-22 15:21:14 -07:00
David Capwell 8014eec7aa Merge branch 'cassandra-5.0' into trunk 2025-07-18 16:06:36 -07:00
David Capwell 169c37b030 Merge branch 'cassandra-4.1' into cassandra-5.0 2025-07-18 16:04:46 -07:00
Yuqi Yan 4962f3da3f IntrusiveStack.accumulate is not accumulating correctly
patch by Yuqi Yan; reviewed by Benedict Elliott Smith, David Capwell for CASSANDRA-20670
2025-07-18 16:02:17 -07:00
Benedict Elliott Smith d229eaf667 ninja: remove accidental file 2025-07-18 13:10:38 +01:00
Stefan Miklosovic 418e3a4899
Merge branch 'cassandra-5.0' into trunk 2025-07-18 07:52:31 +02:00
Stefan Miklosovic 5fb3ae4473
Merge branch 'cassandra-4.1' into cassandra-5.0 2025-07-18 07:41:32 +02:00
Yuqi Yan 5c94bc8820
Add nodetool get/setguardrailsconfig commands
patch by Yuqi Yan; reviewed by Brandon Williams, Maxwell Guo, Stefan Miklosovic for CASSANDRA-19552

Co-authored-by: Stefan Miklosovic <smiklosovic@apache.org>
2025-07-18 07:25:17 +02:00
Benedict Elliott Smith 0e9f0fab78 Improve:
- PreLoadContext descriptions
 - Introduce LoadKeysFor so we can avoid loading range transactions except where necessary

patch by Benedict; reviewed by Alex Petrov for CASSANDRA-20758
2025-07-17 07:21:04 +01:00
Benedict Elliott Smith 37b6ade9c9 Accord: Fix minDecidedId calculation
Also fix:
 - slowCoordinatorDelay calculation for locally truncated command throws ISE
 - Bad cast of Truncated during replay
 - Don't throw IllegalStateException in Invariants.expect if accord.testing == false
 - Replay of empty segment throws NPE
Also improve:
 - Support tracing of recovery and home progress

patch by Benedict; reviewed by Alex Petrov for CASSANDRA-20771
2025-07-16 19:37:26 +01:00
Ariel Weisberg e6cf2132ab Add testing of consensus live migration to simulator
Patch by Ariel Weisberg; Reviewed by Benedict Elliott Smith for CASSANDRA-20587
2025-07-16 11:31:54 -04:00
Alex Petrov 54a9c407bd Improve Interval B-Tree test coverage; add pprinter for B-Trees
Patch by Alex Petrov; reviewed by Benedict Elliott Smith for CASSANDRA-20766
2025-07-16 11:32:51 +02:00
Benedict Elliott Smith e8bd2319f4 Accord Fixes:
- Don't try to update metrics when no TxnId (e.g. GetMaxConflict)
 - maybeExecuteImmediately did not guarantee mutual exclusivity
 - Cancellation of a running 'plain' task could corrupt AccordExecutor state
 - GetLatestDeps message serializer
 - txn_blocked_by StackOverflowError
 - Not updating CommandsForKey in all cases on restart
Also Improve:
 - TableId.from/toString
 - Route toString methods
 - Tracing coverage of FetchRoute
 - Replay command store parallelism

patch by Benedict; reviewed by Alex Petrov for CASSANDRA-20763
2025-07-14 23:22:45 +01:00
Alex Petrov ccf12ef845 Add accord journal standalone dump tool
Patch by Alex Petrov; reviewed by Benedict Elliott Smith for CASSANDRA-20738
2025-07-12 13:38:53 +02:00
Benedict Elliott Smith 674b8d5100 Deduplicate ranges in WatermarkCollector
Also: collect ranges before submitting to TopologyManager to avoid quadratic complexity

patch by Benedict; reviewed by Alex Petrov for CASSANDRA-20757
2025-07-12 10:36:53 +01:00
David Capwell 28edf72bce Fix flakey test org.apache.cassandra.utils.SimpleBitSetSerializerTest
patch by David Capwell; reviewed by Benedict Elliott Smith, David Capwell for CASSANDRA-20759
2025-07-11 14:09:50 -07:00
Benedict Elliott Smith 7e32d92d06 Fix/Improve IntervalBTree
- Use AsymmetricComparator for seeking
 - Reimplement AbstractTransformer for more efficienct operation when only transforming a subset (i.e. Subtraction)
 - Reuse source node IntervalMaxIndex calculations for first branch level (to save iterating leaves)

patch by Benedict; reviewed by Alex Petrov for CASSANDRA-20756
2025-07-11 19:42:03 +01:00
Benedict Elliott Smith 92cbde1fde Introduce PseudoUtf8Type for virtual tables to search on types that are otherwise treated as strings for pretty printing;
Use this facility to make Accord debug virtual tables more useful

patch by Benedict; reviewed by Alex Petrov for CASSANDRA-20755
2025-07-11 19:00:26 +01:00
Benedict Elliott Smith fd2e11f0d2 Accord Fixes:
- WatermarkCollector should not report same closed/retired epoch N times
 - AccordSyncPropagator can merge pending requests and back-off retries
 - AccordCommandLoader should notify listeners
 - AccordSegmentCompactor should estimate number of keys to ensure bloom filters work
 - txn_blocked_by table should report what it can, not throw IllegalStateException
 - Permit uncompressed system tables

patch by Benedict; reviewed by Alex Petrov for CASSANDRA-20754
2025-07-11 15:19:36 +01:00
David Capwell 8289bb4cde Accord: Topology serializer has a lot of repeated data, can dedup to shrink the cost
patch by David Capwell; reviewed by Benedict Elliott Smith for CASSANDRA-20715
2025-07-10 20:06:02 -07:00
Marcus Eriksson f6c1002e44 Stream individual files in their own transactions and hand over ownership to a parent transaction on completion
patch by Marcus Eriksson; reviewed by Caleb Rackliffe and Jon Meredith for CASSANDRA-20728
2025-07-10 12:57:51 -05:00
Benedict Elliott Smith a866f92fcf Improve:
- Journal debugging vtable support
 - Background task tracing support
Fix:
 - HLC_BOUND only valid for strictly lower HLC
 - HAS_UNIQUE_HLC can only be safely computed if READY_TO_EXECUTE
 - Break recursion in CommandStore.ensureReadyToCoordinate
 - Fix find intersecting shard scheduler
 - Separate adhoc ShardScheduler from normal to avoid overwriting
 - Should still use execution listeners to detect invalid reads for certain transactions even with dataStoreDetectsFutureReads

patch by Benedict; reviewed by Alex Petrov for CASSANDRA-20746
2025-07-04 09:08:50 +01:00
Caleb Rackliffe 30f094080f Merge branch 'cassandra-5.0' into trunk
* cassandra-5.0:
  Ensure CassandraIndexTest sets up properly when using SAI in the wake of CASSANDRA-20732
2025-07-02 12:04:23 -05:00
Caleb Rackliffe 81fcacdfa4 Ensure CassandraIndexTest sets up properly when using SAI in the wake of CASSANDRA-20732
patch by Caleb Rackliffe; reviewed by Stefan Miklosovic for CASSANDRA-20740
2025-07-02 11:52:34 -05:00
Benedict Elliott Smith c9e52399e6 Fix:
- Lock inversion when restarting durability for erased sync point
 - Initialise durability cycle start time
Improve:
 - Improve durability reporting
 - Timeout durability requests
 - DurabilityQueue concurrency should limit only until quorum is achieved
 - Some exceptions that may be thrown when starting coordination may not be propagated

patch by Benedict; reviewed by Alex Petrov for CASSANDRA-20328
2025-07-02 14:09:45 +01:00
Benedict Elliott Smith 0a7797ca85 Fix:
- Cannot shrink CommandsForKey if loading pruned
 - NoSpamLogger to suppress AccordSyncPropagator repeats
 - Minor performance improvement to BTree.Subtraction
 - EphemeralReads should retry in a later epoch if replication factor changes
 - Fix no local epoch for NotAccept
 - Invalidate a command with no route but for which we know we own some key that has applied locally
 - Don't pre-merge existing DurableBefore, to reduce duplicate/redundant persistence
 - Misc purging bugs and improve testing of purging

patch by Benedict; reviewed by Alex Petrov for CASSANDRA-20739
2025-06-29 12:59:09 +01:00
David Capwell 78d6892dfb Fix flakey test AbstractTypeTest#meaninglessEmptyness why is equality so hard
patch by David Capwell; reviewed by Caleb Rackliffe for CASSANDRA-20733
2025-06-27 12:29:00 -07:00