Commit Graph

9019 Commits

Author SHA1 Message Date
Caleb Rackliffe 567e93ba3b ninja-fix: Make sure the MutationJournal starts before trying to create tracked keyspaces in unit-level tests that call SchemaLoader.schemaDefinition() 2025-10-27 15:40:24 -05:00
Caleb Rackliffe 78466146f8 Enable range reads for MultiNodeTableWalkWithMutationTrackingTest and document failure modes for known problematic seeds
patch by Caleb Rackliffe; reviewed by Blake Eggleston for CASSANDRA-20898
2025-10-22 15:48:38 -05:00
Aleksey Yeschenko 1bf039709e Truncate mutation journal as logs get reconciled
patch by Aleksey Yeschenko; reviewed by Alex Petrov for CASSANDRA-20710
2025-10-08 14:02:22 +01:00
Caleb Rackliffe b67f4e2c2a Disable mutation tracking by default in CassandraGenerators
patch by Caleb Rackliffe; reviewed by Blake Eggleston for CASSANDRA-20944
2025-10-06 20:11:32 -05:00
Abe Ratnofsky 7f4bac3efb Fix failing test: org.apache.cassandra.replication.CoordinatorLogOffsetsTest#reconciledBounds 2025-10-01 18:13:25 -04:00
Blake Eggleston 6067fb62a7 CEP-45: Support host replacement with tracked keyspaces
Patch by Blake Eggleston; Reviewed by Abe Ratnofsky for CASSANDRA-20891
2025-09-29 15:26:41 -07:00
Ariel Weisberg 698ea0cf74 Fix failing unit tests from witness merge
Patch by Ariel Weisberg; Reviewed by Blake Eggleston for CASSANDRA-20934

Co-authored-by: Blake Eggleston <blake@ultrablake.com>
2025-09-29 12:49:04 -07:00
Alex Petrov 31f81c0b2c CEP-45: Integrate Mutation Journal with Memtable flush tracking
* Tracking of Memtable -> SSTable flushes in Mutation Journal segments
  * Replay of Mutation Journal segments that were not fully flushed to SSTables
  * Distinguish between CommitLog and MutationJournal offsets
  * Effectively adds support for node bounces

Patch by Alex Petrov; reviewed by Aleksey Yeschenko for CASSANDRA-20919
2025-09-29 20:59:58 +02:00
Ariel Weisberg 2403bfcaac Initial witness non-serial single partition read/write path
Patch by Ariel Weisberg; Reviewed by Abe Ratnosfky for CASSANDRA-20930
2025-09-26 13:03:19 -07:00
Aleksey Yeschenko 942fa5a3f3 Fix CoordinatorLogOffsetsLifecycleTest 2025-09-25 14:34:44 +01:00
Aleksey Yeschenko bd5a657548 Implement Shard and CoordinatorLog metadata durability
patch by Aleksey Yeschenko; reviewed by Abe Ratnofsky for CASSANDRA-20882
2025-09-19 15:55:11 +01:00
Blake Eggleston 8b7753f7ec Single-partition read fixes:
- filter augmenting mutations and add comment explaining why we don't exclude reconciled mutations when augmenting
- propagate exceptions during acknowledgeReconcile
- fix UnreconciledMutations handling of single key bounds
- fix single offset remove bug

patch by Blake Eggleston; reviewed by Caleb Rackliffe for CASSANDRA-20830
2025-09-05 17:46:15 -05:00
Caleb Rackliffe 4bc014434a Enable single-partition AST fuzz tests w/ mutation tracking
patch by Caleb Rackliffe; reviewed by ? for CASSANDRA-20830
2025-09-05 17:46:15 -05:00
Aleksey Yeschenko c342b8dd07 Fix failing repair tests 2025-08-28 12:18:58 +01:00
Aleksey Yeschenko 418a19dc31 Implement unified log reconciliation
patch by Aleksey Yeschenko; reviewed by Blake Eggleston for
CASSANDRA-20729

Co-authored-by: Aleksey Yeschenko <aleksey@apache.org>
Co-authored-by: Blake Eggleston <blake@ultrablake.com>
2025-08-26 17:35:16 +01:00
Caleb Rackliffe 6430d497bb Fixed LegacySSTableTest to reflect thew new be_index_summary table type 2025-08-14 12:41:08 -05:00
Caleb Rackliffe d8f67d77cf Fixed output format expectations in DescribeStatementTest 2025-08-14 11:48:53 -05:00
Caleb Rackliffe a99663c197 Separate TestParams by use-case (Accord vs. Mutation Journal) 2025-08-14 11:31:24 -05:00
Caleb Rackliffe f9c9fe63b4 Initial round of post-rebase (mostly Accord-related) fixes 2025-08-14 00:19:26 -05:00
Blake Eggleston 83cc3212ee CEP-45: Fix ALLOW FILTERING queries for mutation tracking
Patch by Blake Eggleston; Reviewed by Aleksey Yeschenko for CASSANDRA-20555
2025-08-13 23:31:22 -05:00
Abe Ratnofsky b004207b32 Fully reconciled SSTable promotion
patch by Abe Ratnofsky; reviewed by Ariel Weisberg, Blake Eggleston for CASSANDRA-20381
2025-08-13 23:24:24 -05:00
Abe Ratnofsky 822c5d572a Add mutation tracking summary to SSTables
Patch by Abe Ratnofsky; reviewed by Ariel Weisberg, Blake Eggleston for CASSANDRA-20336
2025-08-13 22:59:26 -05:00
Aleksey Yeschenko ea9e041345 Fix-up offset broadcasting logic
patch by Aleksey Yeschenko; reviewed by Abe Ratnofsky for CASSANDRA-20576
2025-08-13 22:21:42 -05:00
Abe Ratnofsky 88c2c769b4 Fixup inter-DC forwarding of writes from a coordinator-replica (fixup)
Patch by Abe Ratnofsky; Reviewed by Aleksey Yeshchenko for CASSANDRA-20336
2025-08-13 22:21:42 -05:00
Aleksey Yeschenko 59922b9f5d Implement coordinator log offset broadcasting
patch by Aleksey Yeschenko; reviewed by Blake Eggleston for CASSANDRA-20576
2025-08-13 22:21:42 -05:00
Abe Ratnofsky 985aaa5ca8 CEP-45: Query forwarding
Patch by Abe Ratnofsky; Reviewed by Blake Eggleston for CASSANDRA-20309
2025-08-13 22:21:38 -05:00
Blake Eggleston 217753c3d0 Mutation tracking journal integration, read, and write path
Patch by Blake Eggleston & Aleksey Yeshchenko
Reviewed by Blake Eggleston, Aleksey Yeshchenko, & Abe Ratnofsky
 for CASSANDRA-20304, CASSANDRA-20305, & CASSANDRA-20308, CASSANDRA-20373

Co-authored-by: Blake Eggleston <blake@ultrablake.com>
Co-authored-by: Aleksey Yeschenko <aleksey@apache.org>
2025-08-13 16:18:11 -05:00
Aleksey Yeschenko 9ac462ba55 Introduce MutationJournal for coordinator logs
patch by Aleksey Yeschenko; reviewed by Blake Eggleston for
CASSANDRA-20353
2025-08-12 16:28:32 -05:00
Dmitry Konstantinov 7546d9e895 Optimize MessagingService.getVersionOrdinal
The map lookup (with an implicit auto-boxing) can be replaced with a plain arithmetic operation.
A unit test is used to ensure that the assumption about serialization version incrementing for new versions is still correct.
greaterThanOrEqual method is removed as unused.

Patch by Dmitry Konstantinov; reviewed by Stefan Miklosovic for CASSANDRA-20816
2025-08-08 14:20:13 +01:00
Dmitry Konstantinov 19118a6cff Optimize TrieMemtable#getFlushSet
A flushing range is a combination of partial and full shards, for full shards we use write time captured values for count and total size of partition keys, for partial shards we iterate over keys
When we iterate over partial shards we count key bytes only instead of reading them and also try to skip tokens full parsing

Patch by Dmitry Konstantinov; reviewed by Branimir Lambov for CASSANDRA-20760
2025-08-08 13:34:34 +01:00
Stefan Miklosovic 429b2f8be4
Merge branch 'cassandra-5.0' into trunk 2025-08-08 09:16:21 +02:00
Isaac Reath 170bf57eac
Fix ThreadLocalReadAheadBufferTest to avoid FileNotFoundException when -Djava.io.tmpdir set to /tmp
patch by Isaac Reath; reviewed by Paulo Motta, Stefan Miklosovic for CASSANDRA-20819
2025-08-08 09:00:57 +02:00
David Capwell 6470020dc2 Fix org.apache.cassandra.simulator.test.EpochStressTest so it can run again
patch by David Capwell; reviewed by Alex Petrov for CASSANDRA-20812
2025-08-07 11:21:09 -07:00
Benedict Elliott Smith 7b12404a4e Avoid cache modification reentrancy when cancelling loads
Also fix:
 - Catch exceptions thrown by Task.cleanupExclusive to ensure we do not corrupt SequentialExecutor state
 - ConcurrentModificationException with CommandStore.waitingOnSync
 - Exclude promisedHlc that are in the future, even if not equal to Long.MAX_VALUE
 - Filter CFK on load as not expected to be up to date (for later on demand filtering validation)

patch by Benedict; reviewed by Alex Petrov for CASSANDRA-20821
2025-08-07 12:45:05 +01:00
Stefan Miklosovic bb7e706213
Merge branch 'cassandra-5.0' into trunk 2025-08-06 18:37:11 +02:00
Romain Hardouin 3b9669b116
Sort SSTable TOC entries for determinism
patch by Romain Hardouin; reviewed by Stefan Miklosovic, Dmitry Konstantinov for CASSANDRA-20494

Co-authored-by: Stefan Miklosovic <smiklosovic@apache.org>
2025-08-06 18:34:26 +02:00
Benedict Elliott Smith 46bda174d5 Bootstrap must first wait for a quorum to apply the sync point to ensure the DurableBefore Majority condition holds transitively,
since later quorums may include the bootstrapping node which does not participate in the durability of preceding transactions
Also Improve:
 - Apply MaxDecidedRX filtering to CommandsForKey RX dependencies
 - Optimise AbstractRanges.sliceMinimal (burn test hotspot)
 - Don't start shard schedulers on topology changes if not already started
 - Only registerTransitive if waitingOnSync
 - Reserve DurabilityRequest until ShardDurability is started
 - Don't notify progress log if stopped
 - Remove duplicated CFK unmanaged filtering
Also fix:
 - Edge case in notification across bootstrap boundary by resubmitting Unmanaged to be recomputed
 - Serialization of CommandsForKey.TxnInfo.missing() collection when non-identity flag bits are present

patch by Benedict; reviewed by Alex Petrov for CASSANDRA-20811
2025-08-04 08:41:24 +01:00
Benedict Elliott Smith 54b2f6acbc Improve:
- system_accord_debug.txn_update->txn_ops + force_apply, force_update, try_execute
 - system_accord_debug.commands_for_key
 - system_accord_debug.commands_for_key_unmanaged
 - system_accord_debug.redundant_before + locally_durable_to_X_store
 - flush accord caches on shutdown

patch by Benedict; reviewed by Alex Petrov for CASSANDRA-20814
2025-08-03 15:16:00 +01:00
Caleb Rackliffe eb65c0e600 Support manual secondary index selection at the CQL level
patch by Caleb Rackliffe; reviewed by Andres de la Peña for CASSANDRA-18112

Co-authored-by: Caleb Rackliffe <calebrackliffe@gmail.com>
Co-authored-by: Andres de la Peña <a.penya.garcia@gmail.com>
2025-08-01 14:42:27 -05:00
Stefan Miklosovic 82e1c3d1c0
Merge branch 'cassandra-5.0' into trunk 2025-07-31 11:16:37 +02:00
Stefan Miklosovic 345df4e723
Merge branch 'cassandra-4.1' into cassandra-5.0 2025-07-31 11:15:15 +02:00
Dmitry Konstantinov b4210acd3d
Improve CommitLogSegmentReader to skip SyncBlocks correctly in case of CRC errors
patch by Dmitry Konstantinov; reviewed by Stefan Miklosovic for CASSANDRA-20664
2025-07-31 11:09:11 +02:00
David Capwell 6adac494aa When regulars CQL mutations run on Accord use the txn timestamp rather than server timestamp
patch by David Capwell; reviewed by Ariel Weisberg for CASSANDRA-20744
2025-07-30 13:19:41 -07:00
Stefan Miklosovic 5aeaef5f77
Merge branch 'cassandra-5.0' into trunk 2025-07-30 17:23:14 +02:00
Stefan Miklosovic cf83f8b2a4
Merge branch 'cassandra-4.1' into cassandra-5.0 2025-07-30 17:21:45 +02:00
Isaac Reath fcaa1b3e39
Do not crash on first boot with data_disk_usage_max_disk_size set when data directory is not created yet
patch by Isaac Reath; reviewed by Paulo Motta, Stefan Miklosovic for CASSANDRA-20787
2025-07-30 17:19:10 +02:00
David Capwell 8c9627076b When using BEGIN TRANSACTION if a complex mutation exists in the same statement as one that uses a reference, then the complex delete is dropped
patch by David Capwell; reviewed by Caleb Rackliffe for CASSANDRA-20788
2025-07-29 14:18:36 -07:00
Benedict Elliott Smith 4102ea13e5 Fix CFK restore after replay:
- Remove from CFK any unapplied transactions we know cannot apply
 - Force notification of waiting commands in CFK on replay
Also fix:
 - Don't truncateWithOutcome if pre bootstrap or stale
 - Fix RangeDeps.without(RangeDeps)
 - Fix InMemoryCommandStore replay bug with clearing DefaultLocalListeners
 - Ensure SaveStatus and executeAt are updated together to prevent corruption via expunge
Improve:
 - Inform home shard that command is decided if we cannot execute, to avoid recovery contention
 - Don't recover sync points on the fast path
 - Don't calculate recovery info for RX (since we don't use it anymore, so no need to do the work)

patch by Benedict; reviewed by Alex Petrov for CASSANDRA-20797
2025-07-28 13:32:58 +01:00
Stefan Miklosovic 87069e4dec
Merge branch 'cassandra-5.0' into trunk
This commit only contains in-jvm dtest, logically same as in 5.0, which is ignored as it would fail.
It was identified as part of CASSANDRA-19902 review that trunk branch returns STARTING instead of JOINING operation mode.
This discrepancy between 5.x and trunk is meant to be solved under CASSANDRA-20795.
2025-07-28 09:48:18 +02:00
Paulo Motta 07163cfb51
Fix StorageService JMX mbean not available during bootstrap
Patch by Paulo Motta; reviewed by Stefan Miklosovic for CASSANDRA-19902
2025-07-28 09:28:02 +02:00