Commit Graph

7567 Commits

Author SHA1 Message Date
Caleb Rackliffe 567e93ba3b ninja-fix: Make sure the MutationJournal starts before trying to create tracked keyspaces in unit-level tests that call SchemaLoader.schemaDefinition() 2025-10-27 15:40:24 -05:00
Aleksey Yeschenko 1bf039709e Truncate mutation journal as logs get reconciled
patch by Aleksey Yeschenko; reviewed by Alex Petrov for CASSANDRA-20710
2025-10-08 14:02:22 +01:00
Caleb Rackliffe b67f4e2c2a Disable mutation tracking by default in CassandraGenerators
patch by Caleb Rackliffe; reviewed by Blake Eggleston for CASSANDRA-20944
2025-10-06 20:11:32 -05:00
Abe Ratnofsky 7f4bac3efb Fix failing test: org.apache.cassandra.replication.CoordinatorLogOffsetsTest#reconciledBounds 2025-10-01 18:13:25 -04:00
Blake Eggleston 6067fb62a7 CEP-45: Support host replacement with tracked keyspaces
Patch by Blake Eggleston; Reviewed by Abe Ratnofsky for CASSANDRA-20891
2025-09-29 15:26:41 -07:00
Ariel Weisberg 698ea0cf74 Fix failing unit tests from witness merge
Patch by Ariel Weisberg; Reviewed by Blake Eggleston for CASSANDRA-20934

Co-authored-by: Blake Eggleston <blake@ultrablake.com>
2025-09-29 12:49:04 -07:00
Alex Petrov 31f81c0b2c CEP-45: Integrate Mutation Journal with Memtable flush tracking
* Tracking of Memtable -> SSTable flushes in Mutation Journal segments
  * Replay of Mutation Journal segments that were not fully flushed to SSTables
  * Distinguish between CommitLog and MutationJournal offsets
  * Effectively adds support for node bounces

Patch by Alex Petrov; reviewed by Aleksey Yeschenko for CASSANDRA-20919
2025-09-29 20:59:58 +02:00
Ariel Weisberg 2403bfcaac Initial witness non-serial single partition read/write path
Patch by Ariel Weisberg; Reviewed by Abe Ratnosfky for CASSANDRA-20930
2025-09-26 13:03:19 -07:00
Aleksey Yeschenko 942fa5a3f3 Fix CoordinatorLogOffsetsLifecycleTest 2025-09-25 14:34:44 +01:00
Aleksey Yeschenko bd5a657548 Implement Shard and CoordinatorLog metadata durability
patch by Aleksey Yeschenko; reviewed by Abe Ratnofsky for CASSANDRA-20882
2025-09-19 15:55:11 +01:00
Blake Eggleston 8b7753f7ec Single-partition read fixes:
- filter augmenting mutations and add comment explaining why we don't exclude reconciled mutations when augmenting
- propagate exceptions during acknowledgeReconcile
- fix UnreconciledMutations handling of single key bounds
- fix single offset remove bug

patch by Blake Eggleston; reviewed by Caleb Rackliffe for CASSANDRA-20830
2025-09-05 17:46:15 -05:00
Aleksey Yeschenko 418a19dc31 Implement unified log reconciliation
patch by Aleksey Yeschenko; reviewed by Blake Eggleston for
CASSANDRA-20729

Co-authored-by: Aleksey Yeschenko <aleksey@apache.org>
Co-authored-by: Blake Eggleston <blake@ultrablake.com>
2025-08-26 17:35:16 +01:00
Caleb Rackliffe d8f67d77cf Fixed output format expectations in DescribeStatementTest 2025-08-14 11:48:53 -05:00
Caleb Rackliffe a99663c197 Separate TestParams by use-case (Accord vs. Mutation Journal) 2025-08-14 11:31:24 -05:00
Abe Ratnofsky b004207b32 Fully reconciled SSTable promotion
patch by Abe Ratnofsky; reviewed by Ariel Weisberg, Blake Eggleston for CASSANDRA-20381
2025-08-13 23:24:24 -05:00
Abe Ratnofsky 822c5d572a Add mutation tracking summary to SSTables
Patch by Abe Ratnofsky; reviewed by Ariel Weisberg, Blake Eggleston for CASSANDRA-20336
2025-08-13 22:59:26 -05:00
Aleksey Yeschenko ea9e041345 Fix-up offset broadcasting logic
patch by Aleksey Yeschenko; reviewed by Abe Ratnofsky for CASSANDRA-20576
2025-08-13 22:21:42 -05:00
Aleksey Yeschenko 59922b9f5d Implement coordinator log offset broadcasting
patch by Aleksey Yeschenko; reviewed by Blake Eggleston for CASSANDRA-20576
2025-08-13 22:21:42 -05:00
Blake Eggleston 217753c3d0 Mutation tracking journal integration, read, and write path
Patch by Blake Eggleston & Aleksey Yeshchenko
Reviewed by Blake Eggleston, Aleksey Yeshchenko, & Abe Ratnofsky
 for CASSANDRA-20304, CASSANDRA-20305, & CASSANDRA-20308, CASSANDRA-20373

Co-authored-by: Blake Eggleston <blake@ultrablake.com>
Co-authored-by: Aleksey Yeschenko <aleksey@apache.org>
2025-08-13 16:18:11 -05:00
Aleksey Yeschenko 9ac462ba55 Introduce MutationJournal for coordinator logs
patch by Aleksey Yeschenko; reviewed by Blake Eggleston for
CASSANDRA-20353
2025-08-12 16:28:32 -05:00
Dmitry Konstantinov 7546d9e895 Optimize MessagingService.getVersionOrdinal
The map lookup (with an implicit auto-boxing) can be replaced with a plain arithmetic operation.
A unit test is used to ensure that the assumption about serialization version incrementing for new versions is still correct.
greaterThanOrEqual method is removed as unused.

Patch by Dmitry Konstantinov; reviewed by Stefan Miklosovic for CASSANDRA-20816
2025-08-08 14:20:13 +01:00
Dmitry Konstantinov 19118a6cff Optimize TrieMemtable#getFlushSet
A flushing range is a combination of partial and full shards, for full shards we use write time captured values for count and total size of partition keys, for partial shards we iterate over keys
When we iterate over partial shards we count key bytes only instead of reading them and also try to skip tokens full parsing

Patch by Dmitry Konstantinov; reviewed by Branimir Lambov for CASSANDRA-20760
2025-08-08 13:34:34 +01:00
Stefan Miklosovic 429b2f8be4
Merge branch 'cassandra-5.0' into trunk 2025-08-08 09:16:21 +02:00
Isaac Reath 170bf57eac
Fix ThreadLocalReadAheadBufferTest to avoid FileNotFoundException when -Djava.io.tmpdir set to /tmp
patch by Isaac Reath; reviewed by Paulo Motta, Stefan Miklosovic for CASSANDRA-20819
2025-08-08 09:00:57 +02:00
Stefan Miklosovic bb7e706213
Merge branch 'cassandra-5.0' into trunk 2025-08-06 18:37:11 +02:00
Romain Hardouin 3b9669b116
Sort SSTable TOC entries for determinism
patch by Romain Hardouin; reviewed by Stefan Miklosovic, Dmitry Konstantinov for CASSANDRA-20494

Co-authored-by: Stefan Miklosovic <smiklosovic@apache.org>
2025-08-06 18:34:26 +02:00
Benedict Elliott Smith 46bda174d5 Bootstrap must first wait for a quorum to apply the sync point to ensure the DurableBefore Majority condition holds transitively,
since later quorums may include the bootstrapping node which does not participate in the durability of preceding transactions
Also Improve:
 - Apply MaxDecidedRX filtering to CommandsForKey RX dependencies
 - Optimise AbstractRanges.sliceMinimal (burn test hotspot)
 - Don't start shard schedulers on topology changes if not already started
 - Only registerTransitive if waitingOnSync
 - Reserve DurabilityRequest until ShardDurability is started
 - Don't notify progress log if stopped
 - Remove duplicated CFK unmanaged filtering
Also fix:
 - Edge case in notification across bootstrap boundary by resubmitting Unmanaged to be recomputed
 - Serialization of CommandsForKey.TxnInfo.missing() collection when non-identity flag bits are present

patch by Benedict; reviewed by Alex Petrov for CASSANDRA-20811
2025-08-04 08:41:24 +01:00
Benedict Elliott Smith 54b2f6acbc Improve:
- system_accord_debug.txn_update->txn_ops + force_apply, force_update, try_execute
 - system_accord_debug.commands_for_key
 - system_accord_debug.commands_for_key_unmanaged
 - system_accord_debug.redundant_before + locally_durable_to_X_store
 - flush accord caches on shutdown

patch by Benedict; reviewed by Alex Petrov for CASSANDRA-20814
2025-08-03 15:16:00 +01:00
Caleb Rackliffe eb65c0e600 Support manual secondary index selection at the CQL level
patch by Caleb Rackliffe; reviewed by Andres de la Peña for CASSANDRA-18112

Co-authored-by: Caleb Rackliffe <calebrackliffe@gmail.com>
Co-authored-by: Andres de la Peña <a.penya.garcia@gmail.com>
2025-08-01 14:42:27 -05:00
Stefan Miklosovic 82e1c3d1c0
Merge branch 'cassandra-5.0' into trunk 2025-07-31 11:16:37 +02:00
Stefan Miklosovic 345df4e723
Merge branch 'cassandra-4.1' into cassandra-5.0 2025-07-31 11:15:15 +02:00
Dmitry Konstantinov b4210acd3d
Improve CommitLogSegmentReader to skip SyncBlocks correctly in case of CRC errors
patch by Dmitry Konstantinov; reviewed by Stefan Miklosovic for CASSANDRA-20664
2025-07-31 11:09:11 +02:00
David Capwell 6adac494aa When regulars CQL mutations run on Accord use the txn timestamp rather than server timestamp
patch by David Capwell; reviewed by Ariel Weisberg for CASSANDRA-20744
2025-07-30 13:19:41 -07:00
Benedict Elliott Smith 4102ea13e5 Fix CFK restore after replay:
- Remove from CFK any unapplied transactions we know cannot apply
 - Force notification of waiting commands in CFK on replay
Also fix:
 - Don't truncateWithOutcome if pre bootstrap or stale
 - Fix RangeDeps.without(RangeDeps)
 - Fix InMemoryCommandStore replay bug with clearing DefaultLocalListeners
 - Ensure SaveStatus and executeAt are updated together to prevent corruption via expunge
Improve:
 - Inform home shard that command is decided if we cannot execute, to avoid recovery contention
 - Don't recover sync points on the fast path
 - Don't calculate recovery info for RX (since we don't use it anymore, so no need to do the work)

patch by Benedict; reviewed by Alex Petrov for CASSANDRA-20797
2025-07-28 13:32:58 +01:00
Paulo Motta 07163cfb51
Fix StorageService JMX mbean not available during bootstrap
Patch by Paulo Motta; reviewed by Stefan Miklosovic for CASSANDRA-19902
2025-07-28 09:28:02 +02:00
Maxim Muzafarov 94b251f3ce
Migrate all nodetool commands from airline to picocli
Replace io.airlift:airline dependency with info.picocli:picocli across
all nodetool command implementations. This migration includes:

- Convert 160+ nodetool command classes from airline to picocli
- Add AbstractCommand base class for commands and utility methods
- Add NodetoolCommand top-level class for the cli utility
- Update all command classes to use @Command, @Option, and @Parameters
- Add plain text files to test command help output
- Add mock test classes for improved test coverage and parse validation
- Modify test infrastructure to work with picocli-based commands
- Add new layouts for cli formatting to preserve backwards compatibility

The migration maintains backward compatibility while providing improved
command-line parsing, better help system, and more robust argument
validation through picocli enhanced features.

patch by Maxim Muzafarov; reviewed by Caleb Rackliffe, Dmitry Konstantinov, Stefan Miklosovic for CASSANDRA-17445
2025-07-26 18:19:18 +02:00
David Capwell e48d0dcd2c Journal.TopologyUpdate should not store the local topology as it can be inferred from the global on
patch by David Capwell; reviewed by Benedict Elliott Smith for CASSANDRA-20785
2025-07-24 18:51:24 -07:00
Benedict Elliott Smith fbe12e6291 Accord Query Tracing
Also improve:
 - Metric listeners

patch by Benedict; reviewed by David Capwell for CASSANDRA-20773
2025-07-24 20:20:17 +01:00
Stefan Miklosovic e8e312b4d1
Merge branch 'cassandra-5.0' into trunk 2025-07-24 10:27:58 +02:00
Stefan Miklosovic 922819dc08
Merge branch 'cassandra-4.1' into cassandra-5.0 2025-07-24 10:25:54 +02:00
Stefan Miklosovic f5f71b7fae
Rework / simplification of nodetool get/setguardrailsconfig commands
patch by Stefan Miklosovic; reviewed by Maxim Muzafarov for CASSANDRA-20778
2025-07-24 10:22:55 +02:00
Benedict Elliott Smith 54e39a90b0 Limit replay to those records not durably persisted to both command store and data store
Also fix:
 - Limit truncation to TruncatedApplyWithOutcome until data is persisted durably to local store
 - IntervalUpdater not invoking super.close()
 - Do not invoke preRunExclusive without holding lock
 - IntervalUpdater not correctly initialise BranchBuilder.inUse
 - AccordExecutor should notify if more work on unlock of caches
 - Relax paranoid CFK validation during restart
Also improve:
 - Flush logs before System.exit
 - Start/stop progress log explicitly
 - Limit progress log concurrency
 - Clear heavy fields in some messages once processed

patch by Benedict; reviewed by Alex Petrov for CASSANDRA-20780
2025-07-23 12:19:06 +01:00
Alex Petrov 260c9c9334 Add a way to patch accord journal commands via virtual tables
Patch by Alex Petrov; reviewed by Benedict Elliott Smith for CASSANDRA-20765
2025-07-23 11:06:33 +01:00
David Capwell 0574db7bd9 Accord Route Index does not filter based off txn_id range which causes segments which cant contain the data to be queried
patch by David Capwell; reviewed by Benedict Elliott Smith for CASSANDRA-20772
2025-07-22 15:21:14 -07:00
David Capwell 8014eec7aa Merge branch 'cassandra-5.0' into trunk 2025-07-18 16:06:36 -07:00
David Capwell 169c37b030 Merge branch 'cassandra-4.1' into cassandra-5.0 2025-07-18 16:04:46 -07:00
Yuqi Yan 4962f3da3f IntrusiveStack.accumulate is not accumulating correctly
patch by Yuqi Yan; reviewed by Benedict Elliott Smith, David Capwell for CASSANDRA-20670
2025-07-18 16:02:17 -07:00
Benedict Elliott Smith d229eaf667 ninja: remove accidental file 2025-07-18 13:10:38 +01:00
Stefan Miklosovic 418e3a4899
Merge branch 'cassandra-5.0' into trunk 2025-07-18 07:52:31 +02:00
Stefan Miklosovic 5fb3ae4473
Merge branch 'cassandra-4.1' into cassandra-5.0 2025-07-18 07:41:32 +02:00