Commit Graph

31313 Commits

Author SHA1 Message Date
David Capwell 8d641e06fc Accord: BEGIN TRANSACTIONs IF condition logic does not properly support meaningless emptiness and null values
patch by David Capwell; reviewed by Caleb Rackliffe for CASSANDRA-20667
2025-06-24 21:48:31 -07:00
Alex Petrov 6b2331560a Ninja: fix compilation after rebase 2025-06-23 14:59:42 +02:00
Alex Petrov 64934507ae Avoid waiting for topology indefinitely; print out debug string if topology can not be fetched
Patch by Alex Petrov; reviewed by Benedict Elliott Smith for CASSANDRA-20716
2025-06-23 14:54:10 +02:00
Alex Petrov 5746db0384 Accord Fixes / Improvments:
* Fix typos in AccordDebugKeyspace
  * Make durability service listen to topology changes
  * Account for truncated epoch when computing the low bound
  * Add a test to make sure GC issues do not slip past us in the future
  * Fix compilation

 Patch by Alex Petrov; reviewed by Benedict Elliott Smith for CASSANDRA-20718
2025-06-23 14:54:05 +02:00
Alex Petrov 259b1a532e Fix StackOverflow exception on topology replay with a large number of active epochs/topologies
Patch by Alex Petrov; reviewed by Benedict Elliott Smith for CASSANDRA-20702
2025-06-23 14:54:05 +02:00
Alex Petrov f3bf24c89e Make durability service listen to topology changes
Patch by Alex Petrov; reviewed by Benedict Elliott Smith for CASSANDRA-20703
2025-06-23 14:54:05 +02:00
Alex Petrov 9d6fb76cfa Improve journal corruption detection
Patch by Alex Petrov; reviewed by Benedict Elliott Smith for CASSANDRA-20718
2025-06-23 14:54:05 +02:00
Alex Petrov eb4c6b66fb Mark last min known epoch - 1 as "truncated"
Patch by Alex Petrov; reviewed by Benedict Elliott Smith for CASSANDRA-20689
2025-06-23 14:54:04 +02:00
Alex Petrov eafd55170a Accord Improvements:
* Further relax conditions of 'unsafe' startup
  * Allow skipping unfinished allocations
  * Relax assertion: allow same key accross multiple different stores
  * Fix SegmentTest

Patch by Alex Petrov; reviewed by Benedict Elliott Smith for CASSANDRA-20718
2025-06-23 14:54:04 +02:00
Alex Petrov 0936286ebd Fix visiting same key multiple times during replay
* Make sure to wait for Mutation stage quiesence during replay
* Fix visiting same key multiple times during replay

Patch by Alex Petrov; reviewed by Benedict Elliott Smith for CASSANDRA-20675
2025-06-23 14:54:04 +02:00
Benedict Elliott Smith 1c7e96b5d7 Fix journal out of order iteration
Patch by Benedict Elliott Smith; reviewed by Alex Petrov for CASSANDRA-20718
2025-06-23 14:54:04 +02:00
Sam Tunnicliffe 693eab8776 Revert changes to serving FetchCMSLog/FetchPeerLog requests & remove ReconstructLogState
Patch by Sam Tunnicliffe; reviewed by Alex Petrov for CASSANDRA-20719

Co-authored-by: Sam Tunnicliffe <samt@apache.org>
2025-06-23 14:54:04 +02:00
Alex Petrov 6a37d88f5d Allow unsafe startup if accord.journal.failurePolicy is set to ALLOW_UNSAFE_STARTUP
Background: we want to have an escape hatch to pull / copy journal files to investigate locally, but at that stage, we would rather allow the cluster to skip a log entry rather than wipe an entire cluster.

Patch by Alex Petrov; reviewed by Benedict Elliott Smith for CASSANDRA-20702
2025-06-23 14:54:04 +02:00
Alex Petrov b506665478 Fix CRC errors in Journal
Patch by Alex Petrov; reviewed by Benedict Elliott Smith for CASSANDRA-20661
2025-06-23 14:54:04 +02:00
Alex Petrov 9471016c4f Accord Fixes / Improvements:
* Ignore storage compat mode errors
  * Fix restart test (Cannot invoke "org.slf4j.Logger.debug(String, Object, Object)" because "this.inInstancelogger" is null)
  * Always speculate in accord replacement test to avoid read timeouts
  * Make sure not to cleanup fields before persisting them
        Background: we would clean up redundantBefore and some other fields.
        We are calling persistFieldUpdates, during which we "flush" field updates to command store, but then set it back to null:
            fieldUpdates = null;
        so when we get to persistFieldUpdatesInternal in AccordTask, we don't have any field updates.

Patch by Alex Petrov; reviewed by Benedict Elliott Smith for CASSANDRA-20718
2025-06-23 14:54:04 +02:00
Alex Petrov 1350082a13 Avoid re-serializing all-equivalent topologies
patch by Alex Petrov; reviewed by Benedict for CASSANDRA-20721
2025-06-19 12:40:38 +01:00
Benedict Elliott Smith 7ef2ccd4a1 Erase should consistently erase all content
Also Fix:
 - slowCoordinatorDelay should bound its start point, and log more detail if its expectations are breached
 - Erased SaveStatus can be reported for all queried owned participants on a replica (the saved participants have been erased)

patch by Benedict; reviewed by Alex Petrov for CASSANDRA-20722
2025-06-19 12:40:38 +01:00
Maxwell Guo 9a3fda57cd Merge branch 'cassandra-5.0' into trunk
* cassandra-5.0:
  Make Commitlog flush data safely in Direct IO mode
2025-06-17 23:33:23 +08:00
Maxwell Guo 8de4c9250e Make Commitlog flush data safely in Direct IO mode
patch by Maxwell Guo; reviewed by Ariel Weisberg for CASSANDRA-20692
2025-06-17 23:24:06 +08:00
Benedict Elliott Smith df4eae7fb9 ninja follow-up to CASSANDRA-20713: fix SSTableReaderTest 2025-06-17 08:03:48 +01:00
Caleb Rackliffe 0a3e624a28 Merge branch 'cassandra-5.0' into trunk
* cassandra-5.0:
  Get SAI MemtableIndex refs before SSTableIndex refs at query time
2025-06-16 16:06:41 -05:00
Michael Marshall a72dcbc86a Get SAI MemtableIndex refs before SSTableIndex refs at query time
SAI predicate search currently has a bug that could result in missing rows due to a
concurrent flush during a query. The new test created in this PR shows the point of failure.
The problem is that we get the SSTable index references before getting the Memtable index
references. Note that we do it in the correct order in the ANN OF query path, but not
in the WHERE query path.

This commit updates the QueryView object to hold references to the appropriate Memtable indexes.

It also removes the problematic search methods from MemtableIndexManager to prevent future misuse.

patch by Michael Marshall; reviewed by Caleb Rackliffe and Ekaterina Dimitrova for CASSANDRA-20709
2025-06-16 16:00:47 -05:00
Bernardo Botella Corbi e5f99c5cc5 CASSANDRA-20699 Fix SAI virtual table names in documentation
patch by Soheil Rahsaz; reviewed by Bernardo Botella and Caleb Rackliffe for CASSANDRA-20699
2025-06-16 15:50:52 +02:00
Benedict Elliott Smith 464a4f9d79 Accord Fixes:
- AccordJournal should not attempt to construct expunged commands
 - ready/notReady logic can break if we receive a partial Route that does not cover the local store
 - Invariant.require -> TopologyRetiredException
 - validate min/max in TopologyManager.preciseEpochs
Improve:
 - ExecuteSyncPoint should send Apply

patch by Benedict; reviewed by Alex Petrov for CASSANDRA-20713
2025-06-16 11:42:12 +01:00
Benedict Elliott Smith 1b78bbdc35 Fix:
- Do not query local topology when deciding what keys to fetch to avoid TopologyRetiredException
 - Ensure we propagate information back to the requesting CommandStore by using the store id to ensure it is included
 - BurnTest topology fetching was broken by earlier patch
 - Topology callbacks were not being invoked as we were not calling .begin()
 - Topology mismatch failure during notAccept phase was not being reported due to CoordinatePreAccept already having isDone==true

patch by Benedict; reviewed by David Capwell for CASSANDRA-20711
2025-06-12 11:41:59 +01:00
David Capwell 4d141e4b0a Accord: startup race condition where accord journal tries to access the 2i index before its ready
patch by David Capwell; reviewed by Alex Petrov for CASSANDRA-20686
2025-06-10 14:36:06 -07:00
Ariel Weisberg 83f58b4e1b Merge branch 'cassandra-5.0' into trunk 2025-06-10 16:33:34 -04:00
samlightfoot 53ccfd58f2 Fix MAX_SEGMENT_SIZE < chunkSize in MmappedRegions::updateState
patch by samlightfoot, Szymon Mi<C4><99><C5><BC>a<C5><82>; reviewed by Ariel Weisberg, Michael Semb Wever for CASSANDRA-20636
2025-06-10 16:32:11 -04:00
Benedict Elliott Smith 10ed20bdc3 Fixes
- Don't assume stillExecutes applies to remote request - must mark unavailable any keys we don't have the txn definition for
 - Don't exit notifyManagedPreBootstrap notify loop early, as could have later transactions with lower txnId
 - CoordinateEphemeralRead must retry if insufficient responses from replicas that still own the range
 - Burn test terminates while non-recurring tasks pending on command store queues
 - Infinite recovery due to not sending InformDurable when partially truncated
 - Incorrect participants when invoking removeRedundantDependencies
 - Infinite bootstrap loop on retired topology
Improve
 - Do not perform linear filters of CommandStores or Topologies
 - Some default implementations of forEach/iterator
 - TopologyRetiredException messages

patch by Benedict; reviewed by Alex Petrov for CASSANDRA-20707
2025-06-10 09:29:23 +01:00
samlightfoot 58ee847968 Adopt Unsafe::invokeCleaner for Direct ByteBuffer cleaning
Patch by Samuel D Lightfoot; Reviewed by Ariel Weisberg and Abe Ratnosfky for CASSANDRA-20677
2025-06-09 14:14:40 -04:00
Marcus Eriksson 7c9af51cfa Merge branch 'cassandra-5.0' into trunk 2025-06-05 14:07:44 +02:00
Marcus Eriksson f07510a303 CASSANDRA-18119 followup - test fix, create STATS component when using BtiFormat
Patch by marcuse; reviewed by Mick Semb Wever for CASSANDRA-18119
2025-06-05 14:04:42 +02:00
maoling 1261ba159c
Add additional metrics around hints
patch by Ling Mao; reviewed by Stefan Miklosovic, Maxim Muzafarov for CASSANDRA-20499

Co-authored-by: Stefan Miklosovic <smiklosovic@apache.org>
2025-06-05 10:55:33 +02:00
Stefan Miklosovic 3a506eb104
Merge branch 'cassandra-5.0' into trunk 2025-06-02 12:31:31 +02:00
Dmitry Konstantinov 3969de6524
Mark JDK 17 as production ready
patch by Dmitry Konstantinov; reviewed by Michael Semb Wever, Stefan Miklosovic for CASSANDRA-20681
2025-06-02 12:15:59 +02:00
Ariel Weisberg 5450a1f20e Support for add and replace in IntervalTree
Patch by Ariel Weisberg and Yuqi Yan; Reviewed by Marcus Eriksson for CASSANDRA-20513

Co-authored-by: Yuqi Yan <yukei0509@gmail.com>
2025-05-29 17:10:55 -04:00
Ariel Weisberg d110b16583 Merge branch 'cassandra-5.0' into trunk 2025-05-29 17:03:54 -04:00
Ariel Weisberg 948e6c9879 Merge branch 'cassandra-4.1' into cassandra-5.0 2025-05-29 17:02:18 -04:00
Ariel Weisberg 88ca4f8d9a Backport CASSANDRA-20469 (Paxos repair interrupts running transactions) to 4.1 and 5.0
patch by Ariel Weisberg; reviewed by Benedict Elliott Smith for CASSANDRA-20585
2025-05-29 17:01:18 -04:00
Stefan Miklosovic 3f03dc7263
ninja: remove unused imports in AccordJournalBurnTest to pass ant checkstyle-test 2025-05-29 15:14:54 +02:00
Benedict Elliott Smith b3f035be1e Accord Fixes:
- cfk pruning+prebootstrap=invalid future dependency
 - exclude retired ranges when filtering RX stillTouches
 - propagate uses incorrect lowEpoch when fetch finds additional owned/touched ranges
 - node.withEpoch should callback with TopologyRetiredException, not throw
 - Recovery can race with durable-applied pruning; must not send durable unless latest ballot on apply
 - removeRedundantDependencies was not slicing pre-bootstrap range calculation to participating ranges
 - NPE in TopologyManager.atLeast caused by referencing an epoch that has been GC'd
 - use journal durableBeforePersister in burn test, not NOOP_PERSISTER
 - ServerUtils.cleanupDirectory use tryDeleteRecursive
 - FsyncRunnable shutdown
 - fix NPE in AccordJournalBurnTest

patch by Benedict; reviewed by Alex Petrov for CASSANDRA-20688
2025-05-29 12:52:42 +01:00
Sam Tunnicliffe 8d89e160fd Ninja fix CHANGES.txt 2025-05-29 10:25:25 +01:00
nvharikrishna 53cdefb1de Enabling single sstable uplevel by default
Patch by Venkata Harikrishna Nukala; reviewed by Marcus Eriksson and Sam
Tunnicliffe for CASSANDRA-18509
2025-05-29 10:01:36 +01:00
Brandon Williams 64c7031f46 Merge branch 'cassandra-5.0' into trunk 2025-05-28 17:29:05 -05:00
Brandon Williams f1457e8084 Merge branch 'cassandra-4.1' into cassandra-5.0 2025-05-28 17:28:58 -05:00
Brandon Williams eb4e79500e Merge branch 'cassandra-4.0' into cassandra-4.1 2025-05-28 17:28:51 -05:00
Brandon Williams 00f2b3e9f6 increment version 2025-05-28 17:28:46 -05:00
Caleb Rackliffe fe13726fc6 Merge branch 'cassandra-5.0' into trunk
* cassandra-5.0:
  Ensure replica filtering protection does not trigger unnecessary short read protection reads
2025-05-28 16:53:06 -05:00
Caleb Rackliffe e5c101673a Ensure replica filtering protection does not trigger unnecessary short read protection reads
patch by Caleb Rackliffe; reviewed by Blake Eggleston and Zhao Yang for CASSANDRA-20639
2025-05-28 16:40:37 -05:00
Andy Tolbert 34650af75a Merge branch 'cassandra-5.0' into trunk
* cassandra-5.0:
  Ensure prepared_statement INSERT timestamp precedes eviction DELETE
2025-05-27 22:35:02 -05:00