patch by Andrés Beck-Ruiz, Runtian Liu; reviewed by Zhao Yang, Benedict Elliott Smith for CASSANDRA-21216
Co-authored-by: Runtian Liu <runtian@uber.com>
- now() functions must be deterministic (and derived from Accord timestamp)
- tombstone GC and TTL evaluation must be deterministic and based on both Accord timestamp and Accord GC mechanisms (that guarantee completeness of execution)
patch by Benedict; reviewed by Alex Petrov and Ariel Weisberg for CASSANDRA-21376
- consumer and producer threads wait signal without acquiring the lock first
- lock owners may prepare more work than they need, moving some dynamically-adjusted portion of the work from the prioritised lock-managed structures onto a non-blocking queue, so that other threads may consume work from there; the portion is continually micro-adjusted to target some available work whenever the lock is acquired.
- adopts/supports some features of the SEPExecutor:
- threads may auto-adjust the number of running threads based on how much time is collectively spent waiting, to minimise time spent signalling
- consumers may (timed-sleep) poll rather than await a signal, reducing the number of kernel interactions needed; relying on the auto-adjustment to bound the time the scheduler spends waking threads with no work to do
Also Fix:
- Client Result should not be persisted or included in Command object at any time
patch by Benedict; reviewed by Alex Petrov and Ariel Weisberg for CASSANDRA-21375
The following subcommands have been added:
* abortbootstrap Cancel and undo an inflight bootstrap
* abortmove Cancel and undo an inflight token move
* abortdecommission Cancel and undo an inflight decommission
* assassinate Remove a node from cluster metadata
* resetcms Redefine the CMS membership
* move Modify a node's (single) owned token
* describe Output a summary of the cluster metadata
* forcejoin Force a registered or joining node into a JOINED state
* print Output a full rendering of cluster metadata
* printdirectory Output a full rendering of the directory
* printdataplacements Output the full data placements
Patch by Venkata Harikrishna Nukala; reviewed by Alex Petrov and Sam Tunnicliffe
for CASSANDRA-19151
- TransactionStatement token aware routing
- Mitigate convoy effect:
- ExecuteTxnBacklog v1 can execute single key transaction backlogs and disseminate the result
- Direct local execution possible for single key transactions
- Use shardAppliedBefore instead of gcBefore to cleanup CFK faster
Also Improve:
- Introduce distributed tracing
- Combine RejectBefore and MaxConflicts
- Self-addressed messages are delivered directly
- Disable AccordSyncPropagator by default, as incompatible with large clusters
- FastPathStrategy.SIMPLE should not modify fast path contents; FastPathStrategy.UP introduced to adopt this behaviour
- Configurable queue prioritisation
- ProtocolModifiers mostly final
- Remove unnecessary condition check when serializing TxnUpdate
Also Fix:
- SequentialExecutor ownership bug
- BTree.XUpdater.reset()
patch by Benedict; reviewed by Alex Petrov and Ariel Weisberg for CASSANDRA-21361
use compression ratio to scale the amount of remaining disk usage
CompactionInfo::estimatedRemainingWriteBytes is uncompressed, causing compaction to be rejected due to size when it has space to complete
patch by Matt Byrd; reviewed by Dmitry Konstantinov, Marcus Eriksson for CASSANDRA-21245
Co-authored-by: Jon Meredith <jmeredithco@apple.com>
- Clean Shutdown/Restart
- Rebootstrap to allow nodes to rejoin consensus if they are out of sync, or did not shutdown cleanly
Improve:
- Improve efficiency of BTreeReducingRangeMap
- DurableBefore backed by BTreeReducingRangeMap
- Soft reject new transactions when a replica has a backlog of work
- system_accord_debug.command_store_ops supports starting journal replay
Fix:
- Ensure SequentialExecutor.owner is unset only by the owner
- Detect and avoid taking two AccordExecutor locks simultaneously
- MaxConflicts serializer
- tryToExecuteListening after replay, to handle invalidated dependencies
- TxnNamedRead does not close a RowIterator, leading to native memory leaks
- GetLatestDepsNack serialization
- journal.readLast() did not read last
- DefaultRemoteListeners not correctly synchronised
- RangeTxnScanner cancellation assumes was already started
- Start cfk compaction before replay
- txn_graph should avoid visiting parents twice (possible if two different dependency chains connecting them in the graph)
- Topology.cloneEquivalentWithEpoch should also share nodeLookup and ranges, especially to accelerate computeWaitForEpoch/computeScope
- Do not invoke slowReplicaDelay or slowCoordinatorDelay during restore
- Do not fail if slowReplicaDelay or slowCoordinatorDelay are invoked without knowing the transaction
patch by Benedict; reviewed by Alex Petrov and Ariel Weisberg for CASSANDRA-21355
Best effort attempt to get tableId from CFS, fallback to null which
still allows snapshot to be loaded
End result is that nodetool clearsnapshot and listsnapshot function for such
snapshots on restart
In addition to 5.0 patch we move starting SnapshotManager to after TCM is initiliazed.
The consequence of such a move is that cleanup of ephemeral snapshots happens after startup checks pass.
patch by Matt Byrd; reviewed by Stefan Miklosovic, Brandon Williams and Francisco Guerrero for CASSANDRA-21246
* cassandra-6.0:
Differentiate between legitimate cases where the first entry is the same as the last entry and empty bounds in SSTableCursorWriter#addIndexBlock()
In 6.0, only rename test/data/serialization/5.1 to 6.0
In 7.0, same plus update edges in UpgradeTestBase
patch by Stefan Miklosovic; reviewed by Dmitry Konstantinov for CASSANDRA-21319
Benedict Elliott Smith