When BTreeRow.merge reconciles an update into a row whose existing deletion
shadows the update's cells, it filtered the update (incoming) side of the merge
with Reconciler.retain, which records the removal via
PostReconciliationFunction.delete. On the memtable write path that subtracts the
shadowed cells' on-heap size from the allocator's ownership even though that
incoming data was never allocated to the memtable. Under overwrite/delete churn
that re-applies cells already covered by a newer row/partition deletion (e.g.
repair re-streaming), the allocator's "owns" counter drifts negative and the next
flush trips "AssertionError: Negative released" in MemtablePool$SubPool.released
via MemtableAllocator$SubAllocator.releaseAll during discard.
Filter the update (incoming) side with a non-recording variant,
Reconciler.removeShadowed, and keep retain() for the existing side,
whose data the memtable already owns. The filtered result is identical; only the
erroneous accounting notification is dropped. This mirrors the already-correct
complex-column path in ColumnData.merge.
patch by Stefan Miklosovic; reviewed by Dmitry Konstantinov for CASSANDRA-21469
Assisted-By: Claude Opus 4.8 <noreply@anthropic.com>
CommandChanges.shouldCleanup short-circuits to NO if there is no data, but this is incorrect as Cleanup.EXPUNGE may have dropped the data and the record must receive cleanup EXPUNGE and be reported as ERASED.
Also Fix:
- Populate CFK system table row markers so we can stop reading sstables as soon as we have data
- system_accord_debug.executors has wrong clustering type
- RemoteToLocalVirtualTable should lazily allocate the partition collector to avoid LIMIT clause filtering removing it before it's populated
- ActiveEpochs.withNewEpochs should handle transition from 0 -> more than 1
- RedundantBefore.minGcBefore should be NONE if empty
- Update RangesForEpoch directly, so that we cannot have race conditions where the ownership is unknown
- Avoid reentrancy on local callbacks
- Ensure ReadCoordinator callbacks are invoked on owning thread
- Avoid deadlock when notifying ComplexListener(s)
- Release IntrusivePriorityHeap memory from large capacity heaps when empty
- Prevent SynchronousRecoverAwait reentrancy when invoking onDone (by exposing and invoking invokeOnDone that first sets isDone)
- maybeExecute must invoke either notWaiting or notifyWaiting to ensure tryExecuteListening terminates
Also Improve:
- Configurable execute_waiting_on_start
patch by Benedict; reviewed by Alan Wang and Alex Petrov for CASSANDRA-21440
This commit fixes several issues found using the Cassandra
edge case explorer skill for the BTI feature.
patch by Francisco Guerrero; reviewed by Aleksey Yeschenko for CASSANDRA-21353
org.apache.cassandra.db.rows.RangeTombstoneMarker.Merger is allocated on demand
Optional usage in org.apache.cassandra.db.rows.UnfilteredRowIteratorWithLowerBound is replaced with a boolean field
Double in org.apache.cassandra.io.util.CompressedChunkReader#getCrcCheckChance - specialized supplier is used to avoid boxing
Stack - adjust default from 5 to 6 to avoid extra resizings in org.apache.cassandra.db.transform.Stack#resize
EnumSet for CL guardrail in SelectStatement - add a condition to allocate it only if the guardrail is enabled
EnumSet in ResultMetadata - replace it with plain int flags
patch by Dmitry Konstantinov; reviewed by Francisco Guerrero for CASSANDRA-21360
Current ReadCommand logic returns ArrayCell values, so when we retrieve a cell value as a ByteBuffer we allocate ByteBuffer instances
patch by Dmitry Konstantinov; reviewed by Francisco Guerrero for CASSANDRA-21362
The validation of snapshot names is turned off by default.
It is controlled by system property cassandra.snapshot.validation which is by default
set to false except 7.0 branch.
When the validation is turned on, the allowed characters are AWS safe characters together
with "+" character. A user can not specify any path separators in snapshot names even if
the validation as such is turned off.
patch by Stefan Miklosovic; reviewed by Francisco Guerrero, Matt Byrd for CASSANDRA-21389
patch by Andrés Beck-Ruiz, Runtian Liu; reviewed by Zhao Yang, Benedict Elliott Smith for CASSANDRA-21216
Co-authored-by: Runtian Liu <runtian@uber.com>
Set 1 thread and 1 second for the measurement iteration (note: it is a smoke test only, we do not analyze results from such short measurements)
Exclude some options for heavy benchmarks by default
Shutdown all threads once JMH test is completed to avoid awaiting of a forked JVM till timeout at the end
patch by Dmitry Konstantinov; reviewed by Michael Semb Wever for CASSANDRA-21388
- now() functions must be deterministic (and derived from Accord timestamp)
- tombstone GC and TTL evaluation must be deterministic and based on both Accord timestamp and Accord GC mechanisms (that guarantee completeness of execution)
patch by Benedict; reviewed by Alex Petrov and Ariel Weisberg for CASSANDRA-21376
- consumer and producer threads wait signal without acquiring the lock first
- lock owners may prepare more work than they need, moving some dynamically-adjusted portion of the work from the prioritised lock-managed structures onto a non-blocking queue, so that other threads may consume work from there; the portion is continually micro-adjusted to target some available work whenever the lock is acquired.
- adopts/supports some features of the SEPExecutor:
- threads may auto-adjust the number of running threads based on how much time is collectively spent waiting, to minimise time spent signalling
- consumers may (timed-sleep) poll rather than await a signal, reducing the number of kernel interactions needed; relying on the auto-adjustment to bound the time the scheduler spends waking threads with no work to do
Also Fix:
- Client Result should not be persisted or included in Command object at any time
patch by Benedict; reviewed by Alex Petrov and Ariel Weisberg for CASSANDRA-21375
The following subcommands have been added:
* abortbootstrap Cancel and undo an inflight bootstrap
* abortmove Cancel and undo an inflight token move
* abortdecommission Cancel and undo an inflight decommission
* assassinate Remove a node from cluster metadata
* resetcms Redefine the CMS membership
* move Modify a node's (single) owned token
* describe Output a summary of the cluster metadata
* forcejoin Force a registered or joining node into a JOINED state
* print Output a full rendering of cluster metadata
* printdirectory Output a full rendering of the directory
* printdataplacements Output the full data placements
Patch by Venkata Harikrishna Nukala; reviewed by Alex Petrov and Sam Tunnicliffe
for CASSANDRA-19151
- TransactionStatement token aware routing
- Mitigate convoy effect:
- ExecuteTxnBacklog v1 can execute single key transaction backlogs and disseminate the result
- Direct local execution possible for single key transactions
- Use shardAppliedBefore instead of gcBefore to cleanup CFK faster
Also Improve:
- Introduce distributed tracing
- Combine RejectBefore and MaxConflicts
- Self-addressed messages are delivered directly
- Disable AccordSyncPropagator by default, as incompatible with large clusters
- FastPathStrategy.SIMPLE should not modify fast path contents; FastPathStrategy.UP introduced to adopt this behaviour
- Configurable queue prioritisation
- ProtocolModifiers mostly final
- Remove unnecessary condition check when serializing TxnUpdate
Also Fix:
- SequentialExecutor ownership bug
- BTree.XUpdater.reset()
patch by Benedict; reviewed by Alex Petrov and Ariel Weisberg for CASSANDRA-21361
use compression ratio to scale the amount of remaining disk usage
CompactionInfo::estimatedRemainingWriteBytes is uncompressed, causing compaction to be rejected due to size when it has space to complete
patch by Matt Byrd; reviewed by Dmitry Konstantinov, Marcus Eriksson for CASSANDRA-21245
Co-authored-by: Jon Meredith <jmeredithco@apple.com>
- Clean Shutdown/Restart
- Rebootstrap to allow nodes to rejoin consensus if they are out of sync, or did not shutdown cleanly
Improve:
- Improve efficiency of BTreeReducingRangeMap
- DurableBefore backed by BTreeReducingRangeMap
- Soft reject new transactions when a replica has a backlog of work
- system_accord_debug.command_store_ops supports starting journal replay
Fix:
- Ensure SequentialExecutor.owner is unset only by the owner
- Detect and avoid taking two AccordExecutor locks simultaneously
- MaxConflicts serializer
- tryToExecuteListening after replay, to handle invalidated dependencies
- TxnNamedRead does not close a RowIterator, leading to native memory leaks
- GetLatestDepsNack serialization
- journal.readLast() did not read last
- DefaultRemoteListeners not correctly synchronised
- RangeTxnScanner cancellation assumes was already started
- Start cfk compaction before replay
- txn_graph should avoid visiting parents twice (possible if two different dependency chains connecting them in the graph)
- Topology.cloneEquivalentWithEpoch should also share nodeLookup and ranges, especially to accelerate computeWaitForEpoch/computeScope
- Do not invoke slowReplicaDelay or slowCoordinatorDelay during restore
- Do not fail if slowReplicaDelay or slowCoordinatorDelay are invoked without knowing the transaction
patch by Benedict; reviewed by Alex Petrov and Ariel Weisberg for CASSANDRA-21355
Maxim Muzafarov