Also Fix:
- Restore MaxDecidedRX on replay
- When catchup_on_start_exit_on_failure == false, should startup on any kind of failure, not only timeout
- lazy vtable LIMIT clause regression
- DurabilityService.onEpochRetired
- Command.validate when uniqueHlc differs
- Avoid unsafe publication of AccordExecutor to scheduledFastTasks
- AccordCache hitRate metric names
- use long for return type of DurationSpec.toNanoseconds
- Repair without all replicas should not request all Accord replicas participate
- ExecuteAtSerializer
- SyncPoints should be coordinated in an epoch that contains the ranges
Also Improve:
- Split Accord startup into local+distributed, ensure we
- Add logging to FetchDurableBefore on startup
- Add randomised testing of AbstractLazyVirtualTable
- Add validation of lazy virtual table key ordering
- Don't send requests to faulty replicas
- shrinkOrEvict large objects without holding lock
- Accord dtest shutdown
patch by Benedict; reviewed by Alex Petrov for CASSANDRA-21042
Move TableMetadataRef.get to init part of bulk operations (such as flush, compaction, scrab)
Cache TableMetadata value within TableMetadataRef to reduce overheads for write operations, same for KeyspaceMetadata
Patch by Dmitry Konstantinov; reviewed by Marcus Eriksson for CASSANDRA-20465
* Better handling of DOWN unupgraded nodes
* Test that aborted CMS initialization cleans up state properly
* Clean up orphaned PreInitialize entries in the log on bounce
* Unconditionally reset initiator during abort
* Ensure that metadata log entries aren't exchanged before CMS
initialization is complete
* Precalculate common serialization version, excluding non-upgraded and
LEFT nodes
* Decide if metadata-impacting upgrade is in progress using min common
version
* Add metadata identifier to nodetool cms output
Patch by Sam Tunnicliffe and Marcus Eriksson; reviewed by Sam
Tunnicliffe and Marcus Eriksson for CASSANDRA-21036
Co-authored-by: Marcus Eriksson <marcuse@apache.org>
Co-authored-by: Sam Tunnicliffe <samt@apache.org>
For GC like ShenandoahGC/ZGC etc., there are GC events that do not have STW pauses (Concurrent phases).
Operator might find it reasonable to use lower thresholds for events require STW pauses and higher thresholds for concurrent phases.
gc_concurrent_phase_log_threshold and gc_concurrent_phase_warn_threshold configuration options are introduced in cassandra.yaml
Patch by Yuqi Yan; reviewed by Dmitry Konstantinov, Stefan Miklosovic for CASSANDRA-20980
Co-authored-by: Stefan Miklosovic
- Estimate memory required to allocate for applying a partition update into a memtable and allocate this memory in one shot, then use it as a request-local SLAB
- Reduce contention by switching MemtableAllocator.SubAllocator#owns from updates via AtomicLongFieldUpdater to LongAdder usage. MemtableAllocator.SubAllocator#acquired(..) method updates "owns" value but does not use the updated result.
- Reduce contention by replacing of CAS loop in MemtablePool.SubPool#tryAllocate with allocatedUpdater.addAndGet(this, size)
Patch by Dmitry Konstantinov; reviewed by Michael Semb Wever for CASSANDRA-20226
Before the fix during a capacity extension BufferPoolAllocator returned to BufferPool a sliced ByteBuffer wrapper object instead of the originally allocated one, so the ByteBuffer was not recycled by BufferPool
Adjust BufferPoolAllocatorTest to test the ByteBuf capacity extension with a real BufferPool behavior
Patch by Dmitry Konstantinov; reviewed by Michael Semb Wever for CASSANDRA-20753
Also Introduce:
- Sharded/LogLinearDecayingHistogram
Also Improve:
- Do not take a reference to CFK unless relevant
Also Fix:
- Sharded/LogLinearHistogram
- ExecuteFlags serialization bug in ReadData
patch by Benedict; reviewed by Alex Petrov for CASSANDRA-21017
- All retries are appended to a delay queue so overlaps can be pruned
- Quorum successes are not retried if there are superseding sync points covering the ranges
- User-initiated requests are not purged unless the request has timed out or otherwise completed
- Overlapping requests are queued up against the next to run
Alsp (C*):
- Catch-up with quorums on restart
- Manage an ordered set of keys in cache for faster range searches
Also (Accord):
- Update copy of BTree and import IntervalBTree
- Fix RedundantStatus WAS_OWNED_OVERRIDE_MASK
- Add Catchup mechanism to reach parity with a quorum on restart
patch by Benedict; reviewed by Alex Petrov for CASSANDRA-21013
CASSANDRA-20389 implemented limit on table name length of 222
characters. This commit updates the documentation to reflect this.
It also adds an assert in CreateTableValidationTest to ensure that the
documented limit matches the actual constant.
patch by Ruslan Fomkin; reviewed by Brad Schoening, Dmitry Konstantinov for CASSANDRA-20914
Merge ConfigurationService with TopologyManager to remove cyclic dependency and duplicated work.
Also:
- Improve visibility of work blocking topology processing
- Ensure we cannot double-count peers when deciding an epoch's distributed readiness
- Remove the possibility of distributed stalls, by processing new topologies as soon as a
contiguous sequence is known locally, regardless of whether anyprior local epoch is ready to
coordinate or has recorded this fact to peers. The notification of local readiness continues
to be processed only once all prior epochs are ready, but we can begin using and readying
later epochs immediately.
patch by Benedict; reviewed by Alex Petrov for CASSANDRA-20998
Also fix:
- Paxos should not update minHlc if AccordService is not setup
- Remove EndpointMapper methods that require non-null, to ensure call-sites handle null case explicitly
- AccordRepair should specify the Node Ids that must participate, but these should not affect the durability requirements for GC
- Avoid erroneously marking UniversalOrInvalidated when only known to quorum
- AccordSyncPropagator should not consult FailureDetector as it may throw UnknownEndpointException
- Refuse ReplaceSameAddress if accord enabled
- Reset Accord topologies on restart if no local command stores to ensure we can catch up (as intervening epochs may have otherwise been GC'd)
- MaybeRecover should not abort if found durable if either: 1) prevProgressToken already witnessed durable; or 2) Home state has requested we recover anyway
- Invariant in MaybeRecover is too strong now we permit recovering when known to be stable
- ReadData must respond with InsufficientAndWaiting if not Stable to ensure Durability is inferred correctly by ExecuteTxn
- Avoid StackOverflowException in NotifyWaitingOn
- PreAccept should honour the REJECTED flag of any member of the latest topology if the vote would be relied upon for any prior epoch's quorum
- Resume Accord bootstrap on restart
- Fix burn test non-determinism
- Reset Accord topologies on restart if no local command stores to ensure we can catch up (as intervening epochs may have otherwise been GC'd)
- Exclude stale or removed ids from sync points
- Terminate a failed DurabilityRequest if a required participant is now removed/hardRemoved
Also improve:
- Introduce accord_debug_keyspace.{listeners_txn, listeners_local}
- Filter and record faulty in AbstractCoordination
- If we are the home shard, and our home progress status is done, we should not wait for the home shard to advance the waiting progress state
- Introduce CommandStore.tryExecuteListening to try to execute all waiting transactions and their dependencies
- ExecuteTxn on recovery should not wait, to avoid consuming progress log concurrency slots for transactions that cannot execute
- HomeState should update its phase based on the command status to handle operator request to retry all states
- Separate home/wait queue in DefaultProgressLog
- Additional tracing
- Do not call trySendMore from prerecordFailure
- Force full recovery for sync points if we have lost quorum
patch by Benedict; reviewed by Alex Petrov for CASSANDRA-20921
Benedict Elliott Smith