- Decouple command serialization from TableMetadata version; introduce ColumnMetadata ids; gracefully handle missing TableId
- DataInputPlus.readLeastSignificantBytes must truncate high bits
- Fix RandomPartitioner accord serialization
- Fast path stable commits must not override recovery propose/commit decisions regarding visibility of a transaction
- RejectBefore must mergeMax, not merge, to ensure we maintain epoch and hlc increasing independently
- Bad commitInvalidate decision
- consistent filtering for touches and stillTouches
- ensure TRUNCATE_BEFORE implies SHARD_APPLIED
- TopologyManager.unsyncedOnly off-by-one error
- DurabilityQueue should not retry SyncPointErased
- handle rare case of no deps but none needed
- not updating CFK synchronously on recovery, which can lead to erroneous recovery decisions for other transactions
- Don't return partial read response when one commandStore rejects the commit
- Filter touches/stillTouches consistently
- WaitingState computeLowEpoch must use hasTouched to handle historic key with no route
Improve:
- Use format parameters to defer building Invariants.requireArgument string
- streamline RedundantStatus/RedundantBefore
Also improve:
- Introduce DurabilityService
- Retire SyncPoint, replace Barrier with Write and RX
- MessageType -> enum, restore GetMaxConflict
- Standardise backoff logic with WaitStrategy
- improve TimeoutStrategy/RetryStrategy specification strings
- Forbid KX, remove directKeyDeps
- Introduce UniqueTimeService, permitting hlc reservations for sync points avoid delay when min TxnId is sufficiently in the past
- Remove ListStore custom purge logic
Also fix:
- RejectBefore should reject on both epoch and hlc
- Do not record sync success for removed nodes
- Support GlobalDurability detecting no command store to run on
- Incorrect ballot constructor
- Serializing 15-bit ballot flags incorrectly
- TopologyManager.hasEpoch deadlock
- Computing withOpenEpochs incorrectly, sometimes stopping one epoch short
- PartitionKey serializer should not depend on schema information that can be erased
patch by Benedict; reviewed by Alex Petrov for CASSANDRA-20395
- Bad ArrayBuffers recycling logic
- RX must ensure dependencies TRANSITIVE_VISIBLE
- Permit constructing "antiRange" that spans multiple prefixes
- Not computing range CommandSummary IsDep correctly
- Truncated commands that aren't shard durable could not repopulate CFK on replay, permitting recovery of another command to make an incorrect decision
- NPE on async persist of RX (i.e. supplying no callback)
- NPE in Builder.shouldCleanup when durability is null
patch by Benedict; reviewed by Alex Petrov for CASSANDRA-20370
- Only use persisted RedundantBefore for compaction
- RouteIndex should index only touches, not Route
- Flush RangesForEpoch updates to journal immediately, so we do not rely on the command we are processing succeeding
- DurableBefore updates must wait for the epochs to be known locally
- Shard.mustWitnessEpoch to support guaranteeing to witness relevant non-topology schema changes
- We must propagate RedundantBefore RX shard bounds along with epoch syncs
- Prevent a truncated transaction FetchData infinite loop
- GC_BEFORE status being overwritten by bootstrappedAt, permitting old transaction state to be resurrected
- Avoid CFK.maxUniqueHlc read race on bootstrap
- TopologyManager.awaitEpoch could wait for wrong epoch
- Journal fsync thread could miss notifications
Also improve:
- CommandStores uses SearchableRangeList for finding matching stores
- Refactor RedundantBefore to use a sorted array of TxnId/RedundantStatus pairs (to better fix GC_BEFORE issue)
- Accord debug keyspace operates on keyspace/table, and sorts correctly by token
patch by Benedict; reviewed by Alex Petrov for CASSANDRA-20361
Improve:
- Introduce pre/accept fast execution flags
- Introduce searchable Deps serialization
- Flatten AccordRoutingKey(s) into single type, using sentinel bits
- Introduce new fast byte-comparable serialization methods for Token and TableId to support above
Fix:
- Fix journal re-serialization logic
- Enable RandomPartitioner for Accord by supporting fixed-width serialization for RouteIndex
patch by Benedict; reviewed by Alex Petrov for CASSANDRA-20349
Also fix:
- Topology slicing must declare whether we share/slice node ownership (to assist above)
- CFK.visit removes transitive dependencies too eagerly across epoch change
- apply cleanup to builder consistently, and construct the same value we would produce by purge (so that replay is idempotent)
- Invoke ExecuteTxn.LocalExecute callbacks on originating CommandStore
- misc other minor issues
patch by Benedict; reviewed by Alex Petrov for CASSANDRA-20325
Use initializeTopologyUnsafe to for-load the last topology and create shards rather than replaying all topologies.
Patch by Alex Petrov; reviewed by Benedict Elliott Smith for CASSANDRA-20294
- Even if we can decide autonomously that we took the fast path, we must still wait for earlier transactions to decide themselves
- We must update a command that is in CFK.loadingPruned whether or not it is outOfRange
- We must visit pruned commands that are transitive dependencies of RX to ensure dependencies are propagated
- flagsWithoutDomainAndKind() -> flagsWithoutDomainOrKindOrCardinality() [to fix integration regression]
- Don't invoke uniqueNow() twice when allocate nextTxnId
- BTreeReducingRangeMap edge case that corrupts map when merge function does not return one of its inputs
patch by Benedict; reviewed by Alex Petrov for CASSANDRA-20292
Also fix:
- Truncate command on first access, without participants
- Use Ballot.ZERO when invoking CFK.insertOutOfRange where appropriate
- Don't supply a command's own route to ProgressLog.waiting to ensure new keys are incorporated
- Ensure progress in CommandsForKey by setting vestigial commands to ERASED
- Add any missing owned keys to StoreParticipants.route to ensure fetch can make progress
- Recovery must wait for earlier not-accepted transactions if either has the privileged coordinator optimisation
- Inclusive SyncPoint used incorrect topologies for propose phase
- Barrier must not register local listener without up-to-date topology information
- Stop home shard truncating a TxnId to vestigial rather than Invalidated so other shards can make progress
Also improve:
- Validate commands are constructed with non-empty participants
- Remove some unnecessary synchronized keywords
- Clear ok messages on PreAccept and Accept to free up memory
- Introduce TxnId.Cardinality flag so we can optimise single key queries
- Update CommandsForKey serialization to better handle larger flag space
- Configurable which Txn.Kind can result in a CommandStore being marked stale
- Process DefaultProgressLog queue synchronously when relevant state is resident in memory
- Remove defunct CollectMaxApplied version of ListStore bootstrap
- Standardise linearizability violation reporting
- Improve CommandStore.execute method naming to reduce chance of misuse
- Prune and address some comments
patch by Benedict; reviewed by Alex Petrov for CASSANDRA-20282
- preclude TCM of being behind Accord if newer epoch is reported via withEpoch/fetchTopologyInternal
- improve topology discovery during first boot and replay
- fix races between config service TCM listener reporting topologies, and fetched topologies during
Patch by Alex Petrov, reviewed by Ariel Weisberg and David Capwell for CASSANDRA-20245
- Implement missing parts of protocol optimisations, refine some particulars and remove MEDIUM_PATH_WAIT_ON_RECOVERY
Also fix:
- Deps.txnIds -> Deps.txnIdsWithFlags to make clear unsafety, and validate Command isn't created with flags
- Save a lossy low/high epoch we're waiting on in WaitingState so that we can reconstruct the same Route on callback
- Load any potentially invalidated commands we had in ProgressLog to ensure they are marked invalidated locally and notify any waiters
- Refine Invariant to accept case where a truncated dependency should apply first but both transactions are redundant and only the waiter could not be cleaned up
- only update CFK.maxUniqueHlc using commands we execute
- Don't update uniqueHlc when zero to avoid incompatible transaction invariant check
patch by Benedict; reviewed by Alex Petrov for CASSANDRA-20222
TxnNamedRead was incorrectly determining whether a query was a continuation because the comparison was on TokenKey
and when a MinTokenKey was encountered it would discard the paging state. Comparison on token should be sufficient
because multiple queries in the same batch won't share the same left token bound, only the continuation query will.
Patch by Ariel Weisberg; Reviewed by David Capwell for CASSANDRA-20261
With paging it's possible for the range Accord needs to read from to have the start and end token be the same in which case the LHS token needs to be converted to a MinTokenKey
Patch by Ariel Weisberg; Reviewed by David Capwell for CASSANDRA-20262
- Fix waiting state callback computes different route to initiator
- Invariants.checkX -> Invariants.requireX (to allow complementary Invariants.expectX as appropriate)
patch by Benedict; reviewed by Alex Petrov for CASSANDRA-20228
Follow-up to CASSANDRA-20228:
- Fix AccordUpdateParameters to correctly supply List cell paths derived from applyAt
- Fix ExecuteAtSerialize.serialiseNullable
- Fix topologies flush regression caused by markRetired forcing a flush on every call
patch by Benedict; reviewed by Ariel Weisberg for CASSANDRA-20228
- detect and save whether RX can be used as a GC bound for HLCs
- have RX durably record if they witnessed any superseding epoch and use this for HLC GC
- rework GC to retain applyAt for writes
Also improve:
- PreCommitted overwrites fast path vote, then loses tie-break with NotAccept, so we don't recover
- track executeAtLeast and uniqueHlc separately from WaitingOn in CommandChange
- Always propose executeAt.hlc() >= txnId.hlc()
- Short-circuit recovery if ApplyAt already known
Also fix:
- Dangling recovery when Commit rejected
- Failed epoch fetch handling in DefaultProgressLog
- CommandsForKey must track Accept executeAt for computing earlierWait collection on recovery
- Report DefaultProgressLog exceptions before retrying
- Fix inverted condition in RecoverWithRoute
- Remove laterNoWait from laterWait collection in recovery
- Invalidate incorrectly selected maxNotTruncated if max had a not truncated component
- WaitingState.fetch not request/propagating prior epochs
- CFK visit active pruning was comparing txn against maxCommittedWriteBefore rather than maxCommittedWriteForEpoch
patch by Benedict; reviewed by Alex Petrov for CASSANDRA-20228
- Privileged coordinator. If the coordinator is a replica we can reduce our quorum sizes by including the coordinator's vote.
- with deps: if we include coordinator's preaccept deps we can reliably reduce quorum size by 1, at the expense of recovery sometimes requiring additional phases and waiting for future txns
- with only vote: if we only include the vote we can avoid any additional recovery phases or waiting for future txns, but can reduce our quorum size for only some configurations
- Medium path. If t=t0 at a simple majority we can take just two rounds.
- with additional phase: on recovery, earlier txns must wait for medium path to be disabled (or committed) so we do not accidentally recover a transaction that won't be witnessed
- with unstable deps: on slow path commit, deps not found in accept as committed as unstable and do not affect recovery decisions for earlier transactions
Also improve:
- Recovery of await conditions simply recalculates the rejects flag rather than restarting to ensure faster forward progress
- refactor Command hierarchy
- tweak: don't save Writes for non-write transactions
Also fix:
- isOutOfRange when invoked from callback for some txn < Committed
- handle loadingPruned that is pre-bootstrap on update to bootstrappedAt
- journal replay can overwrite in memory state loaded for a command not yet replayed
- subtle ordering bug in LatestDeps.mergeProposal
- fix occasional class initialisation order bug
- handle race condition on learning of topology
patch by Benedict; reviewed by Aleksey Yeschenko for CASSANDRA-20222
Improve:
- Remove GetMaxConflict; use local MaxConflict collection
Fix:
- Invalidated should retain StoreParticipants so we can update CFK on journal replay
- loading pruned uninitialised commands via CFK: make sure hasTouched contains key so that if invalidated we are notified
- updateExecuteAtLeast should always be higher than TxnId
- Async CFK callbacks treated pruned transactions incorrectly
- node.withEpoch when ExecuteEphemeralRead in futureEpoch
- Deps.without should be key/range aware
- Durably mark bootstrapBeganAt and safeToReadAt in MaxConflicts
- Don't attempt to calculate local deps when DepsErased in GetLatestDeps (command has durably applied to this shard)
- filter StoreParticipants before invoking shouldCleanup
patch by Benedict; reviewed by Alex Petrov for CASSANDRA-20183
Alex Petrov