For correctness, the dependencies we adopt on joining a new topology must exclude the possibility of respondents accepting additional transactions with a lower TxnId, so proxying on the existing `ExclusiveSyncPoint` mechanisms is logical for the time-being. This patch removes the `FetchMajorityDeps` logic in favour of simply waiting for a suitable `ExclusiveSyncPoint` to be proposed.
patch by Benedict, reviewed by Alex Petrov for CASSANDRA-20056
Patch by Alex Petrov; reviewed by Ariel Weisberg for CASSANDRA-20032
Accord Deps tests have incorrect range semantics
patch by David Capwell; reviewed by Ariel Weisberg for CASSANDRA-20029
update durability scheduling and majority deps fetching
do not deserialize deps in CommandsForRangesLoader unless required
AccordJournalPurger should use shouldCleanupPartial
load historical transactions when loading topology
split JournalKey in journal table so we can index it
reorder journal fields so we can easily index on route (when present)
use Message.expiresAtNanos for callback expiration
do not notify slow for range barriers
Accord: Do not contact faulty replicas, and promptly report slow replies for preaccept/read. Do not wait for stale or left nodes for durability.
* reconstruct CFK, TFK, progressLog
* migrate CommandStore collection state from Accord table to the log
* make memtable writes non-durable; reconstruct memtable state from Writes
Patch by Alex Petrov and Benedict Elliott Smith; reviewed by Benedict Elliott Smith and Alex Petrov for CASSANDRA-19869
(Accord): C* stores table in Range which will cause ranges to be removed from Accord when DROP TABLE is performed
patch by David Capwell, Sam Tunnicliffe; reviewed by Sam Tunnicliffe for CASSANDRA-18675
CEP-15: (Accord) sequence EpochReady.coordinating to allow syncComplete to be learned from newer epochs
patch by David Capwell; reviewed by Alex Petrov, Blake Eggleston for CASSANDRA-19769
The SimpleProgressLog had a number of problems:
1. It polled for progress with no attempt to determine whether progress could realistically be made, so:
- as the number of pending transactions grew, the proportion of useful work dropped (as many would be unable to make progress without earlier transactions completing)
- each transaction in the chain could recover only on average 1/2 poll interval behind the last transaction to complete
2. It requested full transaction state from every replica on each attempt
3. It maintained a lot of in-memory state
4. Polling happened en-masse, allowing for little per-transaction control
We also separately maintained fairly expensive per-command listener state that negatively affected our command loading and caching.
The new DefaultProgressLog makes use of several new features: LocalListeners, RemoteListeners, Timers and Await messages.
- LocalListeners provide a memory-efficient collection for managing each CommandStore<E2><80><99>s transaction listeners, with dedicated record keeping for inter-transaction relationships.
- RemoteListeners provide a mechanism for request/response pairs that may be separated by longer than the normal Cassandra message timeout, and require minimal state on sender and recipient. This permits replicas to cheaply update their local state machine as soon as distributed information becomes available.
The DefaultProgressLog tracks each transaction with separate timers to handle per-transaction scheduling, backoff etc, and a succinct state machine. To reduce overhead correspondence is preferentially limited to a handful of replicas, and limited to the home shard where appropriate.
patch by Benedict; reviewed by Ariel Weisberg for CASSANDRA-19870
- Remove concept of non-participating home keys; home keys are required to be a participant in the transaction
- Remove covering/covers concept
- Various invalidation/truncation/erase behaviours
patch by Benedict; reviewed by Blake for CASSANDRA-19825
CommandsForKey periodically self-prunes, so as to continue functioning well in-between garbage collections. Once we prune we are left with potentially incomplete information, and have to sometimes load per-command information from disk. But the payoff is ensuring CommandsForKey objects - which drive the majority of the state machine - are kept to a reasonable size.
patch by Benedict; reviewed by Blake Eggleston and David Capwell
Introduce a special kind of non-durable read that provides only per-key linearizable isolation; i.e. strict-serializable isolation for single partition-key reads.
This read creates only a happens-before edge, by collecting dependencies for execution and ensuring that execution happens strictly after these dependencies
have executed, but at no precise time otherwise. So later writes may be witnessed, and if multiple keys are read they may represents different points in time.
patch by Benedict; reviewed by Ariel Weisberg for CASSANDRA-19305
Refactor CommandsForKey for efficiency, and to support transitive dependency elision
patch by Benedict; reviewed by Aleksey Yeshchenko for CASSANDRA-19310
s fc83325fa9 Fix AccordObjectSizes empty sizes
s 44bce6a08c Fix error handling when there are no column families to repair
s f1459540e5 Fix broken nowInSec deserialization
s 91c1befd1c Fix Mutation serializedSize
s b11467c200 Using simulated clock instead of raw nanoTime
s 6412b35924 Accord repair needs to use sentinel tokens
s d19c01ff01 Enable Accord repair with HappyPathFuzzTest
s 2da87f91b8 Implement abort in AccordRepairJob
s e12877c4fa move barrier to accord spec
Patch by Ariel Weisberg; Reviewed by David Capwell for CASSANDRA-19023
s c4bf1533b4 Add more JVM arg checks to the paxos simulation runner
s 1b650a0636 break circular dependency between FileSystemOwnershipCheck and CassandraRelevantProperties
s e0111748dd fix verify with-rng and interrupts
s f8475f2a18 make SimulatedFailureDetector deterministic
s ee77734bbe Deterministic transformation
s 890f23696a Add mechanism forsimulator to intercept password salting
s 5205b99874 Specify Paxos V2 because it is the only linearizable option
Patch by Ariel Weisberg; Reviewed by David Capwell for CASSANDRA-19008
- Avoid serializing full TxnData instances to Accord state tables
patch by Caleb Rackliffe; reviewed by David Capwell, Benedict Elliot Smith, and Ariel Weisberg for CASSANDRA-18355
Accord compaction purgers see random slices of Accord state during compaction (based on randomly selected compaction inputs).
For at least the `durability` column in the `commands` table the tombstone being created when truncating was deleting the latest value since we can get enough information to truncate without actuall yhaving the latest `durability` value.
To fix we can wait to emit a tombstone until we are erasing the entire command row when truncating or truncating with outcome and meanwhile we can drop the extra columns that are no longer needed instead of using a tombstone. We don't need to emit cell tombstones we can drop them from the purger when processing each row.
patch by Ariel Weisberg; reviewed by David Capwell for CASSANDRA-18795
patch by Benedict Elliott Smith; reviewed by Ariel Weisberg, Aleksey Yeschenko, and David Capwell for CASSANDRA-18883
Co-authored-by: Benedict Elliott Smith <benedict@apache.org>
Co-authored-by: Ariel Weisberg <aweisberg@apple.com>
Co-authored-by: Aleksey Yeschenko <aleksey@apache.org>
Co-authored-by: David Capwell <dcapwell@gmail.com>
Patch by Blake Eggleston and Benedict Elliott Smith; Reviewed by David
Capwell for CASSANDRA-17101
CEP-15: Accord TCM integration
Patch by Blake Eggleston; Reviewed by David Capwell for CASSANDRA-18444
David Capwell