* Disable version vector unicast with idempotent transactions
* Respond to review requests
---------
Co-authored-by: Dan Lambright <hlambright@apple.com>
* Track shard moves for version vector
* Don't broadcast to all TL when a different CP had a metadata mutation, unless on shard moves
* update lastShardMove on resolver
* Respond to review comments
---------
Co-authored-by: Dan Lambright <hlambright@apple.com>
Simulation found an assertion failure in SS:
ASSERT(rollbackVersion >= data->storageVersion());
The reason is that storage version is updated to a version larger than the
forced recovery version, due to only 1'000'000 for max_read_transaction_life_versions.
Also added debugging for cumulative checksum mutations.
See rdar://144550725
20250309-185039-jzhou-5145c65b0e8071b7
* During commits with version vector enabled, compute location list only once, as recalcuating could
generate a different random number, hence a different set of locations.
* Respond to review comments.
* Select replicas from locations returned from resolver.
* Respond to review comments
---------
Co-authored-by: Dan Lambright <hlambright@apple.com>
* New restore consolidated commit
This change adds RestoreDispatchPartitionedTaskFunc to restore
from partitioned-format backup.
* ArenaBlock::totalSize parameter pass by ref
* Fix format issues identified by CI
version only after receiving a commit version reply from the sequencer.
Advancing the min committed version prior to that point may result in
invalid DBRecoveryDurability errors (if a recovery happens after the
advancement) in simulation tests.
* disable version vector with range lock tests
* turn off rangeLock if versionvector is on (#11747)
---------
Co-authored-by: Dan Lambright <hlambright@apple.com>
Co-authored-by: Zhe Wang <zhe.wang@wustl.edu>
* range lock framework
* improve the framework
* persist to txnStateStore
* fix bugs
* code clean
* code clean
* bug fix
* address comments
* add complex test workload and fix bugs found by the workload
* add workload correctness check and fix bugs
* code clean up
* add random range lock injection
* fix bugs in RandomRangeLock.actor.cpp
* enable random range lock injection in general workloads
* add rangelockcycle test
* disable random range lock in backup workloads
* nits
* add range lock ownership concept
* enable lock ownership to rangeLock
* api deal with tenant
* fix CI
* add test for multiple rangeLock owners
* nits
* address comments and renaming
* address comments
* - Compute known committed version correctly when version vector unicast
is enabled.
* - Set ProxyCommitData::minKnownCommittedVersion only if the commit
version is above ProxyCommitData::committedVersion.
* acs framework
* code refactor and fix bugs
* add ss crash loop protector
* use sharedptr instead of raw pointer
* fixed critical bugs and add provate mutation acs to the framework
* enable ACS for all mutations except for clear serverTag mutation and fix bugs
* fix restarting tests
* refactor code and fix bugs
* fix AccumulativeChecksumState toString
* fix bugs
* allow all mutations in acs and fixed bugs
* fix bugs and code cleanup
* code clean up for adding recovery support
* simplify code and support recovery
* clear acs state at ss
* fix bug
* terminate validator if ss will be removed in the current batch
* simplify code
* add trace
* address comments
* optimize code
* deep copy when adding mutation to acs validator
* warp encode and decode persist acs key
* make acstable private
* remove unless func
* remove unless func
* remove epoch in ACS validator
* add acs mutation counter in SS metrics
* code cleanup and make knob check better
* make mutation buffer global
* simplify code
* add comments
* make knob randomly set
* address comments
* ss reboot after acs mismatch found
* Fix detection of private mutations in version vector
* add assertion that all tlogs receive changes to txn state in version vector
* Re-suppress version_vector upgrade tests
---------
Co-authored-by: Dan Lambright <hlambright@apple.com>
* throttle hot shards
* expire throttled shards over time
* add backoff
* Parallelize messaging from RK to CP
* Obtain shards from a single SS
* handle expired transactions
* bump transaction_throttled_hot_shard
* Change SevError to SevWarn for CannotMonitorHotShardForSS
* Add log per request
Description
Given Configurable encryption has been checked in and being tested via
simulation for more than a month and also to avoid penalty of accessing
KNOBS in inline commit path, patch retires the KNOB and make
ConfigurationEncryption default EaR mode for FDB.
BlobCipher still supports the old format header and encryption semantics,
will remove the dead code as a followup PR.
Testing
devRunCorrectness - 100K
now() is only updated in the run loop, so timer() is more appropriate
for accurate measurements of latency. This change is limited to the
commit proxy server, where we observed inaccurate latency statistics.
Jingyu Zhou