Found by simulation:
seed: -f tests/slow/ApiCorrectnessAtomicRestore.toml -s 177856328 -b on
Commit: 51ad8428e0
Compiler: clang++
Env: Rhel9 okteto
applyMutations() has processed version 801400000-803141392, and before calling sendCommitTransactionRequest(),
which was going to update apply begin version to 803141392. But DID NOT wait for the transaction commit.
Then there is an update on the apply end version to 845345760, which picks up the PREVIOUS apply begin version 801400000.
Thus started another applyMutation() with version range 801400000-845345760. Note because previous
applyMutation() has finished and didn't wait for the transaction commit, thus the starting version
is wrong. As a result, this applyMutation() re-processed version range 801400000-803141392.
The test failed during re-processing, because mutations are missing for the overlapped range.
The fix is to wait for the transaction to commit in sendCommitTransactionRequest().
This bug probably affects DR as well.
See rdar://146877552
20250317-162835-jzhou-ff4c4d6d7c51bfed
* New restore consolidated commit
This change adds RestoreDispatchPartitionedTaskFunc to restore
from partitioned-format backup.
* ArenaBlock::totalSize parameter pass by ref
* Fix format issues identified by CI
* range lock framework
* improve the framework
* persist to txnStateStore
* fix bugs
* code clean
* code clean
* bug fix
* address comments
* add complex test workload and fix bugs found by the workload
* add workload correctness check and fix bugs
* code clean up
* add random range lock injection
* fix bugs in RandomRangeLock.actor.cpp
* enable random range lock injection in general workloads
* add rangelockcycle test
* disable random range lock in backup workloads
* nits
* add range lock ownership concept
* enable lock ownership to rangeLock
* api deal with tenant
* fix CI
* add test for multiple rangeLock owners
* nits
* address comments and renaming
* address comments
E.g., StatusBuilderPerf and TLogVersionMessagesOverheadFactor are more like
performance tests, which shouldn't be running so many times.
Without the change, a 100k-run has this many for these tests:
1318 tests/rare/CycleWithKills.toml
1591 tests/rare/TLogVersionMessagesOverheadFactor.toml
1647 tests/rare/ConfigDBUnitTest.toml
1839 tests/rare/StatusBuilderPerf.toml
After the change, a 100k-run has:
129 tests/rare/TLogVersionMessagesOverheadFactor.toml
151 tests/rare/CycleWithKills.toml
160 tests/rare/StatusBuilderPerf.toml
375 tests/rare/ConfigDBUnitTest.toml
In the restore test, there are hot ranges with writes to customized range with
prefix "BeforeRestart" or "AfterRestart". As a result, FDB can return
transaction_throttled_hot_shard errors and cause test failure.
* - Compare storage replicas on reads (in "loadBalance()")
* - Do consistency check on reads in loadbalance
* - Do replica consistency check in the case where loadBalance issues
requests to multiple storage servers
* - Address a state variable related bug
* - Code formatting
* - API simplification
* - Simplify code
* - Code formatting
* - Address a review comment
Description
Given Configurable encryption has been checked in and being tested via
simulation for more than a month and also to avoid penalty of accessing
KNOBS in inline commit path, patch retires the KNOB and make
ConfigurationEncryption default EaR mode for FDB.
BlobCipher still supports the old format header and encryption semantics,
will remove the dead code as a followup PR.
Testing
devRunCorrectness - 100K
* EaR: Configurable encryption support for TLog mutations
Description
diff-1 : Address review comments
Major changes includes:
1. Update the code involved in ensuring Tlog mutation encryption to be
compliant with "configurable encryption" feature.
2. Update ENABLE_CONFIGURABLE_ENCRYPTION flag to be 'true' by default
and BUGGIFY it.
Testing
devRunCorrectness - 100K
Update fdbclient/DatabaseBackupAgent.actor.cpp
Co-authored-by: Trevor Clinkenbeard <trevor.clinkenbeard@snowflake.com>
Provide log version information to fdbdr status
Zhe Wang