Changes:
1. Change `isEncryptionOpSupported` to not check against `clientDBInfo.isEncryptionEnabled`, but instead against ENABLE_ENCRYPTION server knob. The problem with clientDBInfo is before its being broadcast to the workers, its content is uninitialized, during which some data (e.g. item 2) is not getting encrypted when they should.
2. Fix CommitProxy not encrypting metadata mutations which are recovered from txnStateStore
3. Fix KeyValueStoreMemory (thus TxnStateStore) partial transaction coming from recovery is not encrypted
4. new CODE_PROBE for the above fixes
5. Logging changes
* fix: the log router could see popped() versions between recoveredAt and the recovery txn version which cause remote tlogs to be stopped at a version in this range, which causes storage servers to read at versions in this range
* fix: a spurious correctness bug resulted from the consistency check running during a recovery
Configuration database data lives on the coordinators. When a change
coordinators command is issued, the data must be sent to the new
coordinators to keep the database consistent.
* 'main' of github.com:sfc-gh-nwijetunga/foundationdb: (32 commits)
Store rocksdb::DBOptions and rocksdb::ColumnFamilyOptions to (#7766)
Update CONTRIBUTING.md
Update tests/rare/SpecificUnitTests.toml
fix ASAN OOM problem
Update CONTRIBUTING.md
Write tracing and ALP special key errors as JSON
Fix: the static tenant map in the Java tester was being accessed concurrently from multiple threads. Make it a concurrent map. (#7805)
Run clang-format
Print SIGNAL output to stdout
Print to stderr only upon errors
Testing upgrades to a future version of FDB (#7780)
Flush gcov coverage upon SIGTERM
Report the unit tests being run in test harness
Fix a bug in a storage wiggler unit test where some servers were added with too recent a timestamp
Fix undefined behavior in versioned btree test due to integer overflow
When a transaction operation gets an unknown tenant error, it needs to reset the tenant ID so it can be updated in the next tenant lookup request.
Don't buggify max tenants per cluster globally; instead buggify it in specific tests
Remove non-existing unittest
Add unit tests to the correctness package
Add comment to INetwork
...
* Enable configuring the next future protocol version as the current protocol version in FDB client, fdbserver, and fdbcli
* Auto format python files used in upgrade tests
* Add a test for upgrading to a future FDB version
* Emphasize that the options for using future protocol version are intended for test purposes only
* Make the global variable for current protocol version visible only locally
* Refactirng to avoid using currentProtocolVersion() in static intialization
* Update go bindings
There was a rare but possible issue where a change coordinators request
could be in flight, but reading `\xff/coordinators` would return the old
set of coordinators, even though a previous attempt to write it returned
`commit_unknown_result`. This happened if there was an unrelated
recovery between setting `\xff/coordinators` and the updated
coordinators reaching each machine. The `commit_unknown_result` caused
by this recovery meant the change coordinators retry loop would read
`\xff/coordinators`, but since the request was ongoing it would read the
old set of coordinators. The fix is to have the cluster controller
ignore any change coordinators request from an old generation, meaning
when the retry loop reads the old coordiantors from `\xff/coordinators`,
it is guaranteed that the in progress change coordinators message will
be rejected when arriving at the cluster controller.
* proof of concept
* use code-probe instead of test
* code probe working on gcc
* code probe implemented
* renamed TestProbe to CodeProbe
* fixed refactoring typo
* support filtered output
* print probes at end of simulation
* fix missed probes print
* fix deduplication
* Fix refactoring issues
* revert bad refactor
* make sure file paths are relative
* fix more wrong refactor changes
Adding encryption support for TxnStateStore. It is done by supporting encryption. for KeyValueStoreMemory. The encryption is currently done on operation level when the operations are being write to the underlying log file. See inline comment for the encrypted data format.
This PR depends on #7252. It is part of the effort to support TLog encryption #6942.
This was not set and can cause infinite loop in simulation where the client
calls getConsistentReadVersion(), in which we do "continue" for stale GRV reply
and retry. Then this repeats forever.
Yi Wu