These were lost, likely due to refactoring. Now TLogMetrics have meaningful
data like:
TLogMetrics ID=59ec9c67b4d07433 Elapsed=5 BytesInput=0 -1 17048 BytesDurable=47.4 225.405 17048 BlockingPeeks=0 -1 0 BlockingPeekTimeouts=0 -1 0 EmptyPeeks=1.6 2.79237 236 NonEmptyPeeks=0 -1 32 ...
We have a recent redesign that no longer required to pass tenant name to get encryption key, and also not allowing optional tenant mode for tenant-aware encryption. This PR clean up Redwood code to remove tenant map usage, and update various checks accordingly.
Changes:
* Cleanup TenantPrefixIndex in TenantAwareEncryptionKeyProvider and related logic in storage server and Redwood for passing the map around.
* Cleanup and update DecodeBoundaryVerifier the reflect the new design.
* A minor fix to writePages() that avoid a page that's default domain encrypted having a lower bound key belonging to a non-default domain.
* Fix TenantAwareEncryptionKeyProvider::getEncryptionDomain() returning wrong prefix long for system domain.
* A minor change to add a context string to IoTimeoutError.
* Extend Tlog persistentStorage to persist encryption state
Description
diff-3: Address review comment.
diff-2: Extend ClusterController endpoints to allow query
cluster's encryptionAtRest status
Update Tlog recovery to ensure on-disk encryption
status matches with cluster's cstate persisted
encryptionAtRest
diff-1: Store encryptionAtRestMode state in Coordinators
Major changes proposed are:
1. Extend TLog persistentStorage to persist encryption state
2. Encryption state persisted is derived from corresponding
db-config and relevant SERVER_KNOBS. In near future, knobs
shall be removed.
3. On TLog startup, the persisted encryption state is compared
against cluster configuration, if mismatch, the TLog is killed
and not allowed to rejoin the cluster.
Testing
devRunCorrectness - 100K
The logic to determine the validity of a process joining a cluster now
belongs on the worker and the cluster controller. It is no longer
restricted to tlogs and storages, but instead applies to all processes
(even stateless ones).
* Init TLogPersistent storage for a sharedTLog
Description
diff-1: Address review comments
Patch udpates the code to intialize TLogPersistent storage for a
shared TLog independent of intializing persistentState for a
versioned Tlog data. Appraoch allows initializing Tlog persistent
storage as well as writing 'persistFormat' key for a shared TLog
earlier in the TLog creation lifecycle.
Testing
devRunCorrectness - 100K
* fix: persistentData->commit() was not protected by the persistentDataCommitLock, meaning it is possible for inconsistent data to be made durable on the tlog
* fixed a compilation error
We've observed the recovery process stuck in initPersistentState,
while waiting to acquire persistentDataCommitLock. All of the
other places in that function which potentially interact with a
disk are guarded by a timeout: TLOG_MAX_CREATE_DURATION.
Since it's possible that the current holder of that lock is
stuck in persistentData->commit(), it makes sense to add
a timeout around the entire function, rather than each of the
places where it might get stuck on an I/O operation.
The end result is that after 10 seconds, this process will fail
and cluster recovery will restart.
* proof of concept
* use code-probe instead of test
* code probe working on gcc
* code probe implemented
* renamed TestProbe to CodeProbe
* fixed refactoring typo
* support filtered output
* print probes at end of simulation
* fix missed probes print
* fix deduplication
* Fix refactoring issues
* revert bad refactor
* make sure file paths are relative
* fix more wrong refactor changes
* Add an internal C API to support memory connection records
* Track shared state in the client using a unique and immutable cluster ID from the cluster
* Add missing code to store the clusterId in the database state object
* Update some arguments to pass by const&
This PR add support for TLog encryption through commit proxy. The encryption is done on per-mutation basis. As CP writes mutations to TLog, it inserts encryption header alongside encrypted mutations. Storage server (and other consumers of TLog such as storage cache and backup worker) decrypts the mutations as they peek TLog.
* Fix comments
* Add simulation value for SERVER_KNOBS->SNAP_CREATE_MAX_TIMEOUT
* A work version with correctness clean
* Remove unnecessay comments; debugging symbols
* Only check secondary address for coordinators, same as before
* Change the trace to SevError and remove the ASSERT(false)
* Remove TLogSnapRequest handling on TlogServer, which is changed to use WorkerSnapRequest
* Add retry for network failures
* Add retry limit for network failures; still allow duplicate snapshots on processes are both tlog and storage to avoid race
* Add retry limit as a knob and make backoff exponentail
* Add getDatabaseConfiguration(Transaction* tr)
* revert back to send request for each role once
* update some comments
Zhe Wu