* throttle the cluster when blob workers fall behind
* do not throttle on blob workers if they are not enabled
* remove an unnecessary actor
* fixed a compile error
* fetch blob worker metrics at the same interval as the rate is updated, avoid fetching the complete blob worker list too frequently
* fixed another compilation bug
* added a 5 second delay before bw throttling to prevent false positives caused by the 100e6 version jump during recovery. Lower the throttling thresholds to react much quicker to bw lag.
* fixed a number of problems
* changed the minBlobVersionRequest to look at storage server versions since this will be a lot more efficient
* fix: do not let desired go backwards
* fix: track the version of notAtLatest changefeeds for throttling
* ratekeeper now throttled blob workers by estimating the transaction per second throughput of the blob workers
* added metrics for blob worker change feeds
* added a knob to disable bw throttling
* fixed the transaction options in blob manager
* Granule purge cannot delete history entry for fully deleting granule until all children are completely done splitting
* Several purging fixes related to granule history
* Fixed typo in refactor
* fixing memory model for purgeRange
* formatting
* weakening granule purge test for now
* cleanup
* First version of force purging granules
* fixing issue in BW range assignment reporting
* Fixing incorrect assert with force purging
* Error handling when checking force purged state
* fixed force purging and recover/reassign range races and check
* Handling force purge + boundary change race
* more places to check for force purged status
* fixed manager restart in the middle of force purge bug
* fixing same-BM purge and assignment races in all cases
* weakening orphaned granule history check a bit because of difficult to solve races
* fixing txn options on retry
* loading force purged ranges at start to avoid resuming a merge that is being force purged
* cleanup
* Enabling purging in granule tests, and adding check for leaked change feeds in force purge
* formatting
* missed parameter in merge conflicts
* Fixing leaked change feed race with merge and force purge
* adding change feed cleanup when new blob manager recovers in-progress merge that raced with force purge
* added forcepurge fdbcli command
* Encrypt BlobGranule delta files
Description
diff-1: Address review comments
Major changes proposed by the patch are:
1. Refactor code to allow caching of 'encryption key ctx' as part of
BlobFilePointerRef. The refactoring allows snapshot and/or delta files
to store their own file encryption context.
2. Enable BlobGranule delta file encryption/decryption semantics.
Testing
BlobGranuleCorrrectness
BlobGranuleCorrectnessClean
BlobGranuleFileUnitTestToml
Description
Testing
Josh Slocum