* Rename and simplify fetch time variables
* Add RefreshTime detail to TenantCacheGetStorageUsageRefreshSlow trace
* Stagger storage estimation requests
* Update the value of a knob in simulation to reduce flakiness
* Improve names of TenantCache and StorageQuota related traces. Add slow refresh time.
* Convert potentially spammy TenantCache traces to SevDebug
* Enable secure allocation mode in Arena
This mode allows zeroing out blocks holding sensitive data after use
* Introduce WipedString to all token-holding memory
Also introduce a option flag "sensitive"
* Make pointer equivalency a hard requirement for non-ASAN builds
So that we can detect when Arena/malloc/memory-wipe behavior changes
* Boilerplate code for tenant-aware DD
* Add an DD tenant-cache-assembly actor
* Add basic tenant list monitoring for tenant cache. Tenant cache is not yet a DD-scoped structure
* Create a DDTenantCache class
* Move DDTenantCache to new files
* Tweak some traces; fix placement of build and monitor actor invocations
* Update DD tenant cache refresh to be more efficient and unit-testable
* Fix incorrect use of prefixToID in the DDTenantCache code; it is not correct when the tenant is within a tenant subspace
* Fix a faulty assertion in a DDTenantCache unit test, and add a server knob to switch of DDTenantCache code for the moment
* Fix a DDTenantCache UT bug where count of tenants could be 0, causing divide-by-0 exceptions in the test code; Fix accidental movement of atrace statement; rename some DD tenant-cache related variables
* Make some member DDTenantCache member functions const, and change the ddtc abbreviation to ddTenantCache, for ddtc could refer to DDTeamCollection
* Remove unnecessary assertion that erase-ing an element from a Map really removes it
* Remove the DD prefix in the tenant cache class name (and associated impl and UT class names); there is nothing specific to DD in it; DD uses it; other modules may use it in the future
* Disable DD tenant awareness by default
* Split shards along tenant keyspace boundaries
* Update some naming, switch to using prefixRange and fix a bug in tenantShardSplit
* Reorganize tenant shard splitter
* Moved new header file to new include dir under fdbserver
* Make shardEvaluator trigger shard merger only if adjacent shards belong to the same tenant, if tenant-keyspace boundaries are being honored
* Fix issues in the way shard merge decisions are made based on tenant boundaries
* Minor reorg and bug fix in shard merger decision-making
* Resolve merge issues
* Fix bug in ShardEvaluator that made it run in a loop rather than waiting for metrics changes
* Fix tenant splitter for when start and end are in different shards and only one of them needs to be split
* Remove some uses of printable
* Remove uses of printable
* Fix code formatting
* Add a TENANT_SPLIT RelocateReason and fix errors from incorrect merge of upstream code
* Move tenantCreationSignal ownership to TenantCache
* Update fdbserver/DataDistributionTracker.actor.cpp
Co-authored-by: Xiaoxi Wang <xiaoxi.wang@snowflake.com>
* Fix a couple of bugs and spelling errors in tenantShardSplitter
* Update fdbserver/DataDistributionTracker.actor.cpp
Co-authored-by: Xiaoxi Wang <xiaoxi.wang@snowflake.com>
* Refactor split-merge feasibility check methods
* Merge upstream changes
* Break down tenantShardSplitter to remove dependency on DataDistributionTracker and make the splitter unit-testable
* Add unit tests for tenant shard splitting
* Factor out some repeated code when finding tenant boundaries within a shard
* Extract fault line verification steps used in UTs into a separate function
Co-authored-by: Xiaoxi Wang <xiaoxi.wang@snowflake.com>
* Enable configuring the next future protocol version as the current protocol version in FDB client, fdbserver, and fdbcli
* Auto format python files used in upgrade tests
* Add a test for upgrading to a future FDB version
* Emphasize that the options for using future protocol version are intended for test purposes only
* Make the global variable for current protocol version visible only locally
* Refactirng to avoid using currentProtocolVersion() in static intialization
* Update go bindings
* Add an DD tenant-cache-assembly actor
* Add basic tenant list monitoring for tenant cache.
* Update DD tenant cache refresh to be more efficient and unit-testable
* Remove the DD prefix in the tenant cache class name (and associated impl and UT class names); there is nothing specific to DD in it; DD uses it; other modules may use it in the future
* Disable DD tenant awareness by default
Syed Paymaan Raza