These features have been previously marked for deletion per PR #12400.
This change necessarily affects a lot of files. In general I found it preferable to cut along the FDB <-> tenant boundary, rather than try to cut tenant into multiple pieces, stitch the Frankenstein tenant implementation back together with FDB, and generally remove the limbs one by one. So it is a single big deletion.
Note that some tenant-related metadata has been written in a non-flag-controlled manner by prior releases and probably must be ignored indefinitely. Fortunately this is isolated to include/fdbclient/ClientLogEvents.h. (Details: deleting an Optional from a serialized struct results in deserialization of garbage in upgrade tests. The serialized nullopt to indicate "no Tenant" is formally part of FDB persistent metadata even in FDB clusters that never would have enabled the tenant feature.)
During the course of testing these changes, many interesting bugs were encountered. I won't discuss details of them here. Causes range from flat out damage (by me) to production code in the course of removing tenant related bits (mainly in NativeAPI.actor.cpp and CommitProxy.actor.cpp), damage to various workload files (particularly FuzzApiCorrectness.actor.cpp, which is very sensitive to changes), and many toml files needing updated test flags/options.
More testing details: https://quip-apple.com/Zr6VAycxoli9
20251209-012852-gglass-8ff850b772d868f2 compressed=True data_size=35311687 duration=21671404 ended=500000 fail_fast=1000 max_runs=500000 pass=500000 priority=100 remaining=0 runtime=2:31:30 sanity=False started=500000 stopped=20251209-040022 submitted=20251209-012852 timeout=5400 username=gglass
* remove some unneeded tests, and remove mentions of deleted tests from tests/CmakeLists.txt
* Initiate removal of metacluster. NOTE: this seems to also want removal of tenant. Consider removing them together.
* work on removing metacluster
* delete files with `Tenant` in the name, having reviewed them to ensure that they basically contain what the name implies
* fdb_c.h: remove prototypes for C API methods which have been deleted (blob granule) or which are so long deprecated that they are outside any reasonable/documented support window
* Surgical removal of tenant references from files in bindings/ top level directory. Compilation not yet attempted.
* Surgical removal of tenant related stuff from fdbcli/ top level directory. Compilation not yet attempted.
* Misc tenant code removal, and other stuff which I think may not be needed. Compilation still not attempted.
* Remove more tenant or tenant-adjacent or blob-granule-adjacent stuff. Or at least stuff that looks adjacent to that stuff. Not compiled or tested.
* Start removing Tenant stuff from fdbclient/. Far from complete. Compilation not attempted.
* Remove tenant references from many source files. There are still about 7 principal fdbclient/ and fdbserver/ files with a lot of tenant logic left to delete. Also, all of fdbserver/workloads needs to be looked at. Still have not attempted compilation.
* Remove tenant entanglement from watch functionality
* Remove tenant stuff from fdbserver/tester.actor.cpp
* Delete metacluster workloads
* Remove tenant related stuff from workloads. Also taken the liberty of removing some functionality that appears unused or untestable by Apple.
* Checkpoint tenant removal from FuzzApiCorrectness.actor.cpp
* NativeAPI.actor.cpp: `Tenant` has left the building.
* SimulatedCluster.actor.cpp: `Tenant` has left the building
* DDShardTracker.actor.cpp: Tenant evicted
* storageserver.actor.cpp: `tenant` has left the building.
* fdbserver/workloads/FuzzApiCorrectness.actor.cpp: remove tenant references, but some lingering cleanup needed in `loadAndRun`
* FileBackupAgent.actor.cpp: tenant has left the building
* CommitProxyServer.actor.cpp: remove tenant
* Remove more tenant references from misc files such as bindings tests, documentation, and some fdbserver headers I left earlier
* Fix missing-file errors in CMakeLists.txt files. This is the first attempt to compile this stuff.
* checkpoint misc changes to fix compile errors
* checkpoint more compile fixes
* StorageServerInterface.h: put back more verify() calls
* More misc compile fixes
* whole bunch of misc fixups including some code put-backs to address compile errors
* More compile fixes
* More compile fixes. Still does not compile.
* incremental compile fixing
* ...
* ...
* Checkpoint a bunch of compile fixes. Not quite there but getting closer
* More compile fixes. There seem to be about 10 files left, mainly CommitProxyServer.actor.cpp and storageserver.actor.cpp
* IT COMPILES NOW. THIS IS STILL ALL UNTESTED. Unsurprisingly, CommitProxyServer.actor.cpp and storageserver.actor.cpp took the most tweaking.
The updates in CMakeLists.txt and workloads/UnitTests.actor.cpp are basically trivial and mainly reflect
the ordering of dependencies -- that stuff didn't get attempted until all of fdbserver compiled.
* Put back one block relating to encryption at rest mode. Simplify some TODO(gglass) instances.
* Put back some encryption related knobs
* remove `enable_tenants` from local_cluster.py to maybe fix some ctests
* Remove tenant related options from toml files.
* feature-status.md: add a line for encryption at rest, which seems to have been added for multi-tenant; status is now in doubt
* Fix a pretty bad bug introduced in tenant deletion; ensure we dont attempt to construct a std::string of negative length
* workloads/FuzzApiCorrectness.actor.cpp: avoid division by zero
* flow/Platform.actor.cpp: add a try/catch wrapper around side threads; emit a better addr2line type command
* NativeAPI.actor.cpp: fix a bug introduced in tenant removal relating to reporting conflicting keys under conflictingKeysRange
* ReportConflictingKeys.actor.cpp: separate an ANDed assert into two asserts
* SpecialKeySPaceCorrectness.actor.cpp: put back some logic removed with tenant removal. This test was failing due to a bug with conflict key range reporting. Fixed separately in NativeAPI.actor.cpp.
* remove QuotaCommand.actor.cpp
* Force disable tenant and encryption on disk in upgrade tests
* Add back file I guess I deleted? who knows
* put back another file
* design/feature-status.md: update the new row for encryption at rest to firm up the claim that it is experimental, unowned, and scheduled for deletion
* Remove EncryptKeyProxyTest since we do not use it
* new file tests/slow/BulkDumpingS3WithChaos.toml: remove tenantModes setting
* Undo damage to pushToBackupMutations() from removing tenant feature. This caused inverted_range errors and failed commits in backup related simulations.
* tests/restarting/from_7.4.0/Snap*-1: ensure that tenantModes = disabled
* Try again on workloads/FuzzApiCorrectness.actor.cpp
* simplify tenant-free (mostly) FuzzApiCorrectness workload code
* try harder to remove lingering tenant-related brokenness from FuzzApiCorrectness.actor.cpp
* Explicitly specify tenantModes = ['disabled'] in all the -1 restart files
* Remove tenantModes from 7.1-based upgrade tests as its an unknown option. Hopefully the code doesnt actually turn on tenant stuff
* do not specify tenantModes in downgrade tests
* Downgrade test to_7.4.5: dont say tenantModes
* more tenantModes updates
* Remove a legacy allowDefaultTenant that no longer is meaningful in downgrade to 8.0
* Put back empty Optional<TenantName> turdlets into serialized log events to avoid breaking ClientTransactionProfilingCorrectness upgrade tests (even with tenantMode = disabled)
* disable encryption on a few more upgrade related test cases. That feature is slated for removal anyway
* Remove unneeded workload files that have been subject to #if 0 for a while. Remove commented out block in ClusterRecovery
* disable encryption in more upgrade tests
* Remove choice four-letter words from commentary
* Format 42 files
* Try to fix a doc bug failing the CI build
* More doc compilation error fixes
* Delete more tenant junk from documentation
* fix spelling mistake in comment
* Remove deleted cross-references from documentation. This necessitated editing release 3.0.0 release notes, which is insane.
* Remove more tenant stuff from bindings tests
* Remove more tenant bits from design/ files
* Remove more tenant related stuff
* Delete more tenant references. Put back ten-ant spellings as tenant now that grep output is substantially reduced.
* Put back some tenant stuff into apitester; its deletion seems to have introduced bugs. Also whine about comments some more, because, really, the comments deserve it.
* Updates to workload files and one other thing based on review comments
* de-actorify decodeKVPairs
* format one source file
* Restore transaction tagging doc
* Restore throttle doc details in administration.rst
* Restore fdbserver/workloads/GetEstimatedRangeSize.actor.cpp and associated toml file, minus tenant stuff
* bindings/c/test/{shim related}: update comments and disable functionality that no longer works post-tenant
* put the cli-throttle tag back in
* bindingtester: fix python syntax errors
* remove useless comment
* Remove comment about useless comments, and remove the useless comments
* get_client_status: add to Java bindings
* Add get_client_status in python binding
* Make python unit tests available as ctest
* Fix file name in the copyright header
* Fix library path name for mac
* get_client_status: Minor changes in Java & Python bindings addressing review comments
* Rename fdb_python_unit_tests to unit_tests
Add atomic op functions to the tenant object in Python.
Fix a Python tester bug for tenant operations.
Fix a build error with the binding tester package.
Only fetch requested worker interfaces in rebootWorkerActor
Add killall test to multiprocess_fdbcli_tests
Make the rebootWorker only sends requests if all the requested workers are valid
Add comments; Organize print texts; Fix the bug clear the map every time run kill/suspend/expensive_data_check
* Initialize cluster version at wall-clock time
Previously, new clusters would begin at version 0. After this change,
clusters will initialize at a version matching wall-clock time. Instead
of using the Unix epoch (or Windows epoch), FDB clusters will use a new
epoch, defaulting to January 1, 2010, 01:00:00+00:00. In the future,
this base epoch will be modifiable through fdbcli, allowing
administrators to advance the cluster version.
Basing the version off of time allows different FDB clusters to share
data without running into version issues.
* Send version epoch to master
* Cleanup
* Update fdbserver/storageserver.actor.cpp
Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>
* Jump directly to expected version if possible
* Fix initial version issue on storage servers
* Add random recovery offset to start version in simulation
* Type fixes
* Disable reference time by default
Enable on a cluster using the fdbcli command `versionepoch add 0`.
* Use correct recoveryTransactionVersion when recovering
* Allow version epoch to be adjusted forwards (to decrease the version)
* Set version epoch in simulation
* Add quiet database check to ensure small version offset
* Fix initial version issue on storage servers
* Disable reference time by default
Enable on a cluster using the fdbcli command `versionepoch add 0`.
* Add fdbcli command to read/write version epoch
* Cause recovery when version epoch is set
* Handle optional version epoch key
* Add ability to clear the version epoch
This causes version advancement to revert to the old methodology whereas
versions attempt to advance by about a million versions per second,
instead of trying to match the clock.
* Update transaction access
* Modify version epoch to use microseconds instead of seconds
* Modify fdbcli version target API
Move commands from `versionepoch` to `targetversion` top level command.
* Add fdbcli tests for
* Temporarily disable targetversion cli tests
* Fix version epoch fetch issue
* Fix Arena issue
* Reduce max version jump in simulation to 1,000,000
* Rework fdbcli API
It now requires two commands to fully switch a cluster to using the
version epoch. First, enable the version epoch with `versionepoch
enable` or `versionepoch set <versionepoch>`. At this point, versions
will be given out at a faster or slower rate in an attempt to reach the
expected version. Then, run `versionepoch commit` to perform a one time
jump to the expected version. This is essentially irreversible.
* Temporarily disable old targetversion tests
* Cleanup
* Move version epoch buggify to sequencer
This will cause some issues with the QuietDatabase check for the version
offset - namely, it won't do anything, since the version epoch is not
being written to the txnStateStore in simulation. This will get fixed in
the future.
Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>
* initial structure for remote IKVS server
* moved struct to .h file, added new files to CMakeList
* happy path implementation, connection error when testing
* saved minor local change
* changed tracing to debug
* fixed onClosed and getError being called before init is finished
* fix spawn process bug, now use absolute path
* added server knob to set ikvs process port number
* added server knob for remote/local kv store
* implement simulator remote process spawning
* fixed bug for simulator timeout
* commit all changes
* removed print lines in trace
* added FlowProcess implementation by Markus
* initial debug of FlowProcess, stuck at parent sending OpenKVStoreRequest to child
* temporary fix for process factory throwing segfault on create
* specify public address in command
* change remote kv store knob to false for jenkins build
* made port 0 open random unused port
* change remote store knob to true for benchmark
* set listening port to randomly opened port
* added print lines for jenkins run open kv store timeout debug
* removed most tracing and print lines
* removed tutorial changes
* update handleIOErrors error handling to handle remote-ikvs cases
* Push all debugging changes
* A version where worker bug exists
* A version where restarting tests fail
* Use both the name and the port to determine the child process
* Remove unnecessary update on local address
* Disable remote-kvs for DiskFailureCycle test
* A version where restarting stuck
* A version where most restarting tests green
* Reset connection with child process explicitly
* Remove change on unnecessary files
* Unify flags from _ to -
* fix merging unexpected changes
* fix trac.error to .errorUnsuppressed
* Add license header
* Remove unnecessary header in FlowProcess.actor.cpp
* Fix Windows build
* Fix Windows build, add missing ;
* Fix a stupid bug caused by code dropped by code merging
* Disable remote kvs by default
* Pass the conn_file path to the flow process, though not needed, but the buildNetwork is difficult to tune
* serialization change on readrange
* Update traces
* Refactor the RemoteIKVS interface
* Format files
* Update sim2 interface to not clog connections between parent and child processes in simulation
* Update comments; remove debugging symbols; Add error handling for remote_kvs_cancelled
* Add comments, format files
* Change method name from isBuggifyDisabled to isStableConnection; Decrease(0.1x) latency for stable connections
* Commit the IConnection interface change, forgot in previous commit
* Fix the issue that onClosed request is cancelled by ActorCollection
* Enable the remote kv store knob
* Remove FlowProcess.actor.cpp and move functions to RemoteIKeyValueStore.actor.cpp; Add remote kv store delay to avoid race; Bind the child process to die with parent process
* Fix the bug where one process starts storage server more than once
* Add a please_reboot_remote_kv_store error to restart the storage server worker if remote kvs died abnormally
* Remove unreachable code path and add comments
* Clang format the code
* Fix a simple wait error
* Clang format after merging the main branch
* Testing mixed mode in simulation if remote_kvs knob is enabled, setting the default to false
* Disable remote kvs for PhysicalShardMove which is for RocksDB
* Cleanup #include orders, remove debugging traces
* Revert the reorder in fdbserver.actor.cpp, which fails the gcc build
Co-authored-by: “Lincoln <“lincoln.xiao@snowflake.com”>
Trevor Clinkenbeard