b4e2aef58b
add tenant_id_prefix to metacluster status ( #10455 )
2023-06-09 15:03:49 -04:00
8b1fe728be
Add configuration option `auto_tenant_assignment` to data clusters ( #10058 )
...
This PR adds auto_tenant_assignment option to register/configure data clusters.
Setting auto_tenant_assignment to disabled means the data cluster is a dedicated one and won't be
used for auto tenant assignment. This option is enabled by default (allowing auto tenant assignment).
Test plan:
simulation tests and metacluster_fdbcli_tests.py
---------
Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>
2023-05-01 21:58:49 -07:00
807646675c
Refactor the metacluster project into smaller files, and reorganize the namespaces. Move some metacluster and tenant testing helpers into the metacluster project.
2023-03-30 16:20:09 -07:00
e61748c7d5
Move metacluster into its own directory and static library
2023-03-30 16:07:49 -07:00
6c337e5c7a
fix metacluster get segfault ( #9693 )
2023-03-23 14:21:25 -04:00
6b2185c707
Fix off-by-one in token hint generator for metacluster restore
2023-03-10 18:50:34 -08:00
cbc330697c
Disallow repopulating a management cluster from a data cluster with matching tenant ID prefix unless forced. Remember the largest used tenant ID on the data cluster and use it to update the management cluster tenant ID when force repopulating the same ID.
2023-03-10 15:36:37 -08:00
eca98afcb0
metacluster check tenant mode
2023-02-24 13:59:54 -08:00
a5a8c57a38
Fix some merge issues and missing updates to use new boolean parameters
2023-02-22 12:52:58 -08:00
ec79ecce73
Add a boolean parameter for ForceRemove; rename ForceJoinNewMetacluster to ForceJoin
2023-02-22 12:26:24 -08:00
7284e691fb
Fix a few minor restore bugs and add a dry-run mode. Some improvements to the fdbcli output.
2023-02-14 12:28:55 -08:00
958ff862e0
Fix some merge issues
2023-02-13 12:59:48 -08:00
98407809d9
Merge branch 'main' into metacluster-mgmt-restore
...
# Conflicts:
# fdbcli/MetaclusterCommands.actor.cpp
# fdbclient/Metacluster.cpp
# fdbclient/include/fdbclient/MetaclusterManagement.actor.h
# fdbserver/workloads/MetaclusterManagementWorkload.actor.cpp
# tests/CMakeLists.txt
2023-02-13 12:30:33 -08:00
e6021f8326
Add Jon's metacluster concurrency test and fix various bugs that it found
2023-02-11 15:15:32 -08:00
4b13c9c211
Make a few minor fixes, refactor some code for clarity, and improve throughput of repopulating a management cluster
2023-02-10 10:41:55 -08:00
f9a68056ac
Add support for modifying a data cluster that is being restored so that we can manage conflicts
2023-02-09 15:33:40 -08:00
b7ef50d1f8
inital commit
2023-02-09 14:32:29 -08:00
2d59c5681d
Bug fixes and test improvements for management cluster restoration
2023-02-09 08:42:23 -08:00
0410f7d9a1
Add intial support for restoring a management cluster in a metacluster
2023-02-07 15:08:46 -08:00
56b716040f
Various fixes and cleanup
2023-01-30 13:34:45 -08:00
be5b0f451d
Expand the restore test; fix some bugs mainly around renaming and changing tenant configuration during a restore
2023-01-30 13:34:45 -08:00
af55feb493
Fix some test issues
2023-01-30 13:34:18 -08:00
91f5347551
Add some simulated testing to the metacluster restore process; fix various issues found by the testing.
2023-01-30 13:34:18 -08:00
7d705ea6cd
Remove duplicate code for connection string parsing now that a few other restore options are removed
2023-01-30 13:25:30 -08:00
8fcc393994
Misc fixes
...
- Added a blurb for the restore command
- Remove unnecessary if checks
- Fix the next loop start values.
2023-01-30 13:25:30 -08:00
226976f0a7
Change option to repopulate_from_data_cluster
2023-01-30 13:25:30 -08:00
22d5546206
Revert to getting the tenants from Management cluster in a loop
2023-01-30 13:25:30 -08:00
718074a016
Metacluster support for restoring a data cluster
2023-01-30 13:24:07 -08:00
d461dbdd7b
Extend `metacluster status` on non-management clusters ( #9179 )
...
The metacluster status command in fdbcli currently reports some useful metacluster information when run on a
management cluster. We should update this command to report a status even on data clusters of a metacluster and
standalone clusters that do not belong to any metacluster.
- On data clusters, this would report that the cluster is a data cluster as well its name and the name of the metacluster it is a part of.
- On standalone clusters, status should report that the cluster is not part of a metacluster.
Test plan:
- CI
- Manual test
- Added new test `metacluster_fdbcli_tests.py` that can be run with ctest `ctest -R metacluster_fdbcli_tests`
2023-01-26 13:04:22 -08:00
0fa462fca9
initial commit to trace metacluster metrics
2022-09-26 15:56:45 -07:00
cd82f65ef5
Change the data cluster entry to use a state field rather than a locked boolean
2022-08-09 11:53:57 -07:00
e5bf838c83
Fix metacluster register command hints, fix bug in cluster registration on retries, extend the length of the tenant management concurrency test
2022-07-28 21:02:12 -07:00
65cfc839a6
Some various cleanup
2022-07-28 13:35:24 -07:00
2990294f6a
Make the metacluster experimental by updating the name of the command used to configure one.
2022-07-24 19:11:25 -07:00
eccf244ba6
Add an index that allows us to allocate tenants from the non-full cluster with the most tenants already assigned to it.
2022-06-27 12:28:55 -07:00
4b69723a2f
Cleanup metadata associated with a cluster when force removing it. Move some metacluster metadata into the normal key-space. Cleanup some todos.
2022-06-24 14:55:57 -07:00
9f3819752f
Change the command to create a metacluster from using 'configure tenant_mode=management' to 'metacluster create <NAME>'. Distribute this name to all processes in a metacluster. Eliminate the tenant mode entirely from metacluster clusters, instead relying on a metacluster registration key.
2022-06-22 12:15:43 -07:00
96fad987eb
Add new command to get total metacluster capacity. Fix bug where removing a data cluster could make it impossible to create new tenants on it. Add knobs for max data clusters and max tenants.
2022-06-17 10:42:03 -07:00
9773261a03
Expand support for tenant groups. Track where each tenant group is assigned and used groups to help assign tenants. Fix tenant state tracking bugs. Add help hints for metacluster remove FORCE.
2022-05-24 11:58:40 -07:00
367e59dc33
Add support for getting tenant and cluster metadata in fdbcli in a JSON format
2022-05-18 14:53:25 -07:00
98c3813431
Refactor the register logic so that only two write transactions take place. The first is to the data cluster, the second to the management cluster.
2022-05-18 11:21:39 -07:00
e8e26c9f7c
Remove the internal special key-space mechanism (at least for now), instead relying on the local logic to implement the metacluster functions. Add ability to force remove a cluster. Improve idempotency of register operation.
2022-05-18 08:18:25 -07:00
80a3c8dd50
All metacluster operations to the management cluster go through the management cluster special key space
2022-05-09 14:50:07 -07:00
7c3b99f5bd
Add support for connection to clusters via a connection string. Start work on multi-cluster operations with data cluster registration.
2022-05-06 09:53:53 -07:00
7e860dc92c
Some initial metacluster implementation
2022-05-03 12:53:35 -07:00
Jon Fu