8b1fe728be
Add configuration option `auto_tenant_assignment` to data clusters ( #10058 )
...
This PR adds auto_tenant_assignment option to register/configure data clusters.
Setting auto_tenant_assignment to disabled means the data cluster is a dedicated one and won't be
used for auto tenant assignment. This option is enabled by default (allowing auto tenant assignment).
Test plan:
simulation tests and metacluster_fdbcli_tests.py
---------
Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>
2023-05-01 21:58:49 -07:00
b258159d3a
Change enum capitalization. Improve error reporting if we cannot read metacluster registration when fetching metacluster metrics. Improve timeliness of metacluster metrics updates.
2023-05-01 11:21:42 -07:00
882ec9998b
Add testing for the metacluster version feature to metacluster management workload. This also adds additional operations to the workload (create/decommission metacluster, lock tenant) to ensure they are covered by this testing.
2023-04-27 10:18:34 -07:00
807646675c
Refactor the metacluster project into smaller files, and reorganize the namespaces. Move some metacluster and tenant testing helpers into the metacluster project.
2023-03-30 16:20:09 -07:00
e61748c7d5
Move metacluster into its own directory and static library
2023-03-30 16:07:49 -07:00
64b6a5d257
Allow boolean parameters to be nested inside of namespaces or classes
2023-03-30 15:09:59 -07:00
d39cda610a
Merge branch 'main' into metacluster-improvements
...
# Conflicts:
# fdbcli/TenantCommands.actor.cpp
2023-03-13 15:58:39 -07:00
cbc330697c
Disallow repopulating a management cluster from a data cluster with matching tenant ID prefix unless forced. Remember the largest used tenant ID on the data cluster and use it to update the management cluster tenant ID when force repopulating the same ID.
2023-03-10 15:36:37 -08:00
de5f2c0fee
Disallow cluster names that start with the `\xff` byte
2023-03-07 11:46:34 -08:00
2898a95c81
Fix two metacluster issues:
...
1. When retrying the transaction to register a restoring cluster, don't choose a new ID if the current ID matches the one recorded for the restoring cluster
2. A metacluster test was incorrectly handling the case where a transaction was retried with unknown result and had committed successfully
2023-02-28 15:40:04 -08:00
8c3ee768a2
Add an option to allow exceeding the tenant group capacity limit when changing tenant configuration
2023-02-24 21:01:36 -08:00
1c71056e26
Merge pull request #9479 from sfc-gh-nwijetunga/nim/enforce-metacluster-tenant-mode
...
Enforce Disabled Tenant Mode in Metacluster
2023-02-24 19:27:57 -08:00
c8b7cff10c
fix api test
2023-02-24 15:26:59 -08:00
eca98afcb0
metacluster check tenant mode
2023-02-24 13:59:54 -08:00
2d936acbdd
Fix memory error from using invalidated iterator
2023-02-24 10:25:38 -08:00
344f6977c9
Restores can now throw a cluster_already_exists error in the metacluster management workload if we timeout a restore and have to retry
2023-02-23 20:32:18 -08:00
33f8e90f9f
Split tenant group metadata ( #9446 )
...
* initial commit to split tenant group metadata
* attempt to fix merge errors
* fix compile errors and adjust existing tests
* fix infinite loop and extra ACTOR tag
* direct assignment instead of store
* direct assign instead of store (missed a few)
2023-02-23 18:11:49 -05:00
9e9a31c0f1
Use error variable consistently
2023-02-23 11:27:53 -08:00
a76af5c696
Improve restore support in the metacluster management workload
2023-02-22 17:42:36 -08:00
a5a8c57a38
Fix some merge issues and missing updates to use new boolean parameters
2023-02-22 12:52:58 -08:00
2d74d10a91
use MetaclusterAPI namespace over TenantAPI for TenantState
2023-02-21 11:33:36 -08:00
f27c3e24a8
fix code formatting
2023-02-21 11:09:14 -08:00
428eb07766
Merge branch 'main' of github.com:apple/foundationdb into split-tenant-metadata
2023-02-21 10:11:35 -08:00
3163201201
Restore ID fixes: we weren't generating a restore ID; we weren't setting the restore ID on the management cluster in some restore modes; it is possible in some test scenarios to encounter a restore conflict, in which case we need to retry.
2023-02-17 21:20:06 -08:00
edb7a51b7e
Revert "let client supply restore id"
...
This reverts commit 5fe32b8503
2023-02-17 14:37:22 -08:00
5fe32b8503
let client supply restore id
2023-02-17 14:01:58 -08:00
0d7b6d626b
update restoreCluster test to account for conflicting_restore
2023-02-17 10:36:28 -08:00
53fc43a3a6
Merge branch 'main' of github.com:apple/foundationdb into split-tenant-metadata
2023-02-16 14:46:03 -08:00
c3d6ae0213
Throw invalid_tenant_configuration when changing assigned cluster ( #9350 )
...
Since we currently do not support tenant movement, we should as well explicitly disallow changing the assigned
cluster of a tenant during configuration by throwing `invalid_tenant_configuration` for now.
Test plan:
- add coverage for changing assigned cluster during tenant configuration
- fdbcli
- simulation tests
2023-02-16 14:20:59 -08:00
036078234e
fix some leftover issues from branch merge
2023-02-16 12:54:45 -08:00
ab15478ef9
Merge branch 'main' of github.com:apple/foundationdb into split-tenant-metadata
2023-02-16 10:04:07 -08:00
8f4aec7d7f
address review comments and fix some test errors
2023-02-15 15:58:19 -08:00
545e08e1af
Try again to fix the logic in choosing the assigned cluster in the metacluster management test
2023-02-14 21:39:42 -08:00
7284e691fb
Fix a few minor restore bugs and add a dry-run mode. Some improvements to the fdbcli output.
2023-02-14 12:28:55 -08:00
98407809d9
Merge branch 'main' into metacluster-mgmt-restore
...
# Conflicts:
# fdbcli/MetaclusterCommands.actor.cpp
# fdbclient/Metacluster.cpp
# fdbclient/include/fdbclient/MetaclusterManagement.actor.h
# fdbserver/workloads/MetaclusterManagementWorkload.actor.cpp
# tests/CMakeLists.txt
2023-02-13 12:30:33 -08:00
0127dd4b5a
Merge pull request #9356 from sfc-gh-ajbeamon/metacluster-concurrency-testing
...
Add metacluster concurrency test and fix various bugs that it found
2023-02-13 11:57:47 -08:00
93b1e04aa9
Merge pull request #9355 from sfc-gh-ajbeamon/fix-assigned-cluster-test-retry-logic
...
Fix logic in MetaclusterManagementWorkload when retrying a tenant creation with an invalid assigned cluster
2023-02-13 10:43:09 -08:00
e6021f8326
Add Jon's metacluster concurrency test and fix various bugs that it found
2023-02-11 15:15:32 -08:00
a9c7632c83
Merge branch 'main' of https://github.com/apple/foundationdb into fix/main/tenantList
2023-02-11 13:54:27 -08:00
b4f45a0a87
Fix logic in MetaclusterManagementWorkload when retrying a tenant creation with an invalid assigned cluster
2023-02-11 12:09:17 -08:00
ee1b48323d
Merge pull request #9346 from sfc-gh-nwijetunga/nim/global-tenant-ids
...
Support for Two Byte Prefix for Tenant IDs
2023-02-11 11:31:24 -08:00
a6b47c1da4
Fix merge issue
2023-02-10 11:12:36 -08:00
1c0e784a7a
attempt to fix broken test workloads
2023-02-10 11:01:59 -08:00
13eee09ce8
Merge branch 'main' into metacluster-mgmt-restore
2023-02-10 10:58:01 -08:00
4b13c9c211
Make a few minor fixes, refactor some code for clarity, and improve throughput of repopulating a management cluster
2023-02-10 10:41:55 -08:00
8a3f3ea674
clean up code
2023-02-10 01:01:16 -08:00
f9a68056ac
Add support for modifying a data cluster that is being restored so that we can manage conflicts
2023-02-09 15:33:40 -08:00
b7ef50d1f8
inital commit
2023-02-09 14:32:29 -08:00
76d8c1555f
accept review comments
2023-02-09 14:02:57 -08:00
47ac2a8fb2
fix the way verifyListFilter detect tenant state change
2023-02-09 11:34:14 -08:00
Yanqin Jin