b6450f9eaa
More ss cf perf fixes main ( #9109 )
...
* changing future version logic for change feed fetch
* Optimizing change feed data structures and accesses
* coalescing change feed request ranges for merge cursor if they're to the same team
* fixing over-read of memory mutations for change feeds
* feed filter mutations common prefix cpu optimiation
* fix formatting
2023-01-12 16:15:05 -06:00
f999623bb1
Add a tenant lookup interface and use it when starting transactions
2023-01-06 15:51:12 -08:00
9ade12d313
Switch the knob that enables shared map to enable client using cluster id key
2022-11-19 12:46:05 -08:00
81fc8e9d2f
Using a knob to guard updateClusterSharedStateMap
2022-11-19 12:46:05 -08:00
ca572f58fb
Combine *_COST_BYTE_FACTOR knobs into a single TAG_THROTTLING_PAGE_SIZE knob
2022-11-08 07:44:01 -08:00
ce3038bfe5
Use a network option for retaining temporary client lib copies instead of a client knob ( #8630 )
2022-11-07 14:46:17 +01:00
d68cd3493e
making read caching configurable for blob store ( #8607 )
2022-11-03 16:42:46 -05:00
ffbd81cfcf
Merge branch 'main' of github.com:apple/foundationdb into monitorusage
2022-11-01 16:51:27 -07:00
80ee79e39b
Merge remote-tracking branch 'origin/main' into debug
2022-11-01 12:37:27 -07:00
5fd8d05810
Make PROXY_MAX_TAG_THROTTLE_DURATION a server knob
2022-11-01 11:00:45 -07:00
b1a2239612
Increase the value of `UNKNOWN_TENANT_RETRY_DELAY`
2022-10-31 16:46:38 -07:00
0eb1598afa
Merge remote-tracking branch 'origin/main' into expose-txn-cost
2022-10-30 09:36:37 -07:00
4d3553481f
Blob connection provider test ( #8478 )
...
* Refactoring test blob metadata creation
* Implementing BlobConnectionProviderTest
* createRandomTestBlobMetadata supports blobstore and works outside simulation
2022-10-27 10:44:06 -05:00
04ae47b9b9
Addressed review comments
2022-10-21 22:17:33 -07:00
dfc11001af
Reject transactions that have been tag throttled for too long
2022-10-18 15:16:24 -07:00
445e871cf2
Use fungibility ratio in getWriteOperationCost
2022-10-16 21:58:08 -07:00
1231ffbbe4
knob name change/rebase
2022-10-12 20:26:44 -04:00
ef3efbc641
Merge remote-tracking branch 'origin/main' into tracelib
2022-10-07 21:32:26 -04:00
2402b813e9
do not unlink fdb_c in tmp dir if knob set
2022-10-07 16:32:28 -04:00
7fc5c196c4
Make read and write quotas fungible
2022-09-25 21:00:11 -07:00
d831c87d14
Add encryption metrics ( #8070 )
...
Adding the following metrics:
* BlobCipherKeyCache hit/miss
* EKP: KMS requests latencies
* For each component that using encryption, they now need to pass a UsageType enum to the encryption helper methods (GetEncryptCipherKeys/GetLatestEncryptCipherKey/encrypt/decrypt) and those methods will help to log get cipher key latency samples and encryption/decryption cpu times accordingly.
2022-09-09 18:43:09 -07:00
72ccc681be
move tenant entry cache to client
2022-09-01 10:10:47 -07:00
8314e80371
Fixed a few bugs which caused ratekeeper to unnecessarily throttle a cluster ( #8006 )
...
* do not count recently created change feeds for throttling
* fix: blocked assignments were not decremented when force purging
* fix: created needs to be updated when the changefeed is reset
* added asserts to detect if ratekeeper is throttled on blob workers
2022-08-26 15:38:31 -07:00
46b02cab49
Blob granule summary implementation (in native client) ( #7981 )
...
* implemented blob granule summary call in native client
* clean up prints
2022-08-26 14:04:59 -05:00
8fb6d59e94
Use error_code_grv_proxy_memory_limit_exceeded
...
instead of error_code_proxy_memory_limit_exceeded
2022-08-12 11:36:17 -07:00
ed1b7ef173
Add a client knob GRV_ERROR_RETRY_DELAY
2022-08-12 11:13:32 -07:00
5929ac1d65
Merge pull request #7847 from xis19/knobCheck
...
Cleanup the knobs that are not being used
2022-08-11 21:52:52 -07:00
85836cbec9
Cleanup the knobs that are not being used
2022-08-10 16:51:01 -07:00
b33a9da7ce
Don't buggify max tenants per cluster globally; instead buggify it in specific tests
2022-08-06 06:10:45 -07:00
ddbf32d35e
adding knob and fixing blobrange cli check bug with too many granules ( #7793 )
2022-08-05 11:19:17 -05:00
ff23d5994e
Merge pull request #7729 from sfc-gh-ajbeamon/feature-metacluster
...
Metacluster
2022-08-04 15:29:44 -07:00
a2e66c9695
Don't buggify the assignment availability timeout too low or it will become impossible to make an assignment
2022-08-03 19:39:20 -07:00
fa418fd784
Change SHARD_ENCODE_LOCATION_METADATA to a server knob. ( #7770 )
...
Co-authored-by: He Liu <heliu@apple.com>
2022-08-03 13:51:40 -07:00
a69164d160
Merge branch 'main' into feature-metacluster
2022-07-29 11:53:06 -07:00
cfc13e7018
Adding more blobrange cli commands and a couple other tweaks ( #7727 )
2022-07-29 08:20:45 -07:00
7f53b340f5
Merge branch 'main' into feature-metacluster
...
# Conflicts:
# bindings/python/tests/fdbcli_tests.py
# fdbcli/TenantCommands.actor.cpp
# fdbclient/ClientKnobs.cpp
# fdbclient/include/fdbclient/ClientKnobs.h
# fdbclient/include/fdbclient/Tenant.h
# fdbserver/BlobManager.actor.cpp
# fdbserver/BlobWorker.actor.cpp
# fdbserver/TenantCache.actor.cpp
# fdbserver/storageserver.actor.cpp
# fdbserver/workloads/TenantManagementWorkload.actor.cpp
# flow/include/flow/error_definitions.h
2022-07-28 11:43:21 -07:00
7c6b3fb0b8
Merge branch 'main' into feature-metacluster
2022-07-27 08:55:10 -07:00
d39c0b773a
Add a limit to the number of tenants that can be created in a cluster
2022-07-27 08:21:03 -07:00
dc9599f2e9
Get network interfaces ready for https proxy. ( #7556 )
...
* Move HTTP from fdbclient/ to fdbrpc/.
* Move md5 and libb64 to contrib/.
* Get network interfaces ready for https proxy.
* Rebase
2022-07-25 17:08:32 -07:00
077003c4f5
Cleanup tenant tombstones periodically
2022-07-24 18:24:19 -07:00
41c1bef3bf
Merge branch 'feature-tenant-groups' into feature-metacluster
2022-07-22 14:26:02 -07:00
703aa1d279
Mess with timeout values
2022-07-22 10:37:29 -07:00
40d403ed5f
Reduce global configuration system key reads from proxy
...
Clients now poll the proxy for the latest global config for a specific
version. The proxy now periodically requests the latest global
configuration data and stores it in memory, enabling it to respond
immediately to clients with the appropriate version.
2022-07-22 10:37:29 -07:00
279296c29f
Merge branch 'tenant-metadata-change' into feature-metacluster
...
# Conflicts:
# fdbclient/SystemData.cpp
# fdbclient/Tenant.cpp
# fdbclient/include/fdbclient/SystemData.h
# fdbclient/include/fdbclient/Tenant.h
# fdbclient/include/fdbclient/TenantManagement.actor.h
# fdbserver/TenantCache.actor.cpp
# fdbserver/storageserver.actor.cpp
# fdbserver/workloads/TenantManagementWorkload.actor.cpp
2022-07-20 09:18:27 -07:00
537ceff8ac
Remove the ability to configure a tenant subspace. Rename the prefixes used for tenant metadata.
2022-07-19 14:32:05 -07:00
70bc67ce34
Update tenant assignment to check data cluster availability. Cleanup errors when performing an invalid operation on a non-metacluster and remove redundant function.
2022-07-18 16:15:33 -07:00
cb499fa5db
Merge branch 'main' into feature-metacluster
2022-07-13 15:28:34 -07:00
bc5bfaffda
Shard based move ( #6981 )
...
* Shard based move.
* Clean up.
* Clear results on retry in getInitialDataDistribution.
* Remove assertion on SHARD_ENCODE_LOCATION_METADATA for compatibility.
* Resolved comments.
Co-authored-by: He Liu <heliu@apple.com>
2022-07-07 20:49:16 -07:00
e1a93988ef
Merge branch 'main' into feature-metacluster
2022-06-28 14:58:07 -07:00
4b69723a2f
Cleanup metadata associated with a cluster when force removing it. Move some metacluster metadata into the normal key-space. Cleanup some todos.
2022-06-24 14:55:57 -07:00
Josh Slocum