6752a99b0c
fix: when requests are dropped from the queue the txnRequestOut becomes inconsistent with txnRequestIn
2023-08-02 09:07:07 -07:00
6e86f94cb9
Decouple token bucket knobs for different types of throttlers
2023-06-28 01:27:19 -07:00
09ad864eb5
Merge remote-tracking branch 'origin/main' into expose-tag-throttled-duration
2023-02-08 11:25:14 -08:00
b9e00d03df
fix on GRV Proxy converts broken_promise errors to master_failed or tlog_failed
2023-01-27 15:19:44 -08:00
f1bf983c6f
GRV Proxy converts broken_promise errors to master_failed or tlog_failed
2023-01-27 10:23:13 -08:00
0f14647bbf
Merge remote-tracking branch 'origin/main' into expose-tag-throttled-duration
2023-01-05 08:10:49 -08:00
86d6106dc1
format source code after switch to clang 15
2022-12-08 17:26:45 +00:00
4b6098931c
Merge remote-tracking branch 'origin/main' into expose-tag-throttled-duration
2022-12-04 08:40:04 -08:00
b8fd56ec86
Remove duplicate PROXY_MAX_TAG_THROTTLE_DURATION server knob
2022-11-28 09:55:28 -08:00
3c97f43138
Change Histogram::Unit::microseconds to milliseconds
2022-11-21 08:03:56 -08:00
23706c957b
Use DDSketch for Sample Data.
2022-11-12 13:45:46 -08:00
2361acbe22
Change SevWarn to SevInfo for injected errors
2022-11-10 12:10:00 -08:00
64733d2e39
s/GrvProxyTransactionTagThrottler/GrvProxyTagThrottler
2022-11-02 07:35:05 -07:00
36731efe1d
Apply clang-format
2022-11-01 12:53:15 -07:00
047578b3d9
Merge remote-tracking branch 'origin/main' into debug
2022-11-01 12:05:45 -07:00
5fd8d05810
Make PROXY_MAX_TAG_THROTTLE_DURATION a server knob
2022-11-01 11:00:45 -07:00
736cf4a22f
Add GrvProxyTransactionTagThrottler::maxThrottleDuration field
2022-11-01 10:44:32 -07:00
6ae0aac153
Merge remote-tracking branch 'origin/main' into add-tag-throttling-latency-bands
2022-10-27 14:07:51 -07:00
08251329a9
Update tag throttling latency bands in GrvProxyData::updateLatencyBandConfig
2022-10-26 10:48:51 -07:00
3c5d3f7a94
Fix SpanContext for GP:getLiveCommittedVersion ( #8565 )
...
* Fix SpanContext for GP:getLiveCommittedVersion
2022-10-26 16:29:28 +02:00
8766809cee
Add count parameter to LatencyBands::addMeasurement
2022-10-25 17:10:59 -07:00
74212eeacf
Encapsulate CounterCollection
2022-10-25 10:17:15 -07:00
b90722baa6
Add comment for GrvProxyStats latency bands and samples
2022-10-14 15:58:15 -07:00
6f01d97006
Subtract proxyTagThrottleDuration from measured GRV latencies
2022-10-12 14:41:32 -07:00
f1cb4e40f5
Avoid adding untagged requests to GrvProxyTransactionTagThrottler
2022-10-10 22:10:08 -07:00
24a0dd9f17
Change GrvProxy tag throttling algorithm.
...
The new algorithm assumes there is only one tag per request,
so queues are partitioned by tag. This is a more efficient approach than
the old algorithm.
2022-10-10 22:10:08 -07:00
bbf69b2d0a
Rename TagQueue::runEpoch to TagQueue::releaseTransactions
2022-10-10 22:10:08 -07:00
31e8fb0490
Rename epoch to release window in GrvTransactionRateInfo
...
The term "epoch" was used in too many places
2022-10-10 22:10:08 -07:00
34857bd2bf
Remove outSystemPriority parameter from TagQueue::runEpoch
2022-10-10 22:10:08 -07:00
e313edbb85
Add proxyTagThrottledDuration field to GetReadVersion[Request|Reply]
2022-10-10 22:10:08 -07:00
2684464cc6
Use TagQueue in GRV proxy when ENFORCE_TAG_THROTTLING_ON_PROXIES is true
2022-10-10 22:10:08 -07:00
4c973c11ad
Modularize and comment GrvTransactionRateInfo
2022-10-10 22:10:05 -07:00
4fd64630e8
Convert literal string ref instances to use _sr suffix
2022-09-19 11:35:58 -07:00
82adc1e856
Make g_simulator a pointer
2022-09-15 09:00:33 -07:00
eb16947860
Merge pull request #7844 from jzhou77/grv-error
...
Buggify GRV proxy to return errors and fix bugs found
2022-08-15 13:59:21 -06:00
a9d3c9f9b3
Added throttling when a blob worker falls behind ( #7751 )
...
* throttle the cluster when blob workers fall behind
* do not throttle on blob workers if they are not enabled
* remove an unnecessary actor
* fixed a compile error
* fetch blob worker metrics at the same interval as the rate is updated, avoid fetching the complete blob worker list too frequently
* fixed another compilation bug
* added a 5 second delay before bw throttling to prevent false positives caused by the 100e6 version jump during recovery. Lower the throttling thresholds to react much quicker to bw lag.
* fixed a number of problems
* changed the minBlobVersionRequest to look at storage server versions since this will be a lot more efficient
* fix: do not let desired go backwards
* fix: track the version of notAtLatest changefeeds for throttling
* ratekeeper now throttled blob workers by estimating the transaction per second throughput of the blob workers
* added metrics for blob worker change feeds
* added a knob to disable bw throttling
* fixed the transaction options in blob manager
2022-08-12 13:15:56 -07:00
a32822a51d
Add fault injection to GRV proxy to return error response
...
This turns out to fail a lot of tests that we need to fix.
2022-08-12 11:13:32 -07:00
0326d53965
Split proxy_memory_limit_exceeded to commit/grv specific exceptions
...
Currently GRV is reporting proxy_memory_limit_exceeded error which has
error message claiming Commit proxy failing. This split should remove
such confusion.
2022-08-12 00:45:57 -07:00
1bd47a07b2
Add ENFORCE_TAG_THROTTLING_ON_PROXIES knob
2022-08-05 00:40:10 -07:00
7fb427f4ec
Update fdbserver/GrvProxyServer.actor.cpp
...
Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>
2022-07-22 10:37:29 -07:00
a26344b877
Update fdbserver/GrvProxyServer.actor.cpp
...
Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>
2022-07-22 10:37:29 -07:00
8395c6cf3f
Fix Tuple error
2022-07-22 10:37:29 -07:00
5fa8e3f7d0
Simplify refresh condition
2022-07-22 10:37:29 -07:00
703aa1d279
Mess with timeout values
2022-07-22 10:37:29 -07:00
40d403ed5f
Reduce global configuration system key reads from proxy
...
Clients now poll the proxy for the latest global config for a specific
version. The proxy now periodically requests the latest global
configuration data and stores it in memory, enabling it to respond
immediately to clients with the appropriate version.
2022-07-22 10:37:29 -07:00
56dfdbda83
Add migration timeout
2022-07-22 10:37:29 -07:00
e024db85a0
Update fdbserver/GrvProxyServer.actor.cpp
...
Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>
2022-07-22 10:37:29 -07:00
7a48a53778
Perform migration automatically on proxy boot
2022-07-22 10:37:29 -07:00
2e99d5f6cc
Batch global config refresh requests
2022-07-22 10:37:29 -07:00
c33e44b0f4
Proxy GlobalConfig reads through GRV proxies
...
Clients should avoid reading system keys unless authorized. Under global
config, each client reads from the system keyspace to check for new
global config keys. This commit moves these reads to a server role (the
GRV proxies) and sends the results back to GlobalConfig for an in-memory
update.
2022-07-22 10:37:29 -07:00
Evan Tschannen