0536e56725
The tenant cache uses a simpler eviction policy. Some various improvement around the use of futures. Add documentation for C API and fix inaccurate statement in special keys documentation.
2022-03-15 09:23:30 -07:00
06f088e088
Pass along versions with GetKeyServerLocationsRequests, if we have them.
2022-03-15 09:23:30 -07:00
68069c9784
Undo some unused changes. Fix tenant cache eviction.
2022-03-15 09:23:30 -07:00
8bc2b283e1
Create a special keys API to create, delete, and read tenants. Remove the C API to create/delete tenants.
2022-03-15 09:23:30 -07:00
1e1098ca9a
Don't check whether tenants are enabled when watch is called. Support unknown_tenant error.
2022-03-15 09:23:30 -07:00
ecccfd0868
Add cache invalidation to tenant cache. Send tenant ID along with tenant name in requests to validate that the tenant hasn't changed. Fix a few bugs.
2022-03-15 09:23:30 -07:00
fb73e1857a
Some get key requests should not happen within a tenant. Don't set read_system_keys in watch logic unless necessary. Set access_system_keys in a couple movekeys functions.
2022-03-15 09:23:30 -07:00
2a21126028
Don't apply read prefixes on the client. Cache tenant data locally.
2022-03-15 09:23:30 -07:00
a168faf219
Fix bugs in serialization and tenant prefix management on the client.
2022-03-15 09:21:27 -07:00
c635dcd3ad
Add tenant support in the FDB native client
2022-03-15 09:21:27 -07:00
800478ad5e
Add a new client error code stale_version_vector
2022-03-14 20:50:12 -07:00
8d88b7ca41
making range assigner boundary changes fully synchronous to ensure no races with things reading assignments
2022-03-14 14:39:00 -05:00
98eeaac1b7
More debugging and fixing a stuck merge cursor
2022-03-14 11:57:56 -05:00
8dcac2f76d
Fix typos
2022-03-13 10:02:11 -03:00
9b8c101158
Fix a rare race with version vector cache
...
The events that led to the assertion failure are:
Many txn started, some get ReadVersion1, some get ReadVersion2.
Recovery happened.
GRV proxy change detected and the client version cache is cleared.
Old GRV responded with ReadVersion1, and client version cache is updated to ReadVersion1.
A txn starts reading with ReadVersion2, triggering the assertion.
The fix is to throw an error to trigger transaction restart so that a new GRV
will update the version cache.
2022-03-12 12:03:01 -08:00
82eef0c7f7
Better CF tracing for client and server
2022-03-11 16:50:38 -06:00
2a506f2dff
Updating SS desired every mutation to avoid CF merge deadlock
2022-03-10 20:11:49 -06:00
211e5b579a
Adding back debug stream uid to help track down bugs
2022-03-10 13:55:42 -06:00
903f7adbc4
more cleanup
2022-03-10 13:53:06 -06:00
e2c7c30faf
GetMappedRange support serializable & check RYW & continuation ( #6181 )
2022-03-10 10:05:44 -08:00
1f964ac085
BM focused cleanup
2022-03-09 15:04:17 -06:00
b21d0943b9
client-focused cleanup
2022-03-09 10:01:25 -06:00
e71b3533f9
Merge branch 'main' into blob_integration
2022-03-09 08:59:56 -06:00
402fa4dd9e
Merge pull request #6428 from sfc-gh-ajbeamon/fdb-tenant-add-interfaces
...
Add server-side and protocol support for tenants
2022-03-08 15:33:18 -08:00
63b8b838a2
Merge pull request #6558 from sfc-gh-ajbeamon/special-keys-limits
...
Add support for a limits hint in special keys range reads
2022-03-08 15:32:59 -08:00
6f7adb04a5
Send a version along with GetKeyServerLocationsRequests to allow the proxy to reject missing tenants more quickly. Change the commit proxy tenant map to be unversioned.
2022-03-08 13:33:56 -08:00
58f5cea39b
fix typo
2022-03-07 11:20:24 -06:00
04f678d515
Fixing sample logic
2022-03-07 11:20:24 -06:00
665e182bcc
TSS Metrics improvements
2022-03-07 11:20:24 -06:00
c99c7cf55f
Apply tenant prefixes on the storage server. The commit proxy can serve tenant location requests. Fix some bugs.
2022-03-06 21:54:21 -08:00
5fa9d3e1b7
Add a tenant parameter to read and commit requests. Store a map of all tenants on commit proxy and storage servers. Add an option to require tenant mode.
2022-03-06 21:54:21 -08:00
36435af4f5
Add support for a limits hint in special keys. Fix a typo and resolve some comment formatting weirdness.
2022-03-06 21:52:39 -08:00
1a5bf25b5c
Update code base to use fmt 8.1.1
2022-03-04 15:52:06 -08:00
1555c0e2aa
Fix conflict between two recent merges by using the new errorUnsuppressed function.
2022-03-04 12:41:21 -08:00
cdebda35ab
Merge pull request #5725 from sfc-gh-jfu/jfu-grv-cache
...
Add transaction option for clients to use cached read versions
2022-03-04 09:17:27 -08:00
2b8882ca0a
Adding exponential backoff for consecutive change feed stream failures
2022-03-02 18:46:40 -06:00
e73c0a31e6
add wiggle_server_ids and wiggle_server_addresses in status json
2022-03-02 10:03:23 -08:00
5e39da7741
fixup! Reformat the source
2022-02-28 18:03:52 -08:00
02e396274f
Fix the compile error on NativeAPI.actor.cpp
2022-02-28 18:03:52 -08:00
aadd706a0a
Port back addressed comments for fixing get key location issue
2022-02-28 16:08:57 -08:00
0bdae35a2c
Address comments and move updateFailedEndpointRefreshTime
2022-02-24 16:35:30 -08:00
9c99c59b0c
Fix get key location overloading proxies
2022-02-24 16:35:30 -08:00
e5b4fb3d80
Adding explicit popVersion to change feed reply, like tlogs have
2022-02-24 15:20:18 -06:00
250a88e682
Enforce that trace event suppression calls happen first when using trace event call chaining. Fix various instances where we weren't following this requirement.
2022-02-24 12:25:52 -08:00
8cc9a5af1a
Rebase 02/23
2022-02-23 14:23:28 -05:00
ce1d71472d
Merge branch 'main' of github.com:apple/foundationdb into jfu-grv-cache
2022-02-23 12:35:25 -05:00
622d89b552
Rebase on main.
...
Since we changed ClusterConnectionString's status flag from boolean to enum in #6422 , we need to update this PR correspondingly.
2022-02-22 16:29:59 -08:00
481587a8c6
Turn on hostname logic.
2022-02-22 16:29:59 -08:00
ae071a7211
clean up debug trace lines
2022-02-22 17:16:11 -05:00
b60cc02aa2
fix compile error
2022-02-22 13:10:30 -05:00
31d56b2f50
adjust knob setting, revert trace changes, and disable cache if onError retries transactions
2022-02-22 12:42:53 -05:00
a7a3dccba5
TSS Mismatch Quarantine fix
2022-02-17 19:16:07 -06:00
ef25e4fdfb
Sending empty version back immediately to establish stream connection and acknowledgement queue for change feed stream requests
2022-02-17 18:10:17 -06:00
38a75a8b89
Merge branch 'main' into blob_integration
2022-02-17 17:47:38 -06:00
26480c4705
change feed popping fixes
2022-02-15 15:12:23 -06:00
d399daebed
Merge branch 'main' of github.com:apple/foundationdb into jfu-grv-cache
2022-02-15 15:09:40 -05:00
6f1c3d50bb
add debug traces for testing
2022-02-15 15:08:53 -05:00
092b5cee4b
MVC2.0: Rollback added code
2022-02-14 13:50:42 -08:00
a63d218e9d
simplify test workload and adjust ratekeeper throttling strategy
2022-02-11 16:41:14 -05:00
2e63ac6963
code review changes: additional test checks, cleanup extra code, mark internal option hidden
2022-02-10 17:01:09 -05:00
458e708272
addressed code review comments: renamed variables, small functional changes, style changes
2022-02-10 16:17:54 -05:00
f4c235f134
propagating change_feed_popped correctly
2022-02-09 09:34:13 -06:00
72e06369a4
Merge remote-tracking branch 'apple-upstream/main' into version-vector-prototype
2022-02-08 17:47:57 +00:00
7e5ad88780
formatting for Github CI
2022-02-07 15:31:59 -05:00
9c0a512cf5
Merge branch 'main' of github.com:apple/foundationdb into jfu-grv-cache
2022-02-07 14:51:12 -05:00
ec2bbf0343
clean up some more trace lines and leftover code snippets
2022-02-07 14:50:04 -05:00
9b3f6fe452
More bugs in streaming split
2022-02-07 13:25:14 -06:00
88b1593681
Removing incorrect assert
2022-02-07 11:56:14 -06:00
88cab7fb67
More change feed fetching improvements and optimizations
2022-02-07 11:19:51 -06:00
c48ca9430d
Fixes for streaming split keys
2022-02-07 10:36:22 -06:00
809ec24184
More change feed buffer tuning
2022-02-07 08:19:19 -06:00
1736eabd33
fixes for split metrics stream
2022-02-04 18:24:55 -06:00
41a274ed9c
Fixing bug and refactoring iterative range splitting
2022-02-04 17:10:46 -06:00
ddfc301d74
Improving memory footprint of change feeds and making it configurable
2022-02-04 16:41:25 -06:00
a974773676
Handled change feed getting moved away or cancelled during move
2022-02-04 09:33:04 -06:00
2592c3f0ae
Implemented less efficient version of reply stream closing
2022-02-03 20:07:49 -06:00
9c8afdf35c
Debugged and fixed stuck change feed issue
2022-02-02 15:15:54 -06:00
6d05ecffc2
remove all multi-version client code related to grv cache
2022-02-02 14:39:17 -05:00
d8e7fea421
clean up some comments and debug changes
2022-02-02 14:03:32 -05:00
d7819f8387
Merge pull request #6304 from sfc-gh-ajbeamon/reset-copy-trstate
...
Copy transaction state during reset
2022-01-31 09:11:04 -08:00
89f4ed7481
Merge pull request #6296 from sfc-gh-ajbeamon/mark-external-system-transactions
...
Mark various externally exposed system transactions as read/access system keys
2022-01-28 14:16:40 -08:00
f9f3735f73
Add resolveHostnamesBlocking() in ConnectionString and IClusterConnectionRecord.
...
Also, combine IClusterConnectionRecord::getConnectionString() and IClusterConnectionRecord::getMutableConnectionString() to IClusterConnectionRecord::getConnectionString(), and rename setConnectionString() to setAndPersistConnectionString().
2022-01-28 12:20:41 -08:00
ac1fd056dd
Added change feed popped read guard for blob workers
2022-01-28 10:45:33 -06:00
c26e11c2c3
Adding extra debugid to change feed stream requests to track things better
2022-01-27 18:53:27 -06:00
8418cb2839
Mark various externally exposed system transactions as read/access system keys. Also adjust a few lock aware declarations to read lock aware.
2022-01-27 13:58:33 -08:00
29f0bcda21
Copy transaction state during reset to avoid potential issues with operations that were in-flight during the reset
2022-01-27 13:13:48 -08:00
bc7b76b407
Fix for gotAll in getChangeFeedMutations
2022-01-27 14:25:08 -06:00
1800259d7c
Merge pull request #6136 from sfc-gh-ajbeamon/client-description-trace-field
...
Add ClientDescription field for client processes to identify the client logging an event.
2022-01-26 11:37:31 -08:00
9db49484d4
add reference counting to shared state and create new fdb_c_types header
2022-01-25 16:10:03 -05:00
5d9843ec32
Merge branch 'main' of github.com:apple/foundationdb into jfu-grv-cache
2022-01-24 13:20:51 -05:00
133ee2c113
added fdb_c_internal.h and set up shared space initialization in multiversion api db creation
2022-01-24 12:55:24 -05:00
dd1d48754d
revert create_database signatures and added new shared state struct
2022-01-21 14:32:04 -05:00
3a21438bb8
remove getclusterid api
2022-01-21 13:12:30 -05:00
a8f1d6fa00
StorageMetrics and and halting fixes
2022-01-20 19:27:25 -06:00
915e2f6c1c
Merge branch 'main' of github.com:apple/foundationdb into jfu-grv-cache
2022-01-20 16:17:20 -05:00
ca1839c405
add sim_validation back to check staleness
2022-01-20 15:45:50 -05:00
9544379cdf
rebase
2022-01-20 11:12:33 -05:00
79704aa05f
Fixing change feed private mutation handling
2022-01-18 20:17:39 -06:00
58bc3a78ea
Fixed range assignment bug and fixed printf compiler warnings
2022-01-14 17:46:32 -06:00
e44dae0e5b
Removing incorrect assert
2022-01-14 15:47:51 -06:00
A.J. Beamon