6f7adb04a5
Send a version along with GetKeyServerLocationsRequests to allow the proxy to reject missing tenants more quickly. Change the commit proxy tenant map to be unversioned.
2022-03-08 13:33:56 -08:00
58f5cea39b
fix typo
2022-03-07 11:20:24 -06:00
04f678d515
Fixing sample logic
2022-03-07 11:20:24 -06:00
665e182bcc
TSS Metrics improvements
2022-03-07 11:20:24 -06:00
c99c7cf55f
Apply tenant prefixes on the storage server. The commit proxy can serve tenant location requests. Fix some bugs.
2022-03-06 21:54:21 -08:00
5fa9d3e1b7
Add a tenant parameter to read and commit requests. Store a map of all tenants on commit proxy and storage servers. Add an option to require tenant mode.
2022-03-06 21:54:21 -08:00
36435af4f5
Add support for a limits hint in special keys. Fix a typo and resolve some comment formatting weirdness.
2022-03-06 21:52:39 -08:00
1a5bf25b5c
Update code base to use fmt 8.1.1
2022-03-04 15:52:06 -08:00
1555c0e2aa
Fix conflict between two recent merges by using the new errorUnsuppressed function.
2022-03-04 12:41:21 -08:00
cdebda35ab
Merge pull request #5725 from sfc-gh-jfu/jfu-grv-cache
...
Add transaction option for clients to use cached read versions
2022-03-04 09:17:27 -08:00
2b8882ca0a
Adding exponential backoff for consecutive change feed stream failures
2022-03-02 18:46:40 -06:00
e73c0a31e6
add wiggle_server_ids and wiggle_server_addresses in status json
2022-03-02 10:03:23 -08:00
5e39da7741
fixup! Reformat the source
2022-02-28 18:03:52 -08:00
02e396274f
Fix the compile error on NativeAPI.actor.cpp
2022-02-28 18:03:52 -08:00
aadd706a0a
Port back addressed comments for fixing get key location issue
2022-02-28 16:08:57 -08:00
0bdae35a2c
Address comments and move updateFailedEndpointRefreshTime
2022-02-24 16:35:30 -08:00
9c99c59b0c
Fix get key location overloading proxies
2022-02-24 16:35:30 -08:00
e5b4fb3d80
Adding explicit popVersion to change feed reply, like tlogs have
2022-02-24 15:20:18 -06:00
250a88e682
Enforce that trace event suppression calls happen first when using trace event call chaining. Fix various instances where we weren't following this requirement.
2022-02-24 12:25:52 -08:00
8cc9a5af1a
Rebase 02/23
2022-02-23 14:23:28 -05:00
ce1d71472d
Merge branch 'main' of github.com:apple/foundationdb into jfu-grv-cache
2022-02-23 12:35:25 -05:00
cf31e14904
Merge remote-tracking branch 'origin/main' into features/private-request-streams
2022-02-23 10:29:32 +01:00
622d89b552
Rebase on main.
...
Since we changed ClusterConnectionString's status flag from boolean to enum in #6422 , we need to update this PR correspondingly.
2022-02-22 16:29:59 -08:00
481587a8c6
Turn on hostname logic.
2022-02-22 16:29:59 -08:00
ae071a7211
clean up debug trace lines
2022-02-22 17:16:11 -05:00
b60cc02aa2
fix compile error
2022-02-22 13:10:30 -05:00
31d56b2f50
adjust knob setting, revert trace changes, and disable cache if onError retries transactions
2022-02-22 12:42:53 -05:00
dc973fb67e
Allow List and first test
2022-02-22 11:15:16 +01:00
a7a3dccba5
TSS Mismatch Quarantine fix
2022-02-17 19:16:07 -06:00
ef25e4fdfb
Sending empty version back immediately to establish stream connection and acknowledgement queue for change feed stream requests
2022-02-17 18:10:17 -06:00
38a75a8b89
Merge branch 'main' into blob_integration
2022-02-17 17:47:38 -06:00
26480c4705
change feed popping fixes
2022-02-15 15:12:23 -06:00
d399daebed
Merge branch 'main' of github.com:apple/foundationdb into jfu-grv-cache
2022-02-15 15:09:40 -05:00
6f1c3d50bb
add debug traces for testing
2022-02-15 15:08:53 -05:00
092b5cee4b
MVC2.0: Rollback added code
2022-02-14 13:50:42 -08:00
a63d218e9d
simplify test workload and adjust ratekeeper throttling strategy
2022-02-11 16:41:14 -05:00
2e63ac6963
code review changes: additional test checks, cleanup extra code, mark internal option hidden
2022-02-10 17:01:09 -05:00
458e708272
addressed code review comments: renamed variables, small functional changes, style changes
2022-02-10 16:17:54 -05:00
f4c235f134
propagating change_feed_popped correctly
2022-02-09 09:34:13 -06:00
72e06369a4
Merge remote-tracking branch 'apple-upstream/main' into version-vector-prototype
2022-02-08 17:47:57 +00:00
7e5ad88780
formatting for Github CI
2022-02-07 15:31:59 -05:00
9c0a512cf5
Merge branch 'main' of github.com:apple/foundationdb into jfu-grv-cache
2022-02-07 14:51:12 -05:00
ec2bbf0343
clean up some more trace lines and leftover code snippets
2022-02-07 14:50:04 -05:00
9b3f6fe452
More bugs in streaming split
2022-02-07 13:25:14 -06:00
88b1593681
Removing incorrect assert
2022-02-07 11:56:14 -06:00
88cab7fb67
More change feed fetching improvements and optimizations
2022-02-07 11:19:51 -06:00
c48ca9430d
Fixes for streaming split keys
2022-02-07 10:36:22 -06:00
809ec24184
More change feed buffer tuning
2022-02-07 08:19:19 -06:00
1736eabd33
fixes for split metrics stream
2022-02-04 18:24:55 -06:00
41a274ed9c
Fixing bug and refactoring iterative range splitting
2022-02-04 17:10:46 -06:00
ddfc301d74
Improving memory footprint of change feeds and making it configurable
2022-02-04 16:41:25 -06:00
a974773676
Handled change feed getting moved away or cancelled during move
2022-02-04 09:33:04 -06:00
2592c3f0ae
Implemented less efficient version of reply stream closing
2022-02-03 20:07:49 -06:00
9c8afdf35c
Debugged and fixed stuck change feed issue
2022-02-02 15:15:54 -06:00
6d05ecffc2
remove all multi-version client code related to grv cache
2022-02-02 14:39:17 -05:00
d8e7fea421
clean up some comments and debug changes
2022-02-02 14:03:32 -05:00
d7819f8387
Merge pull request #6304 from sfc-gh-ajbeamon/reset-copy-trstate
...
Copy transaction state during reset
2022-01-31 09:11:04 -08:00
89f4ed7481
Merge pull request #6296 from sfc-gh-ajbeamon/mark-external-system-transactions
...
Mark various externally exposed system transactions as read/access system keys
2022-01-28 14:16:40 -08:00
f9f3735f73
Add resolveHostnamesBlocking() in ConnectionString and IClusterConnectionRecord.
...
Also, combine IClusterConnectionRecord::getConnectionString() and IClusterConnectionRecord::getMutableConnectionString() to IClusterConnectionRecord::getConnectionString(), and rename setConnectionString() to setAndPersistConnectionString().
2022-01-28 12:20:41 -08:00
ac1fd056dd
Added change feed popped read guard for blob workers
2022-01-28 10:45:33 -06:00
c26e11c2c3
Adding extra debugid to change feed stream requests to track things better
2022-01-27 18:53:27 -06:00
8418cb2839
Mark various externally exposed system transactions as read/access system keys. Also adjust a few lock aware declarations to read lock aware.
2022-01-27 13:58:33 -08:00
29f0bcda21
Copy transaction state during reset to avoid potential issues with operations that were in-flight during the reset
2022-01-27 13:13:48 -08:00
bc7b76b407
Fix for gotAll in getChangeFeedMutations
2022-01-27 14:25:08 -06:00
1800259d7c
Merge pull request #6136 from sfc-gh-ajbeamon/client-description-trace-field
...
Add ClientDescription field for client processes to identify the client logging an event.
2022-01-26 11:37:31 -08:00
9db49484d4
add reference counting to shared state and create new fdb_c_types header
2022-01-25 16:10:03 -05:00
5d9843ec32
Merge branch 'main' of github.com:apple/foundationdb into jfu-grv-cache
2022-01-24 13:20:51 -05:00
133ee2c113
added fdb_c_internal.h and set up shared space initialization in multiversion api db creation
2022-01-24 12:55:24 -05:00
dd1d48754d
revert create_database signatures and added new shared state struct
2022-01-21 14:32:04 -05:00
3a21438bb8
remove getclusterid api
2022-01-21 13:12:30 -05:00
a8f1d6fa00
StorageMetrics and and halting fixes
2022-01-20 19:27:25 -06:00
915e2f6c1c
Merge branch 'main' of github.com:apple/foundationdb into jfu-grv-cache
2022-01-20 16:17:20 -05:00
ca1839c405
add sim_validation back to check staleness
2022-01-20 15:45:50 -05:00
9544379cdf
rebase
2022-01-20 11:12:33 -05:00
79704aa05f
Fixing change feed private mutation handling
2022-01-18 20:17:39 -06:00
58bc3a78ea
Fixed range assignment bug and fixed printf compiler warnings
2022-01-14 17:46:32 -06:00
e44dae0e5b
Removing incorrect assert
2022-01-14 15:47:51 -06:00
6b1e387df7
removed debugging
2022-01-14 15:44:31 -06:00
9174bfbb4e
bug fix and extra check in merge cursor
2022-01-14 15:26:37 -06:00
e708072fd8
Cleaned up new merge cursor and fixed memory lifetime issue with cancelling a change feed
2022-01-14 14:47:23 -06:00
be80aa35a4
Fix memory issue with client watch metadata
2022-01-14 09:42:28 -08:00
0e58478c4d
Initial rework of merge cursor
2022-01-14 10:28:54 -06:00
07e5319477
Merge pull request #6165 from sfc-gh-ajbeamon/native-api-refactor
...
Refactor Native API Transactions
2022-01-13 14:53:19 -08:00
661c50d29f
Handled server_overloaded in storageFeedVersionUpdater
2022-01-13 16:29:25 -06:00
cef5d400f9
Fix issue where enabling debug transaction after the transaction has been used could cause an assertion failure.
2022-01-13 13:43:42 -08:00
17415168b6
Make the use provisional proxies parameter be of type UseProvisionalProxies everywhere.
2022-01-13 12:41:20 -08:00
e7d15b2947
more explicit empty version problems
2022-01-13 12:10:44 -06:00
6e361a3283
Need delay(0) for empty check at start of change feed
2022-01-13 09:48:31 -06:00
515dc40153
Fixed issue with out of order empty versions
2022-01-13 08:43:29 -06:00
5dccca4081
Implement initial concept of shared cache space in NativeAPI DatabaseContext
2022-01-12 16:49:00 -05:00
fe9a3d27d8
Not allowing whenAtLeast after change feed end version
2022-01-12 13:16:49 -06:00
da76abbdf6
expose cluster ID through NativeAPI and C API
2022-01-11 17:06:25 -05:00
6b202fc9a8
Fixed stuck change feed storage updater and improved debugging
2022-01-11 15:35:54 -06:00
3b2e58ada8
Fixed rollback of snapshot files bug
2022-01-11 15:35:54 -06:00
b7ca2a656c
remove epoch/generations for grv cache
2022-01-11 13:35:43 -05:00
8b77c8d79a
Remove unused OpenSSL code
2022-01-11 10:34:37 -08:00
64090dd2c0
Suppress a small leak that LSAN is reporting
...
I cannot seem to figure out why LSAN is reporting this, but if it is a
real leak than it's only a few bytes. Better to have the ASAN tests
actually passing IMO.
2022-01-10 13:44:09 -08:00
5bd8350eac
also handling server overloaded in change feed pop
2022-01-10 13:13:00 -06:00
91574fa0b9
fix: poppingChangeFeeds did not properly handle broken promise errors
2022-01-10 10:14:54 -08:00
21309fb55b
Change feed merge cursor whenAtLeast fix
2022-01-07 16:16:29 -06:00
807ec37e84
BG request retry on failed blob worker properly
2022-01-05 17:38:49 -06:00
0f66cca8e0
Fixing change feed race with empty mutation and error
2022-01-05 16:40:07 -06:00
138932c12f
Fixed a memory bug in the watch map. A few other formatting tweaks, etc.
2022-01-05 11:26:43 -08:00
bc69521a91
Several fixes with restarting BW/BM
2022-01-05 12:48:53 -06:00
7fab9ea5c0
adjust sideband workload to expose another consistency error
2022-01-05 12:51:39 -05:00
431d3de43b
Fix some small bugs
2022-01-04 14:23:00 -08:00
329a4b1c85
Handling end of stream correctly in merge cursor optimization
2021-12-22 12:09:20 -06:00
738b72918a
send merge cursor's buffered data if all sub-streams have something at this version
2021-12-21 16:32:56 -06:00
b0aea91895
Broadening explicit disconnect handling to explicit error handling of all types
2021-12-21 14:12:09 -06:00
d337e8fbe8
Handled disconnects explicitly in CF streams
2021-12-21 10:38:04 -06:00
abdaf5c9cd
extra debugging for CF empty versions
2021-12-21 08:58:20 -06:00
d8e161f89e
Refactor NativeAPI transactions to create and pass around a reference counted state object. Watches no longer use the tranasction info object but instead use their own state.
2021-12-17 11:57:39 -08:00
6ffe2067fe
Set the CommitTransactionRequest span context upon construction/reset rather than in tryCommit.
2021-12-16 14:08:08 -08:00
af931ad989
Add support for universal trace fields that apply to all events in a process. Add a universal field on clients called ClientDescription that identifies the client logging the event.
2021-12-10 15:03:14 -08:00
ea7a5114f5
Merge branch 'bw_rollback_grv' into blob_integration
2021-12-10 16:59:59 -06:00
90a652ccfb
fix merge conflict
2021-12-10 16:43:58 -06:00
e9d1f36cae
More cleanup.
2021-12-10 16:43:58 -06:00
310d990b12
Add debugging.
2021-12-10 16:43:58 -06:00
307d049c9d
Cleaning up some memory lifetime issues
2021-12-10 16:12:06 -06:00
ff2cd691cd
Switching back to GRV for committed version checking, with proper rollback checking
2021-12-10 15:27:25 -06:00
9fbccb21d2
Remove checks from native API.
2021-12-10 14:00:34 -06:00
fdc94d3356
Fix blobGranuleVerifier setup.
2021-12-10 14:00:34 -06:00
0f3318a8ba
add TODO
2021-12-10 14:00:34 -06:00
8dc5f79dc7
Made blob worker assume no empty mutations, and made merge cursor not send them
2021-12-09 15:00:46 -06:00
1ee0b16bfa
Fixed bug in merge cursor whenAtLeast
2021-12-09 14:19:00 -06:00
86f6f73518
Merge cursor debugging and fix in BW::waitForVersion
2021-12-09 10:51:31 -06:00
c24826430f
Removing this check as it still won't work if the knob is set
2021-12-08 18:13:41 -06:00
ef74d309d8
Adding delay after onEmpty for merge cursor as well
2021-12-08 12:48:30 -06:00
6c79750412
Added a bunch of debugging for tracking whenAtLatest problems, and fixed a bug there
2021-12-08 12:41:45 -06:00
4a9c2f4172
fixing merge conflicts
2021-12-08 08:47:58 -06:00
c5b2b384da
Fixing ASAN issues
2021-12-08 08:42:35 -06:00
3ee6d69f3d
Wait for empty even for whenAtLatest>0
2021-12-08 08:42:35 -06:00
75267aba80
fix: do not update lastReturnedVersion until the client has consumed the mutations
2021-12-07 13:03:50 -08:00
fdd35070a1
fix: merge cursors can have more mutations sent to self->mutations after it goes empty
2021-12-07 09:57:39 -08:00
def41697bf
Merge pull request #6083 from sfc-gh-tclinkenbeard/remove-temporaries
...
Avoid creating unnecessary temporary objects
2021-12-06 13:24:56 -08:00
528df86a28
fix: refactored and fixed changeFeedWhenAtLatest returning early
2021-12-06 14:06:17 -06:00
02c650c6e7
fix: sending to mutations can cause whenAtLeast to update lastReturnedVersion before we have a chance to
2021-12-05 16:16:20 -08:00
da9c42fd9e
do not log transaction metrics in simulation
2021-12-05 14:59:24 -08:00
f711ef56a3
fix: whenAtLeast updates lastReturnedVersion
2021-12-03 14:27:11 -08:00
2d7c44af31
Fixing bug with readVersionOut being null
2021-12-02 16:42:14 -06:00
bc16e23d2c
Merge branch 'master' into blob_integration
2021-12-02 15:51:18 -06:00
d85eb330e0
retooling some waitForVersion stuff and adding asserts
2021-12-02 14:52:16 -06:00
f43169cb7b
bug fixes
2021-12-02 12:40:37 -06:00
632f701c8a
First pass at using ChangeFeedData in blob worker
2021-12-02 11:42:16 -06:00
85c6e30aec
Fix: several empty version issues with large single-version change feed entries
2021-12-02 09:56:15 -06:00
3b5d23ef88
Remove unnecessary temporary objects when appending to vector of pairs
2021-12-01 18:15:16 -08:00
2c63d0ae03
cleaning up prints
2021-12-01 17:17:19 -06:00
a82845af43
Merge branch 'master' into bg_bindings
2021-12-01 16:55:28 -06:00
13a4743e76
Check whether connRecord is null before logging it.
2021-11-29 14:22:44 -08:00
dcb0fc82e9
remove future wait optimization with NotifiedDouble and use regular GRV path if cache is too stale
2021-11-29 15:13:48 -05:00
A.J. Beamon