Commit Graph

1360 Commits

Author SHA1 Message Date
Josh Slocum 6b1e387df7 removed debugging 2022-01-14 15:44:31 -06:00
Josh Slocum 9174bfbb4e bug fix and extra check in merge cursor 2022-01-14 15:26:37 -06:00
Josh Slocum e708072fd8 Cleaned up new merge cursor and fixed memory lifetime issue with cancelling a change feed 2022-01-14 14:47:23 -06:00
A.J. Beamon be80aa35a4 Fix memory issue with client watch metadata 2022-01-14 09:42:28 -08:00
Josh Slocum 0e58478c4d Initial rework of merge cursor 2022-01-14 10:28:54 -06:00
A.J. Beamon 07e5319477
Merge pull request #6165 from sfc-gh-ajbeamon/native-api-refactor
Refactor Native API Transactions
2022-01-13 14:53:19 -08:00
Josh Slocum 661c50d29f Handled server_overloaded in storageFeedVersionUpdater 2022-01-13 16:29:25 -06:00
A.J. Beamon cef5d400f9 Fix issue where enabling debug transaction after the transaction has been used could cause an assertion failure. 2022-01-13 13:43:42 -08:00
A.J. Beamon 17415168b6 Make the use provisional proxies parameter be of type UseProvisionalProxies everywhere. 2022-01-13 12:41:20 -08:00
Josh Slocum e7d15b2947 more explicit empty version problems 2022-01-13 12:10:44 -06:00
Josh Slocum 6e361a3283 Need delay(0) for empty check at start of change feed 2022-01-13 09:48:31 -06:00
Josh Slocum 515dc40153 Fixed issue with out of order empty versions 2022-01-13 08:43:29 -06:00
Jon Fu 5dccca4081 Implement initial concept of shared cache space in NativeAPI DatabaseContext 2022-01-12 16:49:00 -05:00
Josh Slocum fe9a3d27d8 Not allowing whenAtLeast after change feed end version 2022-01-12 13:16:49 -06:00
Jon Fu da76abbdf6 expose cluster ID through NativeAPI and C API 2022-01-11 17:06:25 -05:00
Josh Slocum 6b202fc9a8 Fixed stuck change feed storage updater and improved debugging 2022-01-11 15:35:54 -06:00
Josh Slocum 3b2e58ada8 Fixed rollback of snapshot files bug 2022-01-11 15:35:54 -06:00
Jon Fu b7ca2a656c remove epoch/generations for grv cache 2022-01-11 13:35:43 -05:00
Markus Pilman 8b77c8d79a Remove unused OpenSSL code 2022-01-11 10:34:37 -08:00
Andrew Noyes 64090dd2c0 Suppress a small leak that LSAN is reporting
I cannot seem to figure out why LSAN is reporting this, but if it is a
real leak than it's only a few bytes. Better to have the ASAN tests
actually passing IMO.
2022-01-10 13:44:09 -08:00
Josh Slocum 5bd8350eac also handling server overloaded in change feed pop 2022-01-10 13:13:00 -06:00
Evan Tschannen 91574fa0b9 fix: poppingChangeFeeds did not properly handle broken promise errors 2022-01-10 10:14:54 -08:00
Josh Slocum 21309fb55b Change feed merge cursor whenAtLeast fix 2022-01-07 16:16:29 -06:00
Josh Slocum 807ec37e84 BG request retry on failed blob worker properly 2022-01-05 17:38:49 -06:00
Josh Slocum 0f66cca8e0 Fixing change feed race with empty mutation and error 2022-01-05 16:40:07 -06:00
A.J. Beamon 138932c12f Fixed a memory bug in the watch map. A few other formatting tweaks, etc. 2022-01-05 11:26:43 -08:00
Josh Slocum bc69521a91 Several fixes with restarting BW/BM 2022-01-05 12:48:53 -06:00
Jon Fu 7fab9ea5c0 adjust sideband workload to expose another consistency error 2022-01-05 12:51:39 -05:00
A.J. Beamon 431d3de43b Fix some small bugs 2022-01-04 14:23:00 -08:00
Josh Slocum 329a4b1c85 Handling end of stream correctly in merge cursor optimization 2021-12-22 12:09:20 -06:00
Josh Slocum 738b72918a send merge cursor's buffered data if all sub-streams have something at this version 2021-12-21 16:32:56 -06:00
Josh Slocum b0aea91895 Broadening explicit disconnect handling to explicit error handling of all types 2021-12-21 14:12:09 -06:00
Josh Slocum d337e8fbe8 Handled disconnects explicitly in CF streams 2021-12-21 10:38:04 -06:00
Josh Slocum abdaf5c9cd extra debugging for CF empty versions 2021-12-21 08:58:20 -06:00
A.J. Beamon d8e161f89e Refactor NativeAPI transactions to create and pass around a reference counted state object. Watches no longer use the tranasction info object but instead use their own state. 2021-12-17 11:57:39 -08:00
A.J. Beamon 6ffe2067fe Set the CommitTransactionRequest span context upon construction/reset rather than in tryCommit. 2021-12-16 14:08:08 -08:00
A.J. Beamon af931ad989 Add support for universal trace fields that apply to all events in a process. Add a universal field on clients called ClientDescription that identifies the client logging the event. 2021-12-10 15:03:14 -08:00
Josh Slocum ea7a5114f5 Merge branch 'bw_rollback_grv' into blob_integration 2021-12-10 16:59:59 -06:00
Suraj Gupta 90a652ccfb fix merge conflict 2021-12-10 16:43:58 -06:00
Suraj Gupta e9d1f36cae More cleanup. 2021-12-10 16:43:58 -06:00
Suraj Gupta 310d990b12 Add debugging. 2021-12-10 16:43:58 -06:00
Josh Slocum 307d049c9d Cleaning up some memory lifetime issues 2021-12-10 16:12:06 -06:00
Josh Slocum ff2cd691cd Switching back to GRV for committed version checking, with proper rollback checking 2021-12-10 15:27:25 -06:00
Suraj Gupta 9fbccb21d2 Remove checks from native API. 2021-12-10 14:00:34 -06:00
Suraj Gupta fdc94d3356 Fix blobGranuleVerifier setup. 2021-12-10 14:00:34 -06:00
Suraj Gupta 0f3318a8ba add TODO 2021-12-10 14:00:34 -06:00
Josh Slocum 8dc5f79dc7 Made blob worker assume no empty mutations, and made merge cursor not send them 2021-12-09 15:00:46 -06:00
Josh Slocum 1ee0b16bfa Fixed bug in merge cursor whenAtLeast 2021-12-09 14:19:00 -06:00
Josh Slocum 86f6f73518 Merge cursor debugging and fix in BW::waitForVersion 2021-12-09 10:51:31 -06:00
Josh Slocum c24826430f Removing this check as it still won't work if the knob is set 2021-12-08 18:13:41 -06:00
Josh Slocum ef74d309d8 Adding delay after onEmpty for merge cursor as well 2021-12-08 12:48:30 -06:00
Josh Slocum 6c79750412 Added a bunch of debugging for tracking whenAtLatest problems, and fixed a bug there 2021-12-08 12:41:45 -06:00
Josh Slocum 4a9c2f4172 fixing merge conflicts 2021-12-08 08:47:58 -06:00
Josh Slocum c5b2b384da Fixing ASAN issues 2021-12-08 08:42:35 -06:00
Josh Slocum 3ee6d69f3d Wait for empty even for whenAtLatest>0 2021-12-08 08:42:35 -06:00
Evan Tschannen 75267aba80 fix: do not update lastReturnedVersion until the client has consumed the mutations 2021-12-07 13:03:50 -08:00
Evan Tschannen fdd35070a1 fix: merge cursors can have more mutations sent to self->mutations after it goes empty 2021-12-07 09:57:39 -08:00
Andrew Noyes def41697bf
Merge pull request #6083 from sfc-gh-tclinkenbeard/remove-temporaries
Avoid creating unnecessary temporary objects
2021-12-06 13:24:56 -08:00
Josh Slocum 528df86a28 fix: refactored and fixed changeFeedWhenAtLatest returning early 2021-12-06 14:06:17 -06:00
Evan Tschannen 02c650c6e7 fix: sending to mutations can cause whenAtLeast to update lastReturnedVersion before we have a chance to 2021-12-05 16:16:20 -08:00
Evan Tschannen da9c42fd9e do not log transaction metrics in simulation 2021-12-05 14:59:24 -08:00
Evan Tschannen f711ef56a3 fix: whenAtLeast updates lastReturnedVersion 2021-12-03 14:27:11 -08:00
Josh Slocum 2d7c44af31 Fixing bug with readVersionOut being null 2021-12-02 16:42:14 -06:00
Josh Slocum bc16e23d2c Merge branch 'master' into blob_integration 2021-12-02 15:51:18 -06:00
Josh Slocum d85eb330e0 retooling some waitForVersion stuff and adding asserts 2021-12-02 14:52:16 -06:00
Josh Slocum f43169cb7b bug fixes 2021-12-02 12:40:37 -06:00
Josh Slocum 632f701c8a First pass at using ChangeFeedData in blob worker 2021-12-02 11:42:16 -06:00
Josh Slocum 85c6e30aec Fix: several empty version issues with large single-version change feed entries 2021-12-02 09:56:15 -06:00
sfc-gh-tclinkenbeard 3b5d23ef88 Remove unnecessary temporary objects when appending to vector of pairs 2021-12-01 18:15:16 -08:00
Josh Slocum 2c63d0ae03 cleaning up prints 2021-12-01 17:17:19 -06:00
Josh Slocum a82845af43 Merge branch 'master' into bg_bindings 2021-12-01 16:55:28 -06:00
A.J. Beamon 13a4743e76 Check whether connRecord is null before logging it. 2021-11-29 14:22:44 -08:00
Jon Fu dcb0fc82e9 remove future wait optimization with NotifiedDouble and use regular GRV path if cache is too stale 2021-11-29 15:13:48 -05:00
Jon Fu b5b0afb96d narrow down places where to advance cache generation 2021-11-29 14:38:52 -05:00
Jon Fu 1fb537cde5 Add cache generations to create time boundaries when commits time out or fail 2021-11-29 14:16:49 -05:00
A.J. Beamon 264c75b9a6 Add some extra client logging details:
1. Add a trace event when a database is created and move the cluster file / connection string from ClientStart to the new trace event
2. Add a detail for the path to the image being loaded
3. Add a detail for whether a client library is primary or not
4. Set a thread name for each external client thread that includes the release version
2021-11-29 09:57:10 -08:00
Jon Fu 2b0ade5250 Change throttling threshold to count loop iterations instead of time 2021-11-25 13:31:55 -05:00
Jon Fu bba7ed8ac0 Add panic knob to shut down cache from outside txn options 2021-11-25 13:22:28 -05:00
Josh Slocum efb21ca6a1 Merge branch 'master' into blob_integration 2021-11-24 13:17:54 -06:00
Evan Tschannen e3422b2443 fix: change feed streams were not being closed properly 2021-11-23 15:01:44 -08:00
Evan Tschannen 3360000a91 fix: an empty mutation vector did not mean that all of the data was returned 2021-11-23 15:01:44 -08:00
Evan Tschannen 9df287ea06 fixed empty version merge logic 2021-11-23 15:01:44 -08:00
Evan Tschannen 6d9f134bf0 fix: merge needs to update on empty versions to release other mutations blocked by the empty version 2021-11-23 15:01:44 -08:00
Evan Tschannen 014fb2f31c fix: onEmpty must check all merged streams 2021-11-23 15:01:44 -08:00
Evan Tschannen a0afe530f8 protect against the changeFeedStorageData getting destroyed 2021-11-23 15:01:44 -08:00
Evan Tschannen 547c5899f8 whenAtLeast could return an incorrect result when the connection closed 2021-11-23 15:01:44 -08:00
Evan Tschannen 31b2414706 fixed a crash while popping change feeds 2021-11-23 15:01:44 -08:00
Jon Fu 3f24128da4 Merge branch 'master' of github.com:apple/foundationdb into jfu-grv-cache 2021-11-19 14:46:55 -05:00
Jon Fu e9c58d9f86 Check for sustained throttling in the proxy to lower threshold time and avoid false positives 2021-11-19 14:33:06 -05:00
Trevor Clinkenbeard 3d0ed2083c
Merge pull request #6000 from sfc-gh-tclinkenbeard/setup-flow-knobs
Setup global flow knobs in `setupNetwork`
2021-11-18 19:30:26 -08:00
sfc-gh-tclinkenbeard 2613ec7561 Expand use of fmt to get rid of %ld usage 2021-11-17 17:03:32 -08:00
Jon Fu 1e2060a52b add extra tracking in dbcontext to periodically contact GRV proxies 2021-11-17 15:24:37 -05:00
sfc-gh-tclinkenbeard 766a05d33c Merge remote-tracking branch 'origin/master' into add-format-warning 2021-11-17 12:14:01 -08:00
sfc-gh-tclinkenbeard e04526b345 Allow knobs to be set via network options before network is setup 2021-11-16 13:47:37 -08:00
Jon Fu 8714480370 fix some compile issues 2021-11-16 13:59:35 -05:00
Jon Fu 16e551e834 change knob to be an adjustable probability instead of binary on/off 2021-11-16 13:41:19 -05:00
sfc-gh-tclinkenbeard deb0eb85e8 Initialize global flow knobs in setupNetwork 2021-11-16 00:41:34 -08:00
Josh Slocum f140028086 Merge branch 'master' into blob_integration 2021-11-15 15:09:40 -06:00
Jon Fu 416d94d5b0 add some more trace details and remove spammy trace throttling temporarily 2021-11-15 13:45:23 -05:00
Evan Tschannen 94a51e57a5 Merge branch 'master' into feature-changefeed-empty-versions
# Conflicts:
#	fdbclient/StorageServerInterface.h
2021-11-14 19:13:05 -08:00
Evan Tschannen 6909754b21 changefeeds now have a whenAtLeast function for efficiently learning when the version has updated but no mutations have been committed 2021-11-14 19:08:46 -08:00
sfc-gh-tclinkenbeard 62efeb6812 Merge remote-tracking branch 'origin/master' into add-format-warning 2021-11-12 11:50:36 -08:00
Josh Slocum bab8756e17 fixed bugs in small splits 2021-11-11 13:50:19 -06:00
Josh Slocum 329091e14f Merge branch 'master' into bg_bindings 2021-11-11 10:13:37 -06:00
Josh Slocum b8ac4213a1 Switched BG APIs to transaction instead of database 2021-11-11 08:59:06 -06:00
Markus Pilman 0dfb72176e
Merge pull request #5857 from sfc-gh-vgasiunas/notify-client-lib-changes
A mechanism to notify MVC about relevant client library status changes on the cluster
2021-11-11 07:43:20 -07:00
Lukas Joswiak 1da288822f Remove distributed trace database option 2021-11-10 13:33:49 -08:00
Lukas Joswiak 8bc0c3e8a2 Convert distributed trace sampling rate to a database option 2021-11-10 13:33:49 -08:00
Lukas Joswiak c93052121f Fix issue where transaction spans would not be recorded 2021-11-10 13:33:49 -08:00
Vaidas Gasiunas 51b8ccf7d3 Merge remote-tracking branch 'apple/master' into notify-client-lib-changes 2021-11-10 18:40:34 +01:00
Tao Lin fdb3b72e35 Introduce GetRangeAndFlatMap to push computations down to FDB
Re-introduce #5609
2021-11-09 13:52:28 -08:00
Jon Fu e984b2f5d0 remove sevdebug temporarily and add extra trace to verify knob values 2021-11-09 15:13:50 -05:00
Jingyu Zhou 6540bef9ee Clear client VV cache after recovery
This is to prevent the client from sending older VV to the new GRV proxies,
because the client side cache can be stale, i.e., not knowing commits for a tag.

This was found in simulation:
A client has a VV cache of [{0:1,841885664} maxversion: 841885664]; client
issues a transaction commits at 841905696. There is a recovery. The client got
Delta=[ maxversion: -1] back. Client issues a read at 946662688, which uses the
cached version 841885664 < 841905696 , thus reading corrupted data.
2021-11-09 08:37:22 -08:00
Tao Lin 586cc3b102
Revert "Introduce GetRangeAndFlatMap to push computations down to FDB" 2021-11-04 08:46:56 -07:00
Tao Lin 6c98e35893 Rename Hop to FlatMap 2021-11-03 13:32:01 -07:00
Tao Lin 0853661d13 Introduce getRangeAndHop to push computations down to FDB 2021-11-03 13:21:16 -07:00
Josh Slocum e3839e198a Cleanup 2021-11-03 13:41:41 -05:00
Josh Slocum 382882f1c1 mako successfully calls read_blob_granules and gets stuff back 2021-11-02 13:43:42 -05:00
Josh Slocum d6a31078fe C API for blob granules 2021-11-02 10:01:23 -05:00
sfc-gh-tclinkenbeard 13bb7838aa Enable clang -Wformat warning 2021-10-30 21:07:38 -07:00
Dan Lambright befe1993c4 fix conflict on rebase 2021-10-29 12:25:26 -04:00
Vaidas Gasiunas 875824b186 MVC2.0: Notify clients about relevant changes of client libraries 2021-10-27 23:43:40 +02:00
Josh Slocum 19076ad4d2 Changed blob manager checkin to by async and to have blob worker speculatively consume change feed ahead of re-snapshot 2021-10-27 16:23:51 -05:00
Suraj Gupta fd50a011b1 Address PR comments. 2021-10-26 21:39:41 -04:00
Suraj Gupta 5a9d9921d0 Fixes and final cleanup for BM failure handling 2021-10-26 16:16:00 -04:00
Suraj Gupta 17b30f188a Working impl 2021-10-26 16:16:00 -04:00
Josh Slocum 0ff8ddc2b6 Merge branch 'master' into blob_full_clean 2021-10-25 13:38:48 -05:00
Jon Fu 7273d352a2 Add extra debug traces and ignore "IMMEDIATE" priority txns for rk throttling 2021-10-25 13:44:45 -04:00
Evan Tschannen 6f7558b8ea Merge branch 'master' of https://github.com/apple/foundationdb into feature-range-feed
# Conflicts:
#	tests/CMakeLists.txt
2021-10-24 21:06:33 -07:00
Evan Tschannen 0e327d3d0a fix: do not duplicate lastEpochEnd from different servers 2021-10-24 19:17:11 -07:00
Evan Tschannen 4039dbd8da fix: combine mutations from the same version into a single MutationsAndVesionRef 2021-10-24 13:46:45 -07:00
A.J. Beamon e882eb33fc Abstract the cluster file into a cluster connection record that can be backed by something other than the filesystem. 2021-10-22 11:05:18 -07:00
Jon Fu 6d74239760 Track throttling by measuring time spent left in queue on the proxy 2021-10-22 13:55:01 -04:00
Josh Slocum 773886515e Merge branch 'feature-range-feed' into blob_full_clean 2021-10-22 11:07:51 -05:00
Evan Tschannen d778cea9b5 fix: prevent infinite loop 2021-10-21 15:16:22 -07:00
Evan Tschannen f1158371a7 Merge branch 'master' of https://github.com/apple/foundationdb into feature-range-feed
# Conflicts:
#	flow/error_definitions.h
2021-10-21 00:55:12 -07:00
Evan Tschannen 4e79296a9f fixed a few bugs with fetching change feeds 2021-10-21 00:44:51 -07:00
Evan Tschannen 3ebabb6edc fixed incorrect use of change feed errors 2021-10-20 22:37:31 -07:00
Jon Fu f1c8d3fbc8 Add code to disable cache when ratekeeper begins throttling 2021-10-20 15:52:43 -04:00
Evan Tschannen 3f7df58a77 fixed a number of issues 2021-10-19 13:56:52 -07:00
A.J. Beamon abab45760d Add some additional logging if the network thread finishes, fails with an error, gets stopped, or is blocked. 2021-10-19 10:05:29 -07:00
Josh Slocum 912ef76f1c cleanup before merge 2021-10-18 17:11:14 -05:00
Sreenath Bodagala 246f035afe Merge remote-tracking branch 'apple-upstream/master' into version-vector-prototype 2021-10-18 19:07:33 +00:00
Jon Fu f75c8b25ad add small random delay before returning a cached read version 2021-10-18 13:59:04 -04:00
Suraj Gupta e2e852e515 Mitigate transitive includes. 2021-10-18 10:49:25 -04:00
Josh Slocum b5074fd597 Reworked all of the system data to encode granule data more efficiently for persistence 2021-10-13 16:28:04 -05:00
Josh Slocum 5f0ec0612a Merge branch 'feature-range-feed' into blob_full 2021-10-13 15:44:35 -05:00
Suraj Gupta bde90f058b Gate the client API. 2021-10-13 13:26:27 -04:00
Yao Xiao e543570343
Add locality cache size to TransactionMetrics. (#5758)
Co-authored-by: Yao Xiao <yaoxiao@Yaos-MacBook-Pro.local>
2021-10-12 13:10:15 -07:00
Evan Tschannen 282f84c807 added handling for broken promise 2021-10-12 10:36:10 -07:00
Markus Pilman 5066d67dec
Merge pull request #5715 from sfc-gh-ljoswiak/fixes/tracing-various
Add knob to control distributed trace recording percentage
2021-10-11 14:03:53 -06:00
Evan Tschannen d51edf18dc fixed merge conflicts 2021-10-09 19:47:24 -07:00
Evan Tschannen 5c642f706e Merge branch 'master' of https://github.com/apple/foundationdb into feature-range-feed
# Conflicts:
#	fdbcli/fdbcli.actor.cpp
2021-10-09 19:34:16 -07:00
Evan Tschannen efc4cec53f fixed a variety of bugs with change feeds 2021-10-09 19:24:01 -07:00
Markus Pilman 4fbc193e19 fix priority 2021-10-09 16:40:24 -06:00
Markus Pilman 9077e15b11 Address review comments 2021-10-09 16:40:24 -06:00
Markus Pilman 4e723ba3d9 Apply suggestions from code review
Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>
2021-10-09 16:40:24 -06:00
Markus Pilman ffdba4a133 assert we never get a stale read version 2021-10-09 16:40:24 -06:00
Jon Fu 2b2d059039 Specify option to specifically skip grv cache for the background updater when using debug knob 2021-10-08 17:32:44 -04:00
Jon Fu b770936cde move background updater actor initialization 2021-10-08 16:38:21 -04:00
Jon Fu ecdafba336 add debug traces to background grv updater 2021-10-08 15:17:28 -04:00
Dan Lambright f68d018a06 conflicts 2021-10-08 12:39:39 -04:00
Yao Xiao efd75ccc74
Add histograms to client GRV batcher. (#5727) 2021-10-07 13:30:25 -07:00
Josh Slocum 31664ddb78 Fixing change feed error retry in pop 2021-10-07 10:07:18 -05:00
Josh Slocum 54d5e8366d fixing another change feed hang error 2021-10-06 16:04:01 -05:00
Jon Fu cc38c48123 Add debug knob to use grv cache for performance testing 2021-10-06 14:35:42 -04:00
Josh Slocum bbe7c7ca9f found change feed liveness bug 2021-10-06 12:32:16 -05:00
Jon Fu 9a18cc8f41 Change debug time bound update logic to be located in masterserver 2021-10-05 17:31:31 -04:00
Jon Fu 44a854772f Merge branch 'master' of github.com:apple/foundationdb into jfu-grv-cache 2021-10-05 12:55:02 -04:00
Jon Fu ae31f01dc4 remove cache update from extractreadversion and set lower time bound to 0 2021-10-05 12:54:22 -04:00
Jon Fu d560eb1fea debug time bounds using sim_validation 2021-10-04 14:12:31 -04:00
Lukas Joswiak e034f66fc2 Add knob to control trace recording percentage 2021-10-01 16:30:46 -07:00
Josh Slocum fa1fe5f08b added blob worker rollbacks that handle (most) cases 2021-09-24 17:52:36 -05:00
Josh Slocum a986679b7f More bug fixes 2021-09-24 10:02:02 -05:00
A.J. Beamon 32a834b0c2 The computation of tag throttling backoff was taking a minimum with a knob for how long a throttle lasts. When this knob was buggified to 0 in simulation, the backoff would then be 0, and this could cause some transactions to get stuck in a retry loop that failed to advance time. Eventually the test would timeout or potentially run out of memory. 2021-09-23 10:03:57 -07:00
Jon Fu 4cd5d138ff Fixed segfault involving missing futures and infinite looping in background actor 2021-09-22 13:13:35 -04:00
Vaidas Gasiunas 16171d8252 Refactoring well-known endpoint registration
- List all well-known endpoints of FDB in a single enum
- Identify well-known endpoints by plain IDs
2021-09-21 11:05:31 -06:00
Sreenath Bodagala d4f9a09a4f - Address test failures 2021-09-17 21:21:44 +00:00
Sreenath Bodagala 2aa3b44d4e Merge remote-tracking branch 'apple-upstream/master' into version-vector-prototype
- Conflicts:
	fdbserver/LogSystem.h
	fdbserver/LogSystemConfig.h
	fdbserver/TagPartitionedLogSystem.actor.cpp

- Files modified during merge:

modified:   fdbserver/LogSystem.cpp
modified:   fdbserver/LogSystemConfig.cpp
2021-09-17 19:36:18 +00:00
Xiaoge Su abf73047ca Enforce std:: specifier rather than using namespace 2021-09-16 19:40:28 -07:00
Sreenath Bodagala 35621a255e - Address review comments 2021-09-16 12:00:26 -04:00
Sreenath Bodagala ffbdc21ca8 - Address review comments 2021-09-16 12:00:26 -04:00
Sreenath Bodagala 852fc96200 Address simulation test failures caused by:
- Assertion failures in MoveKeys.actor.cpp
- Wrong results returned by getRange()

Changes:

DatabaseContext.h, NativeAPI.actor.[h,cpp]:
- Introduce a new flag, TransactionInfo::readVersionObtainedFromGrvProxy.
- Set this flag to true by default, and clear it when the read version of a
transaction is explicitly set (by using setVersion()).
- Modify getLatestCommitVersions() to not populate "latestCommitVersions" if
this flag is not set. (This will cause storage server to read at the specified
read version.)

- Modify getRange() actor to always use the specified version as the read
version (except when the specified version is latestVersion).

- Modify waitForCommittedVersion(), getRawVersion(), and getConsistentReadVersion()
to update local version vector cache after receiving GetReadVersionReply.

IClientApi.h, IConfigTransaction.h, ISingleThreadTransaction.h,
MultiVersionTransaction[.actor].[h,cpp], ThreadSafeTransaction.[h,cpp],
ApiWorkload.h:
- Add methods to get the spanID of a transaction and also the version vector
cached in a transaction. (Likely to be useful for debugging simulation test
failures.)

VersionVector.h:
- Update "maxVersion" when populating/applying a delta. (Note that empty
mutation messages only update VersionVector::maxVersion.)

BackupWorker.actor.cpp:
- Update local version vector cache after receiving GetReadVersionReply message.

Status.actor.cpp:
- Update local version vector cache and
TransactionInfo::info.readVersionObtainedFromGrvProxy after setting the
read version.
2021-09-16 12:00:26 -04:00
Josh Slocum 6673540553 TSS Metrics fix 2021-09-14 13:15:05 -05:00
Jon Fu 87fac6e301 adjust to changed function signature after merge 2021-09-13 13:04:55 -04:00
Jon Fu d971b63bbf Merge branch 'master' of github.com:apple/foundationdb into jfu-grv-cache 2021-09-13 12:47:42 -04:00
Jon Fu cfbe5eca9c Use notifieddouble to track GRV request time and include grvdelay in timing calculations 2021-09-13 11:57:19 -04:00
Josh Slocum cefb66d64c fixing a couple bugs 2021-09-10 12:52:33 -05:00
Josh Slocum c2d1d1704f Merge branch 'feature-range-feed' into blob_full 2021-09-10 11:21:52 -05:00
Evan Tschannen ac5b580e2d Merge branch 'master' into feature-range-feed
# Conflicts:
#	fdbcli/fdbcli.actor.cpp
#	fdbclient/StorageServerInterface.cpp
#	fdbclient/StorageServerInterface.h
#	fdbserver/ApplyMetadataMutation.cpp
#	fdbserver/TLogServer.actor.cpp
#	flow/error_definitions.h
2021-09-09 23:13:22 -07:00
Jon Fu 1ce2457f7e initial commit 2021-09-09 14:01:42 -04:00
Josh Slocum eb76343dfb Added blob granule reassignment and splitting 2021-09-08 14:09:14 -05:00
Evan Tschannen d9782bc29d fixed a few bugs with merging change feeds on the client 2021-09-07 21:35:00 -07:00
Evan Tschannen f53ddf6720 fix: handle the case where I change feed is added while a fetch is in progress 2021-09-06 13:51:53 -07:00
Evan Tschannen af0792b1f5 fixes that allow change feeds to pass correctness in tests with no failures 2021-09-03 17:24:33 -07:00
Evan Tschannen b02e8d99c6 added a workload to test change feeds 2021-09-03 14:19:27 -07:00
Evan Tschannen 8309c2ce26 filter mutations in change feeds to the requested range 2021-09-03 12:06:13 -07:00
A.J. Beamon afd8df4bb9 Merge commit '7f2b6c2c7ec2efcd7339e3c6531e4906322477c3' into network-thread-busyness-burstiness-score 2021-09-03 09:48:11 -07:00
Evan Tschannen 4bbae59bb0 Fixed a few bugs and added a backup popping mechanism 2021-09-02 21:11:44 -07:00
Evan Tschannen 27151b0831 support reads from range feeds that span multiple storage servers 2021-09-01 14:35:51 -07:00