1736eabd33
fixes for split metrics stream
2022-02-04 18:24:55 -06:00
41a274ed9c
Fixing bug and refactoring iterative range splitting
2022-02-04 17:10:46 -06:00
ddfc301d74
Improving memory footprint of change feeds and making it configurable
2022-02-04 16:41:25 -06:00
a974773676
Handled change feed getting moved away or cancelled during move
2022-02-04 09:33:04 -06:00
2592c3f0ae
Implemented less efficient version of reply stream closing
2022-02-03 20:07:49 -06:00
9c8afdf35c
Debugged and fixed stuck change feed issue
2022-02-02 15:15:54 -06:00
6d05ecffc2
remove all multi-version client code related to grv cache
2022-02-02 14:39:17 -05:00
d8e7fea421
clean up some comments and debug changes
2022-02-02 14:03:32 -05:00
d7819f8387
Merge pull request #6304 from sfc-gh-ajbeamon/reset-copy-trstate
...
Copy transaction state during reset
2022-01-31 09:11:04 -08:00
89f4ed7481
Merge pull request #6296 from sfc-gh-ajbeamon/mark-external-system-transactions
...
Mark various externally exposed system transactions as read/access system keys
2022-01-28 14:16:40 -08:00
f9f3735f73
Add resolveHostnamesBlocking() in ConnectionString and IClusterConnectionRecord.
...
Also, combine IClusterConnectionRecord::getConnectionString() and IClusterConnectionRecord::getMutableConnectionString() to IClusterConnectionRecord::getConnectionString(), and rename setConnectionString() to setAndPersistConnectionString().
2022-01-28 12:20:41 -08:00
ac1fd056dd
Added change feed popped read guard for blob workers
2022-01-28 10:45:33 -06:00
c26e11c2c3
Adding extra debugid to change feed stream requests to track things better
2022-01-27 18:53:27 -06:00
8418cb2839
Mark various externally exposed system transactions as read/access system keys. Also adjust a few lock aware declarations to read lock aware.
2022-01-27 13:58:33 -08:00
29f0bcda21
Copy transaction state during reset to avoid potential issues with operations that were in-flight during the reset
2022-01-27 13:13:48 -08:00
bc7b76b407
Fix for gotAll in getChangeFeedMutations
2022-01-27 14:25:08 -06:00
1800259d7c
Merge pull request #6136 from sfc-gh-ajbeamon/client-description-trace-field
...
Add ClientDescription field for client processes to identify the client logging an event.
2022-01-26 11:37:31 -08:00
9db49484d4
add reference counting to shared state and create new fdb_c_types header
2022-01-25 16:10:03 -05:00
5d9843ec32
Merge branch 'main' of github.com:apple/foundationdb into jfu-grv-cache
2022-01-24 13:20:51 -05:00
133ee2c113
added fdb_c_internal.h and set up shared space initialization in multiversion api db creation
2022-01-24 12:55:24 -05:00
dd1d48754d
revert create_database signatures and added new shared state struct
2022-01-21 14:32:04 -05:00
3a21438bb8
remove getclusterid api
2022-01-21 13:12:30 -05:00
a8f1d6fa00
StorageMetrics and and halting fixes
2022-01-20 19:27:25 -06:00
915e2f6c1c
Merge branch 'main' of github.com:apple/foundationdb into jfu-grv-cache
2022-01-20 16:17:20 -05:00
ca1839c405
add sim_validation back to check staleness
2022-01-20 15:45:50 -05:00
9544379cdf
rebase
2022-01-20 11:12:33 -05:00
79704aa05f
Fixing change feed private mutation handling
2022-01-18 20:17:39 -06:00
58bc3a78ea
Fixed range assignment bug and fixed printf compiler warnings
2022-01-14 17:46:32 -06:00
e44dae0e5b
Removing incorrect assert
2022-01-14 15:47:51 -06:00
6b1e387df7
removed debugging
2022-01-14 15:44:31 -06:00
9174bfbb4e
bug fix and extra check in merge cursor
2022-01-14 15:26:37 -06:00
e708072fd8
Cleaned up new merge cursor and fixed memory lifetime issue with cancelling a change feed
2022-01-14 14:47:23 -06:00
be80aa35a4
Fix memory issue with client watch metadata
2022-01-14 09:42:28 -08:00
0e58478c4d
Initial rework of merge cursor
2022-01-14 10:28:54 -06:00
07e5319477
Merge pull request #6165 from sfc-gh-ajbeamon/native-api-refactor
...
Refactor Native API Transactions
2022-01-13 14:53:19 -08:00
661c50d29f
Handled server_overloaded in storageFeedVersionUpdater
2022-01-13 16:29:25 -06:00
cef5d400f9
Fix issue where enabling debug transaction after the transaction has been used could cause an assertion failure.
2022-01-13 13:43:42 -08:00
17415168b6
Make the use provisional proxies parameter be of type UseProvisionalProxies everywhere.
2022-01-13 12:41:20 -08:00
e7d15b2947
more explicit empty version problems
2022-01-13 12:10:44 -06:00
6e361a3283
Need delay(0) for empty check at start of change feed
2022-01-13 09:48:31 -06:00
515dc40153
Fixed issue with out of order empty versions
2022-01-13 08:43:29 -06:00
5dccca4081
Implement initial concept of shared cache space in NativeAPI DatabaseContext
2022-01-12 16:49:00 -05:00
fe9a3d27d8
Not allowing whenAtLeast after change feed end version
2022-01-12 13:16:49 -06:00
da76abbdf6
expose cluster ID through NativeAPI and C API
2022-01-11 17:06:25 -05:00
6b202fc9a8
Fixed stuck change feed storage updater and improved debugging
2022-01-11 15:35:54 -06:00
3b2e58ada8
Fixed rollback of snapshot files bug
2022-01-11 15:35:54 -06:00
b7ca2a656c
remove epoch/generations for grv cache
2022-01-11 13:35:43 -05:00
8b77c8d79a
Remove unused OpenSSL code
2022-01-11 10:34:37 -08:00
64090dd2c0
Suppress a small leak that LSAN is reporting
...
I cannot seem to figure out why LSAN is reporting this, but if it is a
real leak than it's only a few bytes. Better to have the ASAN tests
actually passing IMO.
2022-01-10 13:44:09 -08:00
5bd8350eac
also handling server overloaded in change feed pop
2022-01-10 13:13:00 -06:00
91574fa0b9
fix: poppingChangeFeeds did not properly handle broken promise errors
2022-01-10 10:14:54 -08:00
21309fb55b
Change feed merge cursor whenAtLeast fix
2022-01-07 16:16:29 -06:00
807ec37e84
BG request retry on failed blob worker properly
2022-01-05 17:38:49 -06:00
0f66cca8e0
Fixing change feed race with empty mutation and error
2022-01-05 16:40:07 -06:00
138932c12f
Fixed a memory bug in the watch map. A few other formatting tweaks, etc.
2022-01-05 11:26:43 -08:00
bc69521a91
Several fixes with restarting BW/BM
2022-01-05 12:48:53 -06:00
7fab9ea5c0
adjust sideband workload to expose another consistency error
2022-01-05 12:51:39 -05:00
431d3de43b
Fix some small bugs
2022-01-04 14:23:00 -08:00
329a4b1c85
Handling end of stream correctly in merge cursor optimization
2021-12-22 12:09:20 -06:00
738b72918a
send merge cursor's buffered data if all sub-streams have something at this version
2021-12-21 16:32:56 -06:00
b0aea91895
Broadening explicit disconnect handling to explicit error handling of all types
2021-12-21 14:12:09 -06:00
d337e8fbe8
Handled disconnects explicitly in CF streams
2021-12-21 10:38:04 -06:00
abdaf5c9cd
extra debugging for CF empty versions
2021-12-21 08:58:20 -06:00
d8e161f89e
Refactor NativeAPI transactions to create and pass around a reference counted state object. Watches no longer use the tranasction info object but instead use their own state.
2021-12-17 11:57:39 -08:00
6ffe2067fe
Set the CommitTransactionRequest span context upon construction/reset rather than in tryCommit.
2021-12-16 14:08:08 -08:00
af931ad989
Add support for universal trace fields that apply to all events in a process. Add a universal field on clients called ClientDescription that identifies the client logging the event.
2021-12-10 15:03:14 -08:00
ea7a5114f5
Merge branch 'bw_rollback_grv' into blob_integration
2021-12-10 16:59:59 -06:00
90a652ccfb
fix merge conflict
2021-12-10 16:43:58 -06:00
e9d1f36cae
More cleanup.
2021-12-10 16:43:58 -06:00
310d990b12
Add debugging.
2021-12-10 16:43:58 -06:00
307d049c9d
Cleaning up some memory lifetime issues
2021-12-10 16:12:06 -06:00
ff2cd691cd
Switching back to GRV for committed version checking, with proper rollback checking
2021-12-10 15:27:25 -06:00
9fbccb21d2
Remove checks from native API.
2021-12-10 14:00:34 -06:00
fdc94d3356
Fix blobGranuleVerifier setup.
2021-12-10 14:00:34 -06:00
0f3318a8ba
add TODO
2021-12-10 14:00:34 -06:00
8dc5f79dc7
Made blob worker assume no empty mutations, and made merge cursor not send them
2021-12-09 15:00:46 -06:00
1ee0b16bfa
Fixed bug in merge cursor whenAtLeast
2021-12-09 14:19:00 -06:00
86f6f73518
Merge cursor debugging and fix in BW::waitForVersion
2021-12-09 10:51:31 -06:00
c24826430f
Removing this check as it still won't work if the knob is set
2021-12-08 18:13:41 -06:00
ef74d309d8
Adding delay after onEmpty for merge cursor as well
2021-12-08 12:48:30 -06:00
6c79750412
Added a bunch of debugging for tracking whenAtLatest problems, and fixed a bug there
2021-12-08 12:41:45 -06:00
4a9c2f4172
fixing merge conflicts
2021-12-08 08:47:58 -06:00
c5b2b384da
Fixing ASAN issues
2021-12-08 08:42:35 -06:00
3ee6d69f3d
Wait for empty even for whenAtLatest>0
2021-12-08 08:42:35 -06:00
75267aba80
fix: do not update lastReturnedVersion until the client has consumed the mutations
2021-12-07 13:03:50 -08:00
fdd35070a1
fix: merge cursors can have more mutations sent to self->mutations after it goes empty
2021-12-07 09:57:39 -08:00
def41697bf
Merge pull request #6083 from sfc-gh-tclinkenbeard/remove-temporaries
...
Avoid creating unnecessary temporary objects
2021-12-06 13:24:56 -08:00
528df86a28
fix: refactored and fixed changeFeedWhenAtLatest returning early
2021-12-06 14:06:17 -06:00
02c650c6e7
fix: sending to mutations can cause whenAtLeast to update lastReturnedVersion before we have a chance to
2021-12-05 16:16:20 -08:00
da9c42fd9e
do not log transaction metrics in simulation
2021-12-05 14:59:24 -08:00
f711ef56a3
fix: whenAtLeast updates lastReturnedVersion
2021-12-03 14:27:11 -08:00
2d7c44af31
Fixing bug with readVersionOut being null
2021-12-02 16:42:14 -06:00
bc16e23d2c
Merge branch 'master' into blob_integration
2021-12-02 15:51:18 -06:00
d85eb330e0
retooling some waitForVersion stuff and adding asserts
2021-12-02 14:52:16 -06:00
f43169cb7b
bug fixes
2021-12-02 12:40:37 -06:00
632f701c8a
First pass at using ChangeFeedData in blob worker
2021-12-02 11:42:16 -06:00
85c6e30aec
Fix: several empty version issues with large single-version change feed entries
2021-12-02 09:56:15 -06:00
3b5d23ef88
Remove unnecessary temporary objects when appending to vector of pairs
2021-12-01 18:15:16 -08:00
2c63d0ae03
cleaning up prints
2021-12-01 17:17:19 -06:00
a82845af43
Merge branch 'master' into bg_bindings
2021-12-01 16:55:28 -06:00
13a4743e76
Check whether connRecord is null before logging it.
2021-11-29 14:22:44 -08:00
dcb0fc82e9
remove future wait optimization with NotifiedDouble and use regular GRV path if cache is too stale
2021-11-29 15:13:48 -05:00
b5b0afb96d
narrow down places where to advance cache generation
2021-11-29 14:38:52 -05:00
1fb537cde5
Add cache generations to create time boundaries when commits time out or fail
2021-11-29 14:16:49 -05:00
264c75b9a6
Add some extra client logging details:
...
1. Add a trace event when a database is created and move the cluster file / connection string from ClientStart to the new trace event
2. Add a detail for the path to the image being loaded
3. Add a detail for whether a client library is primary or not
4. Set a thread name for each external client thread that includes the release version
2021-11-29 09:57:10 -08:00
2b0ade5250
Change throttling threshold to count loop iterations instead of time
2021-11-25 13:31:55 -05:00
bba7ed8ac0
Add panic knob to shut down cache from outside txn options
2021-11-25 13:22:28 -05:00
efb21ca6a1
Merge branch 'master' into blob_integration
2021-11-24 13:17:54 -06:00
e3422b2443
fix: change feed streams were not being closed properly
2021-11-23 15:01:44 -08:00
3360000a91
fix: an empty mutation vector did not mean that all of the data was returned
2021-11-23 15:01:44 -08:00
9df287ea06
fixed empty version merge logic
2021-11-23 15:01:44 -08:00
6d9f134bf0
fix: merge needs to update on empty versions to release other mutations blocked by the empty version
2021-11-23 15:01:44 -08:00
014fb2f31c
fix: onEmpty must check all merged streams
2021-11-23 15:01:44 -08:00
a0afe530f8
protect against the changeFeedStorageData getting destroyed
2021-11-23 15:01:44 -08:00
547c5899f8
whenAtLeast could return an incorrect result when the connection closed
2021-11-23 15:01:44 -08:00
31b2414706
fixed a crash while popping change feeds
2021-11-23 15:01:44 -08:00
3f24128da4
Merge branch 'master' of github.com:apple/foundationdb into jfu-grv-cache
2021-11-19 14:46:55 -05:00
e9c58d9f86
Check for sustained throttling in the proxy to lower threshold time and avoid false positives
2021-11-19 14:33:06 -05:00
3d0ed2083c
Merge pull request #6000 from sfc-gh-tclinkenbeard/setup-flow-knobs
...
Setup global flow knobs in `setupNetwork`
2021-11-18 19:30:26 -08:00
2613ec7561
Expand use of fmt to get rid of %ld usage
2021-11-17 17:03:32 -08:00
1e2060a52b
add extra tracking in dbcontext to periodically contact GRV proxies
2021-11-17 15:24:37 -05:00
766a05d33c
Merge remote-tracking branch 'origin/master' into add-format-warning
2021-11-17 12:14:01 -08:00
e04526b345
Allow knobs to be set via network options before network is setup
2021-11-16 13:47:37 -08:00
8714480370
fix some compile issues
2021-11-16 13:59:35 -05:00
16e551e834
change knob to be an adjustable probability instead of binary on/off
2021-11-16 13:41:19 -05:00
deb0eb85e8
Initialize global flow knobs in setupNetwork
2021-11-16 00:41:34 -08:00
f140028086
Merge branch 'master' into blob_integration
2021-11-15 15:09:40 -06:00
416d94d5b0
add some more trace details and remove spammy trace throttling temporarily
2021-11-15 13:45:23 -05:00
94a51e57a5
Merge branch 'master' into feature-changefeed-empty-versions
...
# Conflicts:
# fdbclient/StorageServerInterface.h
2021-11-14 19:13:05 -08:00
6909754b21
changefeeds now have a whenAtLeast function for efficiently learning when the version has updated but no mutations have been committed
2021-11-14 19:08:46 -08:00
62efeb6812
Merge remote-tracking branch 'origin/master' into add-format-warning
2021-11-12 11:50:36 -08:00
bab8756e17
fixed bugs in small splits
2021-11-11 13:50:19 -06:00
329091e14f
Merge branch 'master' into bg_bindings
2021-11-11 10:13:37 -06:00
b8ac4213a1
Switched BG APIs to transaction instead of database
2021-11-11 08:59:06 -06:00
0dfb72176e
Merge pull request #5857 from sfc-gh-vgasiunas/notify-client-lib-changes
...
A mechanism to notify MVC about relevant client library status changes on the cluster
2021-11-11 07:43:20 -07:00
1da288822f
Remove distributed trace database option
2021-11-10 13:33:49 -08:00
8bc0c3e8a2
Convert distributed trace sampling rate to a database option
2021-11-10 13:33:49 -08:00
c93052121f
Fix issue where transaction spans would not be recorded
2021-11-10 13:33:49 -08:00
51b8ccf7d3
Merge remote-tracking branch 'apple/master' into notify-client-lib-changes
2021-11-10 18:40:34 +01:00
fdb3b72e35
Introduce GetRangeAndFlatMap to push computations down to FDB
...
Re-introduce #5609
2021-11-09 13:52:28 -08:00
e984b2f5d0
remove sevdebug temporarily and add extra trace to verify knob values
2021-11-09 15:13:50 -05:00
6540bef9ee
Clear client VV cache after recovery
...
This is to prevent the client from sending older VV to the new GRV proxies,
because the client side cache can be stale, i.e., not knowing commits for a tag.
This was found in simulation:
A client has a VV cache of [{0:1,841885664} maxversion: 841885664]; client
issues a transaction commits at 841905696. There is a recovery. The client got
Delta=[ maxversion: -1] back. Client issues a read at 946662688, which uses the
cached version 841885664 < 841905696 , thus reading corrupted data.
2021-11-09 08:37:22 -08:00
586cc3b102
Revert "Introduce GetRangeAndFlatMap to push computations down to FDB"
2021-11-04 08:46:56 -07:00
6c98e35893
Rename Hop to FlatMap
2021-11-03 13:32:01 -07:00
0853661d13
Introduce getRangeAndHop to push computations down to FDB
2021-11-03 13:21:16 -07:00
e3839e198a
Cleanup
2021-11-03 13:41:41 -05:00
382882f1c1
mako successfully calls read_blob_granules and gets stuff back
2021-11-02 13:43:42 -05:00
d6a31078fe
C API for blob granules
2021-11-02 10:01:23 -05:00
13bb7838aa
Enable clang -Wformat warning
2021-10-30 21:07:38 -07:00
befe1993c4
fix conflict on rebase
2021-10-29 12:25:26 -04:00
875824b186
MVC2.0: Notify clients about relevant changes of client libraries
2021-10-27 23:43:40 +02:00
19076ad4d2
Changed blob manager checkin to by async and to have blob worker speculatively consume change feed ahead of re-snapshot
2021-10-27 16:23:51 -05:00
fd50a011b1
Address PR comments.
2021-10-26 21:39:41 -04:00
5a9d9921d0
Fixes and final cleanup for BM failure handling
2021-10-26 16:16:00 -04:00
17b30f188a
Working impl
2021-10-26 16:16:00 -04:00
0ff8ddc2b6
Merge branch 'master' into blob_full_clean
2021-10-25 13:38:48 -05:00
7273d352a2
Add extra debug traces and ignore "IMMEDIATE" priority txns for rk throttling
2021-10-25 13:44:45 -04:00
6f7558b8ea
Merge branch 'master' of https://github.com/apple/foundationdb into feature-range-feed
...
# Conflicts:
# tests/CMakeLists.txt
2021-10-24 21:06:33 -07:00
0e327d3d0a
fix: do not duplicate lastEpochEnd from different servers
2021-10-24 19:17:11 -07:00
4039dbd8da
fix: combine mutations from the same version into a single MutationsAndVesionRef
2021-10-24 13:46:45 -07:00
e882eb33fc
Abstract the cluster file into a cluster connection record that can be backed by something other than the filesystem.
2021-10-22 11:05:18 -07:00
6d74239760
Track throttling by measuring time spent left in queue on the proxy
2021-10-22 13:55:01 -04:00
773886515e
Merge branch 'feature-range-feed' into blob_full_clean
2021-10-22 11:07:51 -05:00
d778cea9b5
fix: prevent infinite loop
2021-10-21 15:16:22 -07:00
f1158371a7
Merge branch 'master' of https://github.com/apple/foundationdb into feature-range-feed
...
# Conflicts:
# flow/error_definitions.h
2021-10-21 00:55:12 -07:00
4e79296a9f
fixed a few bugs with fetching change feeds
2021-10-21 00:44:51 -07:00
3ebabb6edc
fixed incorrect use of change feed errors
2021-10-20 22:37:31 -07:00
f1c8d3fbc8
Add code to disable cache when ratekeeper begins throttling
2021-10-20 15:52:43 -04:00
3f7df58a77
fixed a number of issues
2021-10-19 13:56:52 -07:00
abab45760d
Add some additional logging if the network thread finishes, fails with an error, gets stopped, or is blocked.
2021-10-19 10:05:29 -07:00
912ef76f1c
cleanup before merge
2021-10-18 17:11:14 -05:00
246f035afe
Merge remote-tracking branch 'apple-upstream/master' into version-vector-prototype
2021-10-18 19:07:33 +00:00
f75c8b25ad
add small random delay before returning a cached read version
2021-10-18 13:59:04 -04:00
e2e852e515
Mitigate transitive includes.
2021-10-18 10:49:25 -04:00
b5074fd597
Reworked all of the system data to encode granule data more efficiently for persistence
2021-10-13 16:28:04 -05:00
5f0ec0612a
Merge branch 'feature-range-feed' into blob_full
2021-10-13 15:44:35 -05:00
bde90f058b
Gate the client API.
2021-10-13 13:26:27 -04:00
e543570343
Add locality cache size to TransactionMetrics. ( #5758 )
...
Co-authored-by: Yao Xiao <yaoxiao@Yaos-MacBook-Pro.local>
2021-10-12 13:10:15 -07:00
282f84c807
added handling for broken promise
2021-10-12 10:36:10 -07:00
5066d67dec
Merge pull request #5715 from sfc-gh-ljoswiak/fixes/tracing-various
...
Add knob to control distributed trace recording percentage
2021-10-11 14:03:53 -06:00
d51edf18dc
fixed merge conflicts
2021-10-09 19:47:24 -07:00
5c642f706e
Merge branch 'master' of https://github.com/apple/foundationdb into feature-range-feed
...
# Conflicts:
# fdbcli/fdbcli.actor.cpp
2021-10-09 19:34:16 -07:00
efc4cec53f
fixed a variety of bugs with change feeds
2021-10-09 19:24:01 -07:00
4fbc193e19
fix priority
2021-10-09 16:40:24 -06:00
9077e15b11
Address review comments
2021-10-09 16:40:24 -06:00
4e723ba3d9
Apply suggestions from code review
...
Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>
2021-10-09 16:40:24 -06:00
ffdba4a133
assert we never get a stale read version
2021-10-09 16:40:24 -06:00
2b2d059039
Specify option to specifically skip grv cache for the background updater when using debug knob
2021-10-08 17:32:44 -04:00
b770936cde
move background updater actor initialization
2021-10-08 16:38:21 -04:00
ecdafba336
add debug traces to background grv updater
2021-10-08 15:17:28 -04:00
f68d018a06
conflicts
2021-10-08 12:39:39 -04:00
efd75ccc74
Add histograms to client GRV batcher. ( #5727 )
2021-10-07 13:30:25 -07:00
31664ddb78
Fixing change feed error retry in pop
2021-10-07 10:07:18 -05:00
54d5e8366d
fixing another change feed hang error
2021-10-06 16:04:01 -05:00
cc38c48123
Add debug knob to use grv cache for performance testing
2021-10-06 14:35:42 -04:00
bbe7c7ca9f
found change feed liveness bug
2021-10-06 12:32:16 -05:00
9a18cc8f41
Change debug time bound update logic to be located in masterserver
2021-10-05 17:31:31 -04:00
44a854772f
Merge branch 'master' of github.com:apple/foundationdb into jfu-grv-cache
2021-10-05 12:55:02 -04:00
ae31f01dc4
remove cache update from extractreadversion and set lower time bound to 0
2021-10-05 12:54:22 -04:00
d560eb1fea
debug time bounds using sim_validation
2021-10-04 14:12:31 -04:00
e034f66fc2
Add knob to control trace recording percentage
2021-10-01 16:30:46 -07:00
fa1fe5f08b
added blob worker rollbacks that handle (most) cases
2021-09-24 17:52:36 -05:00
a986679b7f
More bug fixes
2021-09-24 10:02:02 -05:00
32a834b0c2
The computation of tag throttling backoff was taking a minimum with a knob for how long a throttle lasts. When this knob was buggified to 0 in simulation, the backoff would then be 0, and this could cause some transactions to get stuck in a retry loop that failed to advance time. Eventually the test would timeout or potentially run out of memory.
2021-09-23 10:03:57 -07:00
4cd5d138ff
Fixed segfault involving missing futures and infinite looping in background actor
2021-09-22 13:13:35 -04:00
16171d8252
Refactoring well-known endpoint registration
...
- List all well-known endpoints of FDB in a single enum
- Identify well-known endpoints by plain IDs
2021-09-21 11:05:31 -06:00
d4f9a09a4f
- Address test failures
2021-09-17 21:21:44 +00:00
2aa3b44d4e
Merge remote-tracking branch 'apple-upstream/master' into version-vector-prototype
...
- Conflicts:
fdbserver/LogSystem.h
fdbserver/LogSystemConfig.h
fdbserver/TagPartitionedLogSystem.actor.cpp
- Files modified during merge:
modified: fdbserver/LogSystem.cpp
modified: fdbserver/LogSystemConfig.cpp
2021-09-17 19:36:18 +00:00
abf73047ca
Enforce std:: specifier rather than using namespace
2021-09-16 19:40:28 -07:00
35621a255e
- Address review comments
2021-09-16 12:00:26 -04:00
ffbdc21ca8
- Address review comments
2021-09-16 12:00:26 -04:00
852fc96200
Address simulation test failures caused by:
...
- Assertion failures in MoveKeys.actor.cpp
- Wrong results returned by getRange()
Changes:
DatabaseContext.h, NativeAPI.actor.[h,cpp]:
- Introduce a new flag, TransactionInfo::readVersionObtainedFromGrvProxy.
- Set this flag to true by default, and clear it when the read version of a
transaction is explicitly set (by using setVersion()).
- Modify getLatestCommitVersions() to not populate "latestCommitVersions" if
this flag is not set. (This will cause storage server to read at the specified
read version.)
- Modify getRange() actor to always use the specified version as the read
version (except when the specified version is latestVersion).
- Modify waitForCommittedVersion(), getRawVersion(), and getConsistentReadVersion()
to update local version vector cache after receiving GetReadVersionReply.
IClientApi.h, IConfigTransaction.h, ISingleThreadTransaction.h,
MultiVersionTransaction[.actor].[h,cpp], ThreadSafeTransaction.[h,cpp],
ApiWorkload.h:
- Add methods to get the spanID of a transaction and also the version vector
cached in a transaction. (Likely to be useful for debugging simulation test
failures.)
VersionVector.h:
- Update "maxVersion" when populating/applying a delta. (Note that empty
mutation messages only update VersionVector::maxVersion.)
BackupWorker.actor.cpp:
- Update local version vector cache after receiving GetReadVersionReply message.
Status.actor.cpp:
- Update local version vector cache and
TransactionInfo::info.readVersionObtainedFromGrvProxy after setting the
read version.
2021-09-16 12:00:26 -04:00
6673540553
TSS Metrics fix
2021-09-14 13:15:05 -05:00
87fac6e301
adjust to changed function signature after merge
2021-09-13 13:04:55 -04:00
d971b63bbf
Merge branch 'master' of github.com:apple/foundationdb into jfu-grv-cache
2021-09-13 12:47:42 -04:00
cfbe5eca9c
Use notifieddouble to track GRV request time and include grvdelay in timing calculations
2021-09-13 11:57:19 -04:00
cefb66d64c
fixing a couple bugs
2021-09-10 12:52:33 -05:00
c2d1d1704f
Merge branch 'feature-range-feed' into blob_full
2021-09-10 11:21:52 -05:00
ac5b580e2d
Merge branch 'master' into feature-range-feed
...
# Conflicts:
# fdbcli/fdbcli.actor.cpp
# fdbclient/StorageServerInterface.cpp
# fdbclient/StorageServerInterface.h
# fdbserver/ApplyMetadataMutation.cpp
# fdbserver/TLogServer.actor.cpp
# flow/error_definitions.h
2021-09-09 23:13:22 -07:00
1ce2457f7e
initial commit
2021-09-09 14:01:42 -04:00
eb76343dfb
Added blob granule reassignment and splitting
2021-09-08 14:09:14 -05:00
d9782bc29d
fixed a few bugs with merging change feeds on the client
2021-09-07 21:35:00 -07:00
f53ddf6720
fix: handle the case where I change feed is added while a fetch is in progress
2021-09-06 13:51:53 -07:00
af0792b1f5
fixes that allow change feeds to pass correctness in tests with no failures
2021-09-03 17:24:33 -07:00
b02e8d99c6
added a workload to test change feeds
2021-09-03 14:19:27 -07:00
8309c2ce26
filter mutations in change feeds to the requested range
2021-09-03 12:06:13 -07:00
afd8df4bb9
Merge commit '7f2b6c2c7ec2efcd7339e3c6531e4906322477c3' into network-thread-busyness-burstiness-score
2021-09-03 09:48:11 -07:00
4bbae59bb0
Fixed a few bugs and added a backup popping mechanism
2021-09-02 21:11:44 -07:00
27151b0831
support reads from range feeds that span multiple storage servers
2021-09-01 14:35:51 -07:00
46adada5ff
Cleaned up debugging and fixed a couple bugs
2021-08-31 12:30:43 -05:00
2fa97f60fe
Update fdbclient/NativeAPI.actor.cpp
...
Co-authored-by: Trevor Clinkenbeard <trevor.clinkenbeard@snowflake.com>
2021-08-31 08:53:37 -07:00
8689e1f106
merge with master
2021-08-30 15:29:08 -04:00
2c788c233d
apply clang-format to *.c, *.cpp, *.h, *.hpp files
2021-08-27 17:07:47 -07:00
2d6fb93316
Compute a burstiness score based on the max busyness interval in a measurement period.
2021-08-27 16:29:15 -07:00
8d49c98a41
Added simulation workload for blob granules and fixed some bugs
2021-08-26 13:48:05 -05:00
714aca4f3c
Refactored blob granule reads to actually be in NativeAPI
2021-08-25 15:01:38 -05:00
b9628bdb50
Merge branch 'master' of github.com:apple/foundationdb into add-flow-db-interface
2021-08-17 20:24:41 +00:00
53e83cf653
Remove change to Database
2021-08-17 20:10:43 +00:00
179c313e08
Use DatabaseContext instead of Database in template functions
2021-08-17 20:02:25 +00:00
925c4a935d
Log size of locationCache in client
...
We observed that multi-threaded client has more memory usage,
and we would like to understand it better.
One suspect is that this shard-to-SS cache is playing a role,
logging the size of it to get better vision.
2021-08-16 10:31:14 -07:00
a278d2977a
renamed range feeds to change feeds, data distribution support almost complete
2021-08-13 14:27:15 -07:00
e444d3781c
Various TSS improvements from snowblower testing
2021-08-13 10:24:15 -05:00
28f3ffd2ec
Fix a crash in debug mode
...
For unknown reasons, The previous implementation crashed while
(according to valgrind and ubsan) trying to interpret uninitialized memory as a
shared_ptr. This only happened when CMAKE_BUILD_TYPE=Debug. This change
works for debug mode and also still addresses this TSAN issue the
original change addressed.
2021-08-12 16:19:20 -07:00
a605fb3852
Merge pull request #5026 from sfc-gh-ljoswiak/fixes/alp6
...
Actor sampling
2021-08-11 13:44:17 -07:00
2de3a1947e
- Address an assertion failure in waitForVersion()
...
Specify the correct "readVersion" when finding the latest commit versions
of storage server replicas (which will serve the read request).
2021-08-11 15:36:35 +00:00
94239098a5
added additional rangefeed commands to fdbcli
2021-08-10 23:59:05 -07:00
7891477a7c
added support for streaming range feeds
2021-08-10 21:28:25 -07:00
618c72f7e2
Draft to use template to rewrite ThrottleApi
2021-08-11 01:07:36 +00:00
e5e8a56b66
Merge branch 'master' of https://github.com/apple/foundationdb into debugging-cleanup
...
# Conflicts:
# fdbbackup/FileDecoder.actor.cpp
2021-08-09 23:36:57 -07:00
ed28aecde0
Merge branch 'master' into feature-range-feed
2021-08-09 20:40:55 -07:00
42ae870c84
added support for querying specific range feed versions
2021-08-09 20:39:28 -07:00
ecc078a42f
Fix TSAN warnings
...
Avoid doing thread-unsafe things in the ReadYourWritesTransaction
default constructor since we now will be calling it in a foreign thread.
Here's the warning we're fixing:
Write of size 8 at 0x7b6800000398 by thread T1:
#0 N2::Net2::updateNow() /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/flow/Net2.actor.cpp:192:33 (libfdb_c.so+0xe74568)
#1 N2::Net2::run() /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/flow/Net2.actor.cpp:1459:3 (libfdb_c.so+0xe74568)
#2 runNetwork() /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/fdbclient/NativeAPI.actor.cpp:2036:13 (libfdb_c.so+0x575004)
#3 ThreadSafeApi::runNetwork() /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/../fdbclient/ThreadSafeTransaction.cpp:445:3 (libfdb_c.so+0xcda9cc)
#4 MultiVersionApi::runNetwork() /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/fdbclient/MultiVersionTransaction.actor.cpp:1741:20 (libfdb_c.so+0x4eab8d)
#5 fdb_run_network /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/../bindings/c/fdb_c.cpp:130:2 (libfdb_c.so+0x4bb425)
#6 decltype(std::__1::forward<int (*)()>(fp)()) std::__1::__invoke<int (*)()>(int (*&&)()) /usr/local/bin/../include/c++/v1/type_traits:3899:1 (fdb_c_unit_tests+0x389816)
#7 void std::__1::__thread_execute<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, int (*)()>(std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, int (*)()>&, std::__1::__tuple_indices<>) /usr/local/bin/../include/c++/v1/thread:280:5 (fdb_c_unit_tests+0x389816)
#8 void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, int (*)()> >(void*) /usr/local/bin/../include/c++/v1/thread:291:5 (fdb_c_unit_tests+0x389816)
Previous read of size 8 at 0x7b6800000398 by main thread:
#0 N2::Net2::now() const /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/flow/Net2.actor.cpp:160:39 (libfdb_c.so+0xe7ea5d)
#1 Span::Span(UID, Location, std::initializer_list<UID> const&) /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/../flow/Tracing.h:38:41 (libfdb_c.so+0x5837b4)
#2 Transaction::Transaction() /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/fdbclient/NativeAPI.actor.cpp:4092:64 (libfdb_c.so+0x5837b4)
#3 ReadYourWritesTransaction::ReadYourWritesTransaction() /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/../fdbclient/ReadYourWrites.h:134:2 (libfdb_c.so+0x7cb4a8)
#4 ISingleThreadTransaction::allocateOnForeignThread(ISingleThreadTransaction::Type) /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/../fdbclient/ISingleThreadTransaction.cpp:29:17 (libfdb_c.so+0xcf785f)
#5 ThreadSafeTransaction::ThreadSafeTransaction(DatabaseContext*, ISingleThreadTransaction::Type) /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/../fdbclient/ThreadSafeTransaction.cpp:149:23 (libfdb_c.so+0xcc9540)
#6 ThreadSafeDatabase::createTransaction() /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/../fdbclient/ThreadSafeTransaction.cpp:50:37 (libfdb_c.so+0xcc4b1c)
#7 fdb_database_create_transaction /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/../bindings/c/fdb_c.cpp:332:2 (libfdb_c.so+0x4bce1d)
#8 fdb::Transaction::Transaction(FDB_database*) /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/../bindings/c/test/unit/fdb_api.cpp:111:24 (fdb_c_unit_tests+0x3925d0)
#9 _DOCTEST_ANON_FUNC_10() /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/../bindings/c/test/unit/unit_tests.cpp:250:19 (fdb_c_unit_tests+0x32a7a8)
#10 doctest::Context::run() /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/doctest/src/doctest/doctest/doctest.h:6112:21 (fdb_c_unit_tests+0x3262e4)
#11 main /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/../bindings/c/test/unit/unit_tests.cpp:2286:20 (fdb_c_unit_tests+0x36303d)
2021-08-09 11:16:52 -07:00
bc9a0e1315
first attempt to add data distribution support for range feeds
2021-08-09 10:05:56 -07:00
2bf7b9b7a9
Remove many calls to .toString(), .printable(), and printable(...) from TraceEvent .detail() values to avoid double-escaping characters such that the same values look different in different events and can't be easily searched for. Also fixed a few warnings found by clang.
2021-08-08 00:04:54 -07:00
a081c0baa5
Merge remote-tracking branch 'apple-upstream/master' into version-vector-prototype
2021-08-05 22:40:32 +00:00
39eff8c569
Add trace_partial_file_suffix network option
2021-08-03 09:43:42 -07:00
5dc9a97230
Merge branch 'master' into fixes/alp6
2021-08-01 20:42:52 -07:00
0989c28a6b
made range feeds durable on the storage server
2021-07-30 15:23:42 -07:00
e9409b02fe
fixed the build
2021-07-28 14:02:12 -07:00
c74047c665
Merge remote-tracking branch 'origin/master' into fix-more-clang-warnings
2021-07-28 11:51:02 -07:00
9f571255bd
Merge pull request #5271 from sfc-gh-ajbeamon/do-not-reset-committed-transaction
...
Do not partially reset a transaction when it is committed or fails to commit with an error
2021-07-28 09:02:54 -07:00
406562b282
Merge branch 'master' of https://github.com/apple/foundationdb into feature-range-feed
2021-07-27 17:04:58 -07:00
3eed4084e2
Merge branch 'master' into fixes/alp6
2021-07-27 11:26:53 -07:00
59d535149e
Merge branch 'master' into fixes/alp6
2021-07-27 10:07:18 -07:00
e9a1679467
Disable sampling everywhere except fdbserver
2021-07-27 09:53:23 -07:00
507c1f11e3
Add .log() to bare TraceEvent() invocations without any .detail()s to avoid clang-tidy warning about immediate destruction of object without use.
2021-07-26 19:55:10 -07:00
34f82e7a15
Do not partially reset a transaction when it is committed or fails to commit with an error.
2021-07-26 09:51:44 -07:00
64dc1dc185
Fix -Wreorder-ctor warnings in NativeAPI.actor.cpp and several other files
2021-07-24 00:23:06 -07:00
1bb01ef591
use version vector in GetRange
2021-07-22 10:49:08 -04:00
6f81155784
Merge remote-tracking branch 'origin/master' into const-serverdbinfo
2021-07-20 10:18:40 -07:00
f596a81073
Rename ::TRUE and ::FALSE in BooleanParams to ::True and ::False so as to not conflict with the TRUE and FALSE macros provided by the Windows and MacOS SDKs.
2021-07-17 00:11:40 -07:00
81001edb2e
- Propagate version vector deltas between processes
2021-07-15 19:49:20 +00:00
b6f89df060
- Propagate the latest commit version of a storage server as part of read request.
...
Make storage server read at the specified version.
2021-07-15 19:48:42 +00:00
4f853b19a6
More const-correctness improvements for Reference<AsyncVar<T>> objects
2021-07-12 21:28:38 -07:00
15fe00927a
Merge branch 'master' of https://github.com/apple/foundationdb into feature-range-feed
...
# Conflicts:
# fdbcli/fdbcli.actor.cpp
# fdbclient/NativeAPI.actor.cpp
# fdbclient/StorageServerInterface.h
# fdbclient/SystemData.cpp
# fdbclient/SystemData.h
# fdbserver/storageserver.actor.cpp
# flow/ProtocolVersion.h
2021-07-12 19:10:14 -07:00
4059215180
Value should not be present for ENABLE_RUN_LOOP_PROFILING
...
This fixes an unintentional change from
79ff07a071
2021-07-12 16:07:56 -07:00
77cd95473d
Merge remote-tracking branch 'origin/master' into const-serverdbinfo
2021-07-12 11:00:22 -07:00
02f243c821
Merge pull request #5110 from sfc-gh-jslocum/tss_streaming_master
...
Added Streaming Read comparison for TSS
2021-07-12 10:44:15 -07:00
6c1d913ab8
Prevent masterServer from modifying db
2021-07-11 21:11:21 -07:00
baceacacf4
- Maintain a cache of the version vector on both the Grv proxy and
...
the client. Update these caches on the (transaction) read path.
2021-07-07 19:35:26 +00:00
ccffd6f47f
Fix option validation for DISABLE_CLIENT_STATISTICS_LOGGING
2021-07-04 19:06:39 -07:00
8cc40e3a2b
Expand use of BOOLEAN_PARAM
2021-07-02 21:41:50 -07:00
79ff07a071
Added *BOOLEAN_PARAM macros to enforce documentation of boolean parameters
2021-07-02 15:04:42 -07:00
9f1afbb709
Added Streaming Read comparison for TSS
2021-07-01 19:00:05 -05:00
ab0ab765c8
Merge pull request #4965 from sfc-gh-jslocum/tss_mapping_changes
...
TSS Mismatch Changes
2021-06-29 08:24:25 -07:00
73752f441b
exclude locality:clang-format, ranged loops, documentation, tracking addStoragesever for exclusion.
2021-06-23 18:03:27 -07:00
62355571d0
exclude servers based on locality match
2021-06-23 18:03:27 -07:00
244e8f3527
Merge branch 'master' of https://github.com/apple/foundationdb into feature-streaming-reply-master
...
# Conflicts:
# fdbclient/CMakeLists.txt
# fdbclient/Knobs.h
# fdbserver/Knobs.h
2021-06-21 16:35:21 -07:00
1cec223434
cherry pick streaming replies to master
2021-06-19 09:47:13 -07:00
2d248a5926
Collect actors waiting on disk and network
2021-06-17 16:50:34 -07:00
826e70916c
Merge remote-tracking branch 'origin/master' into config-db
2021-06-17 09:47:41 -07:00
805c4200ce
Add collector to get running actor name
2021-06-16 18:08:51 -07:00
fe936207a9
Replace lower priority txn request when limit is hit.
2021-06-15 14:00:06 -07:00
0ff041222a
Add removals back
2021-06-13 18:23:59 -07:00
112be7a763
Remove most work
2021-06-13 16:19:02 -07:00
56dadaa428
TSS Mismatch Changes
2021-06-11 23:13:16 +00:00
41c790b299
Merge remote-tracking branch 'origin/master' into config-db
2021-06-10 22:31:23 -07:00
c272304e60
Manage global flow knobs with global knob collection
2021-06-09 22:33:00 -07:00
c6d96c4124
Added TSS Quarantine and fixed TSS mapping bug in CommitProxy
2021-06-09 14:42:50 +00:00
838d847d4e
Merge pull request #4860 from sfc-gh-xwang/ppwtest
...
implement perpetual storage wiggling feature
2021-06-04 16:18:39 -07:00
486a04659f
Lazy inititialization
2021-06-04 15:01:18 -07:00
9b36f69b8d
Merge pull request #4892 from sfc-gh-jslocum/tss_mappingv2
...
TSS Mapping Change
2021-06-04 14:57:51 -07:00
153de33f57
Revert "Merge pull request #4802 from sfc-gh-ljoswiak/revert/actor-lineage"
...
This reverts commit 6499fa178e1512631957
2021-06-04 13:31:55 -07:00
ba25b95c6a
Fix global config not updating on server processes
2021-06-03 14:27:24 -07:00
365b0bc4d8
better api
2021-06-03 19:57:44 +00:00
89dadf1475
Create IKnobCollection interface
2021-06-02 23:40:52 -07:00
351325b3af
comment modification; wait perpetual wiggling close
2021-06-03 05:13:20 +00:00
b3e4f182ef
TSS Mapping Change
2021-06-02 17:30:09 +00:00
4c059670a0
Parse configuration knob values on the client
2021-06-02 10:04:46 -07:00
a775f92fca
Merge remote-tracking branch 'origin/master' into config-db
2021-06-01 15:39:34 -07:00
8b9c8b33fc
manually merge with master
2021-06-01 17:51:42 +00:00
d67184163b
Merge pull request #4556 from sfc-gh-jslocum/tss
...
Testing Storage Server
2021-06-01 09:11:10 -07:00
750901dd1d
Reduce the frequency that buggified reads are failed so that transactions with a lot of reads aren't doomed to almost always fail.
2021-05-27 21:54:59 -07:00
24c0c3361a
change quietdatabase
2021-05-25 21:20:24 +00:00
4257ac2b4d
More TSS Changes/Fixes
2021-05-25 20:37:48 +00:00
ce82c9653e
Testing Storage Server implementation
2021-05-25 20:28:50 +00:00
6165af81fa
Merge branch 'apple:master' into feature-range-feed
2021-05-21 15:21:34 -07:00
bb61a20851
Improve const-correctness of ISingleThreadTransaction
2021-05-15 00:23:21 -07:00
4ea760b2a9
Revert "Merge pull request #4136 from sfc-gh-mpilman/features/actor-lineage"
...
This reverts commit da41534618e6300905d6
2021-05-10 20:26:12 -07:00
331dd2e377
Replace config_transaction option with use_config_database
2021-05-10 13:18:31 -07:00
cdf98f987d
Revert "Fix global config not triggering changes on server processes"
2021-05-10 12:18:28 -07:00
71ba6f4501
Switch std::function to AsyncTrigger
2021-05-07 16:05:03 -07:00
0fc1552683
Fix global config updates not triggering on fdbserver processes
2021-05-06 15:46:34 -07:00
5c2d7b6080
Create RangeResult type alias
2021-05-03 13:14:16 -07:00
8dcd779fc4
Merge branch 'master' into features/actor-lineage
2021-05-02 14:11:42 -07:00
2621c7153a
clang format PR
2021-04-30 10:41:35 -07:00
d652287bae
Merge branch 'master' of https://github.com/apple/foundationdb into feature-range-feed
...
# Conflicts:
# fdbclient/DatabaseContext.h
# fdbclient/NativeAPI.actor.cpp
# fdbclient/StorageServerInterface.h
# fdbserver/storageserver.actor.cpp
2021-04-30 10:29:42 -07:00
f513543305
fix special keyspace register
2021-04-28 16:35:09 -06:00
904a39e473
Merge pull request #4667 from sfc-gh-ajbeamon/feature-mvc-monitor-protocol-version
...
Use fewer connections in the multi-version client
2021-04-28 14:13:17 -07:00
f5b0008346
Merge branch 'features/actor-lineage' into features/actor-lineage-fluentd
2021-04-28 09:31:20 -06:00
54919d4f3b
Merge remote-tracking branch 'sfc/features/actor-lineage' into features/actor-lineage
2021-04-28 09:22:14 -06:00
4fab2ecd30
Merge remote-tracking branch 'origin/master' into features/actor-lineage
2021-04-28 09:20:54 -06:00
2d6fafde64
Implemented configuration
2021-04-27 10:26:42 -06:00
823873a9aa
Address review comments:
...
Use nullptr instead of NULL
Use const& for a parameter
Add some comments
2021-04-26 14:39:27 -07:00
a794fca932
Support 5.0 (and earlier) client versions by adding GRV probing for old versions. Update the C bindings implementation of get_server_protocol to convert the ProtocolVersion object into a uint64_t. Rename a misleading protocol version alias.
2021-04-23 15:00:21 -07:00
52bba82e8e
Add window size configuration key
2021-04-23 14:05:05 -07:00
a6a8a97e1f
Merge remote-tracking branch 'sfc/features/actor-lineage' into features/actor-lineage-instrument
2021-04-22 17:50:36 -06:00
15336ca274
Add callback for specific global configuration key changes
2021-04-20 17:51:38 -07:00
eaaae2e16d
Merge branch master into 'feature-mvc-monitor-protocol-version'
2021-04-20 15:07:02 -07:00
d76b32da18
Annotate read paths on the server side
2021-04-20 15:10:01 -06:00
b34825a0e6
Merge branch 'master' of github.com:apple/foundationdb into add-dd-and-maintenance
2021-04-19 14:52:10 -07:00
51ce278d2d
Merge branch 'features/actor-lineage' into features/actor-lineage-collector
2021-04-19 12:55:29 -06:00
09ddcb3bae
remove old sample thread
2021-04-19 11:55:35 -06:00
7307750e5e
Merge remote-tracking branch 'origin/master' into features/actor-lineage
2021-04-19 11:29:52 -06:00
c8e9e2fab4
Merge remote-tracking branch 'sfc/features/actor-lineage' into features/actor-lineage-collector
2021-04-16 17:33:40 -06:00
551268b0f2
Add well known endpoint for worker communication
2021-04-15 13:50:50 -07:00
486260e944
Fix infinite loop with stable interface protocol monitoring. Fix case where getting an error with a network option didn't properly terminate the database connection. Reduce option lock critical section.
2021-04-15 13:36:31 -07:00
b2d6930103
The multi-version client monitors the cluster's protocol version and only activates the client library that can connect.
2021-04-15 11:45:14 -07:00
d51f94f16e
Merge pull request #4330 from sfc-gh-ljoswiak/features/global-configuration
...
Add global configuration framework implementation
2021-04-14 16:10:08 -06:00
3ed0d614d2
Move fdb_get_server_protocol to be a function on the database object. Add an argument for expected_version that can be used to signal that the function shouldn't return unless the protocol version is different.
2021-04-14 12:50:30 -07:00
7de23918c0
Add comments, fix erase bug, make optimizations
2021-04-14 10:56:33 -07:00
6de28dd916
clang-format
2021-04-14 10:56:33 -07:00
4a799baa1d
Add clear range for global configuration
2021-04-14 10:56:32 -07:00
c3f68831af
Move existing ClientDBInfo variables to global configuration
2021-04-14 10:56:32 -07:00
f1415412f1
Add global configuration framework implementation
2021-04-14 10:56:32 -07:00
eab468fecc
Remove extra line caused by commit issue
2021-04-14 09:32:48 -07:00
1d362d8869
Merge remote-tracking branch 'origin/master' into features/actor-lineage
2021-04-14 09:55:04 -06:00
feede1d2f6
Fix line length of test macro + comments to be within the 120 character limit
2021-04-13 10:48:52 -07:00
ed8d43cb87
Merge pull request #5 from sfc-gh-ljoswiak/features/network-actors
...
Sample actors waiting on network
2021-04-09 15:07:53 -06:00
5c79d29140
Use object lifetimes instead of function calls
2021-04-07 10:59:45 -07:00
c481ba2cfa
Update annotation class name
2021-04-06 17:32:02 -07:00
433872e17d
Sample actors waiting on network
2021-04-06 17:28:28 -07:00
747fa2a353
Merge branch 'master' of github.com:apple/foundationdb into add-dd-and-maintenance
2021-03-26 12:22:37 -07:00
0eff74f205
Add special keys for maintenance and datadistribution
2021-03-26 12:19:33 -07:00
de1c354b12
address pr comments
2021-03-23 20:56:37 +00:00
73571ad52e
Merge branch 'master' of github.com:sfc-gh-nwijetunga/foundationdb into network_busyness
...
* 'master' of github.com:sfc-gh-nwijetunga/foundationdb: (55 commits)
Port lost changes from #4004
Fix markdown
Factor our refreshTransaction gadget
Align FastAllocator memory to 4096 for size >= 4096
Fix heap-use-after-free's
Disable lsan in joshua
Manually apply changes made to BackupContainer.actor.cpp on release-6.3 to the new files where that code is located.
add release branch instructions
Fix PR number in release note link
fixed a simulation bug where a process on an unreliable machine would be considered reliable by the simulator
Add tini as PID 1 to docker image (#4363 )
Fix typo
Fix broken_promise bug
Explain FastAllocator<4096> special case
Inform LSAN that we have unaligned pointers
apply clang-format to flow/Platform.h
Fix typos
Address review comments
Change the macro that forbids exit() calls to be a static assertion
Add versionstamp serialization information
...
2021-03-18 18:58:43 +00:00
822cbf895d
address pr comments
2021-03-16 23:29:02 +00:00
bf5f83d2bf
address pr comments
2021-03-16 21:49:36 +00:00
5f079904d4
add unit test
2021-03-16 18:30:39 +00:00
17f9da6962
remove traces
2021-03-16 16:47:50 +00:00
191230a7b3
Factor our refreshTransaction gadget
2021-03-16 00:51:20 +00:00
f0d79b3d86
Inital implementation of network busyness
2021-03-15 23:23:56 +00:00
d45481decc
Fix broken_promise bug
2021-03-13 01:16:27 +00:00
6cacf3e002
Fix potential heap-use-after-free
...
If we destroy tr here then we might not be able to dereference |self|.
Keep the window where |self| might have been deleted small.
2021-03-12 00:26:26 +00:00
a76c7b9754
Fix several memory leaks and a thread leak
2021-03-11 23:58:32 +00:00
2c316db94d
Merge remote-tracking branch 'apple/master' into master-format-final
...
Also format the changes
2021-03-11 12:54:26 -08:00
be980ebb55
Merge pull request #4354 from sfc-gh-clin/add-advanceversion
...
Add advanceversion into special keys
2021-03-11 11:43:53 -08:00
d7252da951
clang-format: Fix the TEST() macros which require comments in line
2021-03-10 16:50:53 -08:00
7404a29d58
Merge branch 'master' of github.com:apple/foundationdb into add-dd-and-maintenance
2021-03-10 11:47:39 -08:00
df90cc89de
apply clang-format to *.c, *.cpp, *.h, *.hpp files
2021-03-10 10:18:07 -08:00
6cfba6e54b
added support for popping rangefeeds
2021-03-05 11:46:33 -08:00
21822e4949
more fixes
2021-03-04 19:46:32 -08:00
adaa67df63
fix registerRangeFeedActor
2021-03-04 19:42:36 -08:00
bc757f3e4b
basic framework for range feed support
2021-03-04 19:14:54 -08:00
79cec09255
Apply clang-tidy's performance-inefficient-vector-operation fix
...
I ran this command in my build directory after compiling with
OPEN_FOR_IDE. It took a few small tweaks to get it to compile, which is
outside the scope of this commit.
$ python run-clang-tidy.py -j $(nproc) -checks='-*,performance-inefficient-vector-operation' -fix
2021-03-04 03:58:25 +00:00
9ce9cba0e5
Merge branch 'master' of github.com:apple/foundationdb into add-advanceversion
2021-02-19 14:29:08 -08:00
703107332a
Add special keys for profile client get/set
2021-02-19 12:22:00 -08:00
9fb396e372
Merge pull request #4255 from sfc-gh-clin/add-coordinators-into-special-keys
...
Add coordinators into special keys
2021-02-18 18:26:07 -08:00
fa3834f115
fix bug
2021-02-18 18:23:28 -05:00
ac860b3c22
Add advanceversion into special keys
2021-02-18 01:27:14 -08:00
c075415a51
Merge pull request #4344 from sfc-gh-ljoswiak/fixes/remove-async-tracer
...
Remove async tracer
2021-02-17 09:43:24 -08:00
f3a849de10
Update ChangeConfig test for auto coordinators change
2021-02-16 23:55:58 -08:00
30d2d9ff82
Remove async tracer
2021-02-16 16:36:33 -08:00
82cda5448d
Merge pull request #4266 from sfc-gh-nwijetunga/optimize-watches-client
...
Optimize Watches Client
2021-02-16 13:28:51 -07:00
a59f07e6fe
address pr comment
2021-02-16 13:32:03 -05:00
13e4f6d7e1
small formatting fix
2021-02-12 12:57:33 -05:00
058557a3b1
fix pr issues
2021-02-11 15:51:37 -05:00
270a18f089
formatting fixes
2021-02-10 13:19:59 -05:00
7acfeda955
cleanup
2021-02-09 20:57:16 -05:00
386e3b0838
minor change
2021-02-09 18:36:45 -05:00
4982679ef2
change function name
2021-02-09 18:06:36 -05:00
3b48942f9a
resolve pr comments
2021-02-09 17:56:30 -05:00
d2233ffa5a
whitespace change
2021-02-09 13:38:24 -05:00
82e9f54938
small change
2021-02-09 13:36:44 -05:00
2d46cfcb80
remove traces
2021-02-09 13:33:19 -05:00
2560dc0227
add test workload
2021-02-09 12:32:37 -05:00
90686a95a0
format code
2021-02-08 22:17:16 -08:00
61082e70e1
format code, add comments
2021-02-08 19:11:30 -08:00
83af53b1bb
correctness fixes
2021-02-05 19:23:37 -05:00
a85458dd1e
Merge branch 'master' of github.com:apple/foundationdb into add-c-function-snapshot
2021-02-05 14:14:52 -08:00
97925de9d4
Update fdb_database_create_snapshot to take a UID as input
2021-02-05 14:14:38 -08:00
6c687067a6
refactor
2021-02-04 12:51:53 -05:00
ec6226bde9
change logic
2021-02-03 20:01:26 -05:00
b6e7961b78
change to references
2021-02-02 20:29:03 -05:00
c67aac8212
remove the use of pointers
2021-02-02 13:26:12 -05:00
6f38105e48
fix issue with case 4
2021-02-01 19:07:14 -05:00
6bee8ac0aa
Inital implementation
2021-02-01 18:42:13 -05:00
c2aedb0b94
clang-format, remove comments
2021-01-29 18:20:09 -08:00
7d32045506
Merge branch 'master' of github.com:apple/foundationdb into add-coordinators-into-special-keys
2021-01-29 11:46:01 -08:00
e1182f2f86
Add coordinators to special keys
2021-01-29 11:45:52 -08:00
5b5dd0a0d4
Merge pull request #4204 from sfc-gh-ljoswiak/features/network-tracer
...
Add network option to set tracer in fdbclient
2021-01-27 20:00:49 -08:00
5f912f0deb
Add snapshot c function
2021-01-26 11:53:35 -08:00
7558cd8e06
Merge remote-tracking branch 'origin/master' into simplify-global-knobs
2021-01-26 10:00:25 -08:00
a51de2f73d
Merge pull request #4220 from sfc-gh-clin/add-c-function-force-recovery
...
Add c function fdb_database_force_recovery_with_data_loss
2021-01-25 18:22:41 -08:00
29c0e05ea6
Clarify naming of distributed tracing options
2021-01-22 17:30:33 -08:00
500eabf6dd
Add network option to set tracer in fdbclient
2021-01-21 12:04:28 -08:00
fb9b4b7626
Merge remote-tracking branch 'origin/master' into misc-changes
2021-01-21 10:45:50 -08:00
ad99bf0471
Merge remote-tracking branch 'origin' into misc-changes
2021-01-21 10:03:07 -08:00
9cd7c43a2c
Add c function fdb_database_force_recovery_with_data_loss
2021-01-21 00:42:23 -08:00
fd20011276
Remove tags
2021-01-20 17:05:58 -08:00
4d6fbf099c
Use StringRef for span tags
2021-01-20 13:57:36 -08:00
7ae5e98c5a
Add metadata to a few spans
2021-01-20 13:57:36 -08:00
b0f61fb74f
Resolve the simple-looking conflicts
2021-01-15 19:35:10 +00:00
ff7d306b09
Merge branch 'release-6.3' into anoyes/merge-6.3-to-master
...
Include conflict markers for now. Will resolve.
2021-01-15 18:04:09 +00:00
8ce9bd4e8e
Merge branch 'master' of github.com:apple/foundationdb into add-c-function-for-management-commands
2021-01-14 17:31:26 -08:00
e32a0fb742
Resolve conflicts
2021-01-14 20:13:29 +00:00
6d6b2843da
Merge remote-tracking branch 'upstream/release-6.2' into anoyes/merge-release-6.2
...
I'm including the conflict markers for now and I'll resolve them in a
subsequent commit
2021-01-14 17:20:11 +00:00
87b4463305
Solve comments: update c api description; refine code
2021-01-13 14:56:41 -08:00
8ff14878fe
Merge remote-tracking branch 'origin/master' into simplify-global-knobs
2021-01-13 14:39:35 -08:00
58360a192f
Add a counter for the number of status requests made by a client
2021-01-05 10:18:43 -08:00
e097c98956
Remove headed added by IDE, add comments
2021-01-04 00:18:01 -08:00
da7b65cd71
Change the interface back to return int64_t
2021-01-03 22:45:09 -08:00
5b2e88b187
Use structured bindings in for loops
2020-12-27 01:46:20 -04:00
0ecfaf138e
Merge pull request #4150 from sfc-gh-ljoswiak/fixes/sps-tracing
...
Remove "a/" from tracing keys in special-key-space
2020-12-21 13:48:01 -08:00
9db9a84798
Ignore negative duration value
2020-12-21 13:29:16 -08:00
b68d84aaea
Merge remote-tracking branch 'upstream/master' into add-c-function-for-management-commands
2020-12-21 12:32:31 -08:00
7b4f7ce4e0
update comments
2020-12-20 22:47:55 -08:00
9601769b01
Merge pull request #3858 from sfc-gh-rchen/stable_interfaces
...
Stable interfaces
2020-12-11 09:34:27 -08:00
5f57d72a59
remove print statements and format protocol version workload
2020-12-11 04:46:20 +00:00
4cb821cd63
Merge remote-tracking branch 'upstream/release-6.2' into merge-6.2-to-6.3
2020-12-09 15:47:44 -08:00
Josh Slocum