90caa24af1
add function to get correct checksum class. Remove IChecksum::checkSum function
2020-10-23 02:24:29 +00:00
c8eb003ec4
Change to \xff\xff/management/in_progress_exclusion, the snake case naming style
2020-10-22 18:05:54 -07:00
c8b3b37b4a
Remove unused code for #3930
2020-10-22 17:04:08 -07:00
95654ce7e7
Change dbLocked to snake_case naming db_locked
2020-10-22 11:35:39 -07:00
2b4ec39a2f
Avoid passing references from an iterator into an insertion that invalidates the iterator.
2020-10-22 11:30:08 -07:00
1d64b2c335
clang-format
2020-10-22 11:28:03 -07:00
b728fe473e
Add consistencycheck command to special keys
2020-10-22 11:08:54 -07:00
8d937a56f2
Throw operation_failed() if dd not found when calling ddMetricsRequestServer
...
Remove time_out error in DDMetricsList rpc
Add retry logic for dd special key when dd not found
Update FuzzApiCorrectness test
2020-10-21 15:28:03 -07:00
8cc3e4d3c6
Merge release-6.3 into master
2020-10-19 22:51:56 -07:00
c84a5f4014
change IChecksum interface and remove coordinator protocol version optional
2020-10-19 21:39:07 +00:00
a4ccf8c5b6
add checksum class to change checksum in the future. expose get server protocol api
2020-10-19 16:57:11 +00:00
24ea35e56f
Merge pull request #3748 from sfc-gh-ljoswiak/visibility-2
...
Add TLogVersion::V6
2020-10-14 17:35:32 -07:00
bbc6d2aa03
Merge pull request #3872 from sfc-gh-clin:add-lock-unlock-to-special-keys
...
Add lock/unlock to special keys
2020-10-13 09:09:49 -07:00
5d902f9177
Merge pull request #3394 from dongxinEric/feature/range-split-points-based-on-size
...
Feature/range split points based on size
2020-10-08 14:25:11 -07:00
c66a775d64
Merge remote-tracking branch 'upstream/master' into add-lock-unlock-to-special-keys
2020-10-08 14:23:16 -07:00
53d3759e73
Add lock command to special key \xff\xff/dbLocked
2020-10-08 14:23:02 -07:00
f03a355c4e
Merge branch 'release-6.2' into release-6.3
...
# Conflicts:
# CMakeLists.txt
# documentation/sphinx/source/downloads.rst
# fdbrpc/FlowTransport.actor.cpp
# fdbrpc/FlowTransport.h
# flow/Arena.h
# flow/Knobs.cpp
# flow/Knobs.h
# packaging/msi/FDBInstaller.wxs
2020-10-07 12:28:22 -07:00
dea7000970
Merge remote-tracking branch 'upstream/master' into visibility-1
2020-10-06 18:38:15 -07:00
76da0a7cbc
Backport #3426 to release-6.2
2020-10-05 13:53:51 -07:00
b294e148c5
another compile fix
2020-10-04 16:54:43 -07:00
614c8bc895
Get read versions requests must be load balanced on the number of requests because ratekeeper gives out an equal budget to each proxy
2020-10-04 16:20:24 -07:00
bbc7ce581e
Resolve conflicts merging from 6.3 to master
2020-09-25 16:08:31 -07:00
862336de8f
Merge branch 'master' into mengxu/merge-to-master-PR
2020-09-24 17:06:00 -07:00
b37daf88d4
add jitter to the delay
2020-09-22 22:24:40 -07:00
09ff05c629
backed out buggify of apiVersion because there are too many tests that depend on it for a patch release
2020-09-22 21:15:49 -07:00
e0fbadfeb5
Added a buggify so older API versions are run in simulation
2020-09-22 18:58:01 -07:00
c99627a7ab
Merge branch 'release-6.3' of github.com:apple/foundationdb into feature-apiverison-batch-error
2020-09-22 17:06:17 -07:00
7dec47938e
protect api versions older than 630 from seeing batch_transaction_throttled errors
2020-09-22 17:05:09 -07:00
b4e209bd4e
Replace NULL with nullptr in fdbclient
2020-09-20 11:33:09 -07:00
a52ef0acb8
Merge pull request #3677 from sfc-gh-clin/add-setclass-to-special-key-space
...
Add setclass to special keys
2020-09-19 14:55:03 -07:00
71309a79f9
Merge branch 'release-6.3' into range
2020-09-17 17:54:33 -07:00
cc5bc16bd8
Rename more places from proxy to commit proxy
2020-09-15 22:29:49 -07:00
3dfadff6d8
Merge remote-tracking branch 'upstream/master' into add-c-function-for-management-commands
2020-09-14 16:45:47 -07:00
cc32074ed7
Rename actor
2020-09-14 16:45:34 -07:00
35bef73a1c
Rename proxy to commit proxy
2020-09-10 17:44:15 -07:00
d270b15c89
Merge remote-tracking branch 'upstream/master' into add-setclass-to-special-key-space
2020-09-09 16:21:19 -07:00
1ca7fe1a05
Add span metadata message
2020-09-04 15:36:47 -07:00
ae7bf24353
Merge pull request #3549 from yliucode/grv-proxy
...
Separate out a new role GrvProxy to serve GRVs.
2020-09-03 19:03:45 -07:00
f9624b5a1a
Avoid unnecessary copy
2020-09-03 13:56:23 -07:00
b4c96cadc7
Merge branch 'release-6.3' into merge-release-6.3-into-master
2020-09-02 12:45:57 -07:00
93da4957ad
Update fdbclient/NativeAPI.actor.cpp
2020-09-01 13:17:01 -07:00
800ece286d
Results from read hot range requests were not held by the results arena.
2020-09-01 13:14:27 -07:00
fd7198d874
Extend backup container interface to support query restorable files set by key ranges
2020-08-29 19:58:07 -07:00
e87327b33b
Merge master branch and keep master proxy reporting txn cost estimation to ratekeeper
2020-08-29 12:47:35 -07:00
cd86ca6850
Merge branch 'master' into feature/range-split-points-based-on-size
2020-08-27 10:06:02 -07:00
e8e24a8ed6
Resolve comments
2020-08-26 21:04:35 -07:00
a07b9f234d
Add readonly range \xff\xff/configuration/process/class_source, and change \xff\xff/configuration/class/ to \xff\xff/configuration/process/class_type/
2020-08-25 18:18:32 -07:00
63b3612ad5
Merge master branch and resolve conflicts
2020-08-24 16:42:31 -07:00
3afdb44c7a
merge master
2020-08-23 17:09:04 +00:00
b4cec6577a
Fix merge bugs
2020-08-22 15:04:42 -07:00
e1b7dd0c7d
Merge remote-tracking branch 'upstream/release-6.3' into dyoungworth/fixMerge1
2020-08-22 12:25:19 -07:00
23ef23357c
Test code to add c function for kill/suspend
2020-08-21 14:30:52 -07:00
9b096ab58e
Merge branch 'master' of github.com:apple/foundationdb into add-c-function-for-management-commands
2020-08-21 12:20:35 -07:00
3b63d8b01b
remove FIXME; remote tagSet.reset(); trivial changes
2020-08-21 19:17:16 +00:00
5aaf86206a
add counter for expensive clear estimation for audit and trace
2020-08-20 23:00:45 +00:00
556b239057
clang-format
2020-08-20 14:51:41 -07:00
bff20faa42
Merge remote-tracking branch 'upstream/master' into add-setclass-to-special-key-space
2020-08-20 13:50:54 -07:00
e375761581
Merge branch 'master' of https://github.com/apple/foundationdb
2020-08-20 20:29:14 +00:00
9c709b33d6
scale cost
2020-08-19 19:47:41 +00:00
599675cba8
modify some details to get better performance
2020-08-19 04:23:23 +00:00
7b210aee24
Use =delete to disable special member functions
...
This has the benefits of:
- Easier to understand compiler error messages
- Moves some errors from link-time to compile-time
- NonCopyable classes can still specify default move constructors
2020-08-18 11:07:00 -07:00
24dbb3961c
Knob adjustment
2020-08-17 22:38:22 +00:00
130bd7ac22
Fixed a bug that would potentially cause invalid memory access, i.e. segmentation fault.
2020-08-17 13:16:17 -07:00
daead84fdd
Merge master
2020-08-14 14:07:08 -07:00
548ba06510
change map to deque; code style problem
2020-08-14 18:30:58 +00:00
475d4cca72
solve some cr comments
2020-08-14 16:40:16 +00:00
b9474055a5
Add \xff\xff/configuration/class to read process class
2020-08-14 01:01:08 -07:00
e94de126de
merge with master
2020-08-14 01:59:09 +00:00
20700f75f4
Merge master
2020-08-13 14:28:27 -07:00
b2bba8ccbb
Correct fix for debug ID logging
2020-08-13 13:37:15 -04:00
002cca5e58
Merge branch 'master' into tmp/merge-PR
...
Conflicts:
fdbclient/NativeAPI.actor.cpp
fdbserver/StorageMetrics.actor.h
2020-08-12 20:08:13 -07:00
8bc57c16f0
Merge pull request #3637 from dongxinEric/misc/fix-spammy-readhot-logging
...
Increased some knobs to throttle the spammy read hot logging.
2020-08-12 16:46:32 -07:00
8f1deaa47e
Merge branch 'master' of https://github.com/apple/foundationdb
2020-08-12 23:32:23 +00:00
3ac6996844
Increased some knobs to throttle the spammy read hot logging. Also added more details inside the read hot log to make it useful
2020-08-12 14:27:34 -07:00
cc1059be75
Make logging of server trace ID consistent with other code
2020-08-12 19:05:01 +00:00
f3ecf14601
change midShardSize type and other details
2020-08-12 17:49:12 +00:00
16ef2bd3bd
Pending commit
2020-08-12 10:34:07 -07:00
79ce16650d
merge master branch
2020-08-11 19:22:10 -07:00
4782c7c4b1
Merge remote-tracking branch 'origin/master' into features/udp
2020-08-11 15:35:52 -06:00
b66dab8816
Fix span early destruction
...
We want the span to stick around for the full duration of
extractReadVersion since it writes to the log from its destructor.
2020-08-11 11:19:04 -07:00
c0b7ac0b7d
try to throttle write tag and read tag seperatedly
2020-08-10 23:43:58 +00:00
cae6f04070
sample on cost instead of bytes
2020-08-10 22:30:06 +00:00
696e77c94e
query midShardSize from proxy
2020-08-10 20:13:44 +00:00
df9149fea4
ignore transaction tag of immediate transactions
2020-08-07 23:36:17 +00:00
98c39ad9d4
Fix the build error.
2020-08-07 14:03:42 -07:00
a46e620baa
Merge branch 'master' into feature/range-split-points-based-on-size
2020-08-06 15:23:11 -07:00
6d84bcb568
Merge remote-tracking branch 'origin/master' into features/udp
2020-08-06 14:08:34 -06:00
8976694ba1
UDP implementation (untested)
2020-08-06 14:06:50 -06:00
ba66b1f668
trivial changes
2020-08-06 14:53:26 +00:00
d6a23a4d6b
Resolve comments to make GRV proxy a separate process class
2020-08-06 00:01:57 -07:00
13307679c5
use median shard size"
2020-08-05 03:57:25 +00:00
4997754cf8
fix format in monitorDDMetricsChanges
2020-08-04 04:14:11 +00:00
00f2c38e3a
Merge remote-tracking branch 'upstream/master' into add-c-function-for-management-commands
2020-08-03 20:55:21 -07:00
48094ea17e
add INCOMPLETE_SHARD_PLUS knob
2020-08-04 03:18:05 +00:00
52073738c0
fix bugs; change some KNOB values
2020-08-03 22:18:34 +00:00
b903e60cb7
fix monitorDDMetricsChanges bugs
2020-08-03 17:12:36 +00:00
6db7040f17
update DDMetrics periodically
2020-08-03 05:52:38 +00:00
d1cc87452c
merge with master; solve conflicts; solve initialization;
2020-08-02 22:44:07 +00:00
92c1112c74
consider clear single key
2020-08-01 18:20:13 +00:00
4f7dab4951
sample clear op on client
2020-08-01 06:14:52 +00:00
fc3dfc170f
Merge branch 'master' of https://github.com/apple/foundationdb
2020-07-31 19:09:33 +00:00
c3a629588f
add client transaction tag sample
2020-07-31 19:08:42 +00:00
a048e5e159
Change prefix to \xff\xff/management and make decode/encode as pure functions
2020-07-31 02:18:49 -07:00
f2662a9a74
Merge remote-tracking branch 'upstream/master' into add-management-api-into-special-key-space
2020-07-30 19:01:11 -07:00
10540a3c73
Change error name and fix a bug in ExclusionInProgressRangeImpl::getRange()
2020-07-30 18:58:09 -07:00
11cbcffc9a
Temporary code
2020-07-30 16:07:54 -07:00
b5dc6b715f
Add file comment for GrvProxyServer.actor.cpp
2020-07-29 12:10:53 -07:00
30ea639666
Remove debug traces
2020-07-29 07:55:05 -07:00
f7b76a92af
pass joshua
2020-07-29 07:26:55 -07:00
602b0f9dea
Merge branch 'master' into vectorref-emplace-back
2020-07-28 17:17:11 -07:00
a641429d63
Add an enum for implementation type and use for registeration
2020-07-27 12:29:03 -07:00
ec9553b6c6
change \xff\xff/failure to \xff\xff/error_message
2020-07-24 11:38:56 -07:00
525f10e30c
Merge master branch
2020-07-22 16:08:49 -07:00
302cf5c45f
Remove debug trace events
2020-07-22 12:20:22 -07:00
2703cedac5
Fixed known bugs
2020-07-17 22:24:52 -07:00
0df2a8d014
better code style
2020-07-18 01:48:58 +00:00
1c51aaeea5
Use static keys in SpecialKeySpace
2020-07-17 15:14:18 -07:00
96e5a4c827
Replace all \xff\xff/conf with managementApiRange.begin
2020-07-17 11:56:27 -07:00
5fa6785fb8
Add special keys for command options
2020-07-16 18:15:35 -07:00
9d0d189cc8
better serialize; TransactionOption::clear patch
2020-07-15 22:39:21 +00:00
eb44ae0e86
finish local shard estimation
2020-07-15 16:08:00 +00:00
5b06d69d25
Pass watches test
2020-07-15 00:37:41 -07:00
bcb858288b
eliminate cost estimation when tagSet is empty
2020-07-14 23:07:21 +00:00
3f4da830a2
Merge remote-tracking branch 'upstream/master' into add-management-api-into-special-key-space
2020-07-14 10:36:47 -07:00
a310faf9d1
solve some code reviews
2020-07-14 17:19:55 +00:00
d512170cd8
add clear cost estimation
2020-07-14 00:18:52 +00:00
60d1bfc247
Add read impl for inProgressExclusionServers
2020-07-13 13:35:25 -07:00
6ec2f92a8d
merge master
2020-07-13 17:22:37 +00:00
6374cd96c9
add commit cost estimation
2020-07-10 23:07:29 +00:00
2e5ae8eed5
Change iterator name to small case
2020-07-10 10:56:41 -07:00
cc60597739
Merge branch 'master' into bugfixes/visibility-perf-regression
2020-07-09 17:04:04 -06:00
69864c9f96
Make Spans not allocate heap memory
2020-07-09 11:49:33 -06:00
1b68665228
Merge branch 'master' into release-6.3
2020-07-08 18:52:05 -07:00
3eb3e23ae0
Merge pull request #3425 from sfc-gh-anoyes/anoyes/health-metrics
...
Expose health metrics via special keys
2020-07-08 10:37:31 -07:00
0fbe7101c3
Revert "Revert "Request tracing""
...
This reverts commit 327cc31e35
2020-07-07 10:06:13 -06:00
a43afb9d97
Merge remote-tracking branch 'upstream/master' into add-management-api-into-special-key-space
2020-07-06 15:47:00 -07:00
0361eb5079
Add FailedServersImpl
2020-07-06 15:40:21 -07:00
784d0f6d00
A basic feasible version for exclude and include commands
2020-07-06 11:02:48 -07:00
bcbca81dfe
Remove ROW_LIMIT_UNLIMITED and BYTE_LIMIT_UNLIMITED from CLIENT_KNOBS
2020-07-04 12:48:44 -07:00
d689d34634
Enable range split bindings tests
2020-07-02 14:05:12 -07:00
00e9f8b9bf
Added bindings tests; Protected new SSI endpoints under new 7.0 ProtocolVersion
2020-07-02 14:05:12 -07:00
eb28492900
Fixed a bug which leads to inaccurate metrics being reported. Added tests for the new API.
2020-07-02 14:05:11 -07:00
ea08bc5462
Added server side code for range split support
2020-07-02 14:05:11 -07:00
ae63304392
Address Meng's comment
2020-07-01 20:23:18 -07:00
3ecdaf24ce
Use the actual range begin and end key instead of the shard begin and shard end key
2020-07-01 16:07:59 -07:00
409ccf3be2
Use snake_case to match status json convention
2020-06-30 17:05:29 +00:00
e1dfa410c1
Remove "Storages" field from data_distribution_stats
...
It seems like a cool idea, but it feels rushed to me. Reverting for now.
2020-06-30 15:24:16 +00:00
a55b0771c5
Test schemas
...
Also remove an ASSERT_WE_THINK that doesn't seem to hold now that we
include shards
2020-06-30 15:24:16 +00:00
f8bb76a607
Add CLIENT_BUGGIFY
2020-06-30 15:24:16 +00:00
b1d5e99494
Update documentation
2020-06-30 15:24:16 +00:00
1c30adce74
WIP
2020-06-30 15:24:16 +00:00
0db3145e82
Include storage id's in data_distribution_stats
2020-06-30 15:24:15 +00:00
e304697eca
WIP
2020-06-30 15:24:15 +00:00
ffd1467367
Add \xff\xff/metrics/health/*
2020-06-30 15:24:15 +00:00
e2c5938c28
Update module registration for readwrite module
2020-06-29 15:37:46 -07:00
1082ab2caa
fix ClientVersionRef versionString should be passed by ref
2020-06-29 21:23:56 +02:00
18ff6efbf3
Add failure module for special key space
2020-06-27 12:22:32 -07:00
f011ff2c66
Merge remote-tracking branch 'upstream/master' into add-management-api-into-special-key-space
2020-06-23 11:23:18 -07:00
42159ccfe0
Add -Wpessimizing-move and -Wredundant-move for clang
2020-06-22 23:45:03 +00:00
59361cedaa
Rename to have read and write impls and add set, clear, clearrange interfaces
2020-06-22 14:02:03 -07:00
85a284b13c
Use VectorRef::emplace_back in several places
2020-06-19 23:02:28 -07:00
327cc31e35
Revert "Request tracing"
2020-06-16 12:32:42 -07:00
96206a8032
Merge pull request #3368 from apple/release-6.3
...
Merge Release 6.3 to master
2020-06-15 20:15:22 -07:00
fabdffd6e4
Merge pull request #3332 from sfc-gh-tclinkenbeard/fix-memaccess-warnings
...
Fix class-memaccess warnings
2020-06-15 15:22:19 -07:00
3a914c71f2
Add TransactionOptions::clear function
2020-06-13 17:17:06 -07:00
8595813b7d
Merge pull request #3355 from apple/release-6.3
...
Merge Release 6.3 into master branch
2020-06-12 20:08:47 -07:00
80334351c7
Merge pull request #3329 from sfc-gh-mpilman/features/flatbuffers-debugtx
...
Request tracing
2020-06-12 14:50:19 -07:00
ae6b716820
addressed more review comments
2020-06-11 09:58:41 -07:00
9ef93714ac
Address review comments
2020-06-10 15:48:49 -07:00
4ab3441a95
Merge remote-tracking branch 'origin/master' into features/flatbuffers-debugtx
2020-06-10 15:31:29 -07:00
d9ddb84d34
remove weird include (added by clangd)
2020-06-10 13:18:26 -07:00
a332dff2ec
Fix class-memaccess warnings
2020-06-10 12:34:14 -07:00
99bf993815
Replace BOOST_NOEXCEPT with noexcept
2020-06-09 22:39:19 -07:00
c829b12bbf
defined spans for read requests
2020-06-08 17:26:15 -07:00
09c136e434
Framework for transaction tracing
2020-06-08 16:09:37 -07:00
55448a8d5a
Remove call in watch actor instead of Transaction::watch
2020-06-08 18:19:18 +00:00
731b77a8c2
Don't double count watches for too_many_watches
2020-06-07 23:54:22 +00:00
23a565ec63
Few bug fixes.
2020-06-05 16:27:04 -07:00
cf13e00a8f
Merge remote-tracking branch 'origin/release-6.3' into fdb_cache_wo_allocator
2020-06-01 17:38:31 -07:00
ba63a66d12
Merge remote-tracking branch 'origin/master' into features/flatbuffers-debugtx
2020-05-29 15:38:09 -07:00
8a7d98ab21
Started instrumentation
2020-05-27 15:36:04 -07:00
86f712657e
Eliminate the undefined behavior of calling run_network twice, instead returning an error.
2020-05-22 13:31:06 -07:00
a2fd6d46a0
Cancel pending special key space reads on destroy
...
Previously the special key space would take a reference to the
transaction, but this doesn't make sense since the transaction might not
be refcounted
2020-05-21 17:37:11 +00:00
0d9135ee57
Merge remote-tracking branch 'upstream/release-6.3' into add-data-distribution-metrics
2020-05-20 15:21:26 -07:00
06676fc88f
Merge pull request #3166 from tclinken/6.3-avoid-ub
...
Eliminate some undefined behavior and const_cast usage
2020-05-20 14:36:30 -07:00
b589c3c940
Remove IncludeVersions.h, which is no longer needed for cmake-only builds
2020-05-20 13:23:02 -07:00
16990aeb6f
Add dcId to client transaction events.
2020-05-19 16:07:54 -07:00
00dce3ca1d
clang-format
2020-05-18 14:23:17 -07:00
fcca49f4d5
Add back an std::move that was accidentally removed. Some other small tweaks to avoid map lookups when tags aren't used.
2020-05-18 13:45:51 -07:00
bf62ef7c3e
Add dd-stats into special-key-space
2020-05-18 10:38:23 -07:00
6ce574f5ad
Merge remote-tracking branch 'upstream/release-6.3' into add-data-distribution-metrics
2020-05-17 23:36:52 -07:00
8597b38d10
Add an assertion in SingleSpecialKeyImpl
2020-05-15 16:34:43 -07:00
4a13107805
Add a test for special_keys_cross_module_read, fix two bugs, add several TODOs
2020-05-15 16:34:43 -07:00
c61e198e40
Using enum class instead of enum
2020-05-15 16:34:43 -07:00
e1c4e5f619
Add enums for special-key-space modules and refactor related code
2020-05-15 16:34:42 -07:00
f605c8d74f
Narrow keyrange of special-key-space to [\xff\xff, \xff\xff\xff). Initilize new ones in correctness tests
2020-05-15 16:34:42 -07:00
667b224fc8
Several changes to reduce unnecessary moves/copies
2020-05-13 19:59:34 -07:00
da14e1eb04
Fix: debug tagging was always enabled
2020-05-12 14:11:03 -07:00
8bd5dcaff8
Merge branch 'master' into atn34/special-key-versioning
2020-05-09 15:34:20 -07:00
02307ba7b6
Merge branch 'master' into transaction-tagging
...
# Conflicts:
# fdbclient/DatabaseContext.h
2020-05-09 07:50:29 -07:00
4cbf7b0fc0
Fix typo
2020-05-09 03:12:55 +00:00
d1b16eafc5
Only call getKeyRange once
2020-05-09 03:01:56 +00:00
ef724bf939
Merge remote-tracking branch 'upstream/master' into add-data-distribution-metrics
2020-05-08 18:39:28 -07:00
e8b62e48f4
Rename DDMetrics to DDMetricsRef
2020-05-08 17:17:27 -07:00
f0f52fb2be
Merge branch 'master' into feature-small-endpoint
...
# Conflicts:
# fdbclient/StorageServerInterface.h
2020-05-08 16:37:35 -07:00
749fcd13b0
Merge branch 'master' into fdb_cache_wo_allocator
2020-05-08 16:23:29 -07:00
6389f6ac51
Combine \xff\xff/transaction/* into one module
2020-05-08 05:37:37 +00:00
697685574d
Fix worker_interfaces
...
They don't come sorted by address, they come sorted by process id
2020-05-08 05:37:37 +00:00
38830f0eb9
Fix range read bounds bug
2020-05-08 05:37:37 +00:00
76aed13bdc
Add prefix in comparator for worker_interfaces
2020-05-08 05:37:37 +00:00
32d50fc359
Port over all existing special keys to framework
2020-05-08 05:37:37 +00:00
ce0e03a800
Add virtual destructor to SpecialKeyRangeBaseImpl
2020-05-08 05:37:37 +00:00
640eb41ba4
Introduce special key space in api version 630
2020-05-08 05:37:37 +00:00
a08bbcc539
Merge pull request #3046 from atn34/atn34/conflict-ranges
...
Add read/write conflict ranges to special key space
2020-05-07 22:36:18 -07:00
a8e3ad3c15
Fix merge error
2020-05-07 15:10:50 -07:00
92730b2f05
Merge branch 'master' into transaction-tagging
...
# Conflicts:
# fdbclient/NativeAPI.actor.cpp
2020-05-07 14:50:25 -07:00
9ccc417a4f
An indentifier must be set before calling LOG_TRANSACTION
2020-05-07 11:18:03 -07:00
90dbf33182
Merge pull request #3079 from bnamasivayam/client-transaction-info
...
Add read version and commit version info to getVersion and commit cli…
2020-05-07 09:59:35 -07:00
b1055a8501
Merge branch 'master' into transaction-tagging
2020-05-05 16:03:39 -07:00
f329164fb4
Merge pull request #2532 from dongxinEric/feature/hot-read-key-detection-part-2
...
Feature/hot read key detection part 2
2020-05-05 14:33:34 -07:00
37fe05fc76
Merge remote-tracking branch 'upstream/master' into atn34/conflict-ranges
2020-05-05 15:52:07 +00:00
dd033736ed
Merge branch 'master' into fdb_cache_subfeature2
2020-05-04 17:29:43 -07:00
de1f85d106
Fix accounting bug in TagSet. Change serialization back to not including the number of elements. Fix knob usage.
2020-05-04 15:49:55 -07:00
cd2176ac15
Adjust some knobs
2020-05-04 10:37:53 -07:00
36454bb3b8
Merge branch 'master' into transaction-tagging
...
# Conflicts:
# fdbclient/MasterProxyInterface.h
# fdbclient/NativeAPI.actor.cpp
2020-05-04 10:23:25 -07:00
80a235aa80
Add some correctness tests
2020-05-04 10:15:18 -07:00
bb3d4b6b89
Add a bunch of TEST macros and some other little things
2020-05-04 10:11:36 -07:00
78d644f673
Merge branch 'master' into bugfixes/cmake-includes
2020-05-04 02:11:46 -07:00
d6bc5263ad
Add read version and commit version info to getVersion and commit client transaction events.
2020-05-03 22:42:06 -07:00
0c514e91c7
Use approximate write range for set_versionstamped_key
2020-05-04 04:43:50 +00:00
ca1d457c6c
Fix bug where readVersion member future wasn't assigned
2020-05-02 07:59:37 -07:00
6ce4ddf1db
Fix a couple spots where we were missing checks for the new tag_throttled error
2020-05-01 21:47:12 -07:00
bd699f435c
fixed compiler errors
2020-05-01 11:01:09 -07:00
aed2d34bcb
Merge branch 'master' into feature-proxy-load-balance
...
# Conflicts:
# fdbclient/NativeAPI.actor.cpp
# fdbserver/MasterProxyServer.actor.cpp
# flow/Knobs.cpp
2020-05-01 09:19:39 -07:00
e9b9f0e845
onError wasn't properly retrying tag throttled errors
2020-04-30 21:12:34 -07:00
f1d9c0cb97
Refactor/cleanup
2020-04-30 00:30:37 +00:00
fdd6581167
Add write conflict range
2020-04-30 00:30:37 +00:00
e1c3c73d4c
Rudimentary read your own read conflict range
2020-04-30 00:30:37 +00:00
573b10f9f5
Fix a merge bug and cleanup some argument passing.
2020-04-27 15:03:56 -07:00
41c517a5dd
Merge branch 'master' into transaction-tagging
...
# Conflicts:
# fdbclient/NativeAPI.actor.cpp
2020-04-27 13:05:24 -07:00
a65e97209a
Fix bug in flag management. Pass priority into ratekeeper's updateRate and use it when setting autothrottles.
2020-04-27 11:34:12 -07:00
239876351b
Add some initial auto-throttling. Move the definition of the priority enum to a more global place and use it for all transaction priorites (except in ClientLogEvents, because of serialization incompatibilites).
2020-04-24 11:31:16 -07:00
35c18ac60a
Improvements to expiration. Encode throttles with auto/manual and priority in the key to support throttling the same tag with different values in these parameters.
2020-04-23 20:50:40 -07:00
c87aa33941
Merge branch 'release-6.2'
...
# Conflicts:
# CMakeLists.txt
# bindings/go/src/fdb/generated.go
# documentation/sphinx/source/api-common.rst.inc
# documentation/sphinx/source/api-ruby.rst
# documentation/sphinx/source/release-notes.rst
# fdbclient/FailureMonitorClient.actor.cpp
# fdbclient/NativeAPI.actor.cpp
# fdbclient/vexillographer/fdb.options
# fdbrpc/FlowTransport.actor.cpp
# fdbserver/OldTLogServer_6_0.actor.cpp
# fdbserver/TLogServer.actor.cpp
# fdbserver/fdbserver.actor.cpp
# versions.target
2020-04-23 13:47:53 -07:00
434704fbd9
Various bug fixes
2020-04-22 12:28:51 -07:00
444b338f63
Client clears throttles when the proxy responds to a request with a throttle being absent.
2020-04-21 16:24:02 -07:00
2f69eb42a5
Update client last check time for a throttle when the request is sent, not when the response is received. This prevents sending lots of requests when the check time has expired before we get a response.
2020-04-21 16:23:24 -07:00
ee8447765f
Throttled tags on the client periodically check in with the cluster to update their rates if they get heavily throttled.
2020-04-21 11:56:58 -07:00
e51d0365cf
Cleanup: Use the shutdown callback for destroying TLS state.
2020-04-20 13:16:16 -07:00
da8e47ea25
Merge remote-tracking branch 'upstream/release-6.2' into tls-background-eio-thread
2020-04-20 13:15:05 -07:00
2ce539ef6d
Respect flow<->fdbrpc module boundaries.
...
Which fixes a compilation error due to a circular dependency between
flow.a and fdbrpc.a. However, this is now done at the cost of newNet2
users have to remember to add Net2FileSystem::stop() as a callback.
2020-04-20 02:53:07 -07:00
94b4f78ea9
Fix clients crashing in TLS code on exit.
...
If client code initiates an FDB operation to a TLS cluster, and then
immediately exits the main thread, then OpenSSL's atexit handler would
potentially run while the network thread is attempting to do TLS
operations, and thus crash.
This commit removes the OpenSSL atexit hander, and instead relies on a
client intentionally ending the network thread to do TLS cleanup. If
the client code exits without stopping the network thread, then we'll
never free OpenSSL data structures, which is the safer thing to do.
2020-04-18 15:48:02 -07:00
78d48a0dad
Merge branch 'master' into transaction-tagging
...
# Conflicts:
# fdbcli/fdbcli.actor.cpp
# fdbclient/Knobs.h
# fdbclient/NativeAPI.actor.cpp
# fdbclient/fdbclient.vcxproj
# fdbserver/MasterProxyServer.actor.cpp
2020-04-17 09:23:18 -07:00
2b66dcd24a
Some more refactoring. Reduce what is sent from RK->MP->clients
2020-04-17 08:07:01 -07:00
7dd7406c59
Merge branch 'master' into feature/hot-read-key-detection-part-2
2020-04-16 14:54:05 -07:00
0fba8c47be
Checkpoint: Ratekeeper sets absolute limits for tag throttles and enforces them by distributing requests to proxies, who distribute them to clients.
...
A few refactorings.
2020-04-16 14:43:22 -07:00
b011efa8df
Merge remote-tracking branch 'upstream/master' into private-keyspace-framework
2020-04-16 10:58:04 -07:00
7a9bcf8222
Review comments.
2020-04-14 17:45:34 -07:00
b85dc16c6d
Merge branch 'master' into fdb_cache_subfeature2
2020-04-14 17:07:41 -07:00
e104a2e3a6
Merge commit 'cf01233f28a2c42908656a39f458a4475c1d44a3' into run-loop-busy-profiler
...
# Conflicts:
# documentation/sphinx/source/release-notes.rst
# fdbclient/NativeAPI.actor.h
# fdbserver/fdbserver.actor.cpp
# flow/Net2.actor.cpp
2020-04-14 14:02:24 -07:00
ae1de060c2
Reformat SpecialKeyRangeBaseImpl constructor
2020-04-14 09:10:40 -07:00
65620b00f3
Merge remote-tracking branch 'upstream/master' into private-keyspace-framework
2020-04-13 14:21:07 -07:00
1439de37b5
Convert GetRangeLimits() -> TOO_MANY + ASSERT().
2020-04-12 18:23:14 -07:00
db071256db
Merge remote-tracking branch 'upstream/master' into reduce-txn-state-store-size
2020-04-12 18:06:10 -07:00
6508c891fc
Make the TagSet sent to the storage servers optional so we can distinguish no tags from unsampled.
2020-04-10 13:29:28 -07:00
55a0d00ad4
Encoding of tags in the database now supports multiple tags per throttle. Remove throttle prefix search.
2020-04-10 10:12:26 -07:00
ebeca10bce
Change the serialization of tags sent in some messages. Add communication of the sampling rate from cluster to clients.
2020-04-09 16:55:56 -07:00
34cddd675c
clang-format
2020-04-08 14:33:41 -07:00
20b2984841
fix issues according to andrew's comments
2020-04-08 12:43:25 -07:00
52860043c9
Merge remote-tracking branch 'origin' into atomically-update-dependent-knobs
2020-04-08 12:26:21 -07:00
5aee6880a8
fix issues according to andrew's comments
2020-04-08 11:33:07 -07:00
36da61dd9c
Merge branch 'master' into transaction-tagging
...
# Conflicts:
# fdbclient/NativeAPI.actor.cpp
# fdbclient/vexillographer/fdb.options
2020-04-07 21:12:14 -07:00
a1d8623e5f
Various changes to the throttling scheme, including most notably that clients now enforce the throttles with info they receive from the proxy.
2020-04-07 16:28:09 -07:00
e4611e8ae4
fix versions.h stupidity
2020-04-06 10:28:55 -07:00
8b5780c36c
don't include source and binary dir
...
This forces users to use include paths from the sources root.
So `#include "Arena.h"` won't work anymore, only
`#include "flow/Arena.h"` will.
2020-04-06 10:13:49 -07:00
6078fd1b18
Convert UID to Tag in keyServers to reduce txnStateStore size
2020-04-05 14:30:09 -07:00
aad8f850f3
refactor conflicting keys feature using special key space
2020-04-04 13:26:11 -07:00
6d570cd865
Change version from 7.0 to 6.3
2020-04-03 21:38:58 -07:00
5b6ae4da7f
A simple intergration to put conflictingKeys into special-key-space
2020-04-03 16:11:20 -07:00
2336f073ad
Checkpointing a bunch of work on throttles. Rudimentary implementation of auto-throttling. Support for manual throttling via fdbcli. Throttles are stored in the system keyspace.
2020-04-03 15:24:14 -07:00
760dc68b7f
Merge pull request #2869 from dongxinEric/feature/1689/allow-custome-trace-log-file-identifier
...
Allow the user to provide a custome trace log file identifier that wi…
2020-04-03 14:08:17 -07:00
2ef8f1eb96
Fixed the indentation. Addressed review comments
2020-04-03 13:21:57 -07:00
0816423d78
Merge pull request #2853 from Daniel-B-Smith/debug-transactions
...
Add ability to mark transactions to add debug logging
2020-04-03 11:43:45 -07:00
f6a2a7cee9
Move log statement to correct section
2020-04-03 17:31:38 +00:00
5092f52042
Fix location of logging code.
2020-04-03 13:19:48 -04:00
1ec7b1ebe3
Add logging for server tracing
2020-04-02 22:16:58 +00:00
d0210fd54f
Rename option
2020-04-02 21:41:16 +00:00
a0c8946f31
Merge branch 'master' into fdb_cache_subfeature2
2020-04-02 12:27:04 -07:00
884e92bb49
Atomically update dependent knobs
2020-04-01 15:18:49 -07:00
c8abd9c493
Reformat some code
2020-03-31 18:53:36 -07:00
6820167d77
Merge branch 'master' into feature/1689/allow-custome-trace-log-file-identifier
2020-03-31 16:50:46 -07:00
459c2e34cd
Now limit the file identifier to a). has a length no longer than the value defined in knobs; b). can only contain [A-Za-z0-9_] chars.
2020-03-31 16:42:00 -07:00
a63f8b839c
Merge remote-tracking branch 'upstream/master' into private-keyspace-framework
2020-03-31 12:05:42 -07:00
82a2196d7d
update space range
2020-03-31 09:34:53 -07:00
60f6edc3b5
Merge pull request #2860 from zjuLcg/report-conflicting-key-roll-forward
...
Report conflicting key roll forward
2020-03-30 17:33:56 -07:00
e358510e27
Minor compilation fix.
2020-03-26 16:50:58 -07:00
64148469e8
clang-format the pr
2020-03-26 15:52:30 -07:00
acaf91ac47
Merge branch 'master' into fdb_cache_subfeature2
2020-03-26 13:33:08 -07:00
1052b23ee1
Merge pull request #2370 from atn34/test-watch-outliving-transaction
...
Test watch outliving transaction
2020-03-26 12:40:38 -07:00
a0177a9335
Allow the user to provide a custome trace log file identifier that will be used as the prefix of all trace log files created at the client side.
2020-03-26 11:25:05 -07:00
f6af6ec777
Avoid passing self to watch actor
2020-03-26 10:29:23 -07:00
cdb6bbfc85
Test watch outliving transaction
2020-03-26 10:09:03 -07:00
7d5ed53215
Allow trace log group to be set after database is created
2020-03-25 13:40:43 -07:00
289487559d
Revert "Revert "Merge pull request #2257 from zjuLcg/report-conflicting-key""
...
This reverts commit 804fe1b22e
2020-03-24 18:11:15 -07:00
6df92a8b8d
A work version of strict workload test
2020-03-23 23:55:56 -07:00
247ab84323
Merge branch 'master' of https://github.com/apple/foundationdb into add-data-distribution-metrics
2020-03-23 17:01:17 -07:00
197f866824
Don't include internal tools such as fdbcli, backup, and DR in the status client list.
2020-03-23 14:44:04 -07:00
26b7e02d4c
Some initial work to support tagging transactions and passing them around.
2020-03-20 11:23:11 -07:00
804fe1b22e
Revert "Merge pull request #2257 from zjuLcg/report-conflicting-key"
...
This reverts commit 648dc4a933487d131b38
2020-03-19 21:34:28 -07:00
648dc4a933
Merge pull request #2257 from zjuLcg/report-conflicting-key
...
Report conflicting keys
2020-03-18 13:39:42 -07:00
572f08e5fc
Add option to set transaction as debug
2020-03-17 22:21:31 +00:00
e08f0201f1
merge release 6.2 into master
2020-03-17 12:51:47 -07:00
ed4d02a3e4
Merge pull request #2812 from etschannen/feature-proxy-mem-limit
...
Limit the amount of requests the proxy can queue up in memory
2020-03-16 14:56:56 -07:00
a068d4063f
renamed ProxyGetConsistentReadVersion
2020-03-16 12:11:32 -07:00
99a5cb0572
Fix a corruption bug.
2020-03-13 18:52:34 -07:00
031b579ede
Increase priority of the logging of various metrics trace events.
2020-03-13 16:20:23 -07:00
39a37531db
Fix issues according to Andrew's comments
2020-03-13 15:42:15 -07:00
c4c38c5eca
Delete commented code
2020-03-13 12:58:12 -07:00
243c268d9d
Limit the amount of requests the proxy can queue up in memory
2020-03-13 10:17:49 -07:00
6f90228a0b
change to krmSetRangeCoalescing
2020-03-12 11:31:36 -07:00
c2f0c41c52
use krmSetRange
2020-03-11 23:12:38 -07:00
6ae60870fc
use krmSetRange
2020-03-11 13:20:40 -07:00
157d026ba2
debugging
2020-03-10 18:42:49 -07:00
698198a09e
Merge remote-tracking branch 'upstream/master' into report-conflicting-key
2020-03-09 10:50:33 -07:00
303df197cf
Merge branch 'release-6.2'
...
# Conflicts:
# CMakeLists.txt
# bindings/c/test/mako/mako.c
# documentation/sphinx/source/release-notes.rst
# fdbbackup/backup.actor.cpp
# fdbclient/NativeAPI.actor.cpp
# fdbclient/NativeAPI.actor.h
# fdbserver/DataDistributionQueue.actor.cpp
# fdbserver/Knobs.cpp
# fdbserver/Knobs.h
# fdbserver/LogRouter.actor.cpp
# fdbserver/SkipList.cpp
# fdbserver/fdbserver.actor.cpp
# flow/CMakeLists.txt
# flow/Knobs.cpp
# flow/Knobs.h
# flow/flow.vcxproj
# flow/flow.vcxproj.filters
# versions.target
2020-03-06 18:22:46 -08:00
927e59b1ce
Fix a memory bug.
2020-03-06 12:32:56 -08:00
dbfc0cbcc0
Merge pull request #2781 from alexmiller-apple/certificate-refresh
...
Refresh certificates used for handshaking when they change on disk
2020-03-06 11:12:04 -08:00
fd8d569b91
Fix a few typos.
2020-03-05 14:42:07 -08:00
6479034645
Add more metrics to the TransactionMetrics event
2020-03-05 14:00:44 -08:00
595dd77ed1
Merge remote-tracking branch 'upstream/release-6.2' into certificate-refresh
2020-03-04 20:25:42 -08:00
9b5ef3416e
Refactor TLSParams into TLSConfig + LoadedTLSConfig
...
The idea being that we keep around a TLSConfig that the configuration
that the user has provided, and then when we want to intialize an SSL
context, we ask the TLSConfig to load all certificates and return us a
LoadedTLSConfig that is a concrete set of certificate bytes in memory.
initTLS now just takes the in-memory bytes and applies them to the ssl
context.
This is a large refactor to lead up into certificate refeshing, where we
will periodically check for changes to the certificates, and then
re-load them and apply them to a new SSL context.
2020-03-04 20:14:47 -08:00
7a76e9556d
Merge remote-tracking branch 'upstream/master' into report-conflicting-key
2020-03-04 11:24:39 -08:00
9c50b8369d
Change name from private-key-space to special-key-space
2020-03-03 18:35:04 -08:00
94102969fd
Merge code from upstream and change ACTOR to member class
2020-03-03 14:55:03 -08:00
409718c964
Merge remote-tracking branch 'upstream/master' into private-keyspace-framework
2020-03-03 13:37:51 -08:00
4ee9ce15fc
Change from NULL to nullptr
2020-03-03 00:29:40 -08:00
6054c05963
Merge branch 'release-6.2'
...
# Conflicts:
# CMakeLists.txt
# documentation/sphinx/source/release-notes.rst
# fdbserver/fdbserver.actor.cpp
# versions.target
2020-02-28 12:11:05 -08:00
d1e1fea42d
Our binaries that act like clients (fdbcli, backup and DR binaries) were reporting an unknown client version. Clients did not react if the list of supported versions changed.
2020-02-28 09:35:21 -08:00
c3299b8ebe
if tls cannot be initialized, throw an error from createDatabase
2020-02-26 18:53:06 -08:00
d1598e7c99
set_verify_peers throws an error instead of returning a value
2020-02-26 16:06:16 -08:00
2586bade68
re-added support for configuration TLS options with environment variables
2020-02-26 15:33:48 -08:00
7fa1f105c0
Fix compilation error after merging master.
2020-02-26 11:34:41 -08:00
cd949eca71
Merge branch 'master' into fdb_cache_subfeature2
2020-02-26 11:22:08 -08:00
924d335aa7
Merge branch 'release-6.2'
...
# Conflicts:
# documentation/sphinx/source/release-notes.rst
# flow/Knobs.cpp
# flow/Knobs.h
2020-02-25 18:25:19 -08:00
13a523a355
fix: commit on first proxy did not always commit to the first proxy
2020-02-25 12:34:31 -08:00
8c4cdb1f80
More fixes.
2020-02-25 10:48:57 -08:00
7193fa87b8
And some more compilation fixes.
2020-02-25 10:45:02 -08:00
746523b090
Yet more compilation fixes.
2020-02-24 17:47:33 -08:00
0f64505d0b
Merge branch 'release-6.2' of github.com:apple/foundationdb
...
Needed to pull in changes to build docker
2020-02-23 23:27:53 -08:00
96258b9809
Merge branch 'release-6.2'
...
# Conflicts:
# documentation/sphinx/source/release-notes.rst
# fdbcli/fdbcli.actor.cpp
# fdbclient/ManagementAPI.actor.cpp
# fdbrpc/FlowTransport.actor.cpp
# fdbserver/ClusterController.actor.cpp
# fdbserver/DataDistribution.actor.cpp
# fdbserver/DataDistribution.actor.h
# fdbserver/DataDistributionQueue.actor.cpp
# fdbserver/KeyValueStoreMemory.actor.cpp
# fdbserver/MasterProxyServer.actor.cpp
# fdbserver/QuietDatabase.actor.cpp
# fdbserver/SkipList.cpp
# fdbserver/StorageMetrics.actor.h
# fdbserver/TLogServer.actor.cpp
# fdbserver/fdbserver.actor.cpp
# fdbserver/storageserver.actor.cpp
# fdbserver/workloads/KVStoreTest.actor.cpp
# flow/CMakeLists.txt
# flow/Knobs.cpp
# flow/Knobs.h
# flow/genericactors.actor.cpp
# flow/serialize.h
2020-02-21 19:09:16 -08:00
Richard Chen