1ca7fe1a05
Add span metadata message
2020-09-04 15:36:47 -07:00
ae7bf24353
Merge pull request #3549 from yliucode/grv-proxy
...
Separate out a new role GrvProxy to serve GRVs.
2020-09-03 19:03:45 -07:00
f9624b5a1a
Avoid unnecessary copy
2020-09-03 13:56:23 -07:00
b4c96cadc7
Merge branch 'release-6.3' into merge-release-6.3-into-master
2020-09-02 12:45:57 -07:00
93da4957ad
Update fdbclient/NativeAPI.actor.cpp
2020-09-01 13:17:01 -07:00
800ece286d
Results from read hot range requests were not held by the results arena.
2020-09-01 13:14:27 -07:00
fd7198d874
Extend backup container interface to support query restorable files set by key ranges
2020-08-29 19:58:07 -07:00
e87327b33b
Merge master branch and keep master proxy reporting txn cost estimation to ratekeeper
2020-08-29 12:47:35 -07:00
cd86ca6850
Merge branch 'master' into feature/range-split-points-based-on-size
2020-08-27 10:06:02 -07:00
e8e24a8ed6
Resolve comments
2020-08-26 21:04:35 -07:00
a07b9f234d
Add readonly range \xff\xff/configuration/process/class_source, and change \xff\xff/configuration/class/ to \xff\xff/configuration/process/class_type/
2020-08-25 18:18:32 -07:00
63b3612ad5
Merge master branch and resolve conflicts
2020-08-24 16:42:31 -07:00
3afdb44c7a
merge master
2020-08-23 17:09:04 +00:00
b4cec6577a
Fix merge bugs
2020-08-22 15:04:42 -07:00
e1b7dd0c7d
Merge remote-tracking branch 'upstream/release-6.3' into dyoungworth/fixMerge1
2020-08-22 12:25:19 -07:00
23ef23357c
Test code to add c function for kill/suspend
2020-08-21 14:30:52 -07:00
9b096ab58e
Merge branch 'master' of github.com:apple/foundationdb into add-c-function-for-management-commands
2020-08-21 12:20:35 -07:00
3b63d8b01b
remove FIXME; remote tagSet.reset(); trivial changes
2020-08-21 19:17:16 +00:00
5aaf86206a
add counter for expensive clear estimation for audit and trace
2020-08-20 23:00:45 +00:00
556b239057
clang-format
2020-08-20 14:51:41 -07:00
bff20faa42
Merge remote-tracking branch 'upstream/master' into add-setclass-to-special-key-space
2020-08-20 13:50:54 -07:00
e375761581
Merge branch 'master' of https://github.com/apple/foundationdb
2020-08-20 20:29:14 +00:00
9c709b33d6
scale cost
2020-08-19 19:47:41 +00:00
599675cba8
modify some details to get better performance
2020-08-19 04:23:23 +00:00
7b210aee24
Use =delete to disable special member functions
...
This has the benefits of:
- Easier to understand compiler error messages
- Moves some errors from link-time to compile-time
- NonCopyable classes can still specify default move constructors
2020-08-18 11:07:00 -07:00
24dbb3961c
Knob adjustment
2020-08-17 22:38:22 +00:00
130bd7ac22
Fixed a bug that would potentially cause invalid memory access, i.e. segmentation fault.
2020-08-17 13:16:17 -07:00
daead84fdd
Merge master
2020-08-14 14:07:08 -07:00
548ba06510
change map to deque; code style problem
2020-08-14 18:30:58 +00:00
475d4cca72
solve some cr comments
2020-08-14 16:40:16 +00:00
b9474055a5
Add \xff\xff/configuration/class to read process class
2020-08-14 01:01:08 -07:00
e94de126de
merge with master
2020-08-14 01:59:09 +00:00
20700f75f4
Merge master
2020-08-13 14:28:27 -07:00
b2bba8ccbb
Correct fix for debug ID logging
2020-08-13 13:37:15 -04:00
002cca5e58
Merge branch 'master' into tmp/merge-PR
...
Conflicts:
fdbclient/NativeAPI.actor.cpp
fdbserver/StorageMetrics.actor.h
2020-08-12 20:08:13 -07:00
8bc57c16f0
Merge pull request #3637 from dongxinEric/misc/fix-spammy-readhot-logging
...
Increased some knobs to throttle the spammy read hot logging.
2020-08-12 16:46:32 -07:00
8f1deaa47e
Merge branch 'master' of https://github.com/apple/foundationdb
2020-08-12 23:32:23 +00:00
3ac6996844
Increased some knobs to throttle the spammy read hot logging. Also added more details inside the read hot log to make it useful
2020-08-12 14:27:34 -07:00
cc1059be75
Make logging of server trace ID consistent with other code
2020-08-12 19:05:01 +00:00
f3ecf14601
change midShardSize type and other details
2020-08-12 17:49:12 +00:00
16ef2bd3bd
Pending commit
2020-08-12 10:34:07 -07:00
79ce16650d
merge master branch
2020-08-11 19:22:10 -07:00
4782c7c4b1
Merge remote-tracking branch 'origin/master' into features/udp
2020-08-11 15:35:52 -06:00
b66dab8816
Fix span early destruction
...
We want the span to stick around for the full duration of
extractReadVersion since it writes to the log from its destructor.
2020-08-11 11:19:04 -07:00
c0b7ac0b7d
try to throttle write tag and read tag seperatedly
2020-08-10 23:43:58 +00:00
cae6f04070
sample on cost instead of bytes
2020-08-10 22:30:06 +00:00
696e77c94e
query midShardSize from proxy
2020-08-10 20:13:44 +00:00
df9149fea4
ignore transaction tag of immediate transactions
2020-08-07 23:36:17 +00:00
98c39ad9d4
Fix the build error.
2020-08-07 14:03:42 -07:00
a46e620baa
Merge branch 'master' into feature/range-split-points-based-on-size
2020-08-06 15:23:11 -07:00
6d84bcb568
Merge remote-tracking branch 'origin/master' into features/udp
2020-08-06 14:08:34 -06:00
8976694ba1
UDP implementation (untested)
2020-08-06 14:06:50 -06:00
ba66b1f668
trivial changes
2020-08-06 14:53:26 +00:00
d6a23a4d6b
Resolve comments to make GRV proxy a separate process class
2020-08-06 00:01:57 -07:00
13307679c5
use median shard size"
2020-08-05 03:57:25 +00:00
4997754cf8
fix format in monitorDDMetricsChanges
2020-08-04 04:14:11 +00:00
00f2c38e3a
Merge remote-tracking branch 'upstream/master' into add-c-function-for-management-commands
2020-08-03 20:55:21 -07:00
48094ea17e
add INCOMPLETE_SHARD_PLUS knob
2020-08-04 03:18:05 +00:00
52073738c0
fix bugs; change some KNOB values
2020-08-03 22:18:34 +00:00
b903e60cb7
fix monitorDDMetricsChanges bugs
2020-08-03 17:12:36 +00:00
6db7040f17
update DDMetrics periodically
2020-08-03 05:52:38 +00:00
d1cc87452c
merge with master; solve conflicts; solve initialization;
2020-08-02 22:44:07 +00:00
92c1112c74
consider clear single key
2020-08-01 18:20:13 +00:00
4f7dab4951
sample clear op on client
2020-08-01 06:14:52 +00:00
fc3dfc170f
Merge branch 'master' of https://github.com/apple/foundationdb
2020-07-31 19:09:33 +00:00
c3a629588f
add client transaction tag sample
2020-07-31 19:08:42 +00:00
a048e5e159
Change prefix to \xff\xff/management and make decode/encode as pure functions
2020-07-31 02:18:49 -07:00
f2662a9a74
Merge remote-tracking branch 'upstream/master' into add-management-api-into-special-key-space
2020-07-30 19:01:11 -07:00
10540a3c73
Change error name and fix a bug in ExclusionInProgressRangeImpl::getRange()
2020-07-30 18:58:09 -07:00
11cbcffc9a
Temporary code
2020-07-30 16:07:54 -07:00
b5dc6b715f
Add file comment for GrvProxyServer.actor.cpp
2020-07-29 12:10:53 -07:00
30ea639666
Remove debug traces
2020-07-29 07:55:05 -07:00
f7b76a92af
pass joshua
2020-07-29 07:26:55 -07:00
602b0f9dea
Merge branch 'master' into vectorref-emplace-back
2020-07-28 17:17:11 -07:00
a641429d63
Add an enum for implementation type and use for registeration
2020-07-27 12:29:03 -07:00
ec9553b6c6
change \xff\xff/failure to \xff\xff/error_message
2020-07-24 11:38:56 -07:00
525f10e30c
Merge master branch
2020-07-22 16:08:49 -07:00
302cf5c45f
Remove debug trace events
2020-07-22 12:20:22 -07:00
2703cedac5
Fixed known bugs
2020-07-17 22:24:52 -07:00
0df2a8d014
better code style
2020-07-18 01:48:58 +00:00
1c51aaeea5
Use static keys in SpecialKeySpace
2020-07-17 15:14:18 -07:00
96e5a4c827
Replace all \xff\xff/conf with managementApiRange.begin
2020-07-17 11:56:27 -07:00
5fa6785fb8
Add special keys for command options
2020-07-16 18:15:35 -07:00
9d0d189cc8
better serialize; TransactionOption::clear patch
2020-07-15 22:39:21 +00:00
eb44ae0e86
finish local shard estimation
2020-07-15 16:08:00 +00:00
5b06d69d25
Pass watches test
2020-07-15 00:37:41 -07:00
bcb858288b
eliminate cost estimation when tagSet is empty
2020-07-14 23:07:21 +00:00
3f4da830a2
Merge remote-tracking branch 'upstream/master' into add-management-api-into-special-key-space
2020-07-14 10:36:47 -07:00
a310faf9d1
solve some code reviews
2020-07-14 17:19:55 +00:00
d512170cd8
add clear cost estimation
2020-07-14 00:18:52 +00:00
60d1bfc247
Add read impl for inProgressExclusionServers
2020-07-13 13:35:25 -07:00
6ec2f92a8d
merge master
2020-07-13 17:22:37 +00:00
6374cd96c9
add commit cost estimation
2020-07-10 23:07:29 +00:00
2e5ae8eed5
Change iterator name to small case
2020-07-10 10:56:41 -07:00
cc60597739
Merge branch 'master' into bugfixes/visibility-perf-regression
2020-07-09 17:04:04 -06:00
69864c9f96
Make Spans not allocate heap memory
2020-07-09 11:49:33 -06:00
1b68665228
Merge branch 'master' into release-6.3
2020-07-08 18:52:05 -07:00
3eb3e23ae0
Merge pull request #3425 from sfc-gh-anoyes/anoyes/health-metrics
...
Expose health metrics via special keys
2020-07-08 10:37:31 -07:00
0fbe7101c3
Revert "Revert "Request tracing""
...
This reverts commit 327cc31e35
2020-07-07 10:06:13 -06:00
a43afb9d97
Merge remote-tracking branch 'upstream/master' into add-management-api-into-special-key-space
2020-07-06 15:47:00 -07:00
0361eb5079
Add FailedServersImpl
2020-07-06 15:40:21 -07:00
784d0f6d00
A basic feasible version for exclude and include commands
2020-07-06 11:02:48 -07:00
bcbca81dfe
Remove ROW_LIMIT_UNLIMITED and BYTE_LIMIT_UNLIMITED from CLIENT_KNOBS
2020-07-04 12:48:44 -07:00
d689d34634
Enable range split bindings tests
2020-07-02 14:05:12 -07:00
00e9f8b9bf
Added bindings tests; Protected new SSI endpoints under new 7.0 ProtocolVersion
2020-07-02 14:05:12 -07:00
eb28492900
Fixed a bug which leads to inaccurate metrics being reported. Added tests for the new API.
2020-07-02 14:05:11 -07:00
ea08bc5462
Added server side code for range split support
2020-07-02 14:05:11 -07:00
ae63304392
Address Meng's comment
2020-07-01 20:23:18 -07:00
3ecdaf24ce
Use the actual range begin and end key instead of the shard begin and shard end key
2020-07-01 16:07:59 -07:00
409ccf3be2
Use snake_case to match status json convention
2020-06-30 17:05:29 +00:00
e1dfa410c1
Remove "Storages" field from data_distribution_stats
...
It seems like a cool idea, but it feels rushed to me. Reverting for now.
2020-06-30 15:24:16 +00:00
a55b0771c5
Test schemas
...
Also remove an ASSERT_WE_THINK that doesn't seem to hold now that we
include shards
2020-06-30 15:24:16 +00:00
f8bb76a607
Add CLIENT_BUGGIFY
2020-06-30 15:24:16 +00:00
b1d5e99494
Update documentation
2020-06-30 15:24:16 +00:00
1c30adce74
WIP
2020-06-30 15:24:16 +00:00
0db3145e82
Include storage id's in data_distribution_stats
2020-06-30 15:24:15 +00:00
e304697eca
WIP
2020-06-30 15:24:15 +00:00
ffd1467367
Add \xff\xff/metrics/health/*
2020-06-30 15:24:15 +00:00
e2c5938c28
Update module registration for readwrite module
2020-06-29 15:37:46 -07:00
1082ab2caa
fix ClientVersionRef versionString should be passed by ref
2020-06-29 21:23:56 +02:00
18ff6efbf3
Add failure module for special key space
2020-06-27 12:22:32 -07:00
f011ff2c66
Merge remote-tracking branch 'upstream/master' into add-management-api-into-special-key-space
2020-06-23 11:23:18 -07:00
42159ccfe0
Add -Wpessimizing-move and -Wredundant-move for clang
2020-06-22 23:45:03 +00:00
59361cedaa
Rename to have read and write impls and add set, clear, clearrange interfaces
2020-06-22 14:02:03 -07:00
85a284b13c
Use VectorRef::emplace_back in several places
2020-06-19 23:02:28 -07:00
327cc31e35
Revert "Request tracing"
2020-06-16 12:32:42 -07:00
96206a8032
Merge pull request #3368 from apple/release-6.3
...
Merge Release 6.3 to master
2020-06-15 20:15:22 -07:00
fabdffd6e4
Merge pull request #3332 from sfc-gh-tclinkenbeard/fix-memaccess-warnings
...
Fix class-memaccess warnings
2020-06-15 15:22:19 -07:00
3a914c71f2
Add TransactionOptions::clear function
2020-06-13 17:17:06 -07:00
8595813b7d
Merge pull request #3355 from apple/release-6.3
...
Merge Release 6.3 into master branch
2020-06-12 20:08:47 -07:00
80334351c7
Merge pull request #3329 from sfc-gh-mpilman/features/flatbuffers-debugtx
...
Request tracing
2020-06-12 14:50:19 -07:00
ae6b716820
addressed more review comments
2020-06-11 09:58:41 -07:00
9ef93714ac
Address review comments
2020-06-10 15:48:49 -07:00
4ab3441a95
Merge remote-tracking branch 'origin/master' into features/flatbuffers-debugtx
2020-06-10 15:31:29 -07:00
d9ddb84d34
remove weird include (added by clangd)
2020-06-10 13:18:26 -07:00
a332dff2ec
Fix class-memaccess warnings
2020-06-10 12:34:14 -07:00
99bf993815
Replace BOOST_NOEXCEPT with noexcept
2020-06-09 22:39:19 -07:00
c829b12bbf
defined spans for read requests
2020-06-08 17:26:15 -07:00
09c136e434
Framework for transaction tracing
2020-06-08 16:09:37 -07:00
55448a8d5a
Remove call in watch actor instead of Transaction::watch
2020-06-08 18:19:18 +00:00
731b77a8c2
Don't double count watches for too_many_watches
2020-06-07 23:54:22 +00:00
23a565ec63
Few bug fixes.
2020-06-05 16:27:04 -07:00
cf13e00a8f
Merge remote-tracking branch 'origin/release-6.3' into fdb_cache_wo_allocator
2020-06-01 17:38:31 -07:00
ba63a66d12
Merge remote-tracking branch 'origin/master' into features/flatbuffers-debugtx
2020-05-29 15:38:09 -07:00
8a7d98ab21
Started instrumentation
2020-05-27 15:36:04 -07:00
86f712657e
Eliminate the undefined behavior of calling run_network twice, instead returning an error.
2020-05-22 13:31:06 -07:00
a2fd6d46a0
Cancel pending special key space reads on destroy
...
Previously the special key space would take a reference to the
transaction, but this doesn't make sense since the transaction might not
be refcounted
2020-05-21 17:37:11 +00:00
0d9135ee57
Merge remote-tracking branch 'upstream/release-6.3' into add-data-distribution-metrics
2020-05-20 15:21:26 -07:00
06676fc88f
Merge pull request #3166 from tclinken/6.3-avoid-ub
...
Eliminate some undefined behavior and const_cast usage
2020-05-20 14:36:30 -07:00
b589c3c940
Remove IncludeVersions.h, which is no longer needed for cmake-only builds
2020-05-20 13:23:02 -07:00
16990aeb6f
Add dcId to client transaction events.
2020-05-19 16:07:54 -07:00
00dce3ca1d
clang-format
2020-05-18 14:23:17 -07:00
fcca49f4d5
Add back an std::move that was accidentally removed. Some other small tweaks to avoid map lookups when tags aren't used.
2020-05-18 13:45:51 -07:00
bf62ef7c3e
Add dd-stats into special-key-space
2020-05-18 10:38:23 -07:00
6ce574f5ad
Merge remote-tracking branch 'upstream/release-6.3' into add-data-distribution-metrics
2020-05-17 23:36:52 -07:00
8597b38d10
Add an assertion in SingleSpecialKeyImpl
2020-05-15 16:34:43 -07:00
4a13107805
Add a test for special_keys_cross_module_read, fix two bugs, add several TODOs
2020-05-15 16:34:43 -07:00
c61e198e40
Using enum class instead of enum
2020-05-15 16:34:43 -07:00
e1c4e5f619
Add enums for special-key-space modules and refactor related code
2020-05-15 16:34:42 -07:00
f605c8d74f
Narrow keyrange of special-key-space to [\xff\xff, \xff\xff\xff). Initilize new ones in correctness tests
2020-05-15 16:34:42 -07:00
667b224fc8
Several changes to reduce unnecessary moves/copies
2020-05-13 19:59:34 -07:00
da14e1eb04
Fix: debug tagging was always enabled
2020-05-12 14:11:03 -07:00
8bd5dcaff8
Merge branch 'master' into atn34/special-key-versioning
2020-05-09 15:34:20 -07:00
02307ba7b6
Merge branch 'master' into transaction-tagging
...
# Conflicts:
# fdbclient/DatabaseContext.h
2020-05-09 07:50:29 -07:00
4cbf7b0fc0
Fix typo
2020-05-09 03:12:55 +00:00
d1b16eafc5
Only call getKeyRange once
2020-05-09 03:01:56 +00:00
ef724bf939
Merge remote-tracking branch 'upstream/master' into add-data-distribution-metrics
2020-05-08 18:39:28 -07:00
e8b62e48f4
Rename DDMetrics to DDMetricsRef
2020-05-08 17:17:27 -07:00
f0f52fb2be
Merge branch 'master' into feature-small-endpoint
...
# Conflicts:
# fdbclient/StorageServerInterface.h
2020-05-08 16:37:35 -07:00
749fcd13b0
Merge branch 'master' into fdb_cache_wo_allocator
2020-05-08 16:23:29 -07:00
6389f6ac51
Combine \xff\xff/transaction/* into one module
2020-05-08 05:37:37 +00:00
697685574d
Fix worker_interfaces
...
They don't come sorted by address, they come sorted by process id
2020-05-08 05:37:37 +00:00
38830f0eb9
Fix range read bounds bug
2020-05-08 05:37:37 +00:00
76aed13bdc
Add prefix in comparator for worker_interfaces
2020-05-08 05:37:37 +00:00
32d50fc359
Port over all existing special keys to framework
2020-05-08 05:37:37 +00:00
ce0e03a800
Add virtual destructor to SpecialKeyRangeBaseImpl
2020-05-08 05:37:37 +00:00
640eb41ba4
Introduce special key space in api version 630
2020-05-08 05:37:37 +00:00
a08bbcc539
Merge pull request #3046 from atn34/atn34/conflict-ranges
...
Add read/write conflict ranges to special key space
2020-05-07 22:36:18 -07:00
a8e3ad3c15
Fix merge error
2020-05-07 15:10:50 -07:00
92730b2f05
Merge branch 'master' into transaction-tagging
...
# Conflicts:
# fdbclient/NativeAPI.actor.cpp
2020-05-07 14:50:25 -07:00
9ccc417a4f
An indentifier must be set before calling LOG_TRANSACTION
2020-05-07 11:18:03 -07:00
90dbf33182
Merge pull request #3079 from bnamasivayam/client-transaction-info
...
Add read version and commit version info to getVersion and commit cli…
2020-05-07 09:59:35 -07:00
b1055a8501
Merge branch 'master' into transaction-tagging
2020-05-05 16:03:39 -07:00
f329164fb4
Merge pull request #2532 from dongxinEric/feature/hot-read-key-detection-part-2
...
Feature/hot read key detection part 2
2020-05-05 14:33:34 -07:00
37fe05fc76
Merge remote-tracking branch 'upstream/master' into atn34/conflict-ranges
2020-05-05 15:52:07 +00:00
dd033736ed
Merge branch 'master' into fdb_cache_subfeature2
2020-05-04 17:29:43 -07:00
de1f85d106
Fix accounting bug in TagSet. Change serialization back to not including the number of elements. Fix knob usage.
2020-05-04 15:49:55 -07:00
cd2176ac15
Adjust some knobs
2020-05-04 10:37:53 -07:00
36454bb3b8
Merge branch 'master' into transaction-tagging
...
# Conflicts:
# fdbclient/MasterProxyInterface.h
# fdbclient/NativeAPI.actor.cpp
2020-05-04 10:23:25 -07:00
80a235aa80
Add some correctness tests
2020-05-04 10:15:18 -07:00
bb3d4b6b89
Add a bunch of TEST macros and some other little things
2020-05-04 10:11:36 -07:00
78d644f673
Merge branch 'master' into bugfixes/cmake-includes
2020-05-04 02:11:46 -07:00
d6bc5263ad
Add read version and commit version info to getVersion and commit client transaction events.
2020-05-03 22:42:06 -07:00
0c514e91c7
Use approximate write range for set_versionstamped_key
2020-05-04 04:43:50 +00:00
ca1d457c6c
Fix bug where readVersion member future wasn't assigned
2020-05-02 07:59:37 -07:00
6ce4ddf1db
Fix a couple spots where we were missing checks for the new tag_throttled error
2020-05-01 21:47:12 -07:00
bd699f435c
fixed compiler errors
2020-05-01 11:01:09 -07:00
aed2d34bcb
Merge branch 'master' into feature-proxy-load-balance
...
# Conflicts:
# fdbclient/NativeAPI.actor.cpp
# fdbserver/MasterProxyServer.actor.cpp
# flow/Knobs.cpp
2020-05-01 09:19:39 -07:00
e9b9f0e845
onError wasn't properly retrying tag throttled errors
2020-04-30 21:12:34 -07:00
f1d9c0cb97
Refactor/cleanup
2020-04-30 00:30:37 +00:00
Lukas Joswiak