b0f61fb74f
Resolve the simple-looking conflicts
2021-01-15 19:35:10 +00:00
ff7d306b09
Merge branch 'release-6.3' into anoyes/merge-6.3-to-master
...
Include conflict markers for now. Will resolve.
2021-01-15 18:04:09 +00:00
8ce9bd4e8e
Merge branch 'master' of github.com:apple/foundationdb into add-c-function-for-management-commands
2021-01-14 17:31:26 -08:00
e32a0fb742
Resolve conflicts
2021-01-14 20:13:29 +00:00
6d6b2843da
Merge remote-tracking branch 'upstream/release-6.2' into anoyes/merge-release-6.2
...
I'm including the conflict markers for now and I'll resolve them in a
subsequent commit
2021-01-14 17:20:11 +00:00
87b4463305
Solve comments: update c api description; refine code
2021-01-13 14:56:41 -08:00
8ff14878fe
Merge remote-tracking branch 'origin/master' into simplify-global-knobs
2021-01-13 14:39:35 -08:00
58360a192f
Add a counter for the number of status requests made by a client
2021-01-05 10:18:43 -08:00
e097c98956
Remove headed added by IDE, add comments
2021-01-04 00:18:01 -08:00
da7b65cd71
Change the interface back to return int64_t
2021-01-03 22:45:09 -08:00
5b2e88b187
Use structured bindings in for loops
2020-12-27 01:46:20 -04:00
0ecfaf138e
Merge pull request #4150 from sfc-gh-ljoswiak/fixes/sps-tracing
...
Remove "a/" from tracing keys in special-key-space
2020-12-21 13:48:01 -08:00
9db9a84798
Ignore negative duration value
2020-12-21 13:29:16 -08:00
b68d84aaea
Merge remote-tracking branch 'upstream/master' into add-c-function-for-management-commands
2020-12-21 12:32:31 -08:00
7b4f7ce4e0
update comments
2020-12-20 22:47:55 -08:00
9601769b01
Merge pull request #3858 from sfc-gh-rchen/stable_interfaces
...
Stable interfaces
2020-12-11 09:34:27 -08:00
5f57d72a59
remove print statements and format protocol version workload
2020-12-11 04:46:20 +00:00
4cb821cd63
Merge remote-tracking branch 'upstream/release-6.2' into merge-6.2-to-6.3
2020-12-09 15:47:44 -08:00
43fd545f06
Remove a/ from tracing path in special-key-space
2020-12-09 13:40:16 -08:00
9581bc0279
Merge pull request #4129 from sfc-gh-anoyes/anoyes/merge-release-6.3-master
...
Merge release 6.3 master
2020-12-08 08:57:54 -08:00
54237f28ea
Throw error on commit for late trace options
2020-12-07 14:43:44 -08:00
18201d6f4f
Add option to enable transaction tracing
2020-12-07 11:24:14 -08:00
75836d6114
Fix double counting of range reads in TransactionMetrics
2020-12-07 09:01:19 -08:00
7fbc4d7391
Resolve conflicts
2020-12-04 23:58:42 +00:00
76142549e0
Return keys from range read in lexicographic order
2020-12-04 11:21:39 -08:00
877997632d
Merge branch 'release-6.3' into anoyes/merge-release-6.3-master
...
Include conflict markers for review purposes
2020-12-04 01:38:07 +00:00
3669615e4b
Refactor and clean up
2020-12-03 14:06:11 -08:00
88e66545bb
Change the api to return bool instead of int
2020-12-03 12:01:32 -08:00
c77d9e4abe
merge conflicts
2020-12-02 21:53:19 +00:00
1f166bc183
Add options to set custom transaction ID and disable tracing
2020-11-30 14:57:17 -08:00
1f541f02be
Merge branch 'anoyes/merge-6.2-to-6.3' into anoyes/release-6.3-merge
...
Merge, leaving conflict markers for now
2020-11-24 16:55:34 +00:00
2fae7333c0
Add duration parameter in the api
2020-11-23 14:55:29 -08:00
b1e3478267
Simplified global knobs
2020-11-21 13:27:48 -08:00
2320f970c1
Add reboot_worker and pythong binding
2020-11-19 16:43:39 -08:00
aec0398fd0
Merge remote-tracking branch 'upstream/master' into add-c-function-for-management-commands
2020-11-17 11:39:12 -08:00
45c9a0abc7
Revert "Revert "Add limiting health metrics""
...
This reverts commit 209ebcc595
2020-11-13 17:24:57 -08:00
209ebcc595
Revert "Add limiting health metrics"
2020-11-13 17:08:46 -08:00
8c0b4dbe4c
Merge pull request #4067 from sfc-gh-tclinkenbeard/add-limiting-health-metrics
...
Add limiting health metrics
2020-11-13 16:04:44 -08:00
6c4493166f
Add limiting storage queue and durability lag to health metrics
2020-11-12 20:14:41 -08:00
0f205620bf
Merge branch 'master' into features/udp
2020-11-12 13:51:17 -07:00
1343f40117
don't allow empty coments
2020-11-11 14:07:54 -07:00
4669f837fa
Add uses of makeReference
2020-11-07 22:10:18 -08:00
4788544a6f
Revise comments based on review suggestions
...
Ack. Jingyu and Xin for their suggestions.
2020-11-06 08:51:13 -08:00
063700e4d6
Add comments and questions to HA and tLog code reading
...
The comments' correctness need to be confirmed by reviewers.
2020-10-30 12:14:57 -07:00
cf4c8e375f
Merge remote-tracking branch 'origin/release-6.3' into merge
2020-10-29 22:15:41 -07:00
a72bb52eae
Merge branch 'master' of github.com:apple/foundationdb into add-c-function-for-management-commands
2020-10-28 22:32:44 -07:00
bf7bf470d9
Merge branch 'master' of https://github.com/apple/foundationdb into stable_interfaces
2020-10-26 19:36:19 +00:00
a15bc32b14
Merge pull request #3945 from sfc-gh-clin/add-consistencycheck-to-special-keys
...
Add consistencycheck command to special keys
2020-10-26 10:59:17 -07:00
309e38dc06
add unit test and some cleanup
2020-10-23 19:34:18 +00:00
055add9682
conflicts
2020-10-23 06:33:00 +00:00
90caa24af1
add function to get correct checksum class. Remove IChecksum::checkSum function
2020-10-23 02:24:29 +00:00
c8eb003ec4
Change to \xff\xff/management/in_progress_exclusion, the snake case naming style
2020-10-22 18:05:54 -07:00
c8b3b37b4a
Remove unused code for #3930
2020-10-22 17:04:08 -07:00
95654ce7e7
Change dbLocked to snake_case naming db_locked
2020-10-22 11:35:39 -07:00
2b4ec39a2f
Avoid passing references from an iterator into an insertion that invalidates the iterator.
2020-10-22 11:30:08 -07:00
1d64b2c335
clang-format
2020-10-22 11:28:03 -07:00
b728fe473e
Add consistencycheck command to special keys
2020-10-22 11:08:54 -07:00
8d937a56f2
Throw operation_failed() if dd not found when calling ddMetricsRequestServer
...
Remove time_out error in DDMetricsList rpc
Add retry logic for dd special key when dd not found
Update FuzzApiCorrectness test
2020-10-21 15:28:03 -07:00
8cc3e4d3c6
Merge release-6.3 into master
2020-10-19 22:51:56 -07:00
c84a5f4014
change IChecksum interface and remove coordinator protocol version optional
2020-10-19 21:39:07 +00:00
a4ccf8c5b6
add checksum class to change checksum in the future. expose get server protocol api
2020-10-19 16:57:11 +00:00
24ea35e56f
Merge pull request #3748 from sfc-gh-ljoswiak/visibility-2
...
Add TLogVersion::V6
2020-10-14 17:35:32 -07:00
bbc6d2aa03
Merge pull request #3872 from sfc-gh-clin:add-lock-unlock-to-special-keys
...
Add lock/unlock to special keys
2020-10-13 09:09:49 -07:00
5d902f9177
Merge pull request #3394 from dongxinEric/feature/range-split-points-based-on-size
...
Feature/range split points based on size
2020-10-08 14:25:11 -07:00
c66a775d64
Merge remote-tracking branch 'upstream/master' into add-lock-unlock-to-special-keys
2020-10-08 14:23:16 -07:00
53d3759e73
Add lock command to special key \xff\xff/dbLocked
2020-10-08 14:23:02 -07:00
f03a355c4e
Merge branch 'release-6.2' into release-6.3
...
# Conflicts:
# CMakeLists.txt
# documentation/sphinx/source/downloads.rst
# fdbrpc/FlowTransport.actor.cpp
# fdbrpc/FlowTransport.h
# flow/Arena.h
# flow/Knobs.cpp
# flow/Knobs.h
# packaging/msi/FDBInstaller.wxs
2020-10-07 12:28:22 -07:00
dea7000970
Merge remote-tracking branch 'upstream/master' into visibility-1
2020-10-06 18:38:15 -07:00
76da0a7cbc
Backport #3426 to release-6.2
2020-10-05 13:53:51 -07:00
b294e148c5
another compile fix
2020-10-04 16:54:43 -07:00
614c8bc895
Get read versions requests must be load balanced on the number of requests because ratekeeper gives out an equal budget to each proxy
2020-10-04 16:20:24 -07:00
bbc7ce581e
Resolve conflicts merging from 6.3 to master
2020-09-25 16:08:31 -07:00
862336de8f
Merge branch 'master' into mengxu/merge-to-master-PR
2020-09-24 17:06:00 -07:00
b37daf88d4
add jitter to the delay
2020-09-22 22:24:40 -07:00
09ff05c629
backed out buggify of apiVersion because there are too many tests that depend on it for a patch release
2020-09-22 21:15:49 -07:00
e0fbadfeb5
Added a buggify so older API versions are run in simulation
2020-09-22 18:58:01 -07:00
c99627a7ab
Merge branch 'release-6.3' of github.com:apple/foundationdb into feature-apiverison-batch-error
2020-09-22 17:06:17 -07:00
7dec47938e
protect api versions older than 630 from seeing batch_transaction_throttled errors
2020-09-22 17:05:09 -07:00
b4e209bd4e
Replace NULL with nullptr in fdbclient
2020-09-20 11:33:09 -07:00
a52ef0acb8
Merge pull request #3677 from sfc-gh-clin/add-setclass-to-special-key-space
...
Add setclass to special keys
2020-09-19 14:55:03 -07:00
71309a79f9
Merge branch 'release-6.3' into range
2020-09-17 17:54:33 -07:00
cc5bc16bd8
Rename more places from proxy to commit proxy
2020-09-15 22:29:49 -07:00
3dfadff6d8
Merge remote-tracking branch 'upstream/master' into add-c-function-for-management-commands
2020-09-14 16:45:47 -07:00
cc32074ed7
Rename actor
2020-09-14 16:45:34 -07:00
35bef73a1c
Rename proxy to commit proxy
2020-09-10 17:44:15 -07:00
d270b15c89
Merge remote-tracking branch 'upstream/master' into add-setclass-to-special-key-space
2020-09-09 16:21:19 -07:00
1ca7fe1a05
Add span metadata message
2020-09-04 15:36:47 -07:00
ae7bf24353
Merge pull request #3549 from yliucode/grv-proxy
...
Separate out a new role GrvProxy to serve GRVs.
2020-09-03 19:03:45 -07:00
f9624b5a1a
Avoid unnecessary copy
2020-09-03 13:56:23 -07:00
b4c96cadc7
Merge branch 'release-6.3' into merge-release-6.3-into-master
2020-09-02 12:45:57 -07:00
93da4957ad
Update fdbclient/NativeAPI.actor.cpp
2020-09-01 13:17:01 -07:00
800ece286d
Results from read hot range requests were not held by the results arena.
2020-09-01 13:14:27 -07:00
fd7198d874
Extend backup container interface to support query restorable files set by key ranges
2020-08-29 19:58:07 -07:00
e87327b33b
Merge master branch and keep master proxy reporting txn cost estimation to ratekeeper
2020-08-29 12:47:35 -07:00
cd86ca6850
Merge branch 'master' into feature/range-split-points-based-on-size
2020-08-27 10:06:02 -07:00
e8e24a8ed6
Resolve comments
2020-08-26 21:04:35 -07:00
a07b9f234d
Add readonly range \xff\xff/configuration/process/class_source, and change \xff\xff/configuration/class/ to \xff\xff/configuration/process/class_type/
2020-08-25 18:18:32 -07:00
63b3612ad5
Merge master branch and resolve conflicts
2020-08-24 16:42:31 -07:00
3afdb44c7a
merge master
2020-08-23 17:09:04 +00:00
b4cec6577a
Fix merge bugs
2020-08-22 15:04:42 -07:00
e1b7dd0c7d
Merge remote-tracking branch 'upstream/release-6.3' into dyoungworth/fixMerge1
2020-08-22 12:25:19 -07:00
23ef23357c
Test code to add c function for kill/suspend
2020-08-21 14:30:52 -07:00
9b096ab58e
Merge branch 'master' of github.com:apple/foundationdb into add-c-function-for-management-commands
2020-08-21 12:20:35 -07:00
3b63d8b01b
remove FIXME; remote tagSet.reset(); trivial changes
2020-08-21 19:17:16 +00:00
5aaf86206a
add counter for expensive clear estimation for audit and trace
2020-08-20 23:00:45 +00:00
556b239057
clang-format
2020-08-20 14:51:41 -07:00
bff20faa42
Merge remote-tracking branch 'upstream/master' into add-setclass-to-special-key-space
2020-08-20 13:50:54 -07:00
e375761581
Merge branch 'master' of https://github.com/apple/foundationdb
2020-08-20 20:29:14 +00:00
9c709b33d6
scale cost
2020-08-19 19:47:41 +00:00
599675cba8
modify some details to get better performance
2020-08-19 04:23:23 +00:00
7b210aee24
Use =delete to disable special member functions
...
This has the benefits of:
- Easier to understand compiler error messages
- Moves some errors from link-time to compile-time
- NonCopyable classes can still specify default move constructors
2020-08-18 11:07:00 -07:00
24dbb3961c
Knob adjustment
2020-08-17 22:38:22 +00:00
130bd7ac22
Fixed a bug that would potentially cause invalid memory access, i.e. segmentation fault.
2020-08-17 13:16:17 -07:00
daead84fdd
Merge master
2020-08-14 14:07:08 -07:00
548ba06510
change map to deque; code style problem
2020-08-14 18:30:58 +00:00
475d4cca72
solve some cr comments
2020-08-14 16:40:16 +00:00
b9474055a5
Add \xff\xff/configuration/class to read process class
2020-08-14 01:01:08 -07:00
e94de126de
merge with master
2020-08-14 01:59:09 +00:00
20700f75f4
Merge master
2020-08-13 14:28:27 -07:00
b2bba8ccbb
Correct fix for debug ID logging
2020-08-13 13:37:15 -04:00
002cca5e58
Merge branch 'master' into tmp/merge-PR
...
Conflicts:
fdbclient/NativeAPI.actor.cpp
fdbserver/StorageMetrics.actor.h
2020-08-12 20:08:13 -07:00
8bc57c16f0
Merge pull request #3637 from dongxinEric/misc/fix-spammy-readhot-logging
...
Increased some knobs to throttle the spammy read hot logging.
2020-08-12 16:46:32 -07:00
8f1deaa47e
Merge branch 'master' of https://github.com/apple/foundationdb
2020-08-12 23:32:23 +00:00
3ac6996844
Increased some knobs to throttle the spammy read hot logging. Also added more details inside the read hot log to make it useful
2020-08-12 14:27:34 -07:00
cc1059be75
Make logging of server trace ID consistent with other code
2020-08-12 19:05:01 +00:00
f3ecf14601
change midShardSize type and other details
2020-08-12 17:49:12 +00:00
16ef2bd3bd
Pending commit
2020-08-12 10:34:07 -07:00
79ce16650d
merge master branch
2020-08-11 19:22:10 -07:00
4782c7c4b1
Merge remote-tracking branch 'origin/master' into features/udp
2020-08-11 15:35:52 -06:00
b66dab8816
Fix span early destruction
...
We want the span to stick around for the full duration of
extractReadVersion since it writes to the log from its destructor.
2020-08-11 11:19:04 -07:00
c0b7ac0b7d
try to throttle write tag and read tag seperatedly
2020-08-10 23:43:58 +00:00
cae6f04070
sample on cost instead of bytes
2020-08-10 22:30:06 +00:00
696e77c94e
query midShardSize from proxy
2020-08-10 20:13:44 +00:00
df9149fea4
ignore transaction tag of immediate transactions
2020-08-07 23:36:17 +00:00
98c39ad9d4
Fix the build error.
2020-08-07 14:03:42 -07:00
a46e620baa
Merge branch 'master' into feature/range-split-points-based-on-size
2020-08-06 15:23:11 -07:00
6d84bcb568
Merge remote-tracking branch 'origin/master' into features/udp
2020-08-06 14:08:34 -06:00
8976694ba1
UDP implementation (untested)
2020-08-06 14:06:50 -06:00
ba66b1f668
trivial changes
2020-08-06 14:53:26 +00:00
d6a23a4d6b
Resolve comments to make GRV proxy a separate process class
2020-08-06 00:01:57 -07:00
13307679c5
use median shard size"
2020-08-05 03:57:25 +00:00
4997754cf8
fix format in monitorDDMetricsChanges
2020-08-04 04:14:11 +00:00
00f2c38e3a
Merge remote-tracking branch 'upstream/master' into add-c-function-for-management-commands
2020-08-03 20:55:21 -07:00
48094ea17e
add INCOMPLETE_SHARD_PLUS knob
2020-08-04 03:18:05 +00:00
52073738c0
fix bugs; change some KNOB values
2020-08-03 22:18:34 +00:00
b903e60cb7
fix monitorDDMetricsChanges bugs
2020-08-03 17:12:36 +00:00
6db7040f17
update DDMetrics periodically
2020-08-03 05:52:38 +00:00
d1cc87452c
merge with master; solve conflicts; solve initialization;
2020-08-02 22:44:07 +00:00
92c1112c74
consider clear single key
2020-08-01 18:20:13 +00:00
4f7dab4951
sample clear op on client
2020-08-01 06:14:52 +00:00
fc3dfc170f
Merge branch 'master' of https://github.com/apple/foundationdb
2020-07-31 19:09:33 +00:00
c3a629588f
add client transaction tag sample
2020-07-31 19:08:42 +00:00
a048e5e159
Change prefix to \xff\xff/management and make decode/encode as pure functions
2020-07-31 02:18:49 -07:00
f2662a9a74
Merge remote-tracking branch 'upstream/master' into add-management-api-into-special-key-space
2020-07-30 19:01:11 -07:00
10540a3c73
Change error name and fix a bug in ExclusionInProgressRangeImpl::getRange()
2020-07-30 18:58:09 -07:00
11cbcffc9a
Temporary code
2020-07-30 16:07:54 -07:00
b5dc6b715f
Add file comment for GrvProxyServer.actor.cpp
2020-07-29 12:10:53 -07:00
30ea639666
Remove debug traces
2020-07-29 07:55:05 -07:00
f7b76a92af
pass joshua
2020-07-29 07:26:55 -07:00
602b0f9dea
Merge branch 'master' into vectorref-emplace-back
2020-07-28 17:17:11 -07:00
a641429d63
Add an enum for implementation type and use for registeration
2020-07-27 12:29:03 -07:00
ec9553b6c6
change \xff\xff/failure to \xff\xff/error_message
2020-07-24 11:38:56 -07:00
525f10e30c
Merge master branch
2020-07-22 16:08:49 -07:00
302cf5c45f
Remove debug trace events
2020-07-22 12:20:22 -07:00
2703cedac5
Fixed known bugs
2020-07-17 22:24:52 -07:00
0df2a8d014
better code style
2020-07-18 01:48:58 +00:00
1c51aaeea5
Use static keys in SpecialKeySpace
2020-07-17 15:14:18 -07:00
96e5a4c827
Replace all \xff\xff/conf with managementApiRange.begin
2020-07-17 11:56:27 -07:00
5fa6785fb8
Add special keys for command options
2020-07-16 18:15:35 -07:00
9d0d189cc8
better serialize; TransactionOption::clear patch
2020-07-15 22:39:21 +00:00
eb44ae0e86
finish local shard estimation
2020-07-15 16:08:00 +00:00
5b06d69d25
Pass watches test
2020-07-15 00:37:41 -07:00
bcb858288b
eliminate cost estimation when tagSet is empty
2020-07-14 23:07:21 +00:00
3f4da830a2
Merge remote-tracking branch 'upstream/master' into add-management-api-into-special-key-space
2020-07-14 10:36:47 -07:00
a310faf9d1
solve some code reviews
2020-07-14 17:19:55 +00:00
d512170cd8
add clear cost estimation
2020-07-14 00:18:52 +00:00
60d1bfc247
Add read impl for inProgressExclusionServers
2020-07-13 13:35:25 -07:00
6ec2f92a8d
merge master
2020-07-13 17:22:37 +00:00
6374cd96c9
add commit cost estimation
2020-07-10 23:07:29 +00:00
2e5ae8eed5
Change iterator name to small case
2020-07-10 10:56:41 -07:00
cc60597739
Merge branch 'master' into bugfixes/visibility-perf-regression
2020-07-09 17:04:04 -06:00
69864c9f96
Make Spans not allocate heap memory
2020-07-09 11:49:33 -06:00
1b68665228
Merge branch 'master' into release-6.3
2020-07-08 18:52:05 -07:00
3eb3e23ae0
Merge pull request #3425 from sfc-gh-anoyes/anoyes/health-metrics
...
Expose health metrics via special keys
2020-07-08 10:37:31 -07:00
0fbe7101c3
Revert "Revert "Request tracing""
...
This reverts commit 327cc31e35
2020-07-07 10:06:13 -06:00
a43afb9d97
Merge remote-tracking branch 'upstream/master' into add-management-api-into-special-key-space
2020-07-06 15:47:00 -07:00
0361eb5079
Add FailedServersImpl
2020-07-06 15:40:21 -07:00
784d0f6d00
A basic feasible version for exclude and include commands
2020-07-06 11:02:48 -07:00
bcbca81dfe
Remove ROW_LIMIT_UNLIMITED and BYTE_LIMIT_UNLIMITED from CLIENT_KNOBS
2020-07-04 12:48:44 -07:00
d689d34634
Enable range split bindings tests
2020-07-02 14:05:12 -07:00
00e9f8b9bf
Added bindings tests; Protected new SSI endpoints under new 7.0 ProtocolVersion
2020-07-02 14:05:12 -07:00
eb28492900
Fixed a bug which leads to inaccurate metrics being reported. Added tests for the new API.
2020-07-02 14:05:11 -07:00
ea08bc5462
Added server side code for range split support
2020-07-02 14:05:11 -07:00
ae63304392
Address Meng's comment
2020-07-01 20:23:18 -07:00
3ecdaf24ce
Use the actual range begin and end key instead of the shard begin and shard end key
2020-07-01 16:07:59 -07:00
409ccf3be2
Use snake_case to match status json convention
2020-06-30 17:05:29 +00:00
e1dfa410c1
Remove "Storages" field from data_distribution_stats
...
It seems like a cool idea, but it feels rushed to me. Reverting for now.
2020-06-30 15:24:16 +00:00
a55b0771c5
Test schemas
...
Also remove an ASSERT_WE_THINK that doesn't seem to hold now that we
include shards
2020-06-30 15:24:16 +00:00
f8bb76a607
Add CLIENT_BUGGIFY
2020-06-30 15:24:16 +00:00
b1d5e99494
Update documentation
2020-06-30 15:24:16 +00:00
1c30adce74
WIP
2020-06-30 15:24:16 +00:00
0db3145e82
Include storage id's in data_distribution_stats
2020-06-30 15:24:15 +00:00
e304697eca
WIP
2020-06-30 15:24:15 +00:00
ffd1467367
Add \xff\xff/metrics/health/*
2020-06-30 15:24:15 +00:00
e2c5938c28
Update module registration for readwrite module
2020-06-29 15:37:46 -07:00
1082ab2caa
fix ClientVersionRef versionString should be passed by ref
2020-06-29 21:23:56 +02:00
18ff6efbf3
Add failure module for special key space
2020-06-27 12:22:32 -07:00
f011ff2c66
Merge remote-tracking branch 'upstream/master' into add-management-api-into-special-key-space
2020-06-23 11:23:18 -07:00
42159ccfe0
Add -Wpessimizing-move and -Wredundant-move for clang
2020-06-22 23:45:03 +00:00
59361cedaa
Rename to have read and write impls and add set, clear, clearrange interfaces
2020-06-22 14:02:03 -07:00
85a284b13c
Use VectorRef::emplace_back in several places
2020-06-19 23:02:28 -07:00
327cc31e35
Revert "Request tracing"
2020-06-16 12:32:42 -07:00
96206a8032
Merge pull request #3368 from apple/release-6.3
...
Merge Release 6.3 to master
2020-06-15 20:15:22 -07:00
fabdffd6e4
Merge pull request #3332 from sfc-gh-tclinkenbeard/fix-memaccess-warnings
...
Fix class-memaccess warnings
2020-06-15 15:22:19 -07:00
3a914c71f2
Add TransactionOptions::clear function
2020-06-13 17:17:06 -07:00
8595813b7d
Merge pull request #3355 from apple/release-6.3
...
Merge Release 6.3 into master branch
2020-06-12 20:08:47 -07:00
80334351c7
Merge pull request #3329 from sfc-gh-mpilman/features/flatbuffers-debugtx
...
Request tracing
2020-06-12 14:50:19 -07:00
ae6b716820
addressed more review comments
2020-06-11 09:58:41 -07:00
9ef93714ac
Address review comments
2020-06-10 15:48:49 -07:00
4ab3441a95
Merge remote-tracking branch 'origin/master' into features/flatbuffers-debugtx
2020-06-10 15:31:29 -07:00
d9ddb84d34
remove weird include (added by clangd)
2020-06-10 13:18:26 -07:00
a332dff2ec
Fix class-memaccess warnings
2020-06-10 12:34:14 -07:00
99bf993815
Replace BOOST_NOEXCEPT with noexcept
2020-06-09 22:39:19 -07:00
c829b12bbf
defined spans for read requests
2020-06-08 17:26:15 -07:00
09c136e434
Framework for transaction tracing
2020-06-08 16:09:37 -07:00
55448a8d5a
Remove call in watch actor instead of Transaction::watch
2020-06-08 18:19:18 +00:00
731b77a8c2
Don't double count watches for too_many_watches
2020-06-07 23:54:22 +00:00
23a565ec63
Few bug fixes.
2020-06-05 16:27:04 -07:00
cf13e00a8f
Merge remote-tracking branch 'origin/release-6.3' into fdb_cache_wo_allocator
2020-06-01 17:38:31 -07:00
ba63a66d12
Merge remote-tracking branch 'origin/master' into features/flatbuffers-debugtx
2020-05-29 15:38:09 -07:00
8a7d98ab21
Started instrumentation
2020-05-27 15:36:04 -07:00
86f712657e
Eliminate the undefined behavior of calling run_network twice, instead returning an error.
2020-05-22 13:31:06 -07:00
a2fd6d46a0
Cancel pending special key space reads on destroy
...
Previously the special key space would take a reference to the
transaction, but this doesn't make sense since the transaction might not
be refcounted
2020-05-21 17:37:11 +00:00
0d9135ee57
Merge remote-tracking branch 'upstream/release-6.3' into add-data-distribution-metrics
2020-05-20 15:21:26 -07:00
06676fc88f
Merge pull request #3166 from tclinken/6.3-avoid-ub
...
Eliminate some undefined behavior and const_cast usage
2020-05-20 14:36:30 -07:00
b589c3c940
Remove IncludeVersions.h, which is no longer needed for cmake-only builds
2020-05-20 13:23:02 -07:00
16990aeb6f
Add dcId to client transaction events.
2020-05-19 16:07:54 -07:00
00dce3ca1d
clang-format
2020-05-18 14:23:17 -07:00
fcca49f4d5
Add back an std::move that was accidentally removed. Some other small tweaks to avoid map lookups when tags aren't used.
2020-05-18 13:45:51 -07:00
bf62ef7c3e
Add dd-stats into special-key-space
2020-05-18 10:38:23 -07:00
6ce574f5ad
Merge remote-tracking branch 'upstream/release-6.3' into add-data-distribution-metrics
2020-05-17 23:36:52 -07:00
8597b38d10
Add an assertion in SingleSpecialKeyImpl
2020-05-15 16:34:43 -07:00
4a13107805
Add a test for special_keys_cross_module_read, fix two bugs, add several TODOs
2020-05-15 16:34:43 -07:00
c61e198e40
Using enum class instead of enum
2020-05-15 16:34:43 -07:00
e1c4e5f619
Add enums for special-key-space modules and refactor related code
2020-05-15 16:34:42 -07:00
f605c8d74f
Narrow keyrange of special-key-space to [\xff\xff, \xff\xff\xff). Initilize new ones in correctness tests
2020-05-15 16:34:42 -07:00
667b224fc8
Several changes to reduce unnecessary moves/copies
2020-05-13 19:59:34 -07:00
da14e1eb04
Fix: debug tagging was always enabled
2020-05-12 14:11:03 -07:00
8bd5dcaff8
Merge branch 'master' into atn34/special-key-versioning
2020-05-09 15:34:20 -07:00
02307ba7b6
Merge branch 'master' into transaction-tagging
...
# Conflicts:
# fdbclient/DatabaseContext.h
2020-05-09 07:50:29 -07:00
4cbf7b0fc0
Fix typo
2020-05-09 03:12:55 +00:00
d1b16eafc5
Only call getKeyRange once
2020-05-09 03:01:56 +00:00
ef724bf939
Merge remote-tracking branch 'upstream/master' into add-data-distribution-metrics
2020-05-08 18:39:28 -07:00
e8b62e48f4
Rename DDMetrics to DDMetricsRef
2020-05-08 17:17:27 -07:00
f0f52fb2be
Merge branch 'master' into feature-small-endpoint
...
# Conflicts:
# fdbclient/StorageServerInterface.h
2020-05-08 16:37:35 -07:00
749fcd13b0
Merge branch 'master' into fdb_cache_wo_allocator
2020-05-08 16:23:29 -07:00
6389f6ac51
Combine \xff\xff/transaction/* into one module
2020-05-08 05:37:37 +00:00
697685574d
Fix worker_interfaces
...
They don't come sorted by address, they come sorted by process id
2020-05-08 05:37:37 +00:00
38830f0eb9
Fix range read bounds bug
2020-05-08 05:37:37 +00:00
76aed13bdc
Add prefix in comparator for worker_interfaces
2020-05-08 05:37:37 +00:00
32d50fc359
Port over all existing special keys to framework
2020-05-08 05:37:37 +00:00
ce0e03a800
Add virtual destructor to SpecialKeyRangeBaseImpl
2020-05-08 05:37:37 +00:00
640eb41ba4
Introduce special key space in api version 630
2020-05-08 05:37:37 +00:00
a08bbcc539
Merge pull request #3046 from atn34/atn34/conflict-ranges
...
Add read/write conflict ranges to special key space
2020-05-07 22:36:18 -07:00
a8e3ad3c15
Fix merge error
2020-05-07 15:10:50 -07:00
92730b2f05
Merge branch 'master' into transaction-tagging
...
# Conflicts:
# fdbclient/NativeAPI.actor.cpp
2020-05-07 14:50:25 -07:00
9ccc417a4f
An indentifier must be set before calling LOG_TRANSACTION
2020-05-07 11:18:03 -07:00
90dbf33182
Merge pull request #3079 from bnamasivayam/client-transaction-info
...
Add read version and commit version info to getVersion and commit cli…
2020-05-07 09:59:35 -07:00
b1055a8501
Merge branch 'master' into transaction-tagging
2020-05-05 16:03:39 -07:00
f329164fb4
Merge pull request #2532 from dongxinEric/feature/hot-read-key-detection-part-2
...
Feature/hot read key detection part 2
2020-05-05 14:33:34 -07:00
37fe05fc76
Merge remote-tracking branch 'upstream/master' into atn34/conflict-ranges
2020-05-05 15:52:07 +00:00
dd033736ed
Merge branch 'master' into fdb_cache_subfeature2
2020-05-04 17:29:43 -07:00
de1f85d106
Fix accounting bug in TagSet. Change serialization back to not including the number of elements. Fix knob usage.
2020-05-04 15:49:55 -07:00
cd2176ac15
Adjust some knobs
2020-05-04 10:37:53 -07:00
36454bb3b8
Merge branch 'master' into transaction-tagging
...
# Conflicts:
# fdbclient/MasterProxyInterface.h
# fdbclient/NativeAPI.actor.cpp
2020-05-04 10:23:25 -07:00
80a235aa80
Add some correctness tests
2020-05-04 10:15:18 -07:00
bb3d4b6b89
Add a bunch of TEST macros and some other little things
2020-05-04 10:11:36 -07:00
78d644f673
Merge branch 'master' into bugfixes/cmake-includes
2020-05-04 02:11:46 -07:00
d6bc5263ad
Add read version and commit version info to getVersion and commit client transaction events.
2020-05-03 22:42:06 -07:00
0c514e91c7
Use approximate write range for set_versionstamped_key
2020-05-04 04:43:50 +00:00
ca1d457c6c
Fix bug where readVersion member future wasn't assigned
2020-05-02 07:59:37 -07:00
6ce4ddf1db
Fix a couple spots where we were missing checks for the new tag_throttled error
2020-05-01 21:47:12 -07:00
bd699f435c
fixed compiler errors
2020-05-01 11:01:09 -07:00
aed2d34bcb
Merge branch 'master' into feature-proxy-load-balance
...
# Conflicts:
# fdbclient/NativeAPI.actor.cpp
# fdbserver/MasterProxyServer.actor.cpp
# flow/Knobs.cpp
2020-05-01 09:19:39 -07:00
e9b9f0e845
onError wasn't properly retrying tag throttled errors
2020-04-30 21:12:34 -07:00
f1d9c0cb97
Refactor/cleanup
2020-04-30 00:30:37 +00:00
fdd6581167
Add write conflict range
2020-04-30 00:30:37 +00:00
e1c3c73d4c
Rudimentary read your own read conflict range
2020-04-30 00:30:37 +00:00
573b10f9f5
Fix a merge bug and cleanup some argument passing.
2020-04-27 15:03:56 -07:00
41c517a5dd
Merge branch 'master' into transaction-tagging
...
# Conflicts:
# fdbclient/NativeAPI.actor.cpp
2020-04-27 13:05:24 -07:00
a65e97209a
Fix bug in flag management. Pass priority into ratekeeper's updateRate and use it when setting autothrottles.
2020-04-27 11:34:12 -07:00
239876351b
Add some initial auto-throttling. Move the definition of the priority enum to a more global place and use it for all transaction priorites (except in ClientLogEvents, because of serialization incompatibilites).
2020-04-24 11:31:16 -07:00
35c18ac60a
Improvements to expiration. Encode throttles with auto/manual and priority in the key to support throttling the same tag with different values in these parameters.
2020-04-23 20:50:40 -07:00
c87aa33941
Merge branch 'release-6.2'
...
# Conflicts:
# CMakeLists.txt
# bindings/go/src/fdb/generated.go
# documentation/sphinx/source/api-common.rst.inc
# documentation/sphinx/source/api-ruby.rst
# documentation/sphinx/source/release-notes.rst
# fdbclient/FailureMonitorClient.actor.cpp
# fdbclient/NativeAPI.actor.cpp
# fdbclient/vexillographer/fdb.options
# fdbrpc/FlowTransport.actor.cpp
# fdbserver/OldTLogServer_6_0.actor.cpp
# fdbserver/TLogServer.actor.cpp
# fdbserver/fdbserver.actor.cpp
# versions.target
2020-04-23 13:47:53 -07:00
434704fbd9
Various bug fixes
2020-04-22 12:28:51 -07:00
444b338f63
Client clears throttles when the proxy responds to a request with a throttle being absent.
2020-04-21 16:24:02 -07:00
2f69eb42a5
Update client last check time for a throttle when the request is sent, not when the response is received. This prevents sending lots of requests when the check time has expired before we get a response.
2020-04-21 16:23:24 -07:00
ee8447765f
Throttled tags on the client periodically check in with the cluster to update their rates if they get heavily throttled.
2020-04-21 11:56:58 -07:00
e51d0365cf
Cleanup: Use the shutdown callback for destroying TLS state.
2020-04-20 13:16:16 -07:00
da8e47ea25
Merge remote-tracking branch 'upstream/release-6.2' into tls-background-eio-thread
2020-04-20 13:15:05 -07:00
2ce539ef6d
Respect flow<->fdbrpc module boundaries.
...
Which fixes a compilation error due to a circular dependency between
flow.a and fdbrpc.a. However, this is now done at the cost of newNet2
users have to remember to add Net2FileSystem::stop() as a callback.
2020-04-20 02:53:07 -07:00
94b4f78ea9
Fix clients crashing in TLS code on exit.
...
If client code initiates an FDB operation to a TLS cluster, and then
immediately exits the main thread, then OpenSSL's atexit handler would
potentially run while the network thread is attempting to do TLS
operations, and thus crash.
This commit removes the OpenSSL atexit hander, and instead relies on a
client intentionally ending the network thread to do TLS cleanup. If
the client code exits without stopping the network thread, then we'll
never free OpenSSL data structures, which is the safer thing to do.
2020-04-18 15:48:02 -07:00
78d48a0dad
Merge branch 'master' into transaction-tagging
...
# Conflicts:
# fdbcli/fdbcli.actor.cpp
# fdbclient/Knobs.h
# fdbclient/NativeAPI.actor.cpp
# fdbclient/fdbclient.vcxproj
# fdbserver/MasterProxyServer.actor.cpp
2020-04-17 09:23:18 -07:00
2b66dcd24a
Some more refactoring. Reduce what is sent from RK->MP->clients
2020-04-17 08:07:01 -07:00
7dd7406c59
Merge branch 'master' into feature/hot-read-key-detection-part-2
2020-04-16 14:54:05 -07:00
0fba8c47be
Checkpoint: Ratekeeper sets absolute limits for tag throttles and enforces them by distributing requests to proxies, who distribute them to clients.
...
A few refactorings.
2020-04-16 14:43:22 -07:00
b011efa8df
Merge remote-tracking branch 'upstream/master' into private-keyspace-framework
2020-04-16 10:58:04 -07:00
7a9bcf8222
Review comments.
2020-04-14 17:45:34 -07:00
b85dc16c6d
Merge branch 'master' into fdb_cache_subfeature2
2020-04-14 17:07:41 -07:00
e104a2e3a6
Merge commit 'cf01233f28a2c42908656a39f458a4475c1d44a3' into run-loop-busy-profiler
...
# Conflicts:
# documentation/sphinx/source/release-notes.rst
# fdbclient/NativeAPI.actor.h
# fdbserver/fdbserver.actor.cpp
# flow/Net2.actor.cpp
2020-04-14 14:02:24 -07:00
ae1de060c2
Reformat SpecialKeyRangeBaseImpl constructor
2020-04-14 09:10:40 -07:00
65620b00f3
Merge remote-tracking branch 'upstream/master' into private-keyspace-framework
2020-04-13 14:21:07 -07:00
1439de37b5
Convert GetRangeLimits() -> TOO_MANY + ASSERT().
2020-04-12 18:23:14 -07:00
db071256db
Merge remote-tracking branch 'upstream/master' into reduce-txn-state-store-size
2020-04-12 18:06:10 -07:00
6508c891fc
Make the TagSet sent to the storage servers optional so we can distinguish no tags from unsampled.
2020-04-10 13:29:28 -07:00
55a0d00ad4
Encoding of tags in the database now supports multiple tags per throttle. Remove throttle prefix search.
2020-04-10 10:12:26 -07:00
ebeca10bce
Change the serialization of tags sent in some messages. Add communication of the sampling rate from cluster to clients.
2020-04-09 16:55:56 -07:00
34cddd675c
clang-format
2020-04-08 14:33:41 -07:00
20b2984841
fix issues according to andrew's comments
2020-04-08 12:43:25 -07:00
52860043c9
Merge remote-tracking branch 'origin' into atomically-update-dependent-knobs
2020-04-08 12:26:21 -07:00
5aee6880a8
fix issues according to andrew's comments
2020-04-08 11:33:07 -07:00
36da61dd9c
Merge branch 'master' into transaction-tagging
...
# Conflicts:
# fdbclient/NativeAPI.actor.cpp
# fdbclient/vexillographer/fdb.options
2020-04-07 21:12:14 -07:00
a1d8623e5f
Various changes to the throttling scheme, including most notably that clients now enforce the throttles with info they receive from the proxy.
2020-04-07 16:28:09 -07:00
e4611e8ae4
fix versions.h stupidity
2020-04-06 10:28:55 -07:00
8b5780c36c
don't include source and binary dir
...
This forces users to use include paths from the sources root.
So `#include "Arena.h"` won't work anymore, only
`#include "flow/Arena.h"` will.
2020-04-06 10:13:49 -07:00
6078fd1b18
Convert UID to Tag in keyServers to reduce txnStateStore size
2020-04-05 14:30:09 -07:00
aad8f850f3
refactor conflicting keys feature using special key space
2020-04-04 13:26:11 -07:00
6d570cd865
Change version from 7.0 to 6.3
2020-04-03 21:38:58 -07:00
5b6ae4da7f
A simple intergration to put conflictingKeys into special-key-space
2020-04-03 16:11:20 -07:00
2336f073ad
Checkpointing a bunch of work on throttles. Rudimentary implementation of auto-throttling. Support for manual throttling via fdbcli. Throttles are stored in the system keyspace.
2020-04-03 15:24:14 -07:00
760dc68b7f
Merge pull request #2869 from dongxinEric/feature/1689/allow-custome-trace-log-file-identifier
...
Allow the user to provide a custome trace log file identifier that wi…
2020-04-03 14:08:17 -07:00
2ef8f1eb96
Fixed the indentation. Addressed review comments
2020-04-03 13:21:57 -07:00
0816423d78
Merge pull request #2853 from Daniel-B-Smith/debug-transactions
...
Add ability to mark transactions to add debug logging
2020-04-03 11:43:45 -07:00
f6a2a7cee9
Move log statement to correct section
2020-04-03 17:31:38 +00:00
5092f52042
Fix location of logging code.
2020-04-03 13:19:48 -04:00
1ec7b1ebe3
Add logging for server tracing
2020-04-02 22:16:58 +00:00
d0210fd54f
Rename option
2020-04-02 21:41:16 +00:00
a0c8946f31
Merge branch 'master' into fdb_cache_subfeature2
2020-04-02 12:27:04 -07:00
884e92bb49
Atomically update dependent knobs
2020-04-01 15:18:49 -07:00
c8abd9c493
Reformat some code
2020-03-31 18:53:36 -07:00
6820167d77
Merge branch 'master' into feature/1689/allow-custome-trace-log-file-identifier
2020-03-31 16:50:46 -07:00
459c2e34cd
Now limit the file identifier to a). has a length no longer than the value defined in knobs; b). can only contain [A-Za-z0-9_] chars.
2020-03-31 16:42:00 -07:00
a63f8b839c
Merge remote-tracking branch 'upstream/master' into private-keyspace-framework
2020-03-31 12:05:42 -07:00
82a2196d7d
update space range
2020-03-31 09:34:53 -07:00
60f6edc3b5
Merge pull request #2860 from zjuLcg/report-conflicting-key-roll-forward
...
Report conflicting key roll forward
2020-03-30 17:33:56 -07:00
e358510e27
Minor compilation fix.
2020-03-26 16:50:58 -07:00
64148469e8
clang-format the pr
2020-03-26 15:52:30 -07:00
acaf91ac47
Merge branch 'master' into fdb_cache_subfeature2
2020-03-26 13:33:08 -07:00
1052b23ee1
Merge pull request #2370 from atn34/test-watch-outliving-transaction
...
Test watch outliving transaction
2020-03-26 12:40:38 -07:00
a0177a9335
Allow the user to provide a custome trace log file identifier that will be used as the prefix of all trace log files created at the client side.
2020-03-26 11:25:05 -07:00
f6af6ec777
Avoid passing self to watch actor
2020-03-26 10:29:23 -07:00
cdb6bbfc85
Test watch outliving transaction
2020-03-26 10:09:03 -07:00
7d5ed53215
Allow trace log group to be set after database is created
2020-03-25 13:40:43 -07:00
289487559d
Revert "Revert "Merge pull request #2257 from zjuLcg/report-conflicting-key""
...
This reverts commit 804fe1b22e
2020-03-24 18:11:15 -07:00
6df92a8b8d
A work version of strict workload test
2020-03-23 23:55:56 -07:00
247ab84323
Merge branch 'master' of https://github.com/apple/foundationdb into add-data-distribution-metrics
2020-03-23 17:01:17 -07:00
197f866824
Don't include internal tools such as fdbcli, backup, and DR in the status client list.
2020-03-23 14:44:04 -07:00
26b7e02d4c
Some initial work to support tagging transactions and passing them around.
2020-03-20 11:23:11 -07:00
804fe1b22e
Revert "Merge pull request #2257 from zjuLcg/report-conflicting-key"
...
This reverts commit 648dc4a933487d131b38
2020-03-19 21:34:28 -07:00
648dc4a933
Merge pull request #2257 from zjuLcg/report-conflicting-key
...
Report conflicting keys
2020-03-18 13:39:42 -07:00
572f08e5fc
Add option to set transaction as debug
2020-03-17 22:21:31 +00:00
e08f0201f1
merge release 6.2 into master
2020-03-17 12:51:47 -07:00
ed4d02a3e4
Merge pull request #2812 from etschannen/feature-proxy-mem-limit
...
Limit the amount of requests the proxy can queue up in memory
2020-03-16 14:56:56 -07:00
a068d4063f
renamed ProxyGetConsistentReadVersion
2020-03-16 12:11:32 -07:00
99a5cb0572
Fix a corruption bug.
2020-03-13 18:52:34 -07:00
031b579ede
Increase priority of the logging of various metrics trace events.
2020-03-13 16:20:23 -07:00
39a37531db
Fix issues according to Andrew's comments
2020-03-13 15:42:15 -07:00
c4c38c5eca
Delete commented code
2020-03-13 12:58:12 -07:00
243c268d9d
Limit the amount of requests the proxy can queue up in memory
2020-03-13 10:17:49 -07:00
6f90228a0b
change to krmSetRangeCoalescing
2020-03-12 11:31:36 -07:00
c2f0c41c52
use krmSetRange
2020-03-11 23:12:38 -07:00
6ae60870fc
use krmSetRange
2020-03-11 13:20:40 -07:00
157d026ba2
debugging
2020-03-10 18:42:49 -07:00
698198a09e
Merge remote-tracking branch 'upstream/master' into report-conflicting-key
2020-03-09 10:50:33 -07:00
303df197cf
Merge branch 'release-6.2'
...
# Conflicts:
# CMakeLists.txt
# bindings/c/test/mako/mako.c
# documentation/sphinx/source/release-notes.rst
# fdbbackup/backup.actor.cpp
# fdbclient/NativeAPI.actor.cpp
# fdbclient/NativeAPI.actor.h
# fdbserver/DataDistributionQueue.actor.cpp
# fdbserver/Knobs.cpp
# fdbserver/Knobs.h
# fdbserver/LogRouter.actor.cpp
# fdbserver/SkipList.cpp
# fdbserver/fdbserver.actor.cpp
# flow/CMakeLists.txt
# flow/Knobs.cpp
# flow/Knobs.h
# flow/flow.vcxproj
# flow/flow.vcxproj.filters
# versions.target
2020-03-06 18:22:46 -08:00
927e59b1ce
Fix a memory bug.
2020-03-06 12:32:56 -08:00
dbfc0cbcc0
Merge pull request #2781 from alexmiller-apple/certificate-refresh
...
Refresh certificates used for handshaking when they change on disk
2020-03-06 11:12:04 -08:00
fd8d569b91
Fix a few typos.
2020-03-05 14:42:07 -08:00
6479034645
Add more metrics to the TransactionMetrics event
2020-03-05 14:00:44 -08:00
595dd77ed1
Merge remote-tracking branch 'upstream/release-6.2' into certificate-refresh
2020-03-04 20:25:42 -08:00
9b5ef3416e
Refactor TLSParams into TLSConfig + LoadedTLSConfig
...
The idea being that we keep around a TLSConfig that the configuration
that the user has provided, and then when we want to intialize an SSL
context, we ask the TLSConfig to load all certificates and return us a
LoadedTLSConfig that is a concrete set of certificate bytes in memory.
initTLS now just takes the in-memory bytes and applies them to the ssl
context.
This is a large refactor to lead up into certificate refeshing, where we
will periodically check for changes to the certificates, and then
re-load them and apply them to a new SSL context.
2020-03-04 20:14:47 -08:00
7a76e9556d
Merge remote-tracking branch 'upstream/master' into report-conflicting-key
2020-03-04 11:24:39 -08:00
9c50b8369d
Change name from private-key-space to special-key-space
2020-03-03 18:35:04 -08:00
94102969fd
Merge code from upstream and change ACTOR to member class
2020-03-03 14:55:03 -08:00
409718c964
Merge remote-tracking branch 'upstream/master' into private-keyspace-framework
2020-03-03 13:37:51 -08:00
4ee9ce15fc
Change from NULL to nullptr
2020-03-03 00:29:40 -08:00
6054c05963
Merge branch 'release-6.2'
...
# Conflicts:
# CMakeLists.txt
# documentation/sphinx/source/release-notes.rst
# fdbserver/fdbserver.actor.cpp
# versions.target
2020-02-28 12:11:05 -08:00
d1e1fea42d
Our binaries that act like clients (fdbcli, backup and DR binaries) were reporting an unknown client version. Clients did not react if the list of supported versions changed.
2020-02-28 09:35:21 -08:00
c3299b8ebe
if tls cannot be initialized, throw an error from createDatabase
2020-02-26 18:53:06 -08:00
d1598e7c99
set_verify_peers throws an error instead of returning a value
2020-02-26 16:06:16 -08:00
2586bade68
re-added support for configuration TLS options with environment variables
2020-02-26 15:33:48 -08:00
7fa1f105c0
Fix compilation error after merging master.
2020-02-26 11:34:41 -08:00
cd949eca71
Merge branch 'master' into fdb_cache_subfeature2
2020-02-26 11:22:08 -08:00
924d335aa7
Merge branch 'release-6.2'
...
# Conflicts:
# documentation/sphinx/source/release-notes.rst
# flow/Knobs.cpp
# flow/Knobs.h
2020-02-25 18:25:19 -08:00
13a523a355
fix: commit on first proxy did not always commit to the first proxy
2020-02-25 12:34:31 -08:00
8c4cdb1f80
More fixes.
2020-02-25 10:48:57 -08:00
7193fa87b8
And some more compilation fixes.
2020-02-25 10:45:02 -08:00
746523b090
Yet more compilation fixes.
2020-02-24 17:47:33 -08:00
0f64505d0b
Merge branch 'release-6.2' of github.com:apple/foundationdb
...
Needed to pull in changes to build docker
2020-02-23 23:27:53 -08:00
96258b9809
Merge branch 'release-6.2'
...
# Conflicts:
# documentation/sphinx/source/release-notes.rst
# fdbcli/fdbcli.actor.cpp
# fdbclient/ManagementAPI.actor.cpp
# fdbrpc/FlowTransport.actor.cpp
# fdbserver/ClusterController.actor.cpp
# fdbserver/DataDistribution.actor.cpp
# fdbserver/DataDistribution.actor.h
# fdbserver/DataDistributionQueue.actor.cpp
# fdbserver/KeyValueStoreMemory.actor.cpp
# fdbserver/MasterProxyServer.actor.cpp
# fdbserver/QuietDatabase.actor.cpp
# fdbserver/SkipList.cpp
# fdbserver/StorageMetrics.actor.h
# fdbserver/TLogServer.actor.cpp
# fdbserver/fdbserver.actor.cpp
# fdbserver/storageserver.actor.cpp
# fdbserver/workloads/KVStoreTest.actor.cpp
# flow/CMakeLists.txt
# flow/Knobs.cpp
# flow/Knobs.h
# flow/genericactors.actor.cpp
# flow/serialize.h
2020-02-21 19:09:16 -08:00
4c696d5bf2
Merge branch 'release-6.2' into dd-better-rebalance-logging
...
# Conflicts:
# fdbserver/DataDistributionQueue.actor.cpp
2020-02-21 17:41:00 -08:00
6810a03283
Add more logging to valley filler and mountain chopper
2020-02-21 10:55:14 -08:00
08c318d28a
re-added the connect lock in the fdbcli so that the timeout is not spent before a connection has been initiated (because of the handshake lock)
2020-02-20 10:43:34 -08:00
fd8a58b035
re-added support for the TLS_DISABLED flag
2020-02-19 18:37:47 -08:00
761da5a059
code cleanup
2020-02-19 17:59:45 -08:00
a21f29b586
Bugfixes. Identify the cached range correctly.
2020-02-19 16:27:06 -08:00
88d36af9c7
Fix --tls_password and add better error logging
...
This refactors all tls settings into a TLSParams object so that we can
set the password before loading any certificates.
It turns out that the FDBLibTLS code did really nice things with error
logging, but I just didn't understand openssl enough before to realize
what pieces I should be copying.
2020-02-19 00:57:05 -08:00
649fc6ba94
Merge pull request #2329 from davisp/trace-clock-source-network-option
...
Add network option for the trace clock source
2020-02-15 10:43:00 -08:00
32e285a761
Add network option for the trace clock source
...
This option allows clients to select the clock source for trace events
similar to the `--traceclock` command line parameter for `fdbserver`.
Using the `realtime` clock sources makes loading event data into
OpenTracing systems like Jaeger more useful.
2020-02-15 11:30:43 -06:00
1849939bc3
Added a delay to avoid get stuck in a loop because the request is not versioned and thus if a storage server is behind it might not know it has been assigned a shard range that a proxy thinks it has.
2020-02-12 15:01:26 -08:00
2e1d03cbe7
Addressed AJ's review comments
2020-02-12 14:57:40 -08:00
03287a0214
Fix build error.
2020-02-12 14:57:40 -08:00
57f0c11712
Address Evan's review comments
2020-02-12 14:57:40 -08:00
d934aed1d7
Because when the user issue 'getStorageByteSample' on a large key range, which can be as large as the whole DB, we need to change the behavior of 'waitStorageMetricsMultipleLocation' to avoid the case where a target key range got moved/splited by DD and thus the call to 'waitMetircs' on the corresponding storage server will return 'wrong_shard_server' error and thus the whole 'waitStorageMetricsMultipleLocation' will be retried on the large key range. What we want is to do the retry only for the key range that caused the error.
2020-02-12 14:57:40 -08:00
ff44bd2b33
Merge pull request #2639 from atn34/atn34/include-port-in-address-default
...
Enable include_port_in_address by default for api version 700
2020-02-10 09:50:59 -08:00
e71fe44ee3
Merge branch 'master' into features/icc
2020-02-08 21:33:02 -08:00
e390dbd36c
Add a non-FDBLibTLS verify peers framework to new TLS impl
2020-02-06 21:06:52 -08:00
38d8d0d675
fixed simulation
2020-02-06 19:29:31 -08:00
85cc35e81e
Merge branch 'master' into HEAD
2020-02-05 14:59:55 -08:00
058de86b88
Merge branch 'master' of https://github.com/apple/foundationdb into add-data-distribution-metrics
2020-02-05 10:07:36 -08:00
84853dd1fd
switched SSL implementation to use boost ssl
2020-02-04 14:56:40 -08:00
8449badb3e
Merge pull request #1868 from dongxinEric/fix/1827/error_instead_of_timeout
...
Send error back before put the GRV request with PRIORITY_BATCH into t…
2020-02-04 14:32:47 -08:00
d09e07f1f5
Merge remote-tracking branch 'upstream/master' into features/icc
2020-02-04 10:26:18 -08:00
4524831456
Merge pull request #2518 from vishesh/task/failmon-remove-server
...
FailureMonitoring: Server processes no longer need to talk to ClusterController
2020-02-03 17:22:50 -08:00
2ce887012c
Respect api version for include_port_in_address
2020-02-03 15:25:30 -08:00
07a3051f0e
Enable include_port_in_address by default for api version 700
...
Resolves #2607
2020-02-03 15:10:00 -08:00
7016f7903b
Fixed another build error. Do not use timeReplyIgnoreError since we don not want the logging inside that function and thus that's unnecessary anymore. Change to use ready() which basically ignores the error.
2020-01-31 15:48:29 -08:00
c1f992667b
Fix build failure
2020-01-31 14:27:47 -08:00
8d28c2a7f0
Added two new counters for transaction throttled error and remove the verbose trace event logging. Also changed a chain of 'if' statements into 'if-else' statements since they are mutal exclusive
2020-01-31 14:16:39 -08:00
1b313a4f7e
Address review comments. Rebased with latest master
2020-01-30 14:13:56 -08:00
e21426d12a
Send error back to the GRV requests with batch priority when the cluster is saturated, instead of blindly enqueue the requests and let the client timeout.
2020-01-30 14:13:56 -08:00
182dac7cd5
Convert the slow task profiler into a run loop profiler that also logs when the run loop is 100% busy for a knob-configurable duration.
2020-01-28 12:09:37 -08:00
ec418a4dcb
LocationCache should now be updated
2020-01-27 17:03:36 -08:00
291d524ffe
Some bugfixes, started to add logic for locationCache update
2020-01-27 14:13:54 -08:00
8d774d1b81
Refactoring for readability
2020-01-26 21:25:15 -08:00
1cca70d4ea
Added logic to keep locationCache up to date
2020-01-26 20:53:50 -08:00
51717c970d
Fixed management api
2020-01-24 11:00:50 -08:00
c9363e7e28
Merge branch 'master' of https://github.com/apple/foundationdb into add-data-distribution-metrics
2020-01-22 21:02:21 -08:00
daef5f011a
Merge remote-tracking branch 'apple/master' into task/failmon-remove-server
2020-01-21 13:20:15 -08:00
b0a1af1288
Added the actual read hot detection algorithm and logging machanism.
...
- When a shard has a read bandwidth larger than a threshold value(configurable via knob), and it's read-bandwidth/byte-size ratio is also larger than a threshold(configurable via knob), the corresponding shard tracker will run the algorithm
- The algorithm will divide the shard into 10MB(configurable via knob) chunks and try to find the chunk(s) that has large aforementioned ratio
- Then those ranges will be logged into TraceEvents. This will later do more like actually cache them.
2020-01-21 11:19:52 -08:00
eb64eede8d
Make a smaller range inaccessable after writing a versionstamped key
...
A transaction's read version is the lower bound of what a transaction's
commit version could be. Thus, we can narrow the conflict range of a
versionstamped key, and thus reduce the amount of the keyspace that is
rendered inaccessable, by filling in the read version on the
versionstamped key and using that as the lower bound of the conflict
range.
This allows reads to still be done to versionstampled keys lower than
the read version of the current transaction.
2020-01-16 21:41:59 -08:00
3f9d9d8b84
Merge branch 'release-6.2'
...
# Conflicts:
# CMakeLists.txt
# cmake/FlowCommands.cmake
# documentation/sphinx/source/release-notes.rst
# fdbclient/StorageServerInterface.h
# fdbserver/DataDistributionTracker.actor.cpp
# fdbserver/MasterProxyServer.actor.cpp
# fdbserver/fdbserver.actor.cpp
# flow/Knobs.h
# flow/Platform.cpp
# versions.target
2020-01-16 18:37:47 -08:00
db901d626a
Update some var names and make returned list in ResolveTransactionBatchReply as Optional
2020-01-16 14:41:06 -08:00
30b9858cf5
using std::vector instead of VectorRef to avoid unknown arena change behavior
2020-01-16 13:06:36 -08:00
1d6ac716a1
Merge remote-tracking branch 'origin' into add-data-distribution-metrics
2020-01-15 13:20:04 -08:00
72d39a31f1
Change to send back index of read_conflict_range not keys, which speeds up but fail to give narrowed keyrange
2020-01-15 10:21:51 -08:00
c93ca04ea6
Do not merge more than 100 shards together to avoid creating untrackable shards
2020-01-15 09:33:27 -08:00
1c2b116688
Push sorting and union of overlapped keyranges to clients
2020-01-13 18:08:54 -08:00
4aab9b7bc8
fix: clients would waste time attempting to read from a remote region when it was in the process of catching up
2020-01-10 12:23:59 -08:00
1cdb22c4a4
refine code according to comments
2020-01-09 15:42:21 -08:00
83ad9caf54
implemented a load balancing algorithm which evens out the number of requests processes by each proxy
2020-01-08 01:59:01 -08:00
c7de801935
Merge remote-tracking branch 'upstream/master' into features/cache-initialization
2020-01-07 16:41:51 -08:00
6e6cfaff16
Cleanup old Failure Monitoring code
2020-01-07 15:53:32 -08:00
b8ffc72cca
Use a RYW in transactionInfo to track conflicting keys
2020-01-07 13:55:05 -08:00
9c641c7a8e
First part of client handling
2020-01-07 11:42:57 -08:00
10719200c3
A hack way to call API through getRange("\xff\xff/conflicting_keys\<start_key>", "\xff\xff/conflicting_keys\<end_key>").
2020-01-06 11:22:11 -08:00
3bf971ba8b
Merge branch 'release-6.2' of github.com:apple/foundationdb into release_6.2_merge
...
# Conflicts:
# documentation/sphinx/source/release-notes.rst
# fdbserver/storageserver.actor.cpp
2019-12-12 07:13:12 -08:00
290cbb7628
Fix an internal error that occurred when using client buggify and a successful transaction was failed with commit_unknown_result.
...
Move some buggify errors into a try catch block so that the normal error handling path can run.
Initialize startTime so that it's valid in the case of errors thrown early.
2019-12-09 09:44:48 -08:00
b376b947d5
Encode conflicting keys using base64.
2019-12-06 16:21:19 -08:00
eacfdf6da3
Add a simple workload, ReportConflictingKeysWorkload, to test correctness of the API and performance overhead added to the resovler.
2019-12-06 16:21:03 -08:00
edf52e8c97
First version for reporting conflicting keys
2019-12-02 13:45:39 -08:00
ed8d3f163c
Rename hgVersion to sourceVersion.
2019-11-15 12:26:51 -08:00
4d8cfc00f6
added comment explaining quorum
2019-10-18 15:00:00 -07:00
f38d52b82d
re-introduced coordinator quorum and added some comments
2019-10-18 14:52:07 -07:00
f67eb2f371
removed smartquorum for coordinators check
2019-10-18 09:44:23 -07:00
efa2f5df6b
avoid double overwrite on failed servers and reordered removal of keys
2019-09-26 13:52:53 -07:00
450a09e117
Code Review Changes
2019-09-24 15:48:50 -07:00
40ad6f0931
created new reply types to work with flatbuffer serialization requirements
2019-09-18 13:40:18 -07:00
471e283128
Merge branch 'master' of https://github.com/apple/foundationdb into mark-ss-failed
2019-09-18 11:49:07 -07:00
b495cc697b
Merge branch 'release-6.2'
...
# Conflicts:
# CMakeLists.txt
# documentation/sphinx/source/release-notes.rst
# versions.target
2019-09-13 09:25:08 -07:00
41fd3d9467
Merge branch 'master' into remove-unused-ssi-get-version
...
# Conflicts:
# fdbclient/StorageServerInterface.h
# fdbserver/storageserver.actor.cpp
2019-09-10 08:11:43 -07:00
34dedc9a62
Fix whitespace
2019-09-05 16:44:58 -07:00
c18c4c1b83
Use a transaction option to control includePort behavior
2019-09-05 14:58:39 -07:00
11f6adf645
Treat \xff\xff prefix as 'includePort' for get_addresses_for_key
2019-09-04 17:47:40 -07:00
dc1d055b27
Merge pull request #2042 from senthil-ram/snap_cli_fix
...
fix fdbcli --exec 'snapshot create.sh' failure
2019-08-30 13:40:38 -07:00
a7237c4302
Merge pull request #2045 from atn34/disallow-scalar-network-messages
...
Disallow scalar network messages
2019-08-30 13:38:54 -07:00
b3277f2982
Fix #2009 posix compliant args for snapshot binary
2019-08-30 12:54:09 -07:00
b5f9e9f307
Move comment above if
2019-08-28 15:21:58 -07:00
6aa0ada7b1
Replace scalar root types with proper messages
2019-08-28 14:40:50 -07:00
7a9097ea01
make fdbcli --exec 'snapshot create.sh' to succeed
2019-08-27 16:44:19 -07:00
5d87443323
improved error msgs for snapshot cmd
2019-08-27 16:43:52 -07:00
7414ddd891
cleanup merge/rebase error
2019-08-27 14:39:44 -07:00
3666c0c776
added more trace lines and added timeout to safety check in test workload
2019-08-27 14:39:44 -07:00
04d514c483
added a wait to check for master proxies changed and put in a few more trace events
2019-08-27 14:39:44 -07:00
00c2025d4b
fixed removeKeys impl, adjusted test workload, and introduced extra safety checks to NativeAPI and proxy
2019-08-27 14:39:44 -07:00
a07cf5d038
addressed code review comments
2019-08-27 14:39:44 -07:00
5a877d6b14
added safety check on client to prevent removing all servers from a team
2019-08-27 14:39:43 -07:00
64ce0c3285
Remove the unused getVersion from StorageServerInterface.
2019-08-26 13:53:54 -07:00
c4004a4eea
Don't count read version requests if we've already started one. Also avoid some other work that only needs to be done if we haven't started a read version request.
2019-08-12 15:55:48 -07:00
c9fa7237f1
Merge branch 'master' of github.com:apple/foundationdb
2019-08-06 16:40:14 -07:00
ba54508c47
code cleanup
2019-08-06 16:30:30 -07:00
370ba8b841
Remove --object-serializer flag from executables
2019-08-06 09:25:40 -07:00
69e7ed3e53
Merge pull request #1932 from etschannen/master
...
Bug fixes for rare bugs found by simulation
2019-07-30 17:18:30 -07:00
63941e0d96
disable DD with a in-memory flag and use in snapv2
2019-07-30 17:04:51 -07:00
5c978f6129
fix: switchConnectionFile could get the proxies out of the clientInfo and continue connecting to the wrong cluster
2019-07-30 16:32:26 -07:00
5c98dcce6d
revert the proxy forwarding path, because it is no longer necessary as clients keep a persistent connection open with coordinators
2019-07-27 16:46:22 -07:00
b509a441e7
Merge branch 'master' into feature-skip-confirm
...
# Conflicts:
# bindings/flow/tester/Tester.actor.cpp
# bindings/go/src/_stacktester/stacktester.go
# bindings/java/src/test/com/apple/foundationdb/test/AsyncStackTester.java
# bindings/java/src/test/com/apple/foundationdb/test/StackTester.java
# bindings/python/tests/tester.py
# bindings/ruby/tests/tester.rb
# documentation/sphinx/source/api-c.rst
# documentation/sphinx/source/api-python.rst
# documentation/sphinx/source/api-ruby.rst
# documentation/sphinx/source/data-modeling.rst
# documentation/sphinx/source/developer-guide.rst
# fdbclient/vexillographer/fdb.options
# fdbserver/MasterProxyServer.actor.cpp
2019-07-27 15:08:13 -07:00
d1c7ab325b
fix: getConnectionFile could crash when connectionFile is null
2019-07-27 13:02:06 -07:00
1c4028d71e
fixed merge conflict error
2019-07-26 16:09:46 -07:00
90e3b50213
Merge branch 'master' into feature-coordinator-connection
...
# Conflicts:
# fdbclient/DatabaseContext.h
# fdbclient/NativeAPI.actor.cpp
# fdbclient/NativeAPI.actor.h
# fdbserver/workloads/KillRegion.actor.cpp
2019-07-26 15:05:02 -07:00
8149b5b352
Merge pull request #1413 from atn34/change-connection-file
...
Switch cluster file feature
2019-07-26 13:27:37 -07:00
9afd162e2f
remove snap v1 related code
2019-07-25 17:29:31 -07:00
be5d144b8b
added status information on connected clients
2019-07-25 17:15:31 -07:00
a65c9f92ed
get rid of all timeouts and other changes
2019-07-24 15:36:28 -07:00
a2f2ad96ff
code review comments and merge to master changes
2019-07-24 15:36:28 -07:00
869f77aef1
Few cosmetic edits and fixes
2019-07-24 15:36:28 -07:00
ddd4523816
bug fix in timeout & header file re-arrange in DD
2019-07-24 15:36:28 -07:00
0598779a40
snap create timeout for client call
2019-07-24 15:36:28 -07:00
ec9af66fc5
timeout snapshotdatabase call
2019-07-24 15:36:28 -07:00
62c14dae72
disable dd during snap and enable in restore
2019-07-24 15:36:28 -07:00
209448807d
snap v2: fdbclient related changes
2019-07-24 15:36:28 -07:00
9ad9bd4c1f
Merge branch 'master' of https://github.com/apple/foundationdb into change-connection-file
2019-07-24 15:22:26 -07:00
4a866290b7
Clients keep a persistent connection open with coordinators to get updates to the list of proxies
...
Status still needs to be updated with client information with information from the coordinators
2019-07-23 19:22:44 -07:00
e29a6ea280
Merge pull request #1871 from bnamasivayam/tr-priority-add-client-log
...
Track the priority of sampled Transaction as part of GetReadVersion e…
2019-07-22 13:04:52 -07:00
3507bfd52f
Trigger masterProxiesChangeTrigger in switchConnectionFileImpl
2019-07-22 11:13:07 -07:00
af267ba053
Track the priority of sampled Transaction as part of GetReadVersion event.
2019-07-19 17:31:49 -07:00
c3a8ae4752
Merge pull request #1791 from fzhjon/fetch-keys-requests-priority
...
Introduce priority to fetchKeys requests from data distribution
2019-07-19 14:54:51 -07:00
bc5c65e5ab
Merge pull request #1756 from jzhou77/db-option
...
Add transaction getApproximateSize() API
2019-07-19 08:33:24 -07:00
02de53160d
only skip confirm epoch live if CAUSAL_READ_RISKY is enabled
...
time checked on the proxy should be less than the time waited by the master to account for clock speed differences
setting REQUIRED_MIN_RECOVERY_DURATION and ENFORCED_MIN_RECOVERY_DURATION to 0 will go back to the old behavior
2019-07-12 17:58:16 -07:00
d5051b08dd
Make trace event field lengths (and total event sizes) default knobified and configurable. Add a transaction option to control the field length of transaction debug logging. Make the program start command line field less likely to be truncated.
2019-07-12 16:12:35 -07:00
9001f5d23f
Fix bug introduced in merge commit
2019-07-12 14:02:38 -07:00
969957e619
Merge branch 'master' into change-connection-file
2019-07-12 11:39:19 -07:00
f4366e69ca
Unknown options should not be used internally (i.e. underneath thread-safe API). This commit removes various checks that options exist and replaces them with an ASSERT.
2019-07-11 11:25:39 -07:00
652cd77689
fixed merge conflicts and use new TaskPriority enum class
2019-07-11 09:56:59 -07:00
f12a3909f3
renamed workloads and made code style adjustments
2019-07-11 09:56:58 -07:00
1e9d31597c
removed extra parameter from getRange, added knob to guard new changes, and adjusted style/formatting in several places
2019-07-11 09:56:58 -07:00
f707d186fe
added new priority for fetchkeys requests and adjusted ddmetrics workload to run parallel with mako
2019-07-11 09:56:58 -07:00
bbef631872
fix: do not access optionInfo unless the option already exists in the map
2019-07-10 18:48:54 -07:00
0ad2d2d16e
Add binding test for getApproximateSize API
2019-07-10 14:58:07 -07:00
8ef8b59fcc
Use ThreadFuture for getApproximateSize
...
Change return type to int64_t and fix C and Python binding to use the correct
type.
2019-07-10 14:58:07 -07:00
c50a675bf0
Add transaction getApproximateSize() API
...
The size is the summation of expected size of mutations, read conflict ranges,
and write conflict ranges.
2019-07-10 14:51:52 -07:00
7e919e361c
Merge pull request #1817 from etschannen/feature-proxy-forward
...
Proxies will forward clients to the next generation
2019-07-10 13:53:12 -07:00
a174178be1
Merge branch 'master' into fix-mvc-default-options
...
# Conflicts:
# fdbclient/NativeAPI.actor.cpp
2019-07-09 18:33:14 -07:00
7ad0d1a12b
Merge branch 'master' into feature-proxy-forward
...
# Conflicts:
# fdbclient/NativeAPI.actor.cpp
2019-07-09 17:26:15 -07:00
cce00bb413
Merge pull request #1808 from ajbeamon/improved-transaction-metrics
...
Improve TransactionMetrics
2019-07-09 16:46:17 -07:00
15ecba59db
Merge branch 'master' into fix-mvc-default-options
...
# Conflicts:
# documentation/sphinx/source/release-notes.rst
2019-07-09 15:28:12 -07:00
fdd580c878
Restore some variable initializations that were unintentionally removed.
2019-07-09 15:00:11 -07:00
ae6c3e013a
monitorClientInfo: Wait for master proxy endpoint failures than triggers
...
This will not initiate request to get get new set of proxy unless we
know for a fact that endpoint has indeed failed, not just because the
connection to Peer was closed as it was sitting idle.
2019-07-09 14:24:16 -07:00
1582a2a24d
Merge branch 'master' of https://github.com/apple/foundationdb into change-connection-file
2019-07-09 13:41:54 -07:00
a5a6f8431c
Add a random UID to TransactionMetrics in case a client opens multiple connections and also a field to indicate whether the connection is internal. Convert some of the metrics to our Counter object instead of running totals.
2019-07-08 14:01:04 -07:00
c348b3da51
After a proxy dies, it will remain alive for an additional 10 seconds to forward clients to the new proxies
2019-07-08 12:53:40 -07:00
6b6012ee7b
Add a break to setOption() switch statement. Better detection of missing options (and logging for present options).
2019-07-02 15:42:53 -07:00
3f6ba3d737
Remove space
2019-07-02 11:18:45 -07:00
1cf449db10
Undo formatting changes to otherwise unchanged code
2019-07-02 11:17:47 -07:00
e7218bbb28
Restore retry limiting on the client sampling transaction
2019-07-02 11:16:00 -07:00
2035b36257
Make default and persistent options specifyable via annotations to fdb.options. Fix some issues with persisting these options in the multi-version client. Make size limit option not persistent.
2019-06-28 13:24:32 -07:00
bf883d7055
Merge remote-tracking branch 'upstream/master' into flowlock-api
2019-06-25 14:26:50 -07:00
24937d8125
Merge pull request #1744 from vishesh/task/monitor-leader-on-demand
...
Fix setting enClientFailureMonitor global for client
2019-06-25 13:38:59 -07:00
7a500cd37f
A giant translation of TaskFooPriority -> TaskPriority::Foo
...
This is so that APIs that take priorities don't take ints, which are
common and easy to accidentally pass the wrong thing.
2019-06-25 02:47:35 -07:00
f38414e26d
Changes error type on aborted read
...
Throw transaction_too_old instead of all_alternatives_failed when the
cluster file changes while a read request is outstanding
2019-06-24 10:01:41 -07:00
5abc891b12
undo the partial retry logic in NativeAPI
2019-06-24 09:36:07 -07:00
f7cd9438d3
Use camelCase instead of snake_case
2019-06-24 09:32:14 -07:00
70ebcb3baf
Fix quietDatabase timeouts
...
Update the implementation to interact with the new "don't maintain a
connection to the cluster controller unless necessary" change, and
unlock the originalDB at the end of the workload.
2019-06-22 14:27:30 -07:00
e8c75505d3
Merge pull request #1725 from jzhou77/db-option
...
Add transaction size option
2019-06-21 08:25:34 -07:00
afb0dbcd1c
Merge branch 'master' of https://github.com/apple/foundationdb into change-connection-file
2019-06-20 19:11:29 -07:00
3a63d053e9
Address review comments for PR#1725
2019-06-20 14:06:32 -07:00
844dd60202
FDB compiling with intel compiler
2019-06-20 09:29:01 -07:00
ebc7e7cb4a
Use old transaction size limt during reset
2019-06-19 23:16:27 -07:00
357c9ba0fb
Refactor code
2019-06-19 20:41:53 -07:00
8af92a99ed
Abort reads when connection file changes
2019-06-19 16:53:14 -07:00
9c2257a0e5
Add transaction size option
2019-06-19 07:45:23 -07:00
6e753dc614
Fix setting enClientFailureMonitor global for client
2019-06-19 02:44:30 -07:00
978a5b2d14
Buggified `get`, added comment
2019-06-17 10:27:52 -07:00
5b7dc4eeb8
Added proxy_memory_limit_exceeded to possible commit errors
2019-06-17 09:53:43 -07:00
24ca369425
Fixed probability calculation
2019-06-17 09:51:44 -07:00
7483cba26e
Address review comments and added documentation
2019-06-16 09:52:40 -07:00
6ea75713cb
Overall framework and first buggify entries
2019-06-16 09:09:09 -07:00
f9bfd74dd0
Monitor leader only when proxies are unknown or any dies
...
FDB clients talk to coordinators to monitor the leader, and then ask
them for proxies. Once proxies are known, the clients still keep
talking to coordinators. This patch, stops this monitoring and
reconnects to coordinators only if one of the proxy is no longer
available.
2019-06-14 13:27:57 -07:00
e2b6acffce
Merge branch 'master' into add-data-distribution-metrics
2019-06-12 16:12:16 -07:00
b473a8a830
changed on-the-wire format to use serialized flatbuffers, added cycletest to workload, and fixed small bug in trace
2019-06-11 15:45:06 -07:00
bc03421d05
Open Database as switchable only for client
2019-06-11 13:58:22 -07:00
d4578a49d8
Use 'SwitchConnectionFile' language uniformly
2019-06-11 13:58:22 -07:00
b17956fef5
Remove unnecessary TraceEvent
2019-06-11 13:58:22 -07:00
02e173b601
Add changeConnectionFile method to Transaction
...
Also add tests
2019-06-11 13:58:22 -07:00
5bdf5aaeb6
Merge pull request #1662 from etschannen/master
...
Merge 6.1 into master
2019-06-06 13:57:34 -07:00
dbfa746494
Don't set the deterministic random seed with platform::getRandomSeed since that is what it does by default.
2019-06-05 11:48:29 -07:00
31e82a6d6d
Clients should seed the deterministic random number generator on the network thread.
2019-06-05 11:48:29 -07:00
29b96414e2
Merge branch 'release-6.1'
...
# Conflicts:
# documentation/sphinx/source/release-notes.rst
# fdbclient/NativeAPI.actor.cpp
# fdbserver/Coordination.actor.cpp
# flow/Arena.h
# versions.target
2019-06-03 18:49:35 -07:00
a6bee65f11
Merge branch 'master' into add-data-distribution-metrics
2019-05-30 11:11:49 -07:00
362c2bf1e6
improved the cpu efficiency of printable
2019-05-29 14:55:45 -07:00
4bcb590f12
g_random -> deterministicRandom()
2019-05-28 22:07:46 -07:00
b17ad85497
exec op not supported when log_anti_quorum > 0
2019-05-28 22:07:46 -07:00
dcb99c5138
txn to disable tlogPop to be timedout
...
If the disable tlog pop txn takes more than 30 seconds then
tlog will automatically start enabling pop, fix is to timeout
the txn if it takes more than 10 seconds and retry a new
txn to disable tlog pop.
2019-05-28 22:07:46 -07:00
d3a179b6f9
Multiple bug fixes
...
- wait for snapTLogFailKeys in a loop, otherwise in some race
condition it can cause a false assert
- in single region, there does not seem to be a guarantee of
tagLocalityListKey for a given DC ID, avoiding that assert for now
- to find the workers that are coordinators, looking up by primary
address is not sufficient in some cases, hence looking by both
primary and secondary address
- test make files to reflect the location of the new test cases
2019-05-28 22:07:46 -07:00
bb474dc323
if recovery < fully_recovered then fail the exec
...
Will do more cleanup, pushing it for a test run in CI
2019-05-28 22:07:46 -07:00
925499954b
New status cluster_not_fully_recovered
2019-05-28 22:07:46 -07:00
591ff96b93
increase retry and use eat instead of parsing
2019-05-28 22:07:46 -07:00
936ffc2dde
rebase related changes
2019-05-28 22:07:46 -07:00
ec7834e2f7
code re-orgnaization and address comments
2019-05-28 22:07:46 -07:00
61e93a9304
Address review comments and minor fixes
2019-05-28 22:07:46 -07:00
89b7a052f5
Bug fixes for snapping coordinators
2019-05-28 22:07:46 -07:00
17ecba8313
trace cleanup and other indentation changes
2019-05-28 22:07:46 -07:00
898bed66c1
Allow only whitelisted binary path for exec op
2019-05-28 22:07:46 -07:00
aa79480d69
changes to make fdbfork asynchronous
2019-05-28 22:07:46 -07:00
f7ba0635ef
Make Exec op the first op in the batch
2019-05-28 22:07:46 -07:00
382b246930
trace change and retain fitness file after restore
2019-05-28 22:07:46 -07:00
3d5998e9dd
tlog: when pops are disabled, store them & replay
...
In Tlogs, disable pop is done whlie taking snapshots. Earlier, tlogs
were ignoring the pops if it got pop requests when pops were
disabled. In this change, instead of ignoring the pop - it remembers
the list of pops in-memory and plays them once the popping is
enabled.
2019-05-28 22:07:46 -07:00
72dd067173
Trace message changes and fix few FIXMEs
2019-05-28 22:07:46 -07:00
69edefe68b
Snapshot based backup and resotre implementation
2019-05-28 22:07:46 -07:00
03cebd6032
renamed function from storage metrics to data distribution metrics
2019-05-28 09:52:27 -07:00
603721e125
Merge branch 'master' into thread-safe-random-number-generation
...
# Conflicts:
# fdbclient/ManagementAPI.actor.cpp
# fdbrpc/AsyncFileCached.actor.h
# fdbrpc/genericactors.actor.cpp
# fdbrpc/sim2.actor.cpp
# fdbserver/DiskQueue.actor.cpp
# fdbserver/workloads/BulkSetup.actor.h
# flow/ActorCollection.actor.cpp
# flow/Net2.actor.cpp
# flow/Trace.cpp
# flow/flow.cpp
2019-05-23 08:35:47 -07:00
7b4f2907da
addressed code review changes
2019-05-22 16:58:28 -07:00
e339ab890b
added timeout and shard limiting behaviour
2019-05-21 14:13:09 -07:00
0984d272e1
initial implementation of get-dd-metrics
2019-05-21 14:13:09 -07:00
b8e7fc1b84
Refactor: add std:: qualifier and use emplace_back
2019-05-17 09:38:50 -10:00
9eeb48c43d
Allow to turn on object serializer
...
This commit includes functionality to turn on
the object serializer for network communication.
This is done the following way:
- On incoming connections, a process will detect
whether the client supports the object serializer
and will only serialize responses with it, if it does
- On outgoing connections, the command line flag is used
to determine whether the object serializer should be used
to send data.
This way, a cluster can run in mixed mode. To upgrade one
can upgrade one process at a time and set the flag one process
at a time.
This is how this is tested on the simulator:
- The command line flag can take three options: on, off,
and random.
- For off, the object serializer will never we used.
- For on, the object serializer will be always used.
- For random, the simulator will flip a coin for each
process it starts up.
2019-05-13 14:15:22 -07:00
5f55f3f613
Replace g_random and g_nondeterministic_random with functions deterministicRandom() and nondeterministicRandom() that return thread_local random number generators. Delete g_debug_random and trace_random. Allow only deterministicRandom() to be seeded, and require it to be seeded from each thread on which it is used.
2019-05-10 14:01:52 -07:00
781b6ece77
Fix OPEN_FOR_IDE -Wunused-variable warnings
...
CC #1255 , #1173
2019-04-16 15:28:01 -07:00
6207d724f8
Fix all -Wunused-variable warnings
2019-04-15 18:13:00 -07:00
cd5c9d91fa
Merge pull request #1443 from etschannen/master
...
Merge 6.1 into master
2019-04-10 17:43:07 -07:00
21c0ba555c
Merge branch 'release-6.1'
...
# Conflicts:
# documentation/sphinx/source/release-notes.rst
# versions.target
2019-04-08 18:38:42 -07:00
538b431656
Apply suggestions from code review
2019-04-08 14:55:58 -07:00
a7288e1325
Throw process_behind instead of future_version when all storage nodes on a team are behind. process_behind gets the same backoff behavior as not_committed. Add proxy_memory_limit_exceeded to the retryable predicate.
2019-04-08 14:21:24 -07:00
d01cbf3455
Addressed code review comments
2019-04-05 13:12:20 -07:00
39ecbedd74
Fixed compilation errors on OS X & gcc8
2019-04-05 13:12:19 -07:00
1c16f87a4e
Remove trace-calls to printable (in non-workloads)
2019-04-05 13:12:19 -07:00
72203ba47a
Merge commit '56f3f0b1bc60604f965152d856ae29a591227703'
2019-04-01 18:45:38 -07:00
8714394d42
increase the priority of the client’s version batch timeout, so that we prefer issuing the batch over other possible work
2019-04-01 18:37:40 -07:00
d882c060bf
Merge commit '5dd6396eed0de0dfea6cf9eecc307995eff5cedc'
2019-03-28 18:00:55 -07:00
b6008558d3
renamed BinaryWriter.toStringRef() to .toValue(), because the function now returns a Standalone<StringRef>()
...
eliminated an unnecessary copy from the proxy commit path
eliminated an unnecessary copy from buffered peek cursor
2019-03-28 11:52:50 -07:00
7c02ee6fdd
Fix compiler warning about unreferenced exception variable
2019-03-26 13:43:47 -07:00
5e03e178de
Merge pull request #1345 from ajbeamon/support-multiple-client-or-worker-issues
...
Add support for a client or worker having multiple issues.
2019-03-24 17:27:50 -07:00
e3400c13ae
fixed a performance regression related to broadcasting a read version to too many transactions simultaneously
2019-03-22 18:37:39 -07:00
55a9db1994
spaces to tabs 🤮
2019-03-22 12:58:32 -04:00
4eb5715689
Add support for a client or worker having multiple issues.
2019-03-22 08:29:41 -07:00
64e45e6826
retry limit and max delay transaction options are no longer reset after onError
2019-03-21 18:50:02 -04:00
22f592ce6e
reset the timeout only if the API version is less than 610 to allow transactions with longer timeouts than the database default
2019-03-21 16:47:12 -04:00
148a21a31c
set the timeout actor upon transaction creation if necessary
...
initialize lockaware and maxbackoff in options constructor rather than NativeAPI transaction constructor
2019-03-20 11:13:38 -06:00
5f73b7d6ae
Resolves #775 : Support setting Transaction options at the Database level
...
This selects a few useful transaction options that users may want to set for every transaction and adds them to the database options.
2019-03-20 11:13:38 -06:00
f9aad46573
made use_provisional_proxies a transaction option
2019-03-19 18:44:37 -07:00
5b9c45ea0b
clients do not attempt to connect to provisional proxies
2019-03-19 13:37:50 -07:00
19d27b6271
Merge branch 'master' into fix-compiler-warnings
2019-03-15 10:35:36 -07:00
85b3f11e71
Fix various compiler warnings
2019-03-15 10:34:57 -07:00
59b41b03ed
Merge branch 'master' into fix-threadsafedatabase-safety
...
# Conflicts:
# fdbclient/NativeAPI.actor.cpp
2019-03-14 14:50:14 -07:00
e12d242d18
Address review comments.
2019-03-14 13:42:03 -07:00
5a10bf5dfc
Merge branch 'master' into mengxu/tls-switch-status-PR
2019-03-14 10:35:12 -07:00
cab284027d
Be more careful about accessing ThreadSafeDatabase's db member. Preallocate the db on the calling thread so that other callers can access the pointer immediately.
2019-03-14 10:26:22 -07:00
e30e2af1f3
ClientKnobs: Add CHECK_CONNECTED_COORDINATOR_NUM_DELAY
2019-03-13 16:54:56 -07:00
85c24b0067
Merge branch 'master' into mengxu/tls-switch-status-PR
2019-03-12 15:20:54 -07:00
22f5624494
TLS Status: Reduce cluster controller load
...
When the coordinator changes, we use delayedAsyncVar() to reduce
the frequency for cluster controller to send the updated connectedCoordinatorsNumDelayed
to clients.
This help reduce the cluster controllers workload
2019-03-12 15:08:10 -07:00
46f4b02807
TLS Status: Resolve review comments
...
Use connectedCoordinatorsNumDelayed to reduce the load on cluster controller;
Set connectedCoordinatorsNum to null by default for monitorLeader()
2019-03-11 17:10:08 -07:00
2627bcd35e
Merge branch 'master' into feature-metadata-version
2019-03-10 21:13:28 -07:00
04880e3d4d
Merge branch 'master' into mengxu/tls-switch-status-PR
2019-03-06 13:41:16 -08:00
c6a65389ae
Remove noexcept macro and replace with BOOST_NOEXCEPT.
...
BOOST_NOEXCEPT does what the noexcept macro was supposed to do, but in a
way that is correctly maintained over time.
2019-03-05 22:06:12 -08:00
820548223a
Status: connected_coordinators misc minor changes
...
Change the rst document file;
Change the coding style to be consistent with the nearby code;
Ensure we always initilize the connectedCoordinatesNum to 0
even when the variable is not used.
2019-03-05 21:45:18 -08:00
b7a52e81e2
Status: Count connected coordinators per client
...
A client will always try to connect all coordinators.
This commit let Status track the number of connected coordinators
for each client.
This allows us to do canary in coordinators. For example,
when we switch from non-TLS to TLS, we can switch 1 coordinator
from non-TLS to TLS. This can help check if a client has the ability
to connect through TLS.
We can make the non-TLS to TLS switch for each coordinators
one by one. This avoid the risk of losing connection in the switch.
2019-03-05 21:21:23 -08:00
82d957e0bb
Merge pull request #1178 from vishesh/task/issue-963-IPv6
...
IPv6 Support
2019-03-05 17:14:16 -08:00
f1897f3eb6
Merge branch 'master' into feature-metadata-version
...
# Conflicts:
# fdbclient/NativeAPI.actor.cpp
2019-03-04 21:06:16 -08:00
988add9fb5
cache the metadataVersion for commits, so that doing setVersion with the commit version of a different transaction will
2019-03-04 16:48:34 -08:00
cbc5df5e93
Removed unnecessary Trace line
2019-03-04 16:31:40 -08:00
56ae46f89e
Client lazily fetches health metrics from proxies
2019-03-04 14:16:39 -08:00
e93cd0ff21
Add some checks and comments to IPv6 changes #963
2019-03-04 14:12:45 -08:00
41d18db7b9
fix: update the encoding of AddressExclusion in SystemData #963
2019-03-04 14:12:45 -08:00
57832e625d
net: Support IPv6 #963
...
- NetworkAddress now contains IPAddress object which can be either
IPv4 or IPv6 address. 128bits are used even for IPv4 addresses,
however only 32bits are used when using/serializing IPv4 address.
- ConnectPacket is updated to store IPv6 address. Backward compatible
with old format since the first 32bits of IP address field is used
for serialization of IPv4.
- Mainly updates rest of the code to use IPAddress structure instead
of plain uint32_t.
- IPv6 address/pair ports should be represented as `[ip]:port` as per
convention. This applies to both cluster files and command line
arguments.
2019-03-04 14:12:41 -08:00
075fdef31a
Merge branch 'master' into feature-metadata-version
...
# Conflicts:
# fdbclient/DatabaseContext.h
2019-03-03 22:58:45 -08:00
1576d50ee9
Removed unnecessary Trace line
2019-03-03 17:09:03 -08:00
015997f2f9
Simplified updateHealthMetricsActor
2019-03-02 18:01:23 -08:00
39f612d132
Merge branch 'master' of https://github.com/apple/foundationdb into add-health-metrics
2019-03-02 17:07:00 -08:00
6c45ffc56f
Added exception handling to setNetworkOption
...
Handle non-integer option values for sendDetailedHealthMetrics option by
throwing invalid_option_value
2019-03-02 15:51:51 -08:00
c1de93f467
fix: binary search could get stuck in an infinite loop
...
fix: avoid picking a read version which could be before the last real commit
fix: we must wait on metadataVersionKey in case it is not already cached
fixed review comments
2019-03-02 13:55:41 -08:00
94385447bc
Status: Get if client configured TLS
...
To understand if all clients have configured TLS,
we check the tlsoption when a client tries to open database.
This is similar to how we track the versions of multi-version clients.
2019-03-01 15:17:01 -08:00
2168b14834
Merge branch 'master' into feature-metadata-version
2019-02-28 17:45:55 -08:00
3da85f3acd
implemented the \xff/metadataVersion key, which can be used by layers to help them cheaply cache metadata and know when their cache is invalid
2019-02-28 17:45:00 -08:00
af69ba035a
Merge pull request #1200 from bnamasivayam/master
...
User provided Client Transaction id merging
2019-02-28 11:16:44 -05:00
d2bde4e55b
Merge branch 'master' of https://github.com/apple/foundationdb into add-health-metrics
2019-02-27 16:30:25 -08:00
cc0aac588a
Merge branch 'client-transaction-id'
...
* client-transaction-id:
Add user provided transaction id as part of the format in ClientTransactionProfileCorrectness workload.
Addressed review comments.
Apply suggestions from code review
Add support for client provided transaction identifier to be logged as part of trace logging or transaction profiling.
2019-02-27 13:11:15 -08:00
7d7db54515
Addressed review comments.
2019-02-27 12:42:45 -08:00
999ee68609
Improve avoidance of transient issues when logging IncorrectClusterFileContents SevWarnAlways events by making it time based.
2019-02-27 10:08:24 -08:00
ff9a7cb2f1
Combined proxy health metrics replies into single message type
2019-02-23 10:13:43 -08:00
edc0c5bf2b
Client resends health metrics request when proxies change
2019-02-22 12:30:36 -08:00
fa96b8dd33
Merge branch 'master' of https://github.com/apple/foundationdb into add-health-metrics
2019-02-20 16:56:16 -08:00
0bb60e5a3b
Use proper fwd decl in NativeAPI
...
Also NativeAPI.h -> NativeAPI.actor.h
2019-02-19 15:16:59 -08:00
907446d0ce
Merge remote-tracking branch 'apple/master' into task/tls-upgrade
2019-02-14 11:37:38 -08:00
65136a2ecd
Forward declare actors with ACTOR keyword. #1148
...
There are several more occurrences of this, but they're in .h files that
now need to be .actor.h files. This gets the easy ones out of the way.
2019-02-12 17:56:20 -08:00
067a445e06
Replace unused _ variables with wait(success(...))
2019-02-12 17:30:30 -08:00
91d859d791
Add support for client provided transaction identifier to be logged as part of trace logging or transaction profiling.
2019-02-12 13:54:24 -08:00
6da5971e79
Guard all versions.h to not break old WIN32 build
2019-02-08 16:06:00 -08:00
685242fbfc
Merge pull request #1092 from atn34/trace-format-network-option
...
Add trace_format network option
2019-02-08 10:52:50 -08:00
7e26b4ef0d
Address comments from PR
2019-02-07 15:37:04 -08:00
8a94d80deb
fdbservice and fdbrpc now compiling
2019-02-07 15:37:04 -08:00
5d15aa2794
Client must actually update health metrics after receiving GetHealthMetricsReply
2019-02-02 17:04:53 -08:00
c22b3c35ad
Added updateHealthMetricsActor actor to each DatabaseContext object, to poll proxies for health metrics
2019-02-01 13:23:10 -08:00
e549def5a9
Added send_detailed_health_metrics network option, to be enabled when the client wants to receive per-process health metrics.
2019-01-31 18:46:11 -08:00
e9ddd94e27
The failure monitor is given a list of all IP addresses associated with a process
...
The connect packet includes the correct remote address
Did a lot of code cleanup
Simulation test mixed TLS and non-TLS listeners on the same process
2019-01-31 18:20:14 -08:00
e055fdab03
Better diagnostics for unrecognized trace format
2019-01-29 13:08:47 -08:00
768d7678be
Add trace_format network option
2019-01-28 15:14:17 -08:00
1d7fec3074
Merge commit '048bfc5c368063d9e009513078dab88be0cbd5b0' into task/tls-upgrade-2
...
# Conflicts:
# .gitignore
2019-01-24 17:43:06 -08:00
04b94e74c7
Merge pull request #942 from ajbeamon/remove-cluster-from-bindings
...
Remove cluster from bindings
2019-01-24 14:39:09 -08:00
d4d5740282
* Add Optional.map and ErrorOr.map.
...
* Rename Optional/ErrorOr cast_to to castTo.
* Make printable(Optional<T>) templated rather than restricted to StringRef types.
* Fixes bug in (unused) ErrorOr.castTo where an ErrorOr that was not set would lose its error.
2019-01-11 09:03:38 -08:00
51b89ae083
WIP
2019-01-09 07:41:02 -08:00
e8e01b2406
Remove unused localAddress parameter from newNet2 and Net2 classes
2018-12-13 13:36:52 -08:00
890a0f3e82
Merge branch 'master' into remove-cluster-from-bindings
...
# Conflicts:
# bindings/flow/fdb_flow.actor.cpp
2018-11-26 12:01:03 -08:00
4b5d0b4e2c
Merge branch 'release-6.0'
...
# Conflicts:
# documentation/sphinx/source/release-notes.rst
# fdbclient/AsyncFileBlobStore.actor.cpp
# fdbclient/AsyncFileBlobStore.actor.h
# fdbclient/BlobStore.actor.cpp
# fdbclient/BlobStore.h
# fdbclient/HTTP.actor.cpp
# fdbclient/ManagementAPI.actor.cpp
# fdbclient/NativeAPI.actor.cpp
# fdbrpc/LoadBalance.actor.h
# fdbrpc/batcher.actor.h
# fdbrpc/fdbrpc.vcxproj
# fdbrpc/sim2.actor.cpp
# fdbserver/DataDistribution.actor.cpp
# fdbserver/DataDistributionTracker.actor.cpp
# fdbserver/SimulatedCluster.actor.cpp
# fdbserver/TLogServer.actor.cpp
# fdbserver/masterserver.actor.cpp
2018-11-10 13:04:24 -08:00
a654183f63
Merge pull request #791 from ajbeamon/remove-cluster-from-iclientapi
...
Remove cluster from IClientApi (phase 2 of removing DB names)
2018-11-10 10:16:18 -08:00
6a406bae72
Merge pull request #896 from ajbeamon/downgrade-incorrect-cluster-file-event
...
Downgrade the severity of IncorrectClusterFileContents the first time…
2018-11-10 10:06:36 -08:00
187e507e53
Downgrade the severity of IncorrectClusterFileContents the first time it is logged to avoid transient issues that appear like the cluster file hasn't been updated (e.g. the cluster file is shared between multiple processes).
2018-11-05 09:28:08 -08:00
593fa092f3
added a fixme
2018-11-04 19:54:20 -08:00
bf6545a9cf
clients cache storage server interfaces individually, instead of as a team. This is needed because in fearless every shard has storage servers from two separate teams, leading to a lot of possible combinations
...
allAlternatives failed logic was simplified, because we are already doing a global rate limiting, so a per shard limit is unnecessary
reduced unnecessary state variables in waitMetrics requests
2018-11-02 13:15:09 -07:00
268093a96d
Adjust all includes to be relative to the root.
...
Remove the use of relative paths. A header at foo/bar.h could be included by
files under foo/ with "bar.h", but would be included everywhere else as
"foo/bar.h". Adjust so that every include references such a header with the
latter form.
Signed-off-by: Robert Escriva <rescriva@dropbox.com>
2018-10-19 17:35:33 +00:00
d5357d2439
Update all of the bindings to use the new C API and deprecate or remove unneeded parts of their APIs.
2018-10-02 15:28:46 -07:00
3eb4355a48
Some various cleanup and fixes. Added "Cluster" to TransactionMetrics trace event.
2018-09-25 15:06:19 -07:00
c831051474
This removes the idea of clusters from IClientApi.
2018-09-21 15:58:14 -07:00
90301f497f
Merge branch 'release-6.0'
...
# Conflicts:
# fdbclient/ManagementAPI.actor.cpp
# fdbrpc/FlowTransport.actor.cpp
# fdbrpc/TLSConnection.actor.cpp
# fdbserver/DataDistribution.actor.cpp
# fdbserver/Status.actor.cpp
# fdbserver/storageserver.actor.cpp
# fdbserver/workloads/StatusWorkload.actor.cpp
# versions.target
2018-09-05 16:06:33 -07:00
d42f0e9c44
Merge pull request #618 from ajbeamon/log-options
...
Log network, cluster, and database options when they are set.
2018-09-05 10:36:40 -07:00
2a97139d5d
This is the first step in eliminating the usage of database names in our code. The C API remains the same, but underneath that all usage of database names is eliminated.
2018-08-16 10:24:12 -07:00
bc0832bb55
Deprecate TLS_plugin option. Don't remove deprecated methods from Go bindings, since that breaks compatibility with old API versions.
2018-08-15 10:11:08 -07:00
3c03ddf91e
Remove value logging for options. Add some additional logging for setting an external client library and setting knobs.
2018-08-15 09:42:49 -07:00
fb31a6999f
Rewrite all files to have #include actorcompiler.h as the last include.
2018-08-14 15:50:26 -07:00
535b5701e5
Rewrite all `Void _ = wait(...)` -> `wait(...)`.
...
This takes advantage of the new actorcompiler functionality to avoid
having duplicate definitions of `Void _` when trying to feed the
un-actorompiled source through clang.
2018-08-14 15:50:26 -07:00
c38d5247fd
Initialize client TLSOptions before first use.
...
Because initializing them after first use is called SEGFAULT.
This affects both fdbcli and fdbbackup when passed TLS options. I didn't track
down what introduced this, I just fixed it.
2018-08-06 18:08:45 -07:00
538e684f1c
Merge branch 'release-6.0'
...
# Conflicts:
# versions.target
2018-08-03 11:41:46 -07:00
1c29275672
call all methods which could disable a trace event before it is initialized. In practice this means calling .error first, then .suppressFor, then all your details.
2018-08-01 14:30:57 -07:00
d7f26cb744
Initialize client TLSOptions before first use.
...
Because initializing them after first use is called SEGFAULT.
This affects both fdbcli and fdbbackup when passed TLS options. I didn't track
down what introduced this, I just fixed it.
2018-07-31 16:09:11 -07:00
9ea0f0a5ae
Fixed problem with stack initialization of TLS Options class
2018-07-03 15:02:53 -07:00
c3f88dbfe1
Merge branch 'master' of github.com:apple/foundationdb into tls-static
2018-07-01 23:13:57 -07:00
ef8de426d3
Changed the TLS_DISABLED macro
...
Disable TLS within Windows until working
2018-06-26 12:08:32 -07:00
c6a2207577
fix: stop client sampling during the consistency check
2018-06-20 16:24:37 -07:00
f8ce1de601
Added support for compiling TLS into binaries
2018-06-20 09:21:23 -07:00
99c9958db7
Some more trace event normalization
2018-06-08 13:57:00 -07:00
e5488419cc
Attempt to normalize trace events:
...
* Detail names now all start with an uppercase character and contain no underscores. Ideally these should be head-first camel case, though that was harder to check.
* Type names have the same rules, except they allow one underscore (to support a usage pattern Context_Type). The first character after the underscore is also uppercase.
* Use seconds instead of milliseconds in details.
Added a check when events are logged in simulation that logs a message to stderr if the first two rules above aren't followed.
This probably doesn't address every instance of the above problems, but all of the events I was able to hit in simulation pass the check.
2018-06-08 11:11:08 -07:00
216404de45
Merge branch 'release-5.2' of github.com:apple/foundationdb
...
# Conflicts:
# documentation/sphinx/source/release-notes.rst
# versions.target
2018-06-06 15:25:37 -07:00
e4e06321c7
fix: Read-only transactions that get committed would fail if the readOnly option is set. They would also be counted in the transactionsCommitStarted metric.
2018-06-05 12:10:28 -07:00
529d0497f1
Proxy going OOM when applying high volumes of writes to a proxy, particular in a sudden fashion before ratekeeper can control the workload.
...
Address this issue by proactively monitoring the memory used by commit batches and dropping requests if a certain memory limit is exceeded.
2018-06-01 15:21:40 -07:00
d9c702a9e3
Merge release-5.1 into release-5.2
2018-05-30 09:09:55 -07:00
520aaf731d
merge release 5.2 into master
2018-05-10 14:33:08 -07:00
714da951a6
refactored getKeyLocation code to remove duplication
2018-05-10 10:05:41 -07:00
8f984cb2c9
Merge branch 'release-5.2'
...
# Conflicts:
# fdbrpc/TLSConnection.h
2018-05-10 09:13:22 -07:00
e860cc5994
Merge branch 'release-5.2' of github.com:apple/foundationdb into release-5.2
2018-05-09 17:22:19 -07:00
98e07fd92e
fix: clear the client's storage server interface cache on endpoint failure for all request types
2018-05-09 17:11:54 -07:00
d3450ce5b0
Merge pull request #343 from bnamasivayam/tls-plugin
...
Tls plugin
2018-05-09 16:35:53 -07:00
7591931a09
Revert "Make tls_verify_peers as a comma separated string of constraints."
...
This reverts commit 2033847e4b
2018-05-09 14:40:36 -07:00
2033847e4b
Make tls_verify_peers as a comma separated string of constraints.
2018-05-09 14:37:39 -07:00
8de914a81f
use contents() instead of address of in withPrefix and withSuffix ; whitespace fixes
2018-05-09 09:01:22 -07:00
e8b7f4b190
Add password support for tls.
2018-05-08 20:46:31 -07:00
49af5d685b
Restore previous behavior of not specifying peer_verify option means disable checking.
2018-05-08 18:54:44 -07:00
d3b5cfb93c
Support latest TLS plugin.
...
Add support for https in backup.
2018-05-08 16:28:13 -07:00
ca720e1540
Merge pull request #297 from apple/release-5.2
...
Merge 5.2 to Master
2018-05-08 12:04:20 -07:00
8987d21df3
use new semantics when writing position of vs key in native api
2018-05-08 08:57:09 -07:00
464e2cdbf0
change SetVersionstampedKey and SetVersionstampedValue behavior based on API version to make them consistent
2018-05-08 08:57:09 -07:00
f848c68b73
TraceEvents in FileBackupAgent that include BeginFile should be encoding them with printable because of the possible inclusion of the null byte. This causes the detail to terminate prematurely.
2018-05-01 15:35:48 -07:00
290122637b
Using ASSERT_ABORT in destructors
2018-04-23 14:05:10 +02:00
19762b847d
Merge branch 'release-5.2'
...
# Conflicts:
# fdbserver/DatabaseConfiguration.cpp
# fdbserver/SimulatedCluster.actor.cpp
2018-04-10 17:02:43 -07:00
d3fb17d30a
Merge pull request #74 from bnamasivayam/client-profiling-tests
...
Client profiling tests - Part 1
2018-03-23 16:52:49 -07:00
89d7cc1093
Minor Bug fixes...
2018-03-15 11:00:47 -07:00
856d2a0a9d
Add correctness tests for Client transaction profiling data format. It also includes format check across upgrades.
2018-03-14 12:39:50 -07:00
f2c804e14f
Reverting changes from merge of master into release-5.2 ( b25810711c). Note that we never intend to release master into release-5.2, but if we did we would need to revert this commit.
2018-03-06 10:15:04 -08:00
fd4c9d5b54
Fix line endings
2018-03-05 10:37:52 -08:00
37a6a81634
Merge commit '7f6fc3e039c911cd84b8540f7f799fc38a1c1822' into feature-remote-logs
...
# Conflicts:
# fdbserver/workloads/RestartRecovery.actor.cpp
2018-02-23 12:33:28 -08:00
719bb5bd0c
Merge pull request #4 from bnamasivayam/getKeyServers-refactor
...
Having 1000 as the limit for Limit for GetKeyServerLocationsRequest s…
2018-02-22 12:39:48 -08:00
2fe2b522d5
Simple refactor of code...
2018-02-22 12:38:14 -08:00
Andrew Noyes