b589c3c940
Remove IncludeVersions.h, which is no longer needed for cmake-only builds
2020-05-20 13:23:02 -07:00
16990aeb6f
Add dcId to client transaction events.
2020-05-19 16:07:54 -07:00
00dce3ca1d
clang-format
2020-05-18 14:23:17 -07:00
fcca49f4d5
Add back an std::move that was accidentally removed. Some other small tweaks to avoid map lookups when tags aren't used.
2020-05-18 13:45:51 -07:00
bf62ef7c3e
Add dd-stats into special-key-space
2020-05-18 10:38:23 -07:00
6ce574f5ad
Merge remote-tracking branch 'upstream/release-6.3' into add-data-distribution-metrics
2020-05-17 23:36:52 -07:00
8597b38d10
Add an assertion in SingleSpecialKeyImpl
2020-05-15 16:34:43 -07:00
4a13107805
Add a test for special_keys_cross_module_read, fix two bugs, add several TODOs
2020-05-15 16:34:43 -07:00
c61e198e40
Using enum class instead of enum
2020-05-15 16:34:43 -07:00
e1c4e5f619
Add enums for special-key-space modules and refactor related code
2020-05-15 16:34:42 -07:00
f605c8d74f
Narrow keyrange of special-key-space to [\xff\xff, \xff\xff\xff). Initilize new ones in correctness tests
2020-05-15 16:34:42 -07:00
667b224fc8
Several changes to reduce unnecessary moves/copies
2020-05-13 19:59:34 -07:00
da14e1eb04
Fix: debug tagging was always enabled
2020-05-12 14:11:03 -07:00
8bd5dcaff8
Merge branch 'master' into atn34/special-key-versioning
2020-05-09 15:34:20 -07:00
02307ba7b6
Merge branch 'master' into transaction-tagging
...
# Conflicts:
# fdbclient/DatabaseContext.h
2020-05-09 07:50:29 -07:00
4cbf7b0fc0
Fix typo
2020-05-09 03:12:55 +00:00
d1b16eafc5
Only call getKeyRange once
2020-05-09 03:01:56 +00:00
ef724bf939
Merge remote-tracking branch 'upstream/master' into add-data-distribution-metrics
2020-05-08 18:39:28 -07:00
e8b62e48f4
Rename DDMetrics to DDMetricsRef
2020-05-08 17:17:27 -07:00
f0f52fb2be
Merge branch 'master' into feature-small-endpoint
...
# Conflicts:
# fdbclient/StorageServerInterface.h
2020-05-08 16:37:35 -07:00
6389f6ac51
Combine \xff\xff/transaction/* into one module
2020-05-08 05:37:37 +00:00
697685574d
Fix worker_interfaces
...
They don't come sorted by address, they come sorted by process id
2020-05-08 05:37:37 +00:00
38830f0eb9
Fix range read bounds bug
2020-05-08 05:37:37 +00:00
76aed13bdc
Add prefix in comparator for worker_interfaces
2020-05-08 05:37:37 +00:00
32d50fc359
Port over all existing special keys to framework
2020-05-08 05:37:37 +00:00
ce0e03a800
Add virtual destructor to SpecialKeyRangeBaseImpl
2020-05-08 05:37:37 +00:00
640eb41ba4
Introduce special key space in api version 630
2020-05-08 05:37:37 +00:00
a08bbcc539
Merge pull request #3046 from atn34/atn34/conflict-ranges
...
Add read/write conflict ranges to special key space
2020-05-07 22:36:18 -07:00
a8e3ad3c15
Fix merge error
2020-05-07 15:10:50 -07:00
92730b2f05
Merge branch 'master' into transaction-tagging
...
# Conflicts:
# fdbclient/NativeAPI.actor.cpp
2020-05-07 14:50:25 -07:00
9ccc417a4f
An indentifier must be set before calling LOG_TRANSACTION
2020-05-07 11:18:03 -07:00
90dbf33182
Merge pull request #3079 from bnamasivayam/client-transaction-info
...
Add read version and commit version info to getVersion and commit cli…
2020-05-07 09:59:35 -07:00
b1055a8501
Merge branch 'master' into transaction-tagging
2020-05-05 16:03:39 -07:00
f329164fb4
Merge pull request #2532 from dongxinEric/feature/hot-read-key-detection-part-2
...
Feature/hot read key detection part 2
2020-05-05 14:33:34 -07:00
37fe05fc76
Merge remote-tracking branch 'upstream/master' into atn34/conflict-ranges
2020-05-05 15:52:07 +00:00
de1f85d106
Fix accounting bug in TagSet. Change serialization back to not including the number of elements. Fix knob usage.
2020-05-04 15:49:55 -07:00
cd2176ac15
Adjust some knobs
2020-05-04 10:37:53 -07:00
36454bb3b8
Merge branch 'master' into transaction-tagging
...
# Conflicts:
# fdbclient/MasterProxyInterface.h
# fdbclient/NativeAPI.actor.cpp
2020-05-04 10:23:25 -07:00
80a235aa80
Add some correctness tests
2020-05-04 10:15:18 -07:00
bb3d4b6b89
Add a bunch of TEST macros and some other little things
2020-05-04 10:11:36 -07:00
78d644f673
Merge branch 'master' into bugfixes/cmake-includes
2020-05-04 02:11:46 -07:00
d6bc5263ad
Add read version and commit version info to getVersion and commit client transaction events.
2020-05-03 22:42:06 -07:00
0c514e91c7
Use approximate write range for set_versionstamped_key
2020-05-04 04:43:50 +00:00
ca1d457c6c
Fix bug where readVersion member future wasn't assigned
2020-05-02 07:59:37 -07:00
6ce4ddf1db
Fix a couple spots where we were missing checks for the new tag_throttled error
2020-05-01 21:47:12 -07:00
bd699f435c
fixed compiler errors
2020-05-01 11:01:09 -07:00
aed2d34bcb
Merge branch 'master' into feature-proxy-load-balance
...
# Conflicts:
# fdbclient/NativeAPI.actor.cpp
# fdbserver/MasterProxyServer.actor.cpp
# flow/Knobs.cpp
2020-05-01 09:19:39 -07:00
e9b9f0e845
onError wasn't properly retrying tag throttled errors
2020-04-30 21:12:34 -07:00
f1d9c0cb97
Refactor/cleanup
2020-04-30 00:30:37 +00:00
fdd6581167
Add write conflict range
2020-04-30 00:30:37 +00:00
e1c3c73d4c
Rudimentary read your own read conflict range
2020-04-30 00:30:37 +00:00
573b10f9f5
Fix a merge bug and cleanup some argument passing.
2020-04-27 15:03:56 -07:00
41c517a5dd
Merge branch 'master' into transaction-tagging
...
# Conflicts:
# fdbclient/NativeAPI.actor.cpp
2020-04-27 13:05:24 -07:00
a65e97209a
Fix bug in flag management. Pass priority into ratekeeper's updateRate and use it when setting autothrottles.
2020-04-27 11:34:12 -07:00
239876351b
Add some initial auto-throttling. Move the definition of the priority enum to a more global place and use it for all transaction priorites (except in ClientLogEvents, because of serialization incompatibilites).
2020-04-24 11:31:16 -07:00
35c18ac60a
Improvements to expiration. Encode throttles with auto/manual and priority in the key to support throttling the same tag with different values in these parameters.
2020-04-23 20:50:40 -07:00
c87aa33941
Merge branch 'release-6.2'
...
# Conflicts:
# CMakeLists.txt
# bindings/go/src/fdb/generated.go
# documentation/sphinx/source/api-common.rst.inc
# documentation/sphinx/source/api-ruby.rst
# documentation/sphinx/source/release-notes.rst
# fdbclient/FailureMonitorClient.actor.cpp
# fdbclient/NativeAPI.actor.cpp
# fdbclient/vexillographer/fdb.options
# fdbrpc/FlowTransport.actor.cpp
# fdbserver/OldTLogServer_6_0.actor.cpp
# fdbserver/TLogServer.actor.cpp
# fdbserver/fdbserver.actor.cpp
# versions.target
2020-04-23 13:47:53 -07:00
434704fbd9
Various bug fixes
2020-04-22 12:28:51 -07:00
444b338f63
Client clears throttles when the proxy responds to a request with a throttle being absent.
2020-04-21 16:24:02 -07:00
2f69eb42a5
Update client last check time for a throttle when the request is sent, not when the response is received. This prevents sending lots of requests when the check time has expired before we get a response.
2020-04-21 16:23:24 -07:00
ee8447765f
Throttled tags on the client periodically check in with the cluster to update their rates if they get heavily throttled.
2020-04-21 11:56:58 -07:00
e51d0365cf
Cleanup: Use the shutdown callback for destroying TLS state.
2020-04-20 13:16:16 -07:00
da8e47ea25
Merge remote-tracking branch 'upstream/release-6.2' into tls-background-eio-thread
2020-04-20 13:15:05 -07:00
2ce539ef6d
Respect flow<->fdbrpc module boundaries.
...
Which fixes a compilation error due to a circular dependency between
flow.a and fdbrpc.a. However, this is now done at the cost of newNet2
users have to remember to add Net2FileSystem::stop() as a callback.
2020-04-20 02:53:07 -07:00
94b4f78ea9
Fix clients crashing in TLS code on exit.
...
If client code initiates an FDB operation to a TLS cluster, and then
immediately exits the main thread, then OpenSSL's atexit handler would
potentially run while the network thread is attempting to do TLS
operations, and thus crash.
This commit removes the OpenSSL atexit hander, and instead relies on a
client intentionally ending the network thread to do TLS cleanup. If
the client code exits without stopping the network thread, then we'll
never free OpenSSL data structures, which is the safer thing to do.
2020-04-18 15:48:02 -07:00
78d48a0dad
Merge branch 'master' into transaction-tagging
...
# Conflicts:
# fdbcli/fdbcli.actor.cpp
# fdbclient/Knobs.h
# fdbclient/NativeAPI.actor.cpp
# fdbclient/fdbclient.vcxproj
# fdbserver/MasterProxyServer.actor.cpp
2020-04-17 09:23:18 -07:00
2b66dcd24a
Some more refactoring. Reduce what is sent from RK->MP->clients
2020-04-17 08:07:01 -07:00
7dd7406c59
Merge branch 'master' into feature/hot-read-key-detection-part-2
2020-04-16 14:54:05 -07:00
0fba8c47be
Checkpoint: Ratekeeper sets absolute limits for tag throttles and enforces them by distributing requests to proxies, who distribute them to clients.
...
A few refactorings.
2020-04-16 14:43:22 -07:00
b011efa8df
Merge remote-tracking branch 'upstream/master' into private-keyspace-framework
2020-04-16 10:58:04 -07:00
e104a2e3a6
Merge commit 'cf01233f28a2c42908656a39f458a4475c1d44a3' into run-loop-busy-profiler
...
# Conflicts:
# documentation/sphinx/source/release-notes.rst
# fdbclient/NativeAPI.actor.h
# fdbserver/fdbserver.actor.cpp
# flow/Net2.actor.cpp
2020-04-14 14:02:24 -07:00
ae1de060c2
Reformat SpecialKeyRangeBaseImpl constructor
2020-04-14 09:10:40 -07:00
65620b00f3
Merge remote-tracking branch 'upstream/master' into private-keyspace-framework
2020-04-13 14:21:07 -07:00
1439de37b5
Convert GetRangeLimits() -> TOO_MANY + ASSERT().
2020-04-12 18:23:14 -07:00
db071256db
Merge remote-tracking branch 'upstream/master' into reduce-txn-state-store-size
2020-04-12 18:06:10 -07:00
6508c891fc
Make the TagSet sent to the storage servers optional so we can distinguish no tags from unsampled.
2020-04-10 13:29:28 -07:00
55a0d00ad4
Encoding of tags in the database now supports multiple tags per throttle. Remove throttle prefix search.
2020-04-10 10:12:26 -07:00
ebeca10bce
Change the serialization of tags sent in some messages. Add communication of the sampling rate from cluster to clients.
2020-04-09 16:55:56 -07:00
34cddd675c
clang-format
2020-04-08 14:33:41 -07:00
20b2984841
fix issues according to andrew's comments
2020-04-08 12:43:25 -07:00
52860043c9
Merge remote-tracking branch 'origin' into atomically-update-dependent-knobs
2020-04-08 12:26:21 -07:00
5aee6880a8
fix issues according to andrew's comments
2020-04-08 11:33:07 -07:00
36da61dd9c
Merge branch 'master' into transaction-tagging
...
# Conflicts:
# fdbclient/NativeAPI.actor.cpp
# fdbclient/vexillographer/fdb.options
2020-04-07 21:12:14 -07:00
a1d8623e5f
Various changes to the throttling scheme, including most notably that clients now enforce the throttles with info they receive from the proxy.
2020-04-07 16:28:09 -07:00
e4611e8ae4
fix versions.h stupidity
2020-04-06 10:28:55 -07:00
8b5780c36c
don't include source and binary dir
...
This forces users to use include paths from the sources root.
So `#include "Arena.h"` won't work anymore, only
`#include "flow/Arena.h"` will.
2020-04-06 10:13:49 -07:00
6078fd1b18
Convert UID to Tag in keyServers to reduce txnStateStore size
2020-04-05 14:30:09 -07:00
aad8f850f3
refactor conflicting keys feature using special key space
2020-04-04 13:26:11 -07:00
6d570cd865
Change version from 7.0 to 6.3
2020-04-03 21:38:58 -07:00
5b6ae4da7f
A simple intergration to put conflictingKeys into special-key-space
2020-04-03 16:11:20 -07:00
2336f073ad
Checkpointing a bunch of work on throttles. Rudimentary implementation of auto-throttling. Support for manual throttling via fdbcli. Throttles are stored in the system keyspace.
2020-04-03 15:24:14 -07:00
760dc68b7f
Merge pull request #2869 from dongxinEric/feature/1689/allow-custome-trace-log-file-identifier
...
Allow the user to provide a custome trace log file identifier that wi…
2020-04-03 14:08:17 -07:00
2ef8f1eb96
Fixed the indentation. Addressed review comments
2020-04-03 13:21:57 -07:00
0816423d78
Merge pull request #2853 from Daniel-B-Smith/debug-transactions
...
Add ability to mark transactions to add debug logging
2020-04-03 11:43:45 -07:00
f6a2a7cee9
Move log statement to correct section
2020-04-03 17:31:38 +00:00
5092f52042
Fix location of logging code.
2020-04-03 13:19:48 -04:00
1ec7b1ebe3
Add logging for server tracing
2020-04-02 22:16:58 +00:00
d0210fd54f
Rename option
2020-04-02 21:41:16 +00:00
884e92bb49
Atomically update dependent knobs
2020-04-01 15:18:49 -07:00
c8abd9c493
Reformat some code
2020-03-31 18:53:36 -07:00
A.J. Beamon