76fb345dd1
Merge branch 'master' into feature-tree-broadcast
...
# Conflicts:
# fdbrpc/FailureMonitor.actor.cpp
2020-04-29 09:51:22 -07:00
2b2e0b9f75
Merge pull request #2932 from vishesh/task/issue-1017-slow-machine-poisoning
...
FailureMonitoring: Mark unstable connections as failed [1/X]
2020-04-28 13:57:49 -07:00
b7f5f3be48
merge in master
2020-04-28 13:11:47 -07:00
c87aa33941
Merge branch 'release-6.2'
...
# Conflicts:
# CMakeLists.txt
# bindings/go/src/fdb/generated.go
# documentation/sphinx/source/api-common.rst.inc
# documentation/sphinx/source/api-ruby.rst
# documentation/sphinx/source/release-notes.rst
# fdbclient/FailureMonitorClient.actor.cpp
# fdbclient/NativeAPI.actor.cpp
# fdbclient/vexillographer/fdb.options
# fdbrpc/FlowTransport.actor.cpp
# fdbserver/OldTLogServer_6_0.actor.cpp
# fdbserver/TLogServer.actor.cpp
# fdbserver/fdbserver.actor.cpp
# versions.target
2020-04-23 13:47:53 -07:00
c30010f5b8
FlowTransport: Increase delay for connecting to failed connections
2020-04-23 09:44:30 -07:00
a835123680
exit fdbserver after a ReceiverError, because a packet which should be delivered will never be received
2020-04-22 23:38:46 -07:00
810bba2067
cleanup calls to FlowTransport::isClient()
2020-04-22 23:36:40 -07:00
3a5315d10c
FlowTransport: Don't immediately mark connections failed
...
In connectionKeeper(), when a connection is failed for
FAILURE_DETECTION_DELAY, then only mark connection as failed.
This is much closer to the original centralized behaviour, and also
adds more confidence on whether the connection is actually failed.
2020-04-22 19:46:04 -07:00
500a265d20
FlowTransport: Refactor and clear FailureStatus on destroy
2020-04-21 14:11:19 -07:00
992002cf34
FlowTransport: don't start connectionKeeper for local peer
...
getOrOpenPeer() is used for local addresses sometimes, which ends up
starting connectionKeeper() which is unnecessary.
2020-04-16 14:04:32 -07:00
da7d0093ee
Cleanup unused code
2020-04-15 19:48:25 -07:00
023372a226
FailMon: Mark peer failed after retrying
2020-04-15 19:47:24 -07:00
f959af8228
Refactor per review comments
2020-04-14 11:30:40 -07:00
07cc0a8d74
code cleanup
2020-04-10 17:02:11 -07:00
fed5c543d4
Remove leftover TODO code around centralized healthmonitor
2020-04-08 22:56:07 -07:00
36d89db4e7
FlowTransport: Don't always close unused open connection
2020-04-06 23:05:29 -07:00
4214c45218
FlowTransport: Temporarily keep idle connections open for server processes
2020-04-06 21:27:54 -07:00
2a1bd97120
fix compilation errors
2020-04-06 20:58:43 -07:00
477d66b46d
implemented a tree broadcast for txn state message for proxies, and serverDBInfo for workers
2020-04-05 23:09:36 -07:00
fdc1048f75
Add knob to turn off marking unstable connections
2020-04-03 15:53:00 -07:00
613b8bb169
More TraceEvents and remove 'delayed' used for debugging
2020-04-03 15:53:00 -07:00
1d35f2ff5a
Mark a connection as failed for X seconds if closes too often
2020-04-03 15:53:00 -07:00
d90e168e24
Add HealthMonitoring skeleton code
2020-04-03 15:53:00 -07:00
303df197cf
Merge branch 'release-6.2'
...
# Conflicts:
# CMakeLists.txt
# bindings/c/test/mako/mako.c
# documentation/sphinx/source/release-notes.rst
# fdbbackup/backup.actor.cpp
# fdbclient/NativeAPI.actor.cpp
# fdbclient/NativeAPI.actor.h
# fdbserver/DataDistributionQueue.actor.cpp
# fdbserver/Knobs.cpp
# fdbserver/Knobs.h
# fdbserver/LogRouter.actor.cpp
# fdbserver/SkipList.cpp
# fdbserver/fdbserver.actor.cpp
# flow/CMakeLists.txt
# flow/Knobs.cpp
# flow/Knobs.h
# flow/flow.vcxproj
# flow/flow.vcxproj.filters
# versions.target
2020-03-06 18:22:46 -08:00
976c2fc7a8
Update fdbrpc/FlowTransport.actor.cpp
...
Co-Authored-By: Alex Miller <35046903+alexmiller-apple@users.noreply.github.com>
2020-03-04 16:13:59 -08:00
820957025f
accept connections in batches of 20 to improve performance
2020-03-04 14:24:57 -08:00
924d335aa7
Merge branch 'release-6.2'
...
# Conflicts:
# documentation/sphinx/source/release-notes.rst
# flow/Knobs.cpp
# flow/Knobs.h
2020-02-25 18:25:19 -08:00
65fbe0d0bc
revert AcceptSocket priority change because of bad performance results
2020-02-21 19:22:14 -08:00
96258b9809
Merge branch 'release-6.2'
...
# Conflicts:
# documentation/sphinx/source/release-notes.rst
# fdbcli/fdbcli.actor.cpp
# fdbclient/ManagementAPI.actor.cpp
# fdbrpc/FlowTransport.actor.cpp
# fdbserver/ClusterController.actor.cpp
# fdbserver/DataDistribution.actor.cpp
# fdbserver/DataDistribution.actor.h
# fdbserver/DataDistributionQueue.actor.cpp
# fdbserver/KeyValueStoreMemory.actor.cpp
# fdbserver/MasterProxyServer.actor.cpp
# fdbserver/QuietDatabase.actor.cpp
# fdbserver/SkipList.cpp
# fdbserver/StorageMetrics.actor.h
# fdbserver/TLogServer.actor.cpp
# fdbserver/fdbserver.actor.cpp
# fdbserver/storageserver.actor.cpp
# fdbserver/workloads/KVStoreTest.actor.cpp
# flow/CMakeLists.txt
# flow/Knobs.cpp
# flow/Knobs.h
# flow/genericactors.actor.cpp
# flow/serialize.h
2020-02-21 19:09:16 -08:00
dc3826e2fd
fix: tls throttling would re-insert the failure into the map
2020-02-20 18:17:39 -08:00
f04e311a1e
Merge commit 'b46d6e25e24993ab5a5f04091fd3235050b7cd09' into feature-boost-ssl
...
# Conflicts:
# fdbserver/SimulatedCluster.actor.cpp
# flow/Net2.actor.cpp
2020-02-20 17:36:38 -08:00
3c4d551647
improve prioritization of connection monitor and listen given that listen is no longer expensive (because handshake is done separately)
2020-02-19 18:50:21 -08:00
46d5f5e325
do not trigger the resetPing if we cannot actually remove the peer, because it will cause us to reset the timeout, so repeated calls to removePeer can keep a dead peer from being removed
2020-02-19 15:17:50 -08:00
132f5aa9ba
FastRestore:Improve trace name and cosmetic change
2020-02-18 16:41:19 -08:00
31a6ec34b7
Merge branch 'master' into mengxu/fast-restore-agent-PR
2020-02-18 16:17:59 -08:00
1248d2b8b4
Remove USE_OBJECT_SERIALIZER knob
2020-02-12 10:41:52 -08:00
69de430057
separate handshaking from connection to improve pipelining
2020-02-06 16:45:54 -08:00
53d0867a17
limit the number of connections a process can attempt to establish in parallel
2020-02-04 18:15:10 -08:00
84853dd1fd
switched SSL implementation to use boost ssl
2020-02-04 14:56:40 -08:00
3b57bf1781
Merge branch 'master' into mengxu/fast-restore-agent-PR
2020-02-03 17:23:54 -08:00
4524831456
Merge pull request #2518 from vishesh/task/failmon-remove-server
...
FailureMonitoring: Server processes no longer need to talk to ClusterController
2020-02-03 17:22:50 -08:00
ca3b6135d0
FastRestore:Add debug to see why restore role is not connected
...
Reason: restore is a fdbserver who does not register with CC.
The new failure monitor changes how connection works for client and server.
For client, it does not connect to CC to get connected.
For server, it has to connect to CC to get connected.
Restore worker becomes the special role that behaves like a client but is a server.
2020-02-03 17:19:52 -08:00
76e192d490
Merge pull request #2538 from alexmiller-apple/hashlittle2-to-crc32c
...
Convert more hashlittle{,2} uses to crc32c_append
2020-01-23 17:54:38 -08:00
daef5f011a
Merge remote-tracking branch 'apple/master' into task/failmon-remove-server
2020-01-21 13:20:15 -08:00
da73164eda
Move crc32c from fdbrpc to flow
...
So that we can use it from a piece of flow code without breaking module
boundaries.
Also rename generated-constants to crc32c-generated-constants so that
it's more apparent that they're related files.
2020-01-13 18:19:30 -08:00
598b2eaeb0
fdbrpc: Add warning when peer is unavailable for long time
2020-01-08 13:55:13 -08:00
6b8daeae6e
FailureMonitor: Cleanup code around reconnecting to failed connections
...
If there a peer with non-zero reference, keep trying to reconnect when
a connection fails. We already make sure it doesn't happen only once
every 2 seconds.
2020-01-07 15:53:32 -08:00
85c24dc074
Active Failure Monitoring no longer needed at server processes
...
This patch removes active failure monitoring at server processes.
Hence like client processes, servers no longer require continuously
publishing their membership to cluster controller.
When a process is marked as failed, we still need to know if it back
up at certain point, particularly when the reference count is
incremented. In that case, loadBalance may see AllAlternativesFailed
as failed. To overcome this problem, whenever peer references is
incremented and and the address is marked as failed, connectionKeeper
will bypass waiting for data and connects immediately to check if the
process is back up.
2020-01-07 15:53:32 -08:00
9e137d3b49
fix: addPeerReference only marks a connection as healthy if it is the first peerReference
...
added additional logging to long LoadBalance calls, and when the failure monitor state changes for an address
2019-12-19 18:26:29 -08:00
46d10dc7dc
Fix "null passed as argument declared not null"
...
Fix several such reports from ubsan
E.g.
/Users/anoyes/workspace/foundationdb/flow/Arena.h:794:16: runtime error: null pointer passed as argument 1, which is declared to never be null
2019-12-03 14:46:53 -08:00
Evan Tschannen