30bfd606c0
Merge branch 'release-6.2' into release-6.3
...
# Conflicts:
# CMakeLists.txt
# documentation/sphinx/source/downloads.rst
# documentation/sphinx/source/release-notes.rst
# fdbserver/worker.actor.cpp
# packaging/msi/FDBInstaller.wxs
# versions.target
2020-06-04 19:21:32 -07:00
bf072d68ec
Workarounds for strange behaviors in Boost ssl sockets on MacOS and Linux. When writing to the ssl socket, write_some() would sometimes return BrokenPipe instead of WouldBlock unless onWriteable on the raw socket was checked first. On MacOS, even with the onWriteable check using a send size greater than 2016 (determined experimentally) would still result in the error. Also consolidated two identical copies of SendBufferIterator.
2020-06-04 01:44:05 -07:00
86f712657e
Eliminate the undefined behavior of calling run_network twice, instead returning an error.
2020-05-22 13:31:06 -07:00
c2e80fe47b
Linux aarch64 port
2020-05-09 22:14:03 +00:00
95bc24de11
flow: update headers and includes
2020-04-30 18:11:23 +00:00
c87aa33941
Merge branch 'release-6.2'
...
# Conflicts:
# CMakeLists.txt
# bindings/go/src/fdb/generated.go
# documentation/sphinx/source/api-common.rst.inc
# documentation/sphinx/source/api-ruby.rst
# documentation/sphinx/source/release-notes.rst
# fdbclient/FailureMonitorClient.actor.cpp
# fdbclient/NativeAPI.actor.cpp
# fdbclient/vexillographer/fdb.options
# fdbrpc/FlowTransport.actor.cpp
# fdbserver/OldTLogServer_6_0.actor.cpp
# fdbserver/TLogServer.actor.cpp
# fdbserver/fdbserver.actor.cpp
# versions.target
2020-04-23 13:47:53 -07:00
c6df20a179
Use nullptr instead of NULL
2020-04-21 20:39:45 -07:00
a51746b307
Match 6.2.15's behavior in how invalid/unreadable/non-existent certs are handled.
...
Which is to proceed past Net2 creation, and allow certificate refresh to
try and eventually load valid certs. Additionally, fix certificate
refeshing dieing if the certificate is not readable when first called.
In testing, I also found and fixed an issue where if a cert went from
unreadable to readable, we wouldn't reload the TLS context, due to not
considering it as a file change.
2020-04-20 21:38:04 -07:00
20fe068863
Merge branch 'tls-background-eio-thread' into tls-permission-errors
2020-04-20 20:51:05 -07:00
49c6bb90ef
Merge pull request #2982 from alexmiller-apple/tls-log-settings
...
Log Net2TLSConfig with paths and settings when using TLS.
2020-04-20 15:46:26 -07:00
75a4f3b7c9
Remove comment about ignoring runOnMainThread errors.
...
If we got an exception, it wouldn't be of type `Error` anyway, so
it seems like things would crash regardless.
2020-04-20 13:19:42 -07:00
da8e47ea25
Merge remote-tracking branch 'upstream/release-6.2' into tls-background-eio-thread
2020-04-20 13:15:05 -07:00
5c399bf725
Move the callbacks into ::run() right before it exits.
...
stopped=true doesn't cause the run loop to immediately exit.
2020-04-20 13:14:19 -07:00
2ce539ef6d
Respect flow<->fdbrpc module boundaries.
...
Which fixes a compilation error due to a circular dependency between
flow.a and fdbrpc.a. However, this is now done at the cost of newNet2
users have to remember to add Net2FileSystem::stop() as a callback.
2020-04-20 02:53:07 -07:00
cbb6ffb431
Only log OpenSSL error strings for OpenSSL errors.
...
Normal "connection refused" messages would show up with a long verbose
string that doesn't really provide any useful information otherwise.
2020-04-18 20:39:02 -07:00
11eebc4a48
Log Net2TLSConfig with paths and settings when using TLS.
...
There were similar TraceEvents in the FDBLibTLS/LibreSSL TLS
implementaiton that were accidentally dropped in the TLS rewrite.
This makes it so that one does not have to use magic to figure out if a
process was configued with TLS correctly when some of the settings come
from environment variables.
2020-04-18 20:21:10 -07:00
1398e9a82e
Stop background eio threads on Net2::stop().
...
This will stop eio threads for both the client (`fdb_stop_network()`)
and the server. This change is being done more for the former, but I
don't see any harm in doing the latter as well.
2020-04-18 19:40:55 -07:00
d8690d31cd
Merge branch 'master' into per-priority-busy-logging
...
# Conflicts:
# flow/Net2.actor.cpp
2020-04-15 08:31:30 -07:00
b1172417f5
Merge branch 'master' into per-priority-busy-logging
...
# Conflicts:
# flow/Knobs.cpp
# flow/Knobs.h
# flow/Net2.actor.cpp
2020-04-14 14:22:12 -07:00
e104a2e3a6
Merge commit 'cf01233f28a2c42908656a39f458a4475c1d44a3' into run-loop-busy-profiler
...
# Conflicts:
# documentation/sphinx/source/release-notes.rst
# fdbclient/NativeAPI.actor.h
# fdbserver/fdbserver.actor.cpp
# flow/Net2.actor.cpp
2020-04-14 14:02:24 -07:00
cf01233f28
Merge pull request #2549 from ajbeamon/slow-task-and-priority-tracking-improvements
...
Slow task and priority tracker improvements
2020-04-14 13:26:32 -07:00
903128a36f
Fix boost error trace event fields to match other trace events by the same name.
2020-04-01 08:18:50 -07:00
e08f0201f1
merge release 6.2 into master
2020-03-17 12:51:47 -07:00
db5863145a
Merge remote-tracking branch 'upstream/release-6.2' into fdbcli-tlsinfo
2020-03-16 12:33:50 -07:00
75e2fffe5a
Add a ProcessMetrics.TLSPolicyFailures metric
...
This reports the number of policy failures over the past 5s interval.
It also is step 1 towards getting this information into status json.
2020-03-13 02:24:37 -07:00
0c558efcfe
Add a `tlsinfo` command to fdbcli that prints the certificate chain.
...
This requires the certificate chain to load successfully, otherwise
fdbcli will error out at an earlier point due to Net2 not being able to
configure TLS.
2020-03-13 00:11:53 -07:00
303df197cf
Merge branch 'release-6.2'
...
# Conflicts:
# CMakeLists.txt
# bindings/c/test/mako/mako.c
# documentation/sphinx/source/release-notes.rst
# fdbbackup/backup.actor.cpp
# fdbclient/NativeAPI.actor.cpp
# fdbclient/NativeAPI.actor.h
# fdbserver/DataDistributionQueue.actor.cpp
# fdbserver/Knobs.cpp
# fdbserver/Knobs.h
# fdbserver/LogRouter.actor.cpp
# fdbserver/SkipList.cpp
# fdbserver/fdbserver.actor.cpp
# flow/CMakeLists.txt
# flow/Knobs.cpp
# flow/Knobs.h
# flow/flow.vcxproj
# flow/flow.vcxproj.filters
# versions.target
2020-03-06 18:22:46 -08:00
188d9b8239
Don't swallow actor cancellation in certificate refreshing.
2020-03-06 11:09:17 -08:00
9b760fae2d
Rewrite all Errors into tls_errors if they happen as part of initializing TLS.
2020-03-06 11:06:19 -08:00
ac52b6b474
Rework a bit of error and exception handling.
...
I went back and dug through all of the "what functions can throw what
types", and made sane decisions about them. boost errors are
aggressively translated into FDB ones, whcih might result in multiple
lines of logging about errors, but this is in infrequently run code, so
it should be fine.
2020-03-06 02:33:16 -08:00
ccef3f7d05
Attempt to fix TLS_DISABLED compiles.
2020-03-05 17:32:10 -08:00
2d95a1e64d
Implement certificate refreshing
2020-03-05 17:25:33 -08:00
595dd77ed1
Merge remote-tracking branch 'upstream/release-6.2' into certificate-refresh
2020-03-04 20:25:42 -08:00
9b5ef3416e
Refactor TLSParams into TLSConfig + LoadedTLSConfig
...
The idea being that we keep around a TLSConfig that the configuration
that the user has provided, and then when we want to intialize an SSL
context, we ask the TLSConfig to load all certificates and return us a
LoadedTLSConfig that is a concrete set of certificate bytes in memory.
initTLS now just takes the in-memory bytes and applies them to the ssl
context.
This is a large refactor to lead up into certificate refeshing, where we
will periodically check for changes to the certificates, and then
re-load them and apply them to a new SSL context.
2020-03-04 20:14:47 -08:00
7cbabca124
remove printing to stderr from initTLS because that could cause problems on clients
2020-03-04 15:06:22 -08:00
35a1ac6482
prepare net2 for new versions of boost
2020-03-04 14:26:01 -08:00
6054c05963
Merge branch 'release-6.2'
...
# Conflicts:
# CMakeLists.txt
# documentation/sphinx/source/release-notes.rst
# fdbserver/fdbserver.actor.cpp
# versions.target
2020-02-28 12:11:05 -08:00
707fc1ddea
only capture the policy to match prior code
2020-02-26 19:04:49 -08:00
c3299b8ebe
if tls cannot be initialized, throw an error from createDatabase
2020-02-26 18:53:06 -08:00
bf5a95e6df
Merge commit 'dc39bdfbbf94a7f470386f439df08c044d08d90c' into feature-tls-environment-vars
...
# Conflicts:
# flow/Net2.actor.cpp
2020-02-26 18:02:56 -08:00
f035bed870
defer initializing TLS to avoid throwing errors from a constructor and so that errors can be logged to the trace file
2020-02-26 17:50:07 -08:00
f85af10a18
fixed a few problems with tls setup
2020-02-26 16:06:45 -08:00
2586bade68
re-added support for configuration TLS options with environment variables
2020-02-26 15:33:48 -08:00
0f5c999d4b
Better containment of boost errors related to TLS.
2020-02-26 12:26:43 -08:00
96258b9809
Merge branch 'release-6.2'
...
# Conflicts:
# documentation/sphinx/source/release-notes.rst
# fdbcli/fdbcli.actor.cpp
# fdbclient/ManagementAPI.actor.cpp
# fdbrpc/FlowTransport.actor.cpp
# fdbserver/ClusterController.actor.cpp
# fdbserver/DataDistribution.actor.cpp
# fdbserver/DataDistribution.actor.h
# fdbserver/DataDistributionQueue.actor.cpp
# fdbserver/KeyValueStoreMemory.actor.cpp
# fdbserver/MasterProxyServer.actor.cpp
# fdbserver/QuietDatabase.actor.cpp
# fdbserver/SkipList.cpp
# fdbserver/StorageMetrics.actor.h
# fdbserver/TLogServer.actor.cpp
# fdbserver/fdbserver.actor.cpp
# fdbserver/storageserver.actor.cpp
# fdbserver/workloads/KVStoreTest.actor.cpp
# flow/CMakeLists.txt
# flow/Knobs.cpp
# flow/Knobs.h
# flow/genericactors.actor.cpp
# flow/serialize.h
2020-02-21 19:09:16 -08:00
dc3826e2fd
fix: tls throttling would re-insert the failure into the map
2020-02-20 18:17:39 -08:00
f04e311a1e
Merge commit 'b46d6e25e24993ab5a5f04091fd3235050b7cd09' into feature-boost-ssl
...
# Conflicts:
# fdbserver/SimulatedCluster.actor.cpp
# flow/Net2.actor.cpp
2020-02-20 17:36:38 -08:00
927cff3317
Report errors on TLS misconfigurations ... or at least try to.
2020-02-20 16:57:29 -08:00
d7c841a28a
Merge pull request #2589 from etschannen/feature-proxy-delay
...
Improve version pipelining on the proxy
2020-02-20 15:23:30 -08:00
08c318d28a
re-added the connect lock in the fdbcli so that the timeout is not spent before a connection has been initiated (because of the handshake lock)
2020-02-20 10:43:34 -08:00
Evan Tschannen