Commit Graph

55 Commits

Author SHA1 Message Date
Vishesh Yadav 0032465c53
Add gRPC support to FDB (#11782)
* Implement gRPC support

* Move some CMake stuff around.

* Fix typo

* Add some test

* Add async client

* Add test for checking destroy

* [testing] Automatically discover unit-test and register as ctest

This patch adds `collect_unit_tests()` to CMake which searches over
the codebase and finds all the unit-tests written using Flow's TEST_CASE
macro and register as ctest.

The test then can be then run using ctest command or directly via Test
Explorer in VSCode.

* Fix some tests

* Use NetworkAddress

* Add another variant of call method

* Add a failed call test

* Refactor

* Cleanup shutdown

* Start working on streaming

* Implement server streaming

* Cleanup some unnecessary templating

* Cleanup some tests

* WIP Client Streaming

* WIP

* File Transfer WIP

* Remove UnitTest.h

* Take grpc addresses from command line

* startup grpc in fdbserver

* Cancel if future ref is 0

* noop

* Update some Cmake files

* Fix some build/run issues

* Review comments and remove file transfer

* Compile with gRPC present

* format

* Address review comments

* Add  assert

* fix FLOW_GRPC_ENABLED flag

* include grpc/proto headers for generated files

* fix arm build not finding generated proto

* add debug message for protobuf generation

* add generated dir again

* add check for protoc compiler
2024-12-10 20:56:59 -08:00
Syed Paymaan Raza c3e7542cda Update end year in copyright header 2024-08-02 09:40:11 -07:00
Markus Pilman e07b3e35ca Added C++ Coroutine support to Flow 2023-11-14 10:10:11 +01:00
Jingyu Zhou 7e54174725 Add a knob RESOLVE_PREFER_IPV4_ADDR to prefer IPv4 addresses
The default is to prefer IPv6 addresses.
2023-08-23 14:54:07 -07:00
A.J. Beamon 64b6a5d257 Allow boolean parameters to be nested inside of namespaces or classes 2023-03-30 15:09:59 -07:00
Xiaoge Su 50de69c897 Extract IConnection and NetworkAddress out from network.h 2023-01-24 14:48:31 -08:00
Xiaoge Su c11c48fa3f Extract ChaosMetrics out from network.h 2023-01-24 14:47:48 -08:00
Xiaoge Su 3f03a6b12d Extract out IPAddress and IUDPSocket 2023-01-24 14:47:39 -08:00
A.J. Beamon 0a686719d8 Make mapAsync usable without specifying template parameters 2022-12-14 14:33:59 -08:00
Renxuan Wang 316ea60232 Prefer IPv6 in hostname resolving. 2022-07-28 20:07:16 -07:00
Renxuan Wang dc9599f2e9
Get network interfaces ready for https proxy. (#7556)
* Move HTTP from fdbclient/ to fdbrpc/.

* Move md5 and libb64 to contrib/.

* Get network interfaces ready for https proxy.

* Rebase
2022-07-25 17:08:32 -07:00
Markus Pilman 03d913a1de Flow compiling 2022-06-27 17:05:55 -06:00
Renxuan Wang 465ff712b6
Move Hostname to its own files. (#6759)
* Change DNS cache to use std::map.

Revert commit 90c259d84e, because if we use unordered_map, toString() can be inconsistent.

* Move ClientKnob::COORDINATOR_HOSTNAME_RESOLVE_DELAY to FlowKnob::HOSTNAME_RESOLVE_DELAY.

* Move Hostname to its own files.

Also, add resolve-related variables and functions in Hostname.
2022-04-04 19:04:51 -07:00
Renxuan Wang 5a336655f1 Use unordered_map in DNS cache. 2022-04-04 15:08:17 -07:00
Renxuan Wang 7da31857b7 Address comments. 2022-04-04 15:08:17 -07:00
Renxuan Wang ff934ca2ad Change MockDNS to DNSCache. 2022-04-04 15:08:17 -07:00
sfc-gh-tclinkenbeard a71099471b Update copyright header dates 2022-03-21 13:36:23 -07:00
Renxuan Wang 06b1d06d38 Support hostname in coordinators commands. 2022-02-24 23:02:29 -08:00
Renxuan Wang a6c482ee91 Add the support of using hostname in ConnectionString.
This PR comes without simulation tests except some unit tests. The simulation tests will be in the PR that uses hostname in code logic.
2022-01-20 19:18:15 -08:00
Renxuan Wang 28832a99d6 Address comment. 2022-01-18 14:34:18 -08:00
Renxuan Wang 46d17d748f Change member variable fromHostname to type bool.
So that it can be serialized.
2021-11-23 14:25:02 -10:00
Renxuan Wang 85dff214a4 Address comments. 2021-11-04 11:42:28 -07:00
Renxuan Wang f15ceb5489 Add Hostname struct, and fromHostname in NetworkAddress struct. 2021-11-04 11:42:28 -07:00
sfc-gh-tclinkenbeard 399c2c96f0 Remove unnecessary std::string copies from flow 2021-06-09 11:40:01 -07:00
FDB Formatster df90cc89de apply clang-format to *.c, *.cpp, *.h, *.hpp files 2021-03-10 10:18:07 -08:00
sfc-gh-tclinkenbeard a7202a5b42 Use connectExernal in INetworkConnections::connect 2020-12-24 21:20:04 -04:00
sfc-gh-tclinkenbeard 201857541d Remove unnecessary copies from INetworkConnections interface 2020-12-24 21:20:03 -04:00
Markus Pilman 6d84bcb568 Merge remote-tracking branch 'origin/master' into features/udp 2020-08-06 14:08:34 -06:00
Markus Pilman 8976694ba1 UDP implementation (untested) 2020-08-06 14:06:50 -06:00
Steve Atherton 38cade1146 Merged tag release-6.3.3 into tls-background-handshake. 2020-07-17 23:39:57 -07:00
Steve Atherton c4f3553826 TLS handshake limit changed to regular flow lock, added new knob for simultaneous handshakes limit and removed old knobs. Added knob to control how much data the FlowTransport connectionWriter() can write using multiple write() calls before yielding to other tasks, and updated p2p network test to use the same logic. Changed knob defaults for send limit on single writes and TLS handshake thread count. 2020-07-16 02:31:17 -07:00
sfc-gh-tclinkenbeard 60b893d423 Switch from boost::variant to std::variant 2020-07-04 19:28:05 -07:00
Evan Tschannen 37e2b0d353
Update flow/network.cpp 2020-04-14 17:12:07 -07:00
A.J. Beamon b1172417f5 Merge branch 'master' into per-priority-busy-logging
# Conflicts:
#	flow/Knobs.cpp
#	flow/Knobs.h
#	flow/Net2.actor.cpp
2020-04-14 14:22:12 -07:00
Evan Tschannen 96258b9809 Merge branch 'release-6.2'
# Conflicts:
#	documentation/sphinx/source/release-notes.rst
#	fdbcli/fdbcli.actor.cpp
#	fdbclient/ManagementAPI.actor.cpp
#	fdbrpc/FlowTransport.actor.cpp
#	fdbserver/ClusterController.actor.cpp
#	fdbserver/DataDistribution.actor.cpp
#	fdbserver/DataDistribution.actor.h
#	fdbserver/DataDistributionQueue.actor.cpp
#	fdbserver/KeyValueStoreMemory.actor.cpp
#	fdbserver/MasterProxyServer.actor.cpp
#	fdbserver/QuietDatabase.actor.cpp
#	fdbserver/SkipList.cpp
#	fdbserver/StorageMetrics.actor.h
#	fdbserver/TLogServer.actor.cpp
#	fdbserver/fdbserver.actor.cpp
#	fdbserver/storageserver.actor.cpp
#	fdbserver/workloads/KVStoreTest.actor.cpp
#	flow/CMakeLists.txt
#	flow/Knobs.cpp
#	flow/Knobs.h
#	flow/genericactors.actor.cpp
#	flow/serialize.h
2020-02-21 19:09:16 -08:00
Evan Tschannen 693e469003 Changed the handshake lock to a BoundedFlowLock, which will enforce that old handshakes complete before starting to initiate new handshakes 2020-02-14 16:49:52 -08:00
A.J. Beamon abb75f7eb7 Add logging to indicate the time spent at each priority that exceeds some minimum busyness threshold 2020-02-07 14:34:24 -08:00
Evan Tschannen 53d0867a17 limit the number of connections a process can attempt to establish in parallel 2020-02-04 18:15:10 -08:00
Andrew Noyes 2ce887012c Respect api version for include_port_in_address 2020-02-03 15:25:30 -08:00
mpilman b18666d942 statically link libstdc++ on Linux and remove std::variant
this will hopefully fix #1610
2019-07-16 14:53:16 -07:00
A.J. Beamon 603721e125 Merge branch 'master' into thread-safe-random-number-generation
# Conflicts:
#	fdbclient/ManagementAPI.actor.cpp
#	fdbrpc/AsyncFileCached.actor.h
#	fdbrpc/genericactors.actor.cpp
#	fdbrpc/sim2.actor.cpp
#	fdbserver/DiskQueue.actor.cpp
#	fdbserver/workloads/BulkSetup.actor.h
#	flow/ActorCollection.actor.cpp
#	flow/Net2.actor.cpp
#	flow/Trace.cpp
#	flow/flow.cpp
2019-05-23 08:35:47 -07:00
mpilman 69fa3d3903 fixed compilation issues after rebase 2019-05-13 14:15:23 -07:00
A.J. Beamon 5f55f3f613 Replace g_random and g_nondeterministic_random with functions deterministicRandom() and nondeterministicRandom() that return thread_local random number generators. Delete g_debug_random and trace_random. Allow only deterministicRandom() to be seeded, and require it to be seeded from each thread on which it is used. 2019-05-10 14:01:52 -07:00
Evan Tschannen 6997075917 changed back to isV6addr instead of isV4addr for compatibility 2019-03-27 19:55:36 -07:00
Evan Tschannen e5a80f2c94 optimized IPaddress 2019-03-27 18:21:13 -07:00
Vishesh Yadav e93cd0ff21 Add some checks and comments to IPv6 changes #963 2019-03-04 14:12:45 -08:00
Vishesh Yadav 592e224155 net: add/use formatIpPort to format IP:PORT pairs #963 2019-03-04 14:12:45 -08:00
Vishesh Yadav 82b2da4b78 net: Add IPAddress::parse() util #963 2019-03-04 14:12:45 -08:00
Vishesh Yadav 25daabdc02 net: TraceEvent and toIPVectorString for new IPAddress structure #963 2019-03-04 14:12:45 -08:00
Vishesh Yadav 57832e625d net: Support IPv6 #963
- NetworkAddress now contains IPAddress object which can be either
IPv4 or IPv6 address. 128bits are used even for IPv4 addresses,
however only 32bits are used when using/serializing IPv4 address.

- ConnectPacket is updated to store IPv6 address. Backward compatible
with old format since the first 32bits of IP address field is used
for serialization of IPv4.

- Mainly updates rest of the code to use IPAddress structure instead
of plain uint32_t.

- IPv6 address/pair ports should be represented as `[ip]:port` as per
convention. This applies to both cluster files and command line
arguments.
2019-03-04 14:12:41 -08:00