The original `ThreadReturnPromiseStream` had a fundamental flaw: it didn't handle reference counting
in a thread-safe way. This led to subtle and hard-to-debug race conditions, especially since Future
and Promise often live on different threads in this use case.
This patch introduces a new implementation of ThreadReturnPromiseStream that uses an atomics backed
spinlock to safely manage reference counts. As a result, the Flow compiler also needs to support a
new type: `ThreadFutureStream`, in addition to the existing `FutureStream`.
The underlying `NotifiedQueue` now has a parallel implementation called `ThreadNotifiedQueue` to
make this possible. The higher-level thread-safe promise and promise counterpart are built on top of
that.
The new `ThreadFutureStream` is compatible with both Flow and C++ coroutines. The original
non-threaded Flow primitives remain untouched and free from the overhead of atomics.
Joshua:
20250417-000356-vishesh-16ce18efeab40158 compressed=True data_size=41124943 duration=5436274 ended=99998 fail=1 fail_fast=10 max_runs=100000 pass=99997 priority=100 remaining=0:00:00 runtime=1:00:47 sanity=False started=100000 submitted=20250417-000356 timeout=5400 username=vishesh
clang-format
This patch has two set of changes:
- Whenever a service is registered and removed from server, we need to restart gRPC server.
GrpcServer provides some methods that can be used by worker actors so that the life of
services registered by them can tied to the life of the worker role itself.
- Replace asio::thread_pool with AsyncTaskExecutor both in client and server.
Add gRPC file transfer service
* grpc: Add file size check
* grpc: change test addresses
* Fix CI/CD failure
* Disable gRPC for build
* Fixes for new gRPC in new build image
* Move FileTransfer definitions to CPP file
* Implement gRPC support
* Move some CMake stuff around.
* Fix typo
* Add some test
* Add async client
* Add test for checking destroy
* [testing] Automatically discover unit-test and register as ctest
This patch adds `collect_unit_tests()` to CMake which searches over
the codebase and finds all the unit-tests written using Flow's TEST_CASE
macro and register as ctest.
The test then can be then run using ctest command or directly via Test
Explorer in VSCode.
* Fix some tests
* Use NetworkAddress
* Add another variant of call method
* Add a failed call test
* Refactor
* Cleanup shutdown
* Start working on streaming
* Implement server streaming
* Cleanup some unnecessary templating
* Cleanup some tests
* WIP Client Streaming
* WIP
* File Transfer WIP
* Remove UnitTest.h
* Take grpc addresses from command line
* startup grpc in fdbserver
* Cancel if future ref is 0
* noop
* Update some Cmake files
* Fix some build/run issues
* Review comments and remove file transfer
* Compile with gRPC present
* format
* Address review comments
* Add assert
* fix FLOW_GRPC_ENABLED flag
* include grpc/proto headers for generated files
* fix arm build not finding generated proto
* add debug message for protobuf generation
* add generated dir again
* add check for protoc compiler
Trevor Clinkenbeard