### Description
Fix https://github.com/grpc/grpc/issues/24470.
Adding one example which demonstrate the following use cases:
* Generate RPC ID on client side and propagate to server.
* Context propagation from client to server.
* Context propagation between different server interceptors and the
server handler.
## Use:
1. Start server: `python3 -m async_greeter_server_with_interceptor`
2. Start client: `python3 -m async_greeter_client`
### Expected Logs:
* On client side:
```
Sending request with rpc id: 73bb98beff10c2dd7b9f2252a1e2039e
Greeter client received: Hello, you!
```
* On server side:
```
INFO:root:Starting server on [::]:50051
INFO:root:Interceptor1 called with rpc_id: default
INFO:root:Interceptor2 called with rpc_id: Interceptor1-default
INFO:root:Handle rpc with id Interceptor2-Interceptor1-73bb98beff10c2dd7b9f2252a1e2039e in server handler.
```
Without this change, new users get this warning:
```
CMake Warning (dev) at $MY_INSTALL_DIR/lib64/cmake/protobuf/protobuf-options.cmake:6 (option):
Policy CMP0077 is not set: option() honors normal variables. Run "cmake
--help-policy CMP0077" for policy details. Use the cmake_policy command to
set the policy and suppress this warning.
For compatibility with older versions of CMake, option is clearing the
normal variable 'protobuf_MODULE_COMPATIBLE'.
Call Stack (most recent call first):
$MY_INSTALL_DIR/lib64/cmake/protobuf/protobuf-config.cmake:2 (include)
$MY_SRC_PATH/examples/cpp/cmake/common.cmake:99 (find_package)
CMakeLists.txt:24 (include)
This warning is for project developers. Use -Wno-dev to suppress it.
```
release notes: no
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
The very non-trivial upgrade of third_party/protobuf to 22.x
This PR strives to be as small as possible and many changes that were
compatible with protobuf 21.x and didn't have to be merged atomically
with the upgrade were already merged.
Due to the complexity of the upgrade, this PR wasn't created
automatically by a tool, but manually. Subsequent upgraded of
third_party/protobuf with our OSS release script should work again once
this change is merged.
This is best reviewed commit-by-commit, I tried to group changes in
logical areas.
Notable changes:
- the upgrade of third_party/protobuf submodule, the bazel protobuf
dependency itself
- upgrade of UPB dependency to 22.x (in the past, we used to always
upgrade upb to "main", but upb now has release branch as well). UPB
needs to be upgraded atomically with protobuf since there's a de-facto
circular dependency (new protobuf depends on new upb, which depends on
new protobuf for codegen).
- some protobuf and upb bazel rules are now aliases, so `
extract_metadata_from_bazel_xml.py` and `gen_upb_api_from_bazel_xml.py`
had to be modified to be able to follow aliases and reach the actual
aliased targets.
- some protobuf public headers were renamed, so especially
`src/compiler` needed to be updated to use the new headers.
- protobuf and upb now both depend on utf8_range project, so since we
bundle upb with grpc in some languages, we now have to bundle utf8_range
as well (hence changes in build for python, PHP, objC, cmake etc).
- protoc now depends on absl and utf8_range (previously protobuf had
absl dependency, but not for the codegen part), so python's
make_grpcio_tools.py required partial rewrite to be able to handle those
dependencies in the grpcio_tools build.
- many updates and fixes required for C++ distribtests (currently they
all pass, but we'll probably need to follow up, make protobuf's and
grpc's handling of dependencies more aligned and revisit the
distribtests)
- bunch of other changes mostly due to overhaul of protobuf's and upb's
internal build layout.
TODOs:
- [DONE] make sure IWYU and clang_tidy_code pass
- create a list of followups (e.g. work to reenable the few tests I had
to disable and to remove workaround I had to use)
- [DONE in cl/523706129] figure out problem(s) with internal import
---------
Co-authored-by: Craig Tiller <ctiller@google.com>
This PR aims to de-experimentalize the APIs for GCP Observability.
We would have ideally wanted public feedback before declaring the APIs
stable, but we need stable APIs for GA.
Changes made after API review with @markdroth @veblush, @ctiller and the
entire Core/C++ team -
* The old experimental APIs `grpc::experimental::GcpObservabilityInit`
and `grpc::experimental::GcpObservabilityClose` are now deprecated and
will be deleted after v.1.55 release.
* The new API gets rid of the Close method and follows the RAII idiom
with a single `grpc::GcpObservability::Init()` call that returns an
`GcpObservability` object, the lifetime of which controls when
observability data is flushed.
* The `GcpObservability` class could in the future add more methods. For
example, a debug method that shows the current configuration.
* Document that GcpObservability initialization and flushing (on
`GcpObservability` destruction) are blocking calls.
* Document that gRPC is still usable if GcpObservability initialization
failed. (Added a test to prove the same).
* Since we don't have a good way to flush stats and tracing with
OpenCensus, the examples required users to sleep for 25 seconds. This
sleep is now part of `GcpObservability` destruction.
Additional Implementation details -
* `GcpObservability::Init` is now marked with `GRPC_MUST_USE_RESULT` to
make sure that the results are used. We ideally want users to store it,
but this is better than nothing.
* Added a note on GCP Observability lifetime guarantees.
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
Adds a simple hello-world example for GCP Observability purposes along
with a README.
The microservices observability user guide is not yet up, but this still
refers to it anyway.
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
1. All greeter servers now support flag `--port` to customize the listen
port.
2. All client implementation now have `--target` flag to specify the
server location.
3. ABSL is used to parse the flags to reduce the amount of boilerplate
code in the examples.
Fixes: #26989
* Add timeout example
* Add pb2 file to example
* Remove .proto file
* Add keep_alive example
* Add refelction client
* fixes
* Add example for health_check
* Changes based on comments
* Fix pylint
* Add wait_for_ready with client timeout example
* Use with to create channel
* Fix sanity tests
* Changes based on comments
* Change thread to daemon thread
* Changes based on comments
* Revert "Revert "Add support for systemd socket activation (#30485)" (#31617)"
This reverts commit 867dc6cae2.
* Add checks to unix tests
* Ran generate_projects.sh and fixed styling in test
* Fix variable in unit test
* Use reinterpret_cast in test
* Rebase and fix sanity failures
* Add concurrency into python example in route_guide
- replace await single tasks sequentially with await task_group
* Add comments for asyncio.gather for concurrency
* Correct typo
* Update asyncio_route_guide to meet google style
Co-authored-by: Richard Belleville <rbellevi@google.com>
* Update third_party/protobuf
* run tools/distrib/python/make_grpcio_tools.py
* regenerate protos for ruby, php
* update build_handwritten.yaml
* regenerate projects
* Build - Use :well_known_type_protos instead of :well_known_protos
* Fix target
* Update upb
* Update Python for Protobuf 4.21 (#140)
* Update protobuf dependency on grpcio-tools
* Off by one
* Drop python 3.6 support
* Try upgrading pip
* And in the other script
* Try to figure out if we're compatible with abi3
* See what we've already got installed
* Update the requirements.txt file I didn't know existed
* And here too
* See what's installed
* Let's try that again
* Remove
* Try to confirm version
* Let me see the generated code
* Fix non-Bazel test runner
* Work for all test directories
* Regenerate example protos
* Clean up
* Generate .pyi files
* Fix type checking and linting
* Exclude pyi files from isort
* Upgrade to 3.21.4
* Update iwyu to get around messy protobuf IWYU rules
Co-authored-by: Richard Belleville <gnossen@gmail.com>
* replace darwin checks in extconf.rb to exclude TruffleRuby
* inherit RANLIB and STRIP from RbConfig, set LDXX
* enable overriding ranlib command in top-level makefile
* ensure the -no_warning_for_no_symbols flag is only used with Apple's ranlib
* don't embed openssl & zlib on truffleruby
* add RbConfig's cppflag to CPPFLAGS when using TruffleRuby
* this ensure the paths to find the system's OpenSSL are set up correctly with TruffleRuby (includes being able to find an OpenSSL installed via Homebrew etc)
* don't statically link standard libraries on Linux with Truffleruby
* This does not work when compiling to bitcode.
* Prefer SIGTERM to SIGQUIT for graceful shutdown in examples
* Overriding SIGQUIT is suboptimal, for example on JVM where it is very
useful to dump the thread stacktraces.
* Keep the rb_tr_abi_version symbol for TruffleRuby in grpc_c.so
* Otherwise TruffleRuby cannot verify the ABI version is correct.
* See https://github.com/oracle/truffleruby/issues/2386
* Use RbConfig::CONFIG['STRIP'] instead of just `strip`
* Use a local variable for apple_toolchain for consistency
* Add a comment about -static-libgcc -static-libstdc++ and TruffleRuby
* Split comment into two for openssl/zlib
Co-authored-by: Nicolas Laurent <nicolas.laurent@oracle.com>
* Fixing RouteGuide's C++ Reactor example.
The current method involves locking and unlocking a mutex from different
threads, which isn't allowed. Changing the strategy a bit to address
this.
* Automated change: Fix sanity tests
* Switching to absl::Mutex to annotate usage properly.
* Actually, let's not cover the examples with sanity checks.
* Update target frameworks in examples.
* Update target frameworks and NuGets in examples.
* upgrade Xds example
Co-authored-by: Jan Tattermusch <jtattermusch@google.com>
* Add python async example for hellostreamingworld using generator
* add stub.read() client example + fix Copyrights + small improvements
* Format and lint + remove exernal generator
* Apply lint and auto-format
* Add failing end2end test for inconsistent percent-decoding of URIs
* Add passing h2_local_abstract_uds end2end tests
* Add basic abstract UDS example
* add test runner
* add proper bazel build path
* first attempt at CI configuration
* cleanup
* rename CI files for better readability
* Assert Android API >= v21
This precedes a change that would otherwise break on older Android APIs,
but in a more obvious way.
* error if __ANDROID_API__ is not defined
* update all Andriod minSdkVersions to 21
* csharp experimental: android-19 to 21
Due to limitation of JVM, when user want to create binder channel in
threads created in unmanaged native code, they will need to call this
new API first to make sure Java helper classes can correctly be found.
Unused code in jni_utils.cc are also cleaned up in this commit
* [BinderTransport] Avoid depending on NdkBinder at compile time
We would like to make it possible to use BinderTransport in a APK that
has min sdk version lower than 29 (NdkBinder was introduced at 29)
We copies constants and type definitions from Ndk headers, creates a
same name wrapper for every NdkBinder API we use in
grpc_binder::ndk_util namespace.
We will try to load libbinder_ndk.so and resolve the symbol when the
NdkBinder API wrappers are invoked.
* regenerate projects
* Add GRPC_NO_BINDER guard
* Fix all lint errors in repo.
* Use strict buildifier by default
* Whoops. That file does not exist
* Attempt fix to buildifier invocation
* Add missing copyright
A config option is provided so user can pass --define=grpc_no_binder=true to bazel, or passing `-DGRPC_NO_BINDER` to compiler to disable the dependency.
* Expose experimental binder transport API
New headers are added
`grpcpp/create_channel_binder.h `: interfaces for creating client
channel
`grpcpp/security/binder_credentials.h`: interfaces for binder server
credentials
`grpcpp/security/binder_security_policy.h`: interfaces for binder
security policy, which is used by both server and client. Individual
security policies are merged into this single header.
Users can now depend on the `grpc++_binder` target to use the headers
listed above.
* Regenerate projects
* [BinderTransport] Create client channel instead of direct channel
In this commit we create a client channel instead of direct channel.
BinderConnector is added to connect subchannel when the user actually
make RPC call using the channel.
BindToOnDeviceServerService() is not required anymore since now the
actual connection is delay until the channel is used.
* Regenerate projects.
This commit
1. Implements the security policies that does not require JNI. This
includes a security policy that always allow connection and a security
policy that allows connection when remote UID is the same us local UID.
2. Add security policy as an argument in our interfaces. Old interfaces
are temporarily preserved so compilation will not suddenly break when we
import the code to internal repo.
3. Pumping the security policy from the public interfaces to the code
that handles SETUP_TRANSPORT transaction.
4. Abort the transport setup when the security policy is not satisfied.
Since meaningful tests will require to be run in real Android
environment, we will implement it later. For now, this change is
manually tested with example APKs.
For security policies that need to invoke Java to check authorization,
we will implement them in later.
Some code are commented out because the server interface has not been
merged yet.
After this, user should be able to install both apps on Android device
and test if basic unary call works or not
Also try use the API in example apk to make sure the binder transport
code and the rest of gRPC compiles with the Android toolchain
We will properly expose our interface later.
This commit adds an app that will be served as BinderTransport example
later.
Currently the app simply calls C++ function when button is pressed. In
the future the C++ function will run BinderTransport gRPC example
instead.
Tests not included in this commit, later we will add a build test to
CI to make sure this apk is always build-able. (This app will also be
used to make sure our BinderTransport implementation is compile-able
with Android toolchain, on GitHub) For now we exclude this target in
bazel build test.
Changes are made to WORKSPACE file in order to support android builds.
Build instructions are documented at
examples/android/binder/java/io/grpc/binder/cpp/example/README.md
* Add isort_code.sh to sanity tests
* Run tools/distrib/isort_code.sh
* Fine tune the import order for relative imports
* Make pylint and project generation happy
* Fix a few corner cases
* Use --check instead of --diff
* The import order impacts test result somehow
* Make isort print diff and check output at the same time
* Let tools/run_tests/python_utils be firstparty library
* Run isort against latest HEAD
* De-experimentalize callback API
* Make FromServerContext based on ServerContextBase
* Fix lambda
* Fix headers
* De-experimentalize tests
* clang-format
* Fix consistency checks
* wip
* Fix const-ness of callback client read RPC requests
* Fix golden file
* Give full route_guide callback client example with Hold etc
* Complete example route-guide server
* De-experimentalize test services
* Omit unneeded using
* Remove some uses of non-experimental macro from test
* clang-format examples
* De-experimentalize async stub calls
* Remove experimental namespace use in qps, ubms
* De-experimentalize alarms, generic stubs, and context allocator
* De-experimentalize SetContextAllocator
* clang-format
* Fix conflicts
* Leave obsolete API in place until users can be migrated
Xuan Wang