Continue supporting the current grpc-testing that I suppose is used
inside of Google, but also allow to configure a different project to
upload results to.
The format "project_id.dataset_id.table_id" is common for BigQuery so it
seems idiomatic to do it in this way. Adding a separate command line
option would be more complicated because it would require changes all
the way down the chain (at least in the entry point for the test driver
and in the LoadTest controller).
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#35384
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35384 from lepistone:choose_bigquery_project 2355fea28c
PiperOrigin-RevId: 595523944
This PR adds CSM Observability testing capability in the PSM Interop testing framework. This PR mostly changes the framework Python code.
This adds a flag `enable_csm_observability` to the client / server deployment yaml file such that, when enabled, we will create a GMP `PodMonitoring` resource and pass the `--enable_csm_observability` to each language's client / server container (for them to actually enable the Prometheus endpoint)
I added a new test under `tests/csm/csm_observability_test.py`. This is basically a copy of the `tests/baseline_test.py` but with the `enable_csm_observability=True`.
Other PRs for this whole thing to work:
- https://github.com/grpc/grpc/pull/34752: The `PodMonitoring` resource yaml template
- https://github.com/grpc/grpc/pull/34832: Support for the `--enable_csm_observability` flag in the C++ client/server image
Closes#34835
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/34835 from stanley-cheung:csm-o11y-framework-changes 0b3d0eb7ed
PiperOrigin-RevId: 595502496
- `memory_pressure_controller` finally - allows deletion of pid_controller throughout the codebase
- `overload_protection` - one of the http2 rapid reset mitigations
- `red_max_concurrent_streams` - another http2 rapid reset mitigation
Closes#35426
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35426 from ctiller:new-years-cleanse 4651672e7e
PiperOrigin-RevId: 595205029
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#35292
PiperOrigin-RevId: 595188404
Remove the old `switch` library - this used to be an implementation detail of `Seq`, `TrySeq` - but has become unused.
Add a new user facing primitive `Switch` that fills a similar role to `switch` in C++ - selecting a promise to execute based on a primitive discriminator - much like `If` allows selection based on a boolean discriminator now.
A future change will optimize this to actually lower the `Switch` into an actual `switch` statement, but for right now I want to get the functionality in.
Closes#35424
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35424 from ctiller:switchy 5308a914c6
PiperOrigin-RevId: 595140965
Whilst here, eliminate unnecessary mutexes and streamline some complexity in the read variants.
Closes#35409
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35409 from ctiller:pbe 4f9588101a
PiperOrigin-RevId: 595006455
Adds temporary `call.cc` and `connected_channel.cc` scaffolding to run `CallInterceptor`/`CallHandler` style calls.
This will get ripped out as soon as the v3 transition is completed.
Closes#35312
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35312 from ctiller:v3-accept ae0bf81f8b
PiperOrigin-RevId: 594128029
It turned out that the previous change missed two things which this PR
has
- Fix function `_dockerized_genrule` not to have `timeout` and `flaky`
which Bazel doesn't have. (Bazel 7 may either drop these arguments or
become more strict about passing unrecognized ones)
- Disabled python bazel distribution tests with Bazel 7. This needs to
be addressed by https://github.com/grpc/grpc/issues/35391.
- Added Bazel 7 to the support bazel versions.
- Changed the default Bazel version to 7.
- Fixed Android Binder build issue.
Closes#35362
PiperOrigin-RevId: 592946781
This is a prerequisite change to start supporting Bazel 7. Changes are
- Disabled bzlmod which Bazel 7 begins to enable by default. This eventually needs to be done to support bzlmod but not now.
- Upgraded some bazel rule dependencies which are required to support Bazel 7.
- Using Python 3 explcitly as Bazel 7 begins to reject Python 2.
Note that this isn't enough to enable Bazel 7 by default and another PR will follow for that.
Closes#35374
PiperOrigin-RevId: 592931675
Simple `assert` statements don't help much to know what needs to be done. Instead, explicit error messages will let us know what's wrong which is helpful to know what to look at.
Closes#35375
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35375 from veblush:check-work 0733499c31
PiperOrigin-RevId: 592920747
Fix: https://github.com/grpc/grpc/issues/35085
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#35325
PiperOrigin-RevId: 592635611
When an error occurs in `TrySeq`, log the error.
Also, in the trace statements, capture the file/line that the sequence was constructed at, and log that with the traces.
Closes#35319
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35319 from ctiller:trace-seq bc0d47395f
PiperOrigin-RevId: 591999228
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#35280
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35280 from yashykt:UpdateInteropScriptForFindingAdsChannel db213384b4
PiperOrigin-RevId: 591090750
`AllOk` runs a set of promises concurrently, and like `TryJoin` waits for them all to succeed or one to fail.
Unlike `TryJoin` it returns a single unified status of the composition, so cannot handle member promises that might return `StatusOr` or the like.
Closes#35304
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35304 from ctiller:all-review 30f5f809c6
PiperOrigin-RevId: 591031189
Enable OpenSSL 1.0.2 tests and add a container for 1.1.1 so that it is tested during portability testing as well.
Closes#35236
PiperOrigin-RevId: 590345568
`_parallel_compile_patch.py` exist in multiple places, we should use template system to generate the content.
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#34847
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/34847 from XuanWang-Amos:use_template_for__parallel_compile_patch 15b98ea826
PiperOrigin-RevId: 590262888
### Changes in this PR
* Refactor and remove some Core/C++ dependencies to simplify Python Observability package build process.
* Refactored code to read config at Python layer.
* Enable observability build from source.
* Add observability to run_test.
* Currently it's only enabled in Linux.
* Add error handler in run_test loaders.
* Current framework will always visit modules in test directory then decide which tests to skip.
* Since we're not building Observability for MacOS and Windows this step will fail with error `No module named 'grpc_observability'`.
* After the change we'll just skip those modules.
* We still have `_sanity_test` to make sure all tests are loaded correctly for each platform.
* Remov OC dependency as we're migrating to OTel.
* Also removed trace from testing.
* Note that trace propagation function was also removed because of this.
### Testing
* Passed existing tests.
* Tested locally, able to build observability from source using `GRPC_PYTHON_BUILD_WITH_CYTHON=1 pip install .`.
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#34207
PiperOrigin-RevId: 590258014
Starting from Python 3.11, the pipes module produces this warning:
DeprecationWarning: 'pipes' is deprecated and slated for removal in Python 3.13
Turns out that in this repo the pipes module is only used for the
"quote" function which is turn directly taken from the shlex module [1].
The shlex module is not deprecated as of today and is already used in
other places in this repo. The function shlex.quote has been around
since the ancient Python 3.3.
[1] https://github.com/python/cpython/blob/3.11/Lib/pipes.py#L64-L66
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#34941
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/34941 from lepistone:deprecate-python-pipes 233c54c135
PiperOrigin-RevId: 588883480
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#35153
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35153 from yijiem:native_dns_resolver a4107f7d81
PiperOrigin-RevId: 588543137
Yash Tibrewal