Commit Graph

54389 Commits

Author SHA1 Message Date
Tanvi Jagtap 33a4c8c994 Internal configuration cleanup
PiperOrigin-RevId: 590529857
2023-12-14 03:43:22 +00:00
Tanvi Jagtap 8019903a44 Internal configuration cleanup
PiperOrigin-RevId: 590528536
2023-12-14 03:42:30 +00:00
Craig Tiller a04188b29f [inproc] Promise based inproc transport (#35281)
Closes #35281

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35281 from ctiller:cg-inproc 3fe1bce51b
PiperOrigin-RevId: 590425232
2023-12-12 20:08:30 -08:00
Craig Tiller 03403724f5 [fuzzing] Add ability for api-fuzzer to create inproc channels (#35277)
Closes #35277

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35277 from ctiller:inproc-fuzzing 853c11f144
PiperOrigin-RevId: 590420030
2023-12-12 19:55:27 -08:00
Gregory Cooke 12c0058bf2 [Security - Testing] OpenSSL 1.0.2 and 1.1.1 CI Testing (#35236)
Enable OpenSSL 1.0.2 tests and add a container for 1.1.1 so that it is tested during portability testing as well.

Closes #35236

PiperOrigin-RevId: 590345568
2023-12-12 14:52:43 -08:00
Xuan Wang 4bf0109f52 [Python Misc] Use template to generate _parallel_compile_patch.py (#34847)
`_parallel_compile_patch.py` exist in multiple places, we should use template system to generate the content.

<!--

If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the appropriate
lang label.

-->

Closes #34847

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/34847 from XuanWang-Amos:use_template_for__parallel_compile_patch 15b98ea826
PiperOrigin-RevId: 590262888
2023-12-12 11:00:34 -08:00
Xuan Wang 7f2ecdb3f8 [Python Observability] Building package and add to run_test (#34207)
### Changes in this PR
* Refactor and remove some Core/C++ dependencies to simplify Python Observability package build process.
* Refactored code to read config at Python layer.
* Enable observability build from source.
* Add observability to run_test.
  * Currently it's only enabled in Linux.
* Add error handler in run_test loaders.
  * Current framework will always visit modules in test directory then decide which tests to skip.
  * Since we're not building Observability for MacOS and Windows this step will fail with error `No module named 'grpc_observability'`.
  * After the change we'll just skip those modules.
    * We still have `_sanity_test` to make sure all tests are loaded correctly for each platform.
* Remov OC dependency as we're migrating to OTel.
  * Also removed trace from testing.
  * Note that trace propagation function was also removed because of this.

### Testing
* Passed existing tests.
* Tested locally, able to build observability from source using `GRPC_PYTHON_BUILD_WITH_CYTHON=1 pip install .`.

<!--

If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the appropriate
lang label.

-->

Closes #34207

PiperOrigin-RevId: 590258014
2023-12-12 10:51:35 -08:00
Mark D. Roth 2cc1550e77 [release] add 1.60 to interop matrix (#35274)
Closes #35274

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35274 from markdroth:release_interop_matrix_update b1cfcad14b
PiperOrigin-RevId: 590257600
2023-12-12 10:42:20 -08:00
Craig Tiller 75686ed19d [channel-args] Optimize channel args to string (#35286)
One of the more major CPU sinks for b/311358095

Closes #35286

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35286 from ctiller:fuzz-311358095 ad4278937f
PiperOrigin-RevId: 590247206
2023-12-12 10:13:23 -08:00
alto-ruby eaf1a92264 [Ruby] remove #include <sys/time.h> (#34178)
#34168

Closes #34178

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/34178 from alto-ruby:no-sys-time-h 42c5ad0f79
PiperOrigin-RevId: 590223092
2023-12-12 09:05:07 -08:00
Tanvi Jagtap 0353e5b9c4 Internal configuration
PiperOrigin-RevId: 590178979
2023-12-12 06:18:27 -08:00
Craig Tiller 9118cb1a55 Fix fuzzer bug b/309716763 (#35278)
Also cleanup a little so we're not copying redundant frame headers everywhere.

Closes #35278

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35278 from ctiller:fuzz-309716763 52589ff422
PiperOrigin-RevId: 590042072
2023-12-11 20:21:36 -08:00
Craig Tiller ff63ad9413 [call-v3] Convert message compression filter (#35269)
Closes #35269

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35269 from ctiller:cg-compress 0304821f07
PiperOrigin-RevId: 590010564
2023-12-11 17:58:00 -08:00
Craig Tiller a7e90045db [promises] Fix fuzzer found uncancelable call bug in client promise code (#34909)
Closes #34909

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/34909 from ctiller:lambda-mcfly 92f759d763
PiperOrigin-RevId: 589989462
2023-12-11 16:35:55 -08:00
Craig Tiller b63a21b419 [inproc] Call out requirements for inproc transport experiment (#35267)
Closes #35267

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35267 from ctiller:cg-experiment-requires 242a3cd21a
PiperOrigin-RevId: 589932835
2023-12-11 13:38:45 -08:00
Craig Tiller 2b1d06f037 [fuzzing] Add ability to listen to ports to api-fuzzer (#35273)
Unsure when this got dropped!

Closes #35273

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35273 from ctiller:server-port be94e30a06
PiperOrigin-RevId: 589919718
2023-12-11 13:04:19 -08:00
Craig Tiller 57dacad8c7 [call-v3] Idle/max-age filters (#35270)
Closes #35270

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35270 from ctiller:cg-idle 5312003ca6
PiperOrigin-RevId: 589913523
2023-12-11 12:45:34 -08:00
Gregory Cooke a5c71e132e [Security - TLS] DirectoryReloaderCrlProvider construction time bug fix (#35247)
The `DirectoryReloaderProvider` currently segfaults on construction if grpc_init() is not called before construction. This is because when creating the `DirectoryReloaderCrlProvider` we [call GetDefaultEventEngine](a58f3f2df5/src/core/lib/security/credentials/tls/grpc_tls_crl_provider.cc (L152)), and getting the default event engine requires that `grpc_init` is called.

This PR adds a test that catches the segfault and adds `grpc_init` and `grpc_shutdown` to the ctor and dtor of `DirectoryReloaderCrlProvider` so that the test passes.

Closes #35247

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35247 from gtcooke94:crl_provider_init_fix 25f3dc7f27
PiperOrigin-RevId: 589885254
2023-12-11 11:25:49 -08:00
John Cormie 2ffdca65e5 binder: skip all tests on ios (#35226)
They don't build due to -DNO_GRPC_BINDER.

Closes #35226

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35226 from jdcormie:build-skip-ios 408b57bd6e
PiperOrigin-RevId: 589870083
2023-12-11 10:39:50 -08:00
Luwei Ge dd12460018 [tls] Add set min/max TLS version APIs to TLS credentials APIs. (#34861)
Address #28382. This is a recreation of #31368 except e2e tests are not handled here (yet).

Closes #34861

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/34861 from rockspore:tls_version f9a1215ac1
PiperOrigin-RevId: 589847110
2023-12-11 09:27:44 -08:00
Craig Tiller 71367424d6 [call-v3] Make call traces show useful information (#35262)
Closes #35262

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35262 from ctiller:cg-call-trace 16ea4c8807
PiperOrigin-RevId: 589637737
2023-12-10 14:44:02 -08:00
Hannah Shi 42b57fc632 [ObjC] Support use frameworks! (#34921)
1. add swift example configured with `use_frameworks!`.

2. fix [unable to find C++ headers](https://source.cloud.google.com/results/invocations/2100300a-0b61-4e56-8a77-113e362f2c63/targets/grpc%2Fcore%2Fpull_request%2Fmacos%2Fgrpc_basictests_objc_examples/log) by renaming `.m` files to `.mm`.
`/Volumes/BuildData/tmpfs/altsrc/github/grpc/workspace_objc_macos_opt_native/src/objective-c/examples/SwiftUseFrameworks/Build/Build/Build/Products/Debug-iphoneos/gRPC-Core/grpc.framework/Headers/grpc_audit_logging.h:24:10: fatal error: 'memory' file not found #include <memory>`

3. fix [cyclic dependency errors](https://source.cloud.google.com/results/invocations/90972079-eade-4c07-a5c4-20f139c64d4e/targets/github%2Fgrpc%2Frun_tests%2Fobjc_macos_opt_native%2Fios-buildtest-example-switftsample/tests) by making `openssl/time.h` private.
`/Applications/Xcode_14.1.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS16.1.sdk/usr/include/dispatch/dispatch.h:25:10: error: cyclic dependency in module 'Darwin': Darwin -> openssl_grpc -> UIKit -> Foundation -> CoreFoundation -> Dispatch -> Darwin`

"Basic Tests ObjC Examples" succeeds with this PR:
https://source.cloud.google.com/results/invocations/cff74ec0-43ca-43b9-a2ff-12edabc02896/targets

Closes #34921

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/34921 from HannahShiSFB:support-use-frameworks 87ddd23090
PiperOrigin-RevId: 589300261
2023-12-08 18:57:08 -08:00
Craig Tiller 2fde70a6be [call-v3] Generic forwarder from a CallHandler to a CallInterceptor (#35256)
Closes #35256

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35256 from ctiller:cg-fwd-call cdaae8bccd
PiperOrigin-RevId: 589278551
2023-12-08 16:56:33 -08:00
Sergii Tkachenko 16e4edcdcf
[PSM Interop] Configure CODEOWNERS (#35254)
Configures CODEOWNERS for the PSM Interop framework
`/tools/run_tests/xds_k8s_test_driver/` to be @sergiitk, @XuanWang-Amos,
@gnossen.

We need this change to be able to prevent unexpected changes to the the
framework while it's being moved to the dedicated grpc/psm-interop repo.
2023-12-08 16:03:19 -08:00
Yash Tibrewal c2261c6f28
[example] Add WAIT_FOR_READY example (#35219) 2023-12-08 14:34:26 -08:00
Eugene Ostroukhov d9e4d1302e
[examples] Implement custom metrics reporting from server (#35218) 2023-12-08 14:33:59 -08:00
Yijie Ma 86d90f54b0 [EventEngine] Skip `dns_resolver_cooldown_test` for `event_engine_dns` experiment (#35251)
<!--

If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the appropriate
lang label.

-->

Closes #35251

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35251 from yijiem:fix-dns-resolver-cooldown-test 857835200a
PiperOrigin-RevId: 589159895
2023-12-08 09:47:36 -08:00
Vignesh Babu f0d7df79da [EventEngine] Add virtual a MakeSlice method into MemoryAllocatorImpl and move its default implementation to GrpcMemoryAllocatorImpl
PiperOrigin-RevId: 589155443
2023-12-08 09:32:41 -08:00
Craig Tiller 257efc0375 [call-v3] Add a hook to deadline filter (#35252)
Still need to figure a better long term strategy here, but this 'works for now'

Closes #35252

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35252 from ctiller:cg-deadline a413955afc
PiperOrigin-RevId: 588980941
2023-12-07 19:26:01 -08:00
Richard Belleville 25c298224a Pin googleapis-common-protos (#35255)
<!--

If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the appropriate
lang label.

-->

Closes #35255

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35255 from gnossen:freeze_googleapis_bazel 38fb05d8db
PiperOrigin-RevId: 588965912
2023-12-07 18:01:21 -08:00
Sergii Tkachenko 522434e4f9 docs: service_config.md: fix pb syntax, add highlighting (#35232)
1. Protobuf doesn't support `//` comments, only `#`
2. Add syntax highlighting

Closes #35232

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35232 from grpc:sergiitk-service_config e17a9ef14d
PiperOrigin-RevId: 588943566
2023-12-07 16:25:58 -08:00
Hannah Shi 70acb340ce [ObjC] add privacy manifests to grpc podspec (#35042)
Not sure if we need to do the same for all the podspec files.

Will create a separate PR for swift package.

Closes #35042

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35042 from HannahShiSFB:privacy-manifests c9b19d3c20
PiperOrigin-RevId: 588937122
2023-12-07 16:03:14 -08:00
Alisha Nanda 8cfce72868 [server] Remove per-channel registered methods map (#35231)
https://github.com/grpc/grpc/pull/34286 was split into two parts - the first part was submitted in https://github.com/grpc/grpc/pull/34612, to move to absl::flat_hash_map for per-channel registered methods map, and this is the second part to remove the per-channel map and switch to using per-server.

Closes #35231

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35231 from ananda1066:remove_rm2 6b154545bf
PiperOrigin-RevId: 588928459
2023-12-07 15:34:36 -08:00
Craig Tiller 22537cb193 [call-v3] Convert server-call-tracer filter (#35249)
Closes #35249

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35249 from ctiller:cg-server-call-tracer ad52acb64f
PiperOrigin-RevId: 588885615
2023-12-07 13:19:51 -08:00
Xuan Wang de6c437c89 [Python Misc] Revert change to print backtrace in server (#34877)
Fix: https://github.com/grpc/grpc/issues/34853

In order to make debugging easier, we have begun printing backtraces in servers. However, this change has the unintended consequence of printing errors to stderr by default, which may not be expected by some users.

This PR reverts the change. We recommend that users set up a logging sink if they want to see errors. We will add this to our documentation later.

<!--

If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the appropriate
lang label.

-->

Closes #34877

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/34877 from XuanWang-Amos:revert_print_backtrace_change 8942308784
PiperOrigin-RevId: 588885550
2023-12-07 13:10:40 -08:00
Leonardo Pistone dfa9d853ff Fix Python DeprecationWarning: 'pipes' (#34941)
Starting from Python 3.11, the pipes module produces this warning:

DeprecationWarning: 'pipes' is deprecated and slated for removal in Python 3.13

Turns out that in this repo the pipes module is only used for the
"quote" function which is turn directly taken from the shlex module [1].

The shlex module is not deprecated as of today and is already used in
other places in this repo. The function shlex.quote has been around
since the ancient Python 3.3.

[1] https://github.com/python/cpython/blob/3.11/Lib/pipes.py#L64-L66

<!--

If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the appropriate
lang label.

-->

Closes #34941

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/34941 from lepistone:deprecate-python-pipes 233c54c135
PiperOrigin-RevId: 588883480
2023-12-07 13:02:23 -08:00
Craig Tiller 5f92a67f94 [call-v3] Convert message size filter to new API (#35233)
Closes #35233

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35233 from ctiller:cg-msg-size cce51d8bd5
PiperOrigin-RevId: 588793125
2023-12-07 08:05:20 -08:00
Craig Tiller 6c816a4f99 [call-v3] Re-add accidentally deleted comment (#35235)
Closes #35235

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35235 from ctiller:cg-fix-comment2 5abebaaadb
PiperOrigin-RevId: 588544922
2023-12-06 14:33:43 -08:00
Yijie Ma 1abfc97502 [EventEngine] Posix NativeDNSResolver implementation (#35153)
<!--

If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the appropriate
lang label.

-->

Closes #35153

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35153 from yijiem:native_dns_resolver a4107f7d81
PiperOrigin-RevId: 588543137
2023-12-06 14:25:38 -08:00
Anirudh Ramachandra a58f3f2df5 [core] Add support for RefIfNonZero in InternallyRefCounted. (#34869)
The support already exists in RefCounted and DualRefcounted, so expose similar API for InternallyRefCounted class

Closes #34869

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/34869 from anicr7:orphanable_ref_if_nonzero f57c64dc62
PiperOrigin-RevId: 588514955
2023-12-06 13:01:20 -08:00
Mark D. Roth 626bf5351b [experiments] extend a bunch of experiments for a few months (#35230)
Closes #35230

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35230 from markdroth:experiments_extend ea733a4840
PiperOrigin-RevId: 588514902
2023-12-06 12:53:01 -08:00
AJ Heller 8c378461c3
[example] C++ Debugging (#35222) 2023-12-06 11:21:11 -08:00
Craig Tiller 295b665a18 [call-v3] Add CallInitiator, CallHandler wrappers around CallSpine (#35223)
Closes #35223

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35223 from ctiller:cg-initiator 78826d99c5
PiperOrigin-RevId: 588481455
2023-12-06 10:59:47 -08:00
Mark D. Roth e497eed251 [XdsCertificateProvider] clean up some cruft (#35227)
- Remove old-style channel args utility methods, which are unused.
- Change `ChannelArgsCompare()` to delegate to the `Compare()` method.

Closes #35227

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35227 from markdroth:xds_cert_provider_cleanup 57dc0c0757
PiperOrigin-RevId: 588440245
2023-12-06 08:49:17 -08:00
Craig Tiller e8d9f222f5 [server] Make SetRegisteredMethodOnMetadata a method, not a static member fn (#35221)
As part of the call-v3 work I'll be making a call to this code via a different path shortly, and separating the C-style callback piece out helps that work

Closes #35221

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35221 from ctiller:cg-registered-method 4b6d80ee7f
PiperOrigin-RevId: 588200784
2023-12-05 15:08:18 -08:00
Eugene Ostroukhov 1a086609d7 [server] Fix typo in the class name (#35216)
Closes #35216

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35216 from eugeneo:tasks/typo-aync d9f58bc130
PiperOrigin-RevId: 588146516
2023-12-05 12:01:57 -08:00
Craig Tiller 2fa8018d2a [call-v3] Introduce some cancellation helpers to CallSpine (#35212)
These will be used in upcoming changes to allow easy spawning of promises into parties that properly cancel out things when those promises fail.

Closes #35212

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35212 from ctiller:cg-cancel-nicely 32fc7aa09a
PiperOrigin-RevId: 588130225
2023-12-05 11:13:23 -08:00
Craig Tiller 9eadf42dbf [promises] Add an API to access new style filters (#35200)
Will be used during this transition time to run 5-pipe style filters somewhat more natively. Once everything is getting closer to 5-pipes, we'll drop this method and have the channel stack understand how to create an interception-map that can be reused per-call, instead of creating the interception-map every time a call is created.

Closes #35200

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35200 from ctiller:cg-channel-filter-api 2fc11dd273
PiperOrigin-RevId: 587940947
2023-12-04 22:12:37 -08:00
Craig Tiller 01fb4a0fe4 [experiments] Clean up some rolled out experiments (#35195)
Closes #35195

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35195 from ctiller:cleanup-cleanup 1f22298ac9
PiperOrigin-RevId: 587857022
2023-12-04 15:17:42 -08:00
Sergii Tkachenko b570879010 [PSM Interop] Log on debug level when resource deletion failure is 404 (#35131)
Removes noise from the cleanup/teardown ops.

#### GCP APIs

In GCP APIs, change log level for delete operations that failed because the resource doesn't exist (API 404) from `info` to `debug`.  Framework's logging philosophy is to only log external operations (e.g. APIs, RPCs). If no error logged, the op is assumed successful.
In the deletion case, is still possible to discriminate between whether the op was actually performed by observing the `Waiting %s sec for %s operation id: %s` log message.

#### K8s APIs
In K8s APIs:
- For delete operations that failed because the resource doesn't exist (API 404) the log level is changed from `info` to `debug`
- For delete operations that failed for any other reason, the log level is changed from `info` to `warning`
- When `wait_for_deletion` is enabled (it's the default) the delete operation will be confirmed with `logger.info("<resource_kind> %s deleted", name)`. Previously it logged at the `debug` level.

Closes #35131

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35131 from sergiitk:psm-interop-debug-log-on-delete-404 f6629e5132
PiperOrigin-RevId: 587851692
2023-12-04 15:01:51 -08:00