Commit Graph

3211 Commits

Author SHA1 Message Date
Mark D. Roth c43f1a63b0 [RLS] implement non-per-call metrics (#36001)
Closes #36001

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36001 from markdroth:non_per_call_metrics_rls b713980f21
PiperOrigin-RevId: 612532005
2024-03-04 11:58:31 -08:00
Mark D. Roth 0213523907 [build] move channel out of grpc_base (#35924)
This adds the following new targets:
- `channel`: A virtual interface for a channel.
- `legacy_channel`: A channel implementation that supports the filter stack and call v2.
- `channel_create`: A standalone function to create a channel.
- `server_interface`: A base class with a few accessor methods used in surface/call.cc.
- `server`: The actual server implementation.
- `api_trace`, `call_tracer`, `server_call_tracer_filter`, `call_finalization`: These were split out of `grpc_base` to avoid various dependency problems.
- `compression`: This is a combination of the previously existing `compression_internal` target and the compression code that was part of `grpc_base`.

Closes #35924

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35924 from markdroth:channel_interface 94a7fffddb
PiperOrigin-RevId: 612512438
2024-03-04 11:07:17 -08:00
Alisha Nanda fa2bae473a [metadata] Expose metadata limits and annotation info to filter stack (#35970)
<!--

If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the appropriate
lang label.

-->

Closes #35970

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35970 from ananda1066:client_metadata b62a9ae146
PiperOrigin-RevId: 610830228
2024-02-27 12:07:35 -08:00
Mark D. Roth 40577dd585 [WRR] implement non-per-call metrics (#35977)
As per gRFC A78 (https://github.com/grpc/proposal/pull/419).

Note that these new metrics are populated only if the `wrr_delegate_to_pick_first` experiment is enabled, which is the case by default.

Closes #35977

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35977 from markdroth:non_per_call_metrics_wrr 7acea32fc4
PiperOrigin-RevId: 610760874
2024-02-27 08:32:09 -08:00
Gregory Cooke a1b7e1e523 [Security] Crl helpers (#35963)
Re-applies after the revert
Fixes the Openssl 1.0.2 portability errors

Closes #35963

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35963 from gtcooke94:CrlHelpers 9cfb401a47
PiperOrigin-RevId: 610503595
2024-02-26 13:12:36 -08:00
Yousuk Seung 641213ee58 [tracing] Remove debugging annotations
PiperOrigin-RevId: 610469149
2024-02-26 11:22:52 -08:00
Jan Katins d4afc993ed
[python] Fix typo in code comment (#35981)
Found this while reading the code.
2024-02-26 09:22:44 -08:00
Xuan Wang ca36ff84ba [Python o11y] Fix readme (#35988)
Fix `readme.rst`.

Tested locally by running `python setup.py doc`, no more errors from observability.

Only error is related to grpc_status:
```
WARNING: autodoc: failed to import module 'rpc_status' from module 'grpc_status'; the following exception was raised:
cannot import name 'status_pb2' from 'google.rpc' (unknown location)
```

<!--

If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the appropriate
lang label.

-->

Closes #35988

PiperOrigin-RevId: 609820414
2024-02-23 13:26:20 -08:00
Matthew Stevenson 51bccbdcf9 [ssl] Support Windows system roots. (#34874)
This PR is copied from #34276, since I did not have permissions to add commits to it. That PR has been verified to work (see the top-level description). This PR just makes the gRPC tests pass (e.g. adding includes, clang formatting).

Closes #34874

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/34874 from matthewstevenson88:pull_34276 d5fb73e5b2
PiperOrigin-RevId: 609107146
2024-02-21 13:40:46 -08:00
Mark D. Roth cfcbb0a535 [build] move parts of the `grpc_client_channel` BUILD target out into their own targets (#35879)
This breaks the following pieces out of the `grpc_client_channel` BUILD target:
- backend_metric_parser
- oob_backend_metric
- child_policy_handler
- backup_poller
- service_config_channel_arg_filter
- client_channel_channelz
- client_channel_internal_header
- subchannel_connector
- subchannel_pool_interface
- config_selector
- client_channel_service_config_parser
- retry_service_config_parser
- retry_throttle

The code left in the `grpc_client_channel` target will need more work to pull apart.

Closes #35879

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35879 from markdroth:client_channel_build_split f388a37edc
PiperOrigin-RevId: 608806548
2024-02-20 18:06:37 -08:00
Mark D. Roth 179e0f952f [build] move parts of the `grpc_base` BUILD target out into their own targets (#35846)
This breaks the following pieces out of the `grpc_base` BUILD target:
- channelz
- call_trace
- dynamic_annotations
- call_combiner
- resource_quota_api
- iomgr

More work is still needed to pull apart the remaining parts of `grpc_base`.

Closes #35846

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35846 from markdroth:client_channel_build_dep_cycle_fix f1a9b6c2b2
PiperOrigin-RevId: 608680098
2024-02-20 11:36:46 -08:00
Esun Kim 34be0d84a9 [Core] Remove wrap_memcpy (#35826)
gRPC has been having the `wrap_memcpy` workaround to handle the breaking change of `memcpy` on glibc 2.14 ([ref1](https://www.win.tue.nl/~aeb/linux/misc/gcc-semibug.html), [ref2](https://stackoverflow.com/questions/35656696/explanation-of-memcpy-memmove-glibc-2-14-2-2-5)). This was necessary to build more portable artifacts which are expected to run on most linux distributions and we manged to have https://github.com/grpc/grpc/pull/5007 years ago for this.

Since we started to use manylinux2010-based docker images for artifacts, however, this became unnecessary since CentOS 6 has glibc 2.12 which is older than 2.14 so it doesn't have memcpy issue, which is another benefit of using manylinux2010.

Superseding https://github.com/grpc/grpc/pull/23385
Changes after the original PR was made; Ruby is using manylinux images (manylinux2014) enabling this change, no more old docker images in our test env.

Closes #35826

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35826 from veblush:del-wrap-memcpy-2 71e6718356
PiperOrigin-RevId: 607499060
2024-02-15 16:53:19 -08:00
Xuan Wang 2c9b599e5e [Python O11Y] Reapply registered method change (#35850)
This reverts commit a18279db2e.

<!--

If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the appropriate
lang label.

-->

Closes #35850

PiperOrigin-RevId: 607476066
2024-02-15 15:36:27 -08:00
Xuan Wang bb3edd22b3 [Python Modernization] Deprecate pkg_resources (#35849)
Discuss thread about this change: [link](https://mail.google.com/mail/u/0/#sent/QgrcJHsBpWNGRlrMktwbppGGfFTVCFLcQgL?compose=new)
<!--

If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the appropriate
lang label.

-->

Closes #35849

PiperOrigin-RevId: 607144827
2024-02-14 16:44:42 -08:00
Eugene Ostroukhov 6c157f9128 [Release] Bump version to 1.63.0-dev (on master branch) (#35899)
Change was created by the release automation script. See go/grpc-release.

Closes #35899

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35899 from eugeneo:bump_dev_version_202402131133 73950425c9
PiperOrigin-RevId: 606777850
2024-02-13 16:09:36 -08:00
Mark D. Roth 41606054c2 [load_file] remove grpc_load_file() in favor of grpc_core::LoadFile() (#35857)
Closes #35857

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35857 from markdroth:highlander 7b7d95aaa9
PiperOrigin-RevId: 605742734
2024-02-09 15:16:24 -08:00
Craig Tiller d1cb0c0874 [chaotic-good] Bring up core e2e tests (#35663)
Closes #35663

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35663 from ctiller:shush-e2e 0c1f82ed71
PiperOrigin-RevId: 605717663
2024-02-09 13:34:46 -08:00
Mark D. Roth 21cb320080 [reorg] move service config code to src/core/service_config (#35843)
Closes #35843

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35843 from markdroth:client_channel_reorg4 0c50ada6f9
PiperOrigin-RevId: 605466874
2024-02-08 16:58:20 -08:00
Mark D. Roth f22c954ef5 [reorg] move client channel code to src/core/client_channel (#35827)
Also rename client_channel.{h,cc} -> client_channel_filter.{h,cc}.

Closes #35827

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35827 from markdroth:client_channel_reorg3 449bff563f
PiperOrigin-RevId: 605006293
2024-02-07 09:23:26 -08:00
Mark D. Roth 10e83973e7 [reorg] move resolver code to src/core/resolver (#35804)
This new directory combines code from the following locations:
- src/core/ext/filters/client_channel/resolver
- src/core/lib/resolver

Closes #35804

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35804 from markdroth:client_channel_resolver_reorg2 30660e6b00
PiperOrigin-RevId: 604665835
2024-02-06 09:07:56 -08:00
Mark D. Roth 148f59c15a [reorg] move LB policy code to src/core/load_balancing (#35786)
This new directory combines code from the following locations:
- src/core/ext/filters/client_channel/lb_policy
- src/core/lib/load_balancing

Closes #35786

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35786 from markdroth:client_channel_resolver_reorg 98554efb98
PiperOrigin-RevId: 604351832
2024-02-05 10:05:35 -08:00
Craig Tiller 610f439342 [channel] Add CallFactory, CallDestination (#35766)
Closes #35766

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35766 from ctiller:chan3 ea9a3260e3
PiperOrigin-RevId: 603437058
2024-02-01 11:58:14 -08:00
Esun Kim 1f979d2a83 [BoringSSL] Update third_party/boringssl-with-bazel (#35768)
Change was created by the release automation script. See go/grpc-release.

On top of changes made by the script, `setup.py` had to be updated to have `/std:c11` option for MSVC to use c11 features for boringssl.

Closes #35768

PiperOrigin-RevId: 603403329
2024-02-01 10:06:33 -08:00
Xuan Wang 03375e064f [Python AIO] Handle DeprecationWarnings when get current loop (#35583)
Fix: https://github.com/grpc/grpc/issues/35086

<!--

If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the appropriate
lang label.

-->

Closes #35583

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35583 from XuanWang-Amos:fix_aio_eventloop aa0d207e99
PiperOrigin-RevId: 603188288
2024-01-31 16:11:10 -08:00
Craig Tiller 59004b4931 [channel] Separate call size estimation out of `grpc_core::Channel` (#35732)
Closes #35732

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35732 from ctiller:chan 0d7a29fc77
PiperOrigin-RevId: 602865976
2024-01-30 16:01:41 -08:00
Craig Tiller fb14685fd6 [transport] Move call spine related types into their own file (#35689)
Part of a continued hygiene effort for transport.h

(pure code movement, no refactoring at this time)

Closes #35689

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35689 from ctiller:channel-ez 3366310cc3
PiperOrigin-RevId: 602416330
2024-01-29 09:49:23 -08:00
Esun Kim 714640d71f [Deps] Upgrade Abseil to 2024-01-16 (#35589)
Closes #35589

PiperOrigin-RevId: 601647295
2024-01-25 20:14:19 -08:00
Xuan Wang a73293bf4c [Python AIO] Add note for read/write APIs (#35598)
Fix: https://github.com/grpc/grpc/issues/35061

Add note to indicate that iterator and read/write APIs can't be mixed in a single RPC.

<!--

If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the appropriate
lang label.

-->

Closes #35598

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35598 from XuanWang-Amos:aio_mix_apis_doc 4c4cfbf06e
PiperOrigin-RevId: 601580411
2024-01-25 15:16:14 -08:00
Xuan Wang bdfacdec7f [Python AIO] Raise resource_exhausted error in case of concurrent RPC limit exceeded (#35376)
Fix: https://github.com/grpc/grpc/issues/30424

Currently the implementation of `maximum_concurrent_rpcs` flag in AIO is different with docstring, in implementation, we're waiting for RPCs to finish and continue execution instead of raising `resource_exhausted` as mentioned in docstring.

This PR changes the implementation of `maximum_concurrent_rpcs` flag in AIO stack to match the docstring.

<!--

If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the appropriate
lang label.

-->

Closes #35376

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35376 from XuanWang-Amos:fix_aio_concurrent_rpc 5b2b59fb9f
PiperOrigin-RevId: 601490418
2024-01-25 10:32:25 -08:00
Eugene Ostroukhov 2fc9dd1546 [build] Update `envoy-api` and `xds` dependencies (#35603)
Closes: #34966

Closes #35603

PiperOrigin-RevId: 600919492
2024-01-23 15:15:47 -08:00
Xuan Wang ad1dc17030
[Python setuptools] Import error from distutils for lower version of setuptools (#35561)
Fix: https://github.com/grpc/grpc/issues/35555
<!--

If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the
appropriate
lang label.

-->
2024-01-18 10:35:38 -08:00
Stanley Cheung 7c347b96eb [Release] Bump version to 1.62.0-dev (on master branch) (#35580)
Change was created by the release automation script. See go/grpc-release.

Closes #35580

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35580 from stanley-cheung:bump_dev_version_202401171835 b363888ca5
PiperOrigin-RevId: 599267642
2024-01-17 13:19:59 -08:00
Craig Tiller 584c0c0c98 [call-v3] Filter executor (#35533)
A call execution environment for the V3 runtime.

The `CallFilters` class will ultimately be a (private) member of `CallSpine`, and the `StackBuilder` component will be used by a channel when all of the filters it needs are known to allow the call spine to start processing a call.

This is accompanied by a reasonably extensive test suite.

I expect to fine tune semantics, implementation, and tests over the coming weeks/months as we iterate to bring up the rest of the pieces.

Closes #35533

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35533 from ctiller:filters 689c7b527b
PiperOrigin-RevId: 599220150
2024-01-17 10:39:37 -08:00
Xuan Wang e36025d3d2 Revert "[Python Aio] Replace get_event_loop() with new_event_loop() (… (#35570)
…#35493)"

This reverts commit d6579e32a0.

<!--

If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the appropriate
lang label.

-->

Closes #35570

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35570 from XuanWang-Amos:revert_aio_event_loop_change af4c6519e2
PiperOrigin-RevId: 598984869
2024-01-16 15:59:26 -08:00
Xuan Wang d6579e32a0 [Python Aio] Replace get_event_loop() with new_event_loop() (#35493)
Fix: https://github.com/grpc/grpc/issues/35086

Starting Python 3.12, `asyncio.get_event_loop()` will [emit deprecation warning](https://docs.python.org/3/library/asyncio-eventloop.html#asyncio.get_event_loop) if there is no current event loop.

Since we're calling `get_event_loop()` in case there is no running loop, it make sense to use `new_event_loop()` instead.

<!--

If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the appropriate
lang label.

-->

Closes #35493

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35493 from XuanWang-Amos:suppress_event_loop_warnning 23b4a9b7d2
PiperOrigin-RevId: 598886480
2024-01-16 10:41:26 -08:00
Craig Tiller 9f6789e2b2 [chttp2] Rework settings management to be a c++ type (#35449)
Closes #35449

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35449 from ctiller:h2-settings f67e542a8b
PiperOrigin-RevId: 597970333
2024-01-12 15:21:05 -08:00
Xuan Wang a18279db2e [Python O11Y] Revert registered method change (#35522)
We're having some issues internally, rolling this change back for now and wait for the affected users to figure out a solution.

This reverts: https://github.com/grpc/grpc/pull/35002, https://github.com/grpc/grpc/pull/35482 and 6872a7a473

<!--

If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the appropriate
lang label.

-->

Closes #35522

PiperOrigin-RevId: 597671989
2024-01-11 15:40:10 -08:00
Craig Tiller 3020a68b86 [call-v3] Move `grpc_call_final_info` into its own target (#35504)
We probably want to reconsider this types role in the system, but for now removing it would be a large job. Move it to somewhere isolated for now.

Closes #35504

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35504 from ctiller:move-it-stats 9ee755db79
PiperOrigin-RevId: 597382258
2024-01-10 16:13:48 -08:00
Mark D. Roth 20e5b2c4bb [xDS] read connection idle timeout from CDS resource (#35395)
Part of the work needed for in-progress gRFC A75 (https://github.com/grpc/proposal/pull/405).

Closes #35395

PiperOrigin-RevId: 597064473
2024-01-09 15:41:22 -08:00
AJ Heller 3f8af9a74c [EventEngine] Rename Posix native resolver (#35491)
Closes #35491

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35491 from drfloob:cleanup-posix-ee-native-resolver 1c97ff5216
PiperOrigin-RevId: 597035026
2024-01-09 13:55:20 -08:00
Craig Tiller 6872a7a473 Change some text to Implementation Private
What was there trips some tooling and will cause some repo maintenance issues going forward

PiperOrigin-RevId: 596992313
2024-01-09 11:23:15 -08:00
AJ Heller 745a61222d [EventEngine] Implement Windows Native Resolver (#35458)
Closes #35458

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35458 from drfloob:windows-native-ee-resolver 364b710342
PiperOrigin-RevId: 596981099
2024-01-09 10:50:27 -08:00
Xuan Wang d8ae1d28c5 [Python O11Y] Add _registered_method to __init__.py (#35482)
<!--

If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the appropriate
lang label.

-->

Closes #35482

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35482 from XuanWang-Amos:fix_registered_method 49321c83b8
PiperOrigin-RevId: 596967469
2024-01-09 18:44:27 +00:00
Xuan Wang 9a7c0b5f57 [Python Observability] Allow create call with registered method (#35002)
Based on [OpenTelemetry Metrics gRFC](https://github.com/grpc/proposal/blob/master/A66-otel-stats.md#opentelemetry-metrics), we should recored unregistered RPC method name as `other`, this PR adds the ability to pass register method information when creating a call.

We'll consider calls created using generated stubs as registered, note that this won't prevent user from setting `registered_method=True` when creating calls manually.

This is also enabled for simple stub flow but **NOT enabled for AsyncIO**, we'll add that later when start working on AsyncIO Observability.
<!--

If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the appropriate
lang label.

-->

Closes #35002

PiperOrigin-RevId: 596719121
2024-01-08 15:30:17 -08:00
Xuan Wang 48cf940fd1 [Python Otel] Re-apply Otel (#35439)
This reverts commit 96b9e8d3e3.

[Implement OpenTelemetry PR](https://github.com/grpc/grpc/pull/35292) was [reverted](96b9e8d3e3) because some tests started failing after import the changes to g3.

After investigation, we found root cause, it can be fixed both on our side and on gapic API side, we opened an issue to [gapic API team](https://github.com/googleapis/python-api-core/issues/579), this PR will includes the fixes on our side.

<!--

If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the appropriate
lang label.

-->

Closes #35439

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35439 from XuanWang-Amos:reapply_otel 0133564438
PiperOrigin-RevId: 595746222
2024-01-04 10:45:14 -08:00
gRPC Team Bot 96b9e8d3e3 Automated rollback of commit d2da19e99e.
PiperOrigin-RevId: 595245935
2024-01-02 17:25:04 -08:00
Craig Tiller a100e90f97 [chttp2] Remove a bunch of mostly-rolled-out experiments (#35426)
- `memory_pressure_controller` finally - allows deletion of pid_controller throughout the codebase
- `overload_protection` - one of the http2 rapid reset mitigations
- `red_max_concurrent_streams` - another http2 rapid reset mitigation

Closes #35426

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35426 from ctiller:new-years-cleanse 4651672e7e
PiperOrigin-RevId: 595205029
2024-01-02 14:07:55 -08:00
Xuan Wang d2da19e99e [python O11Y] Implement OpenTelemetry (#35292)
<!--

If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the appropriate
lang label.

-->

Closes #35292

PiperOrigin-RevId: 595188404
2024-01-02 12:58:01 -08:00
Mark D. Roth c7101d0867 [xDS] move CDS and EDS watchers into xds resolver (#35011)
Implements gRFC A74 (https://github.com/grpc/proposal/pull/404).

Closes #35011

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35011 from markdroth:xds_watchers_in_xds_resolver a39f71f37f
PiperOrigin-RevId: 595134549
2024-01-02 09:20:06 -08:00
Craig Tiller 585bdccf45 Automated rollback of commit a5a25c7dad.
PiperOrigin-RevId: 591269197
2023-12-15 09:02:15 -08:00