Commit Graph

1147 Commits

Author SHA1 Message Date
Mark D. Roth 733e904491 [XdsClient] implement non-per-call metrics (#36020)
As per gRFC A78 (https://github.com/grpc/proposal/pull/419).

Closes #36020

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36020 from markdroth:non_per_call_metrics_xds c1d9b464bf
PiperOrigin-RevId: 617320495
2024-03-19 16:00:32 -07:00
Tanvi Jagtap 2c49416713 [grpc][Gpr_To_Absl_Logging] Using absl from within gpr logging (#36108)
![Config_Vars_Absl (1)](https://github.com/grpc/grpc/assets/139093547/04d9b75c-dd1c-42ee-81a3-d20e2bbd5672)

<!--

If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the appropriate
lang label.

-->

Closes #36108

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36108 from tanvi-jagtap:tjagtap_config_vars c48a3524f0
PiperOrigin-RevId: 615652000
2024-03-13 22:04:47 -07:00
Mark D. Roth 390fef0590 [RR and WRR] clean up dualstack experiments (#35135)
Closes #35135

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35135 from markdroth:dualstack_rr_wrr_cleanup 438b29df5c
PiperOrigin-RevId: 615208297
2024-03-12 16:26:41 -07:00
Vignesh Babu 7682a318d8 Creating a chaotic good endpoint extension to allow configuring chaotic good endpoints.
The extension first contains methods to enable logging of stats.

PiperOrigin-RevId: 614816675
2024-03-11 15:37:49 -07:00
Vignesh Babu 7533328075 [server] Allow configuring max incoming connections at the server (#36088)
Closes #36088

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36088 from Vignesh2208:server-fix e62a8f7ae7
PiperOrigin-RevId: 614729878
2024-03-11 11:17:34 -07:00
Mark D. Roth c43f1a63b0 [RLS] implement non-per-call metrics (#36001)
Closes #36001

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36001 from markdroth:non_per_call_metrics_rls b713980f21
PiperOrigin-RevId: 612532005
2024-03-04 11:58:31 -08:00
Mark D. Roth 0213523907 [build] move channel out of grpc_base (#35924)
This adds the following new targets:
- `channel`: A virtual interface for a channel.
- `legacy_channel`: A channel implementation that supports the filter stack and call v2.
- `channel_create`: A standalone function to create a channel.
- `server_interface`: A base class with a few accessor methods used in surface/call.cc.
- `server`: The actual server implementation.
- `api_trace`, `call_tracer`, `server_call_tracer_filter`, `call_finalization`: These were split out of `grpc_base` to avoid various dependency problems.
- `compression`: This is a combination of the previously existing `compression_internal` target and the compression code that was part of `grpc_base`.

Closes #35924

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35924 from markdroth:channel_interface 94a7fffddb
PiperOrigin-RevId: 612512438
2024-03-04 11:07:17 -08:00
Tanvi Jagtap e39bd50716 [grpc] Redacting unknown metadata types in debug logs. (#36006)
Redacting unknown metadata types in debug logs.

<!--

If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the appropriate
lang label.

-->

Closes #36006

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36006 from tanvi-jagtap:tjagtap_redact_01 94c5738bfe
PiperOrigin-RevId: 611334483
2024-02-28 21:18:41 -08:00
Alisha Nanda fa2bae473a [metadata] Expose metadata limits and annotation info to filter stack (#35970)
<!--

If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the appropriate
lang label.

-->

Closes #35970

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35970 from ananda1066:client_metadata b62a9ae146
PiperOrigin-RevId: 610830228
2024-02-27 12:07:35 -08:00
Mark D. Roth 40577dd585 [WRR] implement non-per-call metrics (#35977)
As per gRFC A78 (https://github.com/grpc/proposal/pull/419).

Note that these new metrics are populated only if the `wrr_delegate_to_pick_first` experiment is enabled, which is the case by default.

Closes #35977

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35977 from markdroth:non_per_call_metrics_wrr 7acea32fc4
PiperOrigin-RevId: 610760874
2024-02-27 08:32:09 -08:00
Mark D. Roth cfcbb0a535 [build] move parts of the `grpc_client_channel` BUILD target out into their own targets (#35879)
This breaks the following pieces out of the `grpc_client_channel` BUILD target:
- backend_metric_parser
- oob_backend_metric
- child_policy_handler
- backup_poller
- service_config_channel_arg_filter
- client_channel_channelz
- client_channel_internal_header
- subchannel_connector
- subchannel_pool_interface
- config_selector
- client_channel_service_config_parser
- retry_service_config_parser
- retry_throttle

The code left in the `grpc_client_channel` target will need more work to pull apart.

Closes #35879

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35879 from markdroth:client_channel_build_split f388a37edc
PiperOrigin-RevId: 608806548
2024-02-20 18:06:37 -08:00
Esun Kim 527557d03d [Deps] Upgrade abseil podsepc to 1.20240116.1 (#35893)
Closes #35893

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35893 from veblush:absl-osx e01dd83ee2
PiperOrigin-RevId: 606780054
2024-02-13 16:19:15 -08:00
Eugene Ostroukhov 6c157f9128 [Release] Bump version to 1.63.0-dev (on master branch) (#35899)
Change was created by the release automation script. See go/grpc-release.

Closes #35899

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35899 from eugeneo:bump_dev_version_202402131133 73950425c9
PiperOrigin-RevId: 606777850
2024-02-13 16:09:36 -08:00
Mark D. Roth 41606054c2 [load_file] remove grpc_load_file() in favor of grpc_core::LoadFile() (#35857)
Closes #35857

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35857 from markdroth:highlander 7b7d95aaa9
PiperOrigin-RevId: 605742734
2024-02-09 15:16:24 -08:00
Mark D. Roth 21cb320080 [reorg] move service config code to src/core/service_config (#35843)
Closes #35843

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35843 from markdroth:client_channel_reorg4 0c50ada6f9
PiperOrigin-RevId: 605466874
2024-02-08 16:58:20 -08:00
Mark D. Roth f22c954ef5 [reorg] move client channel code to src/core/client_channel (#35827)
Also rename client_channel.{h,cc} -> client_channel_filter.{h,cc}.

Closes #35827

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35827 from markdroth:client_channel_reorg3 449bff563f
PiperOrigin-RevId: 605006293
2024-02-07 09:23:26 -08:00
Mark D. Roth 10e83973e7 [reorg] move resolver code to src/core/resolver (#35804)
This new directory combines code from the following locations:
- src/core/ext/filters/client_channel/resolver
- src/core/lib/resolver

Closes #35804

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35804 from markdroth:client_channel_resolver_reorg2 30660e6b00
PiperOrigin-RevId: 604665835
2024-02-06 09:07:56 -08:00
Mark D. Roth 148f59c15a [reorg] move LB policy code to src/core/load_balancing (#35786)
This new directory combines code from the following locations:
- src/core/ext/filters/client_channel/lb_policy
- src/core/lib/load_balancing

Closes #35786

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35786 from markdroth:client_channel_resolver_reorg 98554efb98
PiperOrigin-RevId: 604351832
2024-02-05 10:05:35 -08:00
Craig Tiller 610f439342 [channel] Add CallFactory, CallDestination (#35766)
Closes #35766

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35766 from ctiller:chan3 ea9a3260e3
PiperOrigin-RevId: 603437058
2024-02-01 11:58:14 -08:00
Craig Tiller 59004b4931 [channel] Separate call size estimation out of `grpc_core::Channel` (#35732)
Closes #35732

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35732 from ctiller:chan 0d7a29fc77
PiperOrigin-RevId: 602865976
2024-01-30 16:01:41 -08:00
Craig Tiller fb14685fd6 [transport] Move call spine related types into their own file (#35689)
Part of a continued hygiene effort for transport.h

(pure code movement, no refactoring at this time)

Closes #35689

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35689 from ctiller:channel-ez 3366310cc3
PiperOrigin-RevId: 602416330
2024-01-29 09:49:23 -08:00
AJ Heller 71fa68f7fb [EventEngine] Add QueryExtension Interfaces for *SupportsFd (#35648)
This PR:
* adds FD extensions to the public headers
* Adds the query extension interface to EventEngine, Listener, and Endpoint, via a new `Extensible` interface
* Refactors the PosixEventEngine to use the Extensible interface.

Closes #35648

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35648 from drfloob:posix-ee-query-interface 7cae28e0b3
PiperOrigin-RevId: 601794970
2024-01-26 10:03:18 -08:00
Esun Kim 714640d71f [Deps] Upgrade Abseil to 2024-01-16 (#35589)
Closes #35589

PiperOrigin-RevId: 601647295
2024-01-25 20:14:19 -08:00
Eugene Ostroukhov 2fc9dd1546 [build] Update `envoy-api` and `xds` dependencies (#35603)
Closes: #34966

Closes #35603

PiperOrigin-RevId: 600919492
2024-01-23 15:15:47 -08:00
Nick Cooke f13a564e50 [CocoaPods] Re-work how privacy manifests are exposed (#35542)
<!--

If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the appropriate
lang label.

-->

Follow-up to add privacy manifests to #35042

- [x] Update podspec template files
- [x] Align on manifest update:
```
git grep mach_absolute_time
src/core/lib/gpr/posix/time.cc:static uint64_t g_time_start = mach_absolute_time();
src/core/lib/gpr/posix/time.cc:                ((double)(mach_absolute_time() - g_time_start)) * g_time_scale;
```

cc: @paulb777

Closes #35542

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35542 from ncooke3:nc/privacy-manifests b0e0e57d2e
PiperOrigin-RevId: 600595032
2024-01-22 15:58:06 -08:00
Craig Tiller 67d6b8ea74 [promise] Context improvements (#35592)
A few improvements to the promise context system (more coming)

Allow subclassed contexts:
If we have multiple different kinds of a base context, allow `GetContext<Derived>()` to mean `down_cast<Derived*>(GetContext<Base>())` everywhere for brevity.

Allow custom context lookup:
For a base context type, allow customization of how that context is looked up.

These two together allow:
1. normalization of activity lookup and context lookup to the same syntax (so we can write `GetContext<Activity>()` everywhere now
2. Party & Activity to share a context, so that anywhere we need to do a party specific operation we can write `GetContext<Party>()->...` and safely know that it's the current activity *and* it's a party.

Closes #35592

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35592 from ctiller:contextual-types 37ef948a36
PiperOrigin-RevId: 599651708
2024-01-18 16:03:46 -08:00
Stanley Cheung 7c347b96eb [Release] Bump version to 1.62.0-dev (on master branch) (#35580)
Change was created by the release automation script. See go/grpc-release.

Closes #35580

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35580 from stanley-cheung:bump_dev_version_202401171835 b363888ca5
PiperOrigin-RevId: 599267642
2024-01-17 13:19:59 -08:00
Craig Tiller 584c0c0c98 [call-v3] Filter executor (#35533)
A call execution environment for the V3 runtime.

The `CallFilters` class will ultimately be a (private) member of `CallSpine`, and the `StackBuilder` component will be used by a channel when all of the filters it needs are known to allow the call spine to start processing a call.

This is accompanied by a reasonably extensive test suite.

I expect to fine tune semantics, implementation, and tests over the coming weeks/months as we iterate to bring up the rest of the pieces.

Closes #35533

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35533 from ctiller:filters 689c7b527b
PiperOrigin-RevId: 599220150
2024-01-17 10:39:37 -08:00
Craig Tiller 9f6789e2b2 [chttp2] Rework settings management to be a c++ type (#35449)
Closes #35449

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35449 from ctiller:h2-settings f67e542a8b
PiperOrigin-RevId: 597970333
2024-01-12 15:21:05 -08:00
ericsalo a6823b6e97 Fix deprecation warnings from upb (#35510)
Replace "upb:collections" with "upb:message"
Replace "upb/collections" with "upb/message"
Replace "upb:upb" with "upb:mem" and/or "upb:base"
Replace "upb/upb.hpp" with "upb/mem/arena.hpp" and/or "upb/base/status.hpp"

Closes #35510

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35510 from ericsalo:master 112392a050
PiperOrigin-RevId: 597872730
2024-01-12 10:15:56 -08:00
Craig Tiller 3020a68b86 [call-v3] Move `grpc_call_final_info` into its own target (#35504)
We probably want to reconsider this types role in the system, but for now removing it would be a large job. Move it to somewhere isolated for now.

Closes #35504

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35504 from ctiller:move-it-stats 9ee755db79
PiperOrigin-RevId: 597382258
2024-01-10 16:13:48 -08:00
Mark D. Roth 20e5b2c4bb [xDS] read connection idle timeout from CDS resource (#35395)
Part of the work needed for in-progress gRFC A75 (https://github.com/grpc/proposal/pull/405).

Closes #35395

PiperOrigin-RevId: 597064473
2024-01-09 15:41:22 -08:00
AJ Heller 3f8af9a74c [EventEngine] Rename Posix native resolver (#35491)
Closes #35491

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35491 from drfloob:cleanup-posix-ee-native-resolver 1c97ff5216
PiperOrigin-RevId: 597035026
2024-01-09 13:55:20 -08:00
AJ Heller 745a61222d [EventEngine] Implement Windows Native Resolver (#35458)
Closes #35458

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35458 from drfloob:windows-native-ee-resolver 364b710342
PiperOrigin-RevId: 596981099
2024-01-09 10:50:27 -08:00
Craig Tiller a100e90f97 [chttp2] Remove a bunch of mostly-rolled-out experiments (#35426)
- `memory_pressure_controller` finally - allows deletion of pid_controller throughout the codebase
- `overload_protection` - one of the http2 rapid reset mitigations
- `red_max_concurrent_streams` - another http2 rapid reset mitigation

Closes #35426

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35426 from ctiller:new-years-cleanse 4651672e7e
PiperOrigin-RevId: 595205029
2024-01-02 14:07:55 -08:00
Mark D. Roth c7101d0867 [xDS] move CDS and EDS watchers into xds resolver (#35011)
Implements gRFC A74 (https://github.com/grpc/proposal/pull/404).

Closes #35011

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35011 from markdroth:xds_watchers_in_xds_resolver a39f71f37f
PiperOrigin-RevId: 595134549
2024-01-02 09:20:06 -08:00
Craig Tiller f703530b6d [call-v3] initial surface call integration (#35312)
Adds temporary `call.cc` and `connected_channel.cc` scaffolding to run `CallInterceptor`/`CallHandler` style calls.
This will get ripped out as soon as the v3 transition is completed.

Closes #35312

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35312 from ctiller:v3-accept ae0bf81f8b
PiperOrigin-RevId: 594128029
2023-12-27 17:46:00 -08:00
Craig Tiller ff63ad9413 [call-v3] Convert message compression filter (#35269)
Closes #35269

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35269 from ctiller:cg-compress 0304821f07
PiperOrigin-RevId: 590010564
2023-12-11 17:58:00 -08:00
Craig Tiller 57dacad8c7 [call-v3] Idle/max-age filters (#35270)
Closes #35270

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35270 from ctiller:cg-idle 5312003ca6
PiperOrigin-RevId: 589913523
2023-12-11 12:45:34 -08:00
Yijie Ma 1abfc97502 [EventEngine] Posix NativeDNSResolver implementation (#35153)
<!--

If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the appropriate
lang label.

-->

Closes #35153

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35153 from yijiem:native_dns_resolver a4107f7d81
PiperOrigin-RevId: 588543137
2023-12-06 14:25:38 -08:00
Craig Tiller 295b665a18 [call-v3] Add CallInitiator, CallHandler wrappers around CallSpine (#35223)
Closes #35223

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35223 from ctiller:cg-initiator 78826d99c5
PiperOrigin-RevId: 588481455
2023-12-06 10:59:47 -08:00
Mark D. Roth 15e2f9e873
[release] Bump version on master to 1.61.0-dev (#34923) 2023-11-13 08:33:46 -08:00
Adam Cozzette 016ed921ce
[deps] Upgrade protobuf version to v25.0 (#34513)
This commit upgrades gRPC to protobuf v25.0 and makes some fixes to
account for upb changes. One major change is that upb has been merged
into the protobuf repo, so we can now drop the separate `@upb`
dependency. Another is that `.upb.c` files no longer exist and there are
new `.upb_minitable.h` and `.upb_minitable.c` files. The longer
filenames exceeded a Windows restriction, so to work around that I
renamed the `upb-generated` directory to just `upb-gen`, and likewise
for `upbdefs-generated`.
2023-11-10 00:11:09 -08:00
Yijie Ma 720d7a0653
[EventEngine] WindowsDNSResolver Implementation (#34400)
Design is documented at
[go/windows-dns-resolver-issue](http://go/windows-dns-resolver-issue)
(note that the design doc is slightly outdated regarding the shared
ownership model of the virtual socket that was implemented in
13bd2b404e).

Passed `//test/cpp/naming:resolver_component_tests_runner_invoker` and
`//test/cpp/naming:cancel_ares_query_test`:
```
C:\Users\yijiem\projects\grpc>bazel --output_base=C:\bazel6 test --dynamic_mode=off --verbose_failures --test_env=GRPC_EXPERIMENTS=event_engine_dns --test_env=GRPC_VERBOSITY=debug --test_env=GRPC_TRACE=cares_resolver --enable_runfiles=yes --nocache_test_results //test/cpp/naming:resolver_component_tests_runner_invoker
INFO: Analyzed target //test/cpp/naming:resolver_component_tests_runner_invoker (1 packages loaded, 8 targets configured).
INFO: Found 1 test target...
INFO: From Compiling src/core/lib/event_engine/windows/windows_engine.cc:
C:\bazel6\execroot\com_github_grpc_grpc\src/core/lib/channel/channel_args.h(287): warning C4312: 'reinterpret_cast': conversion from 'int' to 'void *' of greater size
Target //test/cpp/naming:resolver_component_tests_runner_invoker up-to-date:
  bazel-bin/test/cpp/naming/resolver_component_tests_runner_invoker.exe
INFO: Elapsed time: 230.374s, Critical Path: 228.54s
INFO: 9 processes: 2 internal, 7 local.
INFO: Build completed successfully, 9 total actions
//test/cpp/naming:resolver_component_tests_runner_invoker                PASSED in 221.2s

Executed 1 out of 1 test: 1 test passes.
```

```
C:\Users\yijiem\projects\grpc>bazel --output_base=C:\bazel6 test --dynamic_mode=off --verbose_failures --test_env=GRPC_EXPERIMENTS=event_engine_dns --test_env=GRPC_VERBOSITY=debug --test_env=GRPC_TRACE=cares_resolver --enable_runfiles=yes --nocache_test_results //test/cpp/naming:cancel_ares_query_test
INFO: Analyzed target //test/cpp/naming:cancel_ares_query_test (0 packages loaded, 0 targets configured).
INFO: Found 1 test target...
Target //test/cpp/naming:cancel_ares_query_test up-to-date:
  bazel-bin/test/cpp/naming/cancel_ares_query_test.exe
INFO: Elapsed time: 49.656s, Critical Path: 48.00s
INFO: 6 processes: 2 internal, 4 local.
INFO: Build completed successfully, 6 total actions
//test/cpp/naming:cancel_ares_query_test                                 PASSED in 43.0s

Executed 1 out of 1 test: 1 test passes.
```

<!--

If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the
appropriate
lang label.

-->
2023-11-09 14:22:31 -08:00
Gregory Cooke 0d4e1ef5df
[Security - Revocation] Crl Directory Watcher Implementation (#34749)
This adds the directory reloader implementation of the CrlProvider. This
will periodically reload CRL files in a directory per [gRFC
A69](https://github.com/grpc/proposal/pull/382)

Included in this is the following:
* A public API to create the `DirectoryReloaderCrlProvider`
* A basic directory interface in gprpp and platform specific impls for
getting the list of files in a directory (unfortunately prior C++17,
there is no std::filesystem, so we have to have platform specific impls)
* The implementation of `DirectoryReloaderCrlProvider` takes an
event_engine and a directory interface. This allows us to test using the
fuzzing event engine for time mocking, and to implement a test directory
interface so we avoid having to make temporary directories and files in
the tests. This is notably not in `include`, and the
`CreateDirectoryReloaderCrlProvider` is the only way to construct one
from the public API, so we don't expose the event engine and directory
details to the user.

---------

Co-authored-by: gtcooke94 <gtcooke94@users.noreply.github.com>
2023-11-03 20:48:05 +00:00
Yash Tibrewal 2bc23f1138
[xds] C++ XdsServerBuilder Remove src/... header dependency from public header (#34790) 2023-10-25 11:31:07 -07:00
Yousuk Seung e3ff5f4166
[tracing] Add TcpTracer and HttpAnnotation (#34592)
Add TcpTracer interface for TCP instrumentation. It takes no gRPC
dependencies for use in external TCP implementations. Also add
HttpAnnotation for HTTP transport instrumentation using CallTracer.




<!--

If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the
appropriate
lang label.

-->
2023-10-20 16:04:16 -07:00
Gregory Cooke 9969d820b5
[TLS - Revocation] Crl Provider (#34715)
This reverts commit 7af5efcfd3.
2023-10-19 11:33:41 -07:00
Craig Tiller a0c1027bb3
[transport] Move transport interface to C++ (#34618)
Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2023-10-18 13:13:23 -07:00
apolcyn 7af5efcfd3
Revert "[TLS - Revocation] Crl Provider (#33786)" (#34713)
This reverts commit 0f0396ae92.
2023-10-17 13:53:36 -07:00
Craig Tiller 975184f04b
[channel-stack] Make ordering explicit (#32852)
Ditch the old priority scheme for ordering filters, instead explicitly
mark up before/after constraints.

---------

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2023-10-17 13:52:58 -07:00
Gregory Cooke 0f0396ae92
[TLS - Revocation] Crl Provider (#33786)
The basic APIs for the CRL Reloading features.
This adds external types to represent CRL Providers, CRLs, and
CertificateInfo.
Internally we will use `CrlImpl` - this layer is needed to hide OpenSSL
details from the user.

GRFC - https://github.com/grpc/proposal/pull/382

Things Done
* Add external API for `CrlProvider`, `Crl`, `CertInfo` (`CertInfo` is
used during CRL lookup rather than passing the entire certificate).
* Add code paths in `ssl_transport_security` to utilize CRL providers
* Add `StaticCrlProvider`
* Refactor `crl_ssl_transport_security_test.cc` so it is more extensible
and can be used with providers
2023-10-17 13:03:52 -04:00
Craig Tiller 7c59c09f43
[chttp2] Bound write sizes based on observed write performance (#34665)
Instead of fixing a target size for writes, try to adapt it a little to
observed bandwidth.

The initial algorithm tries to get large writes within 100-1000ms
maximum delay - this range probably wants to be tuned, but let's see.

The hope here is that on slow connections we can not back buffer so much
and so when we need to send a ping-ack it's possible without great
delay.
2023-10-12 16:48:55 -07:00
Craig Tiller 8430af1614
[promises] Sketch out a spot for the in-process transport promise conversion (#34616)
Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2023-10-12 11:01:29 -07:00
Mark D. Roth f15635287b
[xxhash] add a wrapper header to avoid clang-format breakage (#34658) 2023-10-12 10:31:40 -07:00
Craig Tiller c9df0ca470
[cpp] Reland removal of channel_filter code (#34598)
Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2023-10-12 09:44:11 -07:00
Craig Tiller 6a49e953a4
[chttp2] Experiments for rst_stream pushback (#34642)
Experiment 1: On RST_STREAM: reduce MAX_CONCURRENT_STREAMS for one round
trip.
Experiment 2: If a settings frame is outstanding with a lower
MAX_CONCURRENT_STREAMS than is configured, and we receive a new incoming
stream that would exceed the new cap, randomly reject it.

---------

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2023-10-10 19:21:41 -07:00
Craig Tiller 7b17e4aa92
Revert "[cpp] Remove legacy channel filter C++ wrapper" (#34541)
Reverts grpc/grpc#34522
2023-09-28 22:29:53 -07:00
Craig Tiller 727c92a15c
[cpp] Remove legacy channel filter C++ wrapper (#34522)
Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2023-09-28 20:59:56 -07:00
Mark D. Roth 7a06614f95
[resolver and LB policy APIs] reland: change address list to support multiple addresses per endpoint (#34531)
Re-land #33567, which was reverted in #34527.

First commit is a pure revert, second commit is a small fix needed to
avoid breaking internal callers.
2023-09-28 15:40:05 -07:00
Craig Tiller a17f08b49d
[chttp2] Continue refactoring towards promises (#34437)
Isolate ping callback tracking to its own file.
Also takes the opportunity to simplify keepalive code by applying the
ping timeout to all pings.
Adds an experiment to allow multiple pings outstanding too (this was
originally an accidental behavior change of the work, but one that I
think may be useful going forward).

---------

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2023-09-28 13:14:05 -07:00
Mark D. Roth 41f26de3b6
Revert "[resolver and LB policy APIs] change address list to support multiple addresses per endpoint" (#34527)
Reverts grpc/grpc#33567 due to import problems.
2023-09-28 12:15:35 -07:00
Mark D. Roth fd2e8c9462
[resolver and LB policy APIs] change address list to support multiple addresses per endpoint (#33567)
More changes as part of the dualstack design:
- Change resolver and LB policy APIs to support multiple addresses per
endpoint. Specifically, replace `ServerAddress` with
`EndpointAddresses`, which encodes more than one address. Per-address
channel args are retained at the same level, so they are now
per-endpoint. For now, `EndpointAddress` provides a single-address ctor
and a single-address accessor for backward compatibility, so
`ServerAdress` is an alias for `EndpointAddresses`; eventually, this
alias and the single-address methods will be removed.
- Add an `EndpointAddressSet` class, which represents an unordered set
of addresses to be used as a map key. This will be used in a number of
LB policies that need to store per-endpoint state.
- Change the LB policy API's `ChannelControlHelper::CreateSubchannel()`
method to take the address and per-endpoint channel args as separate
parameters, so that we don't need to construct a legacy `ServerAddress`
object as we create a new subchannel for each address in the endpoint.
- Change pick_first to flatten the address list.
- Change ring_hash to use `EndpointAddressSet` as the key for its
endpoint map, and to use the first address of the endpoint as the hash
key.
- Change WRR to use `EndpointAddressSet` as the key for its endpoint
weight map.

Note that support for multiple addresses per endpoint is guarded in RR
by the existing `round_robin_delegate_to_pick_fist` experiment and in
WRR by the existing `wrr_delegate_to_pick_first` experiment.

This PR does *not* include support for multiple addresses per endpoint
for the outlier_detection or xds_override_host LB policies; those will
come in subsequent PRs.
2023-09-28 09:32:36 -07:00
Richard Belleville 3bdd972c4a
[Release] Bump master version to 1.60 (gjallarhorn) (#34488)
<!--

If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the
appropriate
lang label.

-->
2023-09-26 13:10:20 -07:00
Craig Tiller 59d886cb5c
[fuzzing] Expose random number generator to some fuzzers (#34415)
Expand our fuzzing capabilities by allowing fuzzers to choose the bits
that go into random number distribution generators.

---------

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2023-09-25 10:17:58 -07:00
Yash Tibrewal b038da5072
[CSM] Second attempt: Add a server selector based on channel args (#34376)
This reverts commit 2db446aa9a.




<!--

If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the
appropriate
lang label.

-->
2023-09-15 15:28:52 -07:00
nanahpang 2db446aa9a
Revert "[CSM] Add a server selector based on channel args" (#34375)
Reverts grpc/grpc#34312
2023-09-15 11:28:20 -07:00
AJ Heller 2467562e4b
[EventEngine] Delete OriginalThreadPool, remove work_stealing experiment (#34315)
This has been stable for a bit, everywhere that the EventEngine is
enabled. Going forward, I think the event_engine_{client|listener}
experiments can probably be used to regulate thread-pool-specific
issues.

---------

Co-authored-by: drfloob <drfloob@users.noreply.github.com>
2023-09-15 10:28:11 -07:00
Yash Tibrewal c145b7910e
[CSM] Add a server selector based on channel args (#34312)
I've added channel args to `CreateNewServerCallTracer` on the
`ServerCallTracerFactory`.
The motivation is for CSM Observability where the OTel plugin will be
configured to only do stats on servers which are xDS enabled, so I plan
to check this via channel args.
In the future, with the new scopes for metrics, I think I'll be able to
change this to only check once per server or server connection instead
of per call.

<!--

If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the
appropriate
lang label.

-->
2023-09-15 10:19:46 -07:00
Mark D. Roth 1986007e1e
[round_robin] 4th attempt: delegate to pick_first as per dualstack design (#34337)
Most recent attempt was #34320, reverted in #34335.

The first commit here is a pure revert. The second commit fixes the
outlier_detection unit test to pass both with and without the
experiment.
2023-09-14 08:55:51 -07:00
Mark D. Roth 6534f0a6bf
Revert "[round_robin] third attempt: delegate to pick_first as per dualstack design" (#34335)
Reverts grpc/grpc#34320
2023-09-13 11:14:54 -07:00
Yijie Ma dcd0f6b8bd
Revert "Revert "[EventEngine] Make `AresResolver` build compatible with older…" (#34328)
Reverts grpc/grpc#34325

<!--

If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the
appropriate
lang label.

-->
2023-09-12 18:09:26 -07:00
Mark D. Roth d713427cec
[round_robin] third attempt: delegate to pick_first as per dualstack design (#34320)
Previous attempt was #34241, reverted in #34317.

The second commit here makes the experiment disablable, so that we can
roll it out slowly internally.
2023-09-12 17:46:31 -07:00
nanahpang c9db496000
Revert "[EventEngine] Make `AresResolver` build compatible with older… (#34325)
… c-ares versions (#34314)"

This reverts commit eb37b91072.




<!--

If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the
appropriate
lang label.

-->
2023-09-12 13:42:20 -07:00
Yijie Ma eb37b91072
[EventEngine] Make `AresResolver` build compatible with older c-ares versions (#34314)
Tested by changing c-ares to `1.14.0` in Bazel dependency and verify
that the `posix_event_engine_test` build. The test would fail though
since c-ares versions < `1.16.0` does not come with address sorting
capability so the relevant test cases will fail. But this is probably
sufficient to make the Cloud C++ CI job pass and unblock the 1.58
release.

<!--

If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the
appropriate
lang label.

-->
2023-09-12 11:08:32 -07:00
Craig Tiller e6bf7c12cf
Revert "[round_robin] delegate to pick_first as per dualstack design" (#34317)
Reverts grpc/grpc#34241
2023-09-11 19:42:54 -07:00
Mark D. Roth 97571ebf81
[round_robin] delegate to pick_first as per dualstack design (#34241)
Rolls forward the remaining changes from #32692, which were rolled back
in #33718.
2023-09-11 12:09:57 -07:00
Eugene Ostroukhov a5e9feeb04
[HTTP Proxy] Rename source/header and move test (#34221) 2023-09-05 20:50:03 -07:00
Esun Kim 1a1124903c
[Deps] Upgrade Protobuf and Upb to 24.x (#34123)
On top of https://github.com/grpc/grpc/pull/34120
2023-08-29 10:58:48 -07:00
Craig Tiller 12c9748134
[chttp2] New framing layer (#33692)
Building out a new framing layer for chttp2.

The central idea here is to have the framing layer be solely responsible
for serialization of frames, and their deserialization - the framing
layer can reject frames that have invalid syntax - but the enacting of
what that frame means is left to a higher layer.

This class will become foundational for the promise conversion of chttp2
- by eliminating action from the parsing of frames we can reuse this
sensitive code.

Right now the new layer is inactive - there's a test that exercises it
relatively well, and not much more. In the next PRs I'll add an
experiments to enable using this layer or the existing code in the
writing and reading paths.

---------

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2023-08-27 20:48:13 -07:00
Esun Kim a90f30008d
[Release] Bump version to 1.59.0-dev (on master branch) (#34144)
Change was created by the release automation script. See
go/grpc-release.
2023-08-23 15:53:32 -07:00
Mark D. Roth c0e2fa60f9
[xDS] avoid creating duplicate hierarchical path attribute values (#34106) 2023-08-21 14:23:02 -07:00
AJ Heller 108af0a94f
[EventEngine] Improve lock contention in WorkStealingThreadPool (alternative) (#34065)
Proposed alternative to https://github.com/grpc/grpc/pull/34024.

This version has a simpler, faster busy-count implementation based on a
sharded set of atomic counts: fast increment/decrement operations,
relatively slower summation of total counts (which need to happen much
less frequently).
2023-08-17 17:00:02 -07:00
Jan Tattermusch a8508ec465
[build metadata] Bazel to "other build systems" improvements (#33803)
- Extract build metadata for some external dependencies from bazel
build. This is achieved by letting extract_metadata_from_bazel_xml.py
analyze some external libraries and sources. The logic is basically the
same as for internal libraries, I only needed to teach
extract_metadata_from_bazel_xml.py which external libraries it is
allowed to analyze.
* currently, the list of source files is automatically determined for
`z`, `upb`, `re2` and `gtest` dependencies (at least for the case where
we're building in "embedded" mode - e.g. mostly native extensions for
python, php, ruby etc. - cmake has the ability to replace some of these
dependencies by actual cmake dependency.)
- Eliminate the need for manually written gen_build_yaml.py for some
dependencies.
- Make the info on target dependencies in build_autogenerated.yaml more
accurate and complete. Until now, there were some depdendencies that
were allowed to show up in build_autogenerated.yaml and some that were
being skipped. This made generating the CMakeLists.txt and Makefile
quite confusing (since some dependencies are being explicitly mentioned
and some had to be assumed by the build system).
- Overhaul the Makefile
* the Makefile is currently only used internally (e.g. for ruby and PHP
builds)
* until now, the makefile wasn't really using the info about which
targets depend on what libraries, but it was effectively hardcoding the
depedendency data (by magically "knowing" what is the list of all the
stuff that e.g. "grpc" depends on).
* After the overhaul, the Makefile.template now actually looks at the
library dependencies and uses them when generating the makefile. This
gives a more correct and easier to maintain makefile.
* since csharp is no longer on the master branch, remove all mentions of
"csharp" targets in the Makefile.

Other notable changes:
- make extract_metadata_from_bazel_xml.py capable of resolving workspace
bind() rules (so that it knows the real name of the target that is
referred to as e.g. `//external:xyz`)

TODO:
- [DONE] ~~pkgconfig C++ distribtest~~
- [DONE} ~~update third_party/README to reflect changes in how some deps
get updated now.~~


Planned followups:
- cleanup naming of some targets in build metadata and buildsystem
templates: libssl vs boringssl, ares vs cares etc.
- further cleanup of Makefile
- further cleanup of CMakeLists.txt
- remote the need from manually hardcoding extra metadata for targets in
build_autogenerated.yaml. Either add logic that determines the
properties of targets automatically, or use metadata from bazel BUILD.
2023-08-16 19:13:39 +02:00
Yijie Ma e04128ad1d
[Deps] Update abseil to 20230802 (#33974)
<!--

If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the
appropriate
lang label.

-->
2023-08-11 15:29:24 -07:00
Craig Tiller 54651a7168
[promises] New `Seq`, `TrySeq` implementation (#33991)
Our current implementation of `Seq`, `TrySeq` leverage some complicated
template stuff to work, which makes them hard to maintain. I've been
thinking about ways to simplify that for some time and had something
like this in mind - using a code generator that's at least a little more
understandable to code generate most of the complexity into a file that
is checkable.

Concurrently - I have a cool optimization in mind - but it requires that
we can move promises after polling, which is a contract change. I'm
going to work through the set of primitives we have in the coming weeks
and change that contract to enable the optimization.

---------

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2023-08-10 09:40:13 -07:00
Hannah Shi 239a5fce2d
[ObjC] dns service resolver for cf event engine (#33971)
re-submit of #33233 with refactored tests

1. split ios event engine tests to client tests (require oracle engine)
and unit test
2. disable dns server setup in
[dns_test.cc](https://github.com/grpc/grpc/blob/master/test/core/event_engine/test_suite/tests/dns_test.cc#L127)
for ios test, this is what's caused the revert.
3. disable dns_test in cf_event_engine_test for MacOS
2023-08-04 15:51:42 -07:00
AJ Heller 48f455a9a0
Revert "[ObjC] dns service resolver for cf event engine" (#33964)
Reverts grpc/grpc#33233. This is failing 100% of builds
https://source.cloud.google.com/results/invocations/248656d3-15ac-4d42-a2f8-6bec5c411e67/targets
2023-08-02 10:08:06 -07:00
Hannah Shi cd85d7edf2
[ObjC] dns service resolver for cf event engine (#33233)
Implement DNS using dns service for iOS.

Current limitation:
1. Using a custom name server is not supported.
2. Only supports `LookupHostname`. `LookupSRV` and `LookupTXT` are not
implemented.
3. Not tested with single stack (ipv4 or ipv6) environment
4. ~Not tested with multiple ip records per stack~ manually tested with
wsj.com
5. Not tested with multiple interface environment
2023-08-01 08:49:05 -07:00
Craig Tiller dd3279ab53
[promises] Fix ordering problems shown up deploying experiment internally (#33779)
Resolves a set of failures seen rolling out promises - we need to read
all of the incoming payload before doing request matching.

---------

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2023-07-26 14:47:31 -07:00
Craig Tiller 3717ff04ba
[chttp2] Split ping policy from transport (#33703)
Why: Cleanup for chttp2_transport ahead of promise conversion - lots of
logic has become interleaved throughout chttp2, so some effort to
isolate logic out is warranted ahead of that conversion.

What: Split configuration and policy tracking for each of ping rate
throttling and abuse detection into their own modules. Add tests for
them.

Incidentally: Split channel args into their own header so that we can
split the policy stuff into separate build targets.

---------

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2023-07-26 09:19:35 -07:00
AJ Heller 42b0d01e68
[Release] Bump version to 1.58.0-dev (on master branch) (#33825)
Change was created by the release automation script. See go/grpc-release
2023-07-24 10:09:08 -07:00
Yijie Ma a7bf07e86a
[EventEngine] PosixEventEngine DNS Resolver (#32701)
This PR implements a c-ares based DNS resolver for EventEngine with the
reference from the original
[grpc_ares_wrapper.h](../blob/master/src/core/ext/filters/client_channel/resolver/dns/c_ares/grpc_ares_wrapper.h).
The PosixEventEngine DNSResolver is implemented on top of that. Tests
which use the client channel resolver API
([resolver.h](../blob/master/src/core/lib/resolver/resolver.h#L54)) are
ported, namely the
[resolver_component_test.cc](../blob/master/test/cpp/naming/resolver_component_test.cc)
and the
[cancel_ares_query_test.cc](../blob/master/test/cpp/naming/cancel_ares_query_test.cc).
The WindowsEventEngine DNSResolver will use the same EventEngine's
grpc_ares_wrapper and will be worked on next.

The
[resolve_address_test.cc](https://github.com/grpc/grpc/blob/master/test/core/iomgr/resolve_address_test.cc)
which uses the iomgr
[DNSResolver](../blob/master/src/core/lib/iomgr/resolve_address.h#L44)
API has been ported to EventEngine's dns_test.cc. That leaves only 2
tests which use iomgr's API, notably the
[dns_resolver_cooldown_test.cc](../blob/master/test/core/client_channel/resolvers/dns_resolver_cooldown_test.cc)
and the
[goaway_server_test.cc](../blob/master/test/core/end2end/goaway_server_test.cc)
which probably need to be restructured to use EventEngine DNSResolver
(for one thing they override the original grpc_ares_wrapper's free
functions). I will try to tackle these in the next step.
<!--

If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the
appropriate
lang label.

-->
2023-07-21 13:24:16 -07:00
Mark D. Roth 083bbee480
[LB policies] revert changes for dualstack design (#33718)
This reverts the following PRs: #32692 #33087 #33093 #33427 #33568

These changes seem to have introduced some flaky crashes. Reverting
while I investigate.
2023-07-14 15:59:42 -07:00
Craig Tiller 20dfe78d95
[promises] Change physical layout of retry filter (#33479)
In preparation for implementing the promise based version, separate out
the legacy call data from the filter.

There are two commits here, each representing one phase of this code
movement:
66676d398c moves `class RetryFilter` into
the header and the vtable name into that class, as this will be shared
code between the implementations
4c84f115ad then moves `class
RetryFilter::CallData` into `class RetryFilterLegacyCallData`, and moves
*that* into its own file

Doing so makes me less confused as to what I'm editing going forward.

No functionality should be affected.

---------

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2023-07-11 10:23:36 -07:00
Craig Tiller c5bb43ab61
[chttp2] Eliminate grpc_chttp2_stream_map (#33503)
No need for a bespoke type anymore... and a step along the path to
C++ification.

---------

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2023-07-05 15:07:31 -07:00
Esun Kim 415f2f9ee3
[Core] Improve HPACK (#33597) 2023-07-05 13:04:31 +09:00
Eugene Ostroukhov e0bc8a2c85
[xDS LB] xDS pick first support (#33540) 2023-06-30 14:06:08 -07:00
Mark D. Roth f09357ccb4
[ring hash] delegate to pick_first instead of creating subchannels directly (#33093)
More work on the dualstack backend design:
- Change ring_hash policy to delegate to pick_first instead of creating
subchannels directly.
- Note that, as mentioned in the WIP gRFC, because we lazily create the
pick_first child policies, so there's no need to swap over to a new list
as an atomic whole. As a result, we don't use the endpoint_list library
in this policy; instead, we just update a map in-place.
- Remove now-unused subchannel_list library.
2023-06-28 06:49:53 -07:00
Mark D. Roth 27a778fece
[round robin] delegate to pick_first instead of creating subchannels directly (#32692)
More work on the dualstack backend design:
- Change round_robin to delegate to pick_first instead of creating
subchannels directly.
- Change pick_first such that when it is the child of a petiole policy,
it will unconditionally start a health watch.
- Change the client-side health checking code such that if client-side
health checking is not enabled, it will return the subchannel's raw
connectivity state.
- As part of this, we introduce a new endpoint_list library to be used
by petiole policies, which is intended to replace the existing
subchannel_list library. The only policy that will still directly
interact with subchannels is pick_first, so the relevant parts of the
subchannel_list functionality have been copied directly into that
policy. The subchannel_list library will be removed after all petiole
policies are updated to delegate to pick_first.
2023-06-27 09:54:12 -07:00