Commit Graph

2446 Commits

Author SHA1 Message Date
Alexander Polcyn 419a070078 Allow configuring server_notify_client_when_started field in xds e2e test lib
While we're here, also change the Echo service handler to mutate clients before handling the RPC, mainly so that it can be reasoned about in cases involving client cancellation.

PiperOrigin-RevId: 613029541
2024-03-05 17:57:46 -08:00
Alexander Polcyn 6f6f0df2ba Make TestServiceSignaller more generically useful
There are tests in another change I'm working on for which we need to do things like:

1) waiting for N RPCs to concurrently pile up on a server

2) doing 1) multiple times sequentially on the same server

PiperOrigin-RevId: 612934305
2024-03-05 12:40:05 -08:00
Mark D. Roth db9ca10b35 [xDS] add optional locality label to per-call metrics (#36049)
Closes #36049

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36049 from markdroth:xds_locality_label_on_per_call_metrics 9c5aeb7fb1
PiperOrigin-RevId: 612648213
2024-03-04 17:43:49 -08:00
Mark D. Roth c43f1a63b0 [RLS] implement non-per-call metrics (#36001)
Closes #36001

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36001 from markdroth:non_per_call_metrics_rls b713980f21
PiperOrigin-RevId: 612532005
2024-03-04 11:58:31 -08:00
Mark D. Roth 40577dd585 [WRR] implement non-per-call metrics (#35977)
As per gRFC A78 (https://github.com/grpc/proposal/pull/419).

Note that these new metrics are populated only if the `wrr_delegate_to_pick_first` experiment is enabled, which is the case by default.

Closes #35977

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35977 from markdroth:non_per_call_metrics_wrr 7acea32fc4
PiperOrigin-RevId: 610760874
2024-02-27 08:32:09 -08:00
Eugene Ostroukhov 70af2c2fda [xds] xDS client per data plane target (#35730)
Closes #35730

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35730 from eugeneo:tasks/multiple-xds-clients 30b459658e
PiperOrigin-RevId: 610511488
2024-02-26 13:38:21 -08:00
Mark D. Roth cfcbb0a535 [build] move parts of the `grpc_client_channel` BUILD target out into their own targets (#35879)
This breaks the following pieces out of the `grpc_client_channel` BUILD target:
- backend_metric_parser
- oob_backend_metric
- child_policy_handler
- backup_poller
- service_config_channel_arg_filter
- client_channel_channelz
- client_channel_internal_header
- subchannel_connector
- subchannel_pool_interface
- config_selector
- client_channel_service_config_parser
- retry_service_config_parser
- retry_throttle

The code left in the `grpc_client_channel` target will need more work to pull apart.

Closes #35879

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35879 from markdroth:client_channel_build_split f388a37edc
PiperOrigin-RevId: 608806548
2024-02-20 18:06:37 -08:00
apolcyn b855423359 [testing] Add StopListening helper and BidiStream handler to XDS e2e test lib (#35861)
As title. Pulling these additions out from a larger change.

Related: cl/563857636

Closes #35861

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35861 from apolcyn:xds_test_changes a67f64d93c
PiperOrigin-RevId: 607728140
2024-02-16 10:25:59 -08:00
Mark D. Roth 41606054c2 [load_file] remove grpc_load_file() in favor of grpc_core::LoadFile() (#35857)
Closes #35857

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35857 from markdroth:highlander 7b7d95aaa9
PiperOrigin-RevId: 605742734
2024-02-09 15:16:24 -08:00
Mark D. Roth 21cb320080 [reorg] move service config code to src/core/service_config (#35843)
Closes #35843

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35843 from markdroth:client_channel_reorg4 0c50ada6f9
PiperOrigin-RevId: 605466874
2024-02-08 16:58:20 -08:00
Mark D. Roth f22c954ef5 [reorg] move client channel code to src/core/client_channel (#35827)
Also rename client_channel.{h,cc} -> client_channel_filter.{h,cc}.

Closes #35827

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35827 from markdroth:client_channel_reorg3 449bff563f
PiperOrigin-RevId: 605006293
2024-02-07 09:23:26 -08:00
Mark D. Roth 10e83973e7 [reorg] move resolver code to src/core/resolver (#35804)
This new directory combines code from the following locations:
- src/core/ext/filters/client_channel/resolver
- src/core/lib/resolver

Closes #35804

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35804 from markdroth:client_channel_resolver_reorg2 30660e6b00
PiperOrigin-RevId: 604665835
2024-02-06 09:07:56 -08:00
Mark D. Roth 148f59c15a [reorg] move LB policy code to src/core/load_balancing (#35786)
This new directory combines code from the following locations:
- src/core/ext/filters/client_channel/lb_policy
- src/core/lib/load_balancing

Closes #35786

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35786 from markdroth:client_channel_resolver_reorg 98554efb98
PiperOrigin-RevId: 604351832
2024-02-05 10:05:35 -08:00
Yijie Ma 77ad5a786e [CSM O11Y] CSM Service Label Plumbing from LB Policies to CallAttemptTracer (#35210)
<!--

If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the appropriate
lang label.

-->

Closes #35210

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35210 from yijiem:csm-service-label 6a6a7d1774
PiperOrigin-RevId: 597641393
2024-01-11 13:37:44 -08:00
Mark D. Roth aa326c947c [xDS] fix aggregate cluster design (#35313)
Implements the aggregate cluster changes described in gRFC A75 (https://github.com/grpc/proposal/pull/405).

Closes #35313

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35313 from markdroth:xds_aggregate_cluster_fix 85cfd70c59
PiperOrigin-RevId: 597326624
2024-01-10 12:40:56 -08:00
Gregory Cooke b5c5b7a264 [OpenSSL] Disable another test (#35461)
This accidentally slipped through the cracks on https://github.com/grpc/grpc/pull/35444

Closes #35461

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35461 from gtcooke94:fix_ossl102_part2 c8b526cefa
PiperOrigin-RevId: 596135991
2024-01-05 19:42:36 -08:00
Craig Tiller 2694f87c23 [channel] Remove test only feature (#35465)
It's not clear to me that this one unit test of very marginal importance warrants 8 bytes per channel.

Closes #35465

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35465 from ctiller:we-dont-need-this-really e7ee62ccb2
PiperOrigin-RevId: 596091614
2024-01-05 15:03:33 -08:00
Gregory Cooke 61a841b208 Disable failing openssl102 tests (#35354)
There are a select few tests that are failing when building with OpenSSL102 - disable them until we can fix.

Closes #35354

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35354 from gtcooke94:fix_ossl_102 8708d6ce86
PiperOrigin-RevId: 595761932
2024-01-04 11:43:09 -08:00
Mark D. Roth c7101d0867 [xDS] move CDS and EDS watchers into xds resolver (#35011)
Implements gRFC A74 (https://github.com/grpc/proposal/pull/404).

Closes #35011

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35011 from markdroth:xds_watchers_in_xds_resolver a39f71f37f
PiperOrigin-RevId: 595134549
2024-01-02 09:20:06 -08:00
Gregory Cooke a5c71e132e [Security - TLS] DirectoryReloaderCrlProvider construction time bug fix (#35247)
The `DirectoryReloaderProvider` currently segfaults on construction if grpc_init() is not called before construction. This is because when creating the `DirectoryReloaderCrlProvider` we [call GetDefaultEventEngine](a58f3f2df5/src/core/lib/security/credentials/tls/grpc_tls_crl_provider.cc (L152)), and getting the default event engine requires that `grpc_init` is called.

This PR adds a test that catches the segfault and adds `grpc_init` and `grpc_shutdown` to the ctor and dtor of `DirectoryReloaderCrlProvider` so that the test passes.

Closes #35247

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35247 from gtcooke94:crl_provider_init_fix 25f3dc7f27
PiperOrigin-RevId: 589885254
2023-12-11 11:25:49 -08:00
Mark D. Roth 5f41fde4f2 [xDS] add test for mTLS for aggregate clusters (#34927)
Closes #34927

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/34927 from markdroth:xds_mtls_aggregate_cluster_fix 6aa956997f
PiperOrigin-RevId: 582838553
2023-11-15 16:14:04 -08:00
Yijie Ma d3828ebfbd
[Fork] Add ObjectGroupForkHandler fork-handling system (#33733)
<!--

If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the
appropriate
lang label.

-->

---------

Co-authored-by: Bradley Hess <bdhess@google.com>
Co-authored-by: AJ Heller <hork@google.com>
2023-11-07 17:48:13 -08:00
Mark D. Roth 1e111df8be
[grpclb_e2e_test] more test improvements and flake fixes (#34896)
- Fix deadlock in load reporting tests.
- Add timeout to `WaitForLoadReport()`. (Note: this required changing
from `grpc::internal::Mutex` and friends to `grpc_core::Mutex` and
friends.)
- Fix balancer stream shutdown machinery.
- Change `ServerThread` to be a class instead of a struct.
2023-11-07 14:47:15 -08:00
Mark D. Roth 8a000f45f8
[grpclb and fake resolver] clean up e2e tests and simplify fake resolver (#34887)
Changes to fake resolver:
- Add `WaitForReresolutionRequest()` method to fake resolver response
generator to allow tests to tell when re-resolution has been requested.
- Change fake resolver response generator API to have only one mechanism
for injecting results, regardless of whether the result is an error or
whether it's triggered by a re-resolution.

Changes to grpclb_end2end_test:
- Change balancer interface such that instead of setting a list of
responses with fixed delays, the test can control exactly when each
response is set.
- Change balancer impl to always send the initial LB response, as
expected by the grpclb protocol.
- Change balancer impl to always read load reports, even if load
reporting is not expected to be enabled. (The latter case will still
cause the test to fail.) Reads are done in a different thread than
writes.
- Allow each test to directly control how many backends and balancers
are started and the client load reporting interval, so that (a) we don't
waste resources starting servers we don't need and (b) there is no need
to arbitrarily split tests across different test classes.
- Add timeouts to `WaitForAllBackends()` functionality, so that tests
will fail with a useful error rather than timing out.
- Improved ergonomics of various helper functions in the test framework.

In the process of making these changes, I found a couple of bugs:
- A bug in pick_first, which I fixed in #34885.
- A bug in grpclb, in which we were using the wrong condition to decide
whether to propagate a re-resolution request from the child policy,
which I've fixed in this PR. (This bug probably originated way back in
#18344.)

This should address a lot of the flakes seen in grpclb_e2e_test
recently.
2023-11-06 14:42:43 -08:00
Craig Tiller 78825f79ca
[event-engine] Fix channelz tests to hermetically clean up callbacks (#34883)
EventEngine experiments, especially with `work_serializer_dispatch` tend
to cause callbacks to occur later than we've previously seen, so tests
that verify global data structures tend to become flakier when these are
introduced.

Here, the fix is waiting for EventEngine to be closed before starting
the new test.

Whilst here, make some adjustments to the test for better readability on
what's going on:
- if we fail a request to an echo service, we do not actually expect the
messages to match, so don't report that
- if we expect a value of 1 or 2, AnyOf is a better tool: it will report
the actual value too

---------

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2023-11-06 07:00:04 -08:00
Matthew Stevenson 52f9e011f3
[tls] Allow skipping server cert verification when no default roots are present. (#34859)
This PR fixes a bug identified in #29667, where the TLS channel
credentials still require a trust bundle even if the user has explicitly
opted to not verify the server certificate. This PR is based on #29810.
2023-11-03 10:23:10 -07:00
Mark D. Roth e8f2e27b2c
[memory test] reland: add benchmarks with xDS enabled (#34820)
Relands #34785, which was reverted in #34818.

The first commit is the revert. The second commit removes the gtest
dependency from the xds_server library, which should address the
testonly problem internally.
2023-10-31 12:56:40 -07:00
Mark D. Roth b1901eb732
Revert "[memory test] add benchmarks with xDS enabled" (#34818)
Reverts grpc/grpc#34785
2023-10-27 15:25:43 -07:00
Mark D. Roth e0a5190a5e
[memory test] add benchmarks with xDS enabled (#34785)
Initial results:

```
default: call/client: 15038.38208
default: call/server: 12873.64608
default: call/xds_client: 15850.04544
default: call/xds_server: 13177.15968
default: channel/client: 27747.9424
default: channel/server: 19939.328
default: channel/xds_client: 45272.2688
default: channel/xds_server: 20805.632
```

b/297028827
2023-10-27 13:52:05 -07:00
Mark D. Roth 066d5dd2bb
[xDS e2e tests] move xDS bootstrap and resource helpers into their own library (#34773)
This paves the way for reusing these helpers in a standalone xDS server
for memory usage tests.
2023-10-24 12:33:00 -07:00
Mark D. Roth 30a24a37d1
[e2e tests] refactor code for determining local IP address (#34769) 2023-10-23 17:27:21 -07:00
Vignesh Babu 299b4fe3fd
[flakiness] Fix regex comparison which causes client_lb test to flake at a high rate internally when event engine is enabled. (#34607) 2023-10-23 09:05:01 -07:00
Gregory Cooke 9969d820b5
[TLS - Revocation] Crl Provider (#34715)
This reverts commit 7af5efcfd3.
2023-10-19 11:33:41 -07:00
Gregory Cooke 74c1de6512
[Security] Reapply Move ownership of tsi_ssl_client_handshaker_factory (#34726)
This reverts commit 601aaf80b7, which
results in rolling forward #34408.

The 601aaf reversion happened because of a deadlock found in Python. The
root cause ended up being an issue with the Python wrapper and was fixed
in #34712 , so this can be rolled forward again
2023-10-18 17:59:44 -04:00
apolcyn 7af5efcfd3
Revert "[TLS - Revocation] Crl Provider (#33786)" (#34713)
This reverts commit 0f0396ae92.
2023-10-17 13:53:36 -07:00
Gregory Cooke 0f0396ae92
[TLS - Revocation] Crl Provider (#33786)
The basic APIs for the CRL Reloading features.
This adds external types to represent CRL Providers, CRLs, and
CertificateInfo.
Internally we will use `CrlImpl` - this layer is needed to hide OpenSSL
details from the user.

GRFC - https://github.com/grpc/proposal/pull/382

Things Done
* Add external API for `CrlProvider`, `Crl`, `CertInfo` (`CertInfo` is
used during CRL lookup rather than passing the entire certificate).
* Add code paths in `ssl_transport_security` to utilize CRL providers
* Add `StaticCrlProvider`
* Refactor `crl_ssl_transport_security_test.cc` so it is more extensible
and can be used with providers
2023-10-17 13:03:52 -04:00
nanahpang 601aaf80b7
Revert "[Security] Move ownership of tsi_ssl_client_handshaker_factory to grpc_ssl_credentials, version 2." (#34701)
Reverts grpc/grpc#34408
2023-10-16 18:18:44 +00:00
Craig Tiller c9df0ca470
[cpp] Reland removal of channel_filter code (#34598)
Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2023-10-12 09:44:11 -07:00
Yijie Ma bae0c705aa
[Deps] Update to Clang-16 (#34492)
<!--

If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the
appropriate
lang label.

-->
2023-10-11 16:26:32 -07:00
Mark D. Roth bb6a6faa69
[SSA] support multiple addresses per endpoint (#34472) 2023-10-06 15:23:33 -07:00
Eugene Ostroukhov 3ed72c0c60
[reflection] Add v1 reflection (#34599) 2023-10-06 11:40:31 -07:00
Craig Tiller 3b0916fc0a
Revert "[reflection] Add v1 reflection" (#34594)
Reverts grpc/grpc#34535
2023-10-04 19:06:58 -07:00
Matthew Stevenson 539e912ef8
[ssl] Do not crash if ssl session cache capacity is zero. (#34539)
This behavior is dangerous because we will crash when the cache is
created, which is not necessarily on application startup and is likely
when you first try to establish an SSL connection. Instead, we log an
error. If the SSL library attempts to put a session ticket in the cache
it will fail to do so, but everything else will continue as normal. In
particular, we will always seamlessly fall back to a full SSL handshake.

Along the way, we also ensure that you cannot put a null `SSL_SESSION`
into the cache, which would lead to a segfault when it is fetched from
the cache.
2023-10-04 12:45:09 -07:00
Eugene Ostroukhov 5817f0f6c1
[reflection] Add v1 reflection (#34535) 2023-10-04 11:09:59 -07:00
Mark D. Roth 7044af2691
[tests] fix flakes caused by dualstack changes (#34575)
Fix chttp2 too_many_pings test to use only one of IPv4 or IPv6,
depending on test environment.

Also fix dumb reversed conditional bug in some other tests that was
accidentally introduced in #34426.
2023-10-02 13:48:43 -07:00
Mark D. Roth 883ec58237
[client_lb_e2e_test] fix flake in RR HealthChecking test (#34572) 2023-10-02 13:08:32 -07:00
Mark D. Roth 24f995538c
[xDS] add support for multiple addresses per endpoint (#34506)
Co-authored-by: markdroth <markdroth@users.noreply.github.com>
2023-09-29 09:46:45 -07:00
Craig Tiller 7b17e4aa92
Revert "[cpp] Remove legacy channel filter C++ wrapper" (#34541)
Reverts grpc/grpc#34522
2023-09-28 22:29:53 -07:00
Craig Tiller 727c92a15c
[cpp] Remove legacy channel filter C++ wrapper (#34522)
Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2023-09-28 20:59:56 -07:00
Mark D. Roth 7a06614f95
[resolver and LB policy APIs] reland: change address list to support multiple addresses per endpoint (#34531)
Re-land #33567, which was reverted in #34527.

First commit is a pure revert, second commit is a small fix needed to
avoid breaking internal callers.
2023-09-28 15:40:05 -07:00