Commit Graph

276 Commits

Author SHA1 Message Date
Alexander Polcyn 419a070078 Allow configuring server_notify_client_when_started field in xds e2e test lib
While we're here, also change the Echo service handler to mutate clients before handling the RPC, mainly so that it can be reasoned about in cases involving client cancellation.

PiperOrigin-RevId: 613029541
2024-03-05 17:57:46 -08:00
Mark D. Roth db9ca10b35 [xDS] add optional locality label to per-call metrics (#36049)
Closes #36049

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36049 from markdroth:xds_locality_label_on_per_call_metrics 9c5aeb7fb1
PiperOrigin-RevId: 612648213
2024-03-04 17:43:49 -08:00
Mark D. Roth 40577dd585 [WRR] implement non-per-call metrics (#35977)
As per gRFC A78 (https://github.com/grpc/proposal/pull/419).

Note that these new metrics are populated only if the `wrr_delegate_to_pick_first` experiment is enabled, which is the case by default.

Closes #35977

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35977 from markdroth:non_per_call_metrics_wrr 7acea32fc4
PiperOrigin-RevId: 610760874
2024-02-27 08:32:09 -08:00
Eugene Ostroukhov 70af2c2fda [xds] xDS client per data plane target (#35730)
Closes #35730

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35730 from eugeneo:tasks/multiple-xds-clients 30b459658e
PiperOrigin-RevId: 610511488
2024-02-26 13:38:21 -08:00
apolcyn b855423359 [testing] Add StopListening helper and BidiStream handler to XDS e2e test lib (#35861)
As title. Pulling these additions out from a larger change.

Related: cl/563857636

Closes #35861

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35861 from apolcyn:xds_test_changes a67f64d93c
PiperOrigin-RevId: 607728140
2024-02-16 10:25:59 -08:00
Mark D. Roth 41606054c2 [load_file] remove grpc_load_file() in favor of grpc_core::LoadFile() (#35857)
Closes #35857

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35857 from markdroth:highlander 7b7d95aaa9
PiperOrigin-RevId: 605742734
2024-02-09 15:16:24 -08:00
Mark D. Roth f22c954ef5 [reorg] move client channel code to src/core/client_channel (#35827)
Also rename client_channel.{h,cc} -> client_channel_filter.{h,cc}.

Closes #35827

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35827 from markdroth:client_channel_reorg3 449bff563f
PiperOrigin-RevId: 605006293
2024-02-07 09:23:26 -08:00
Mark D. Roth 10e83973e7 [reorg] move resolver code to src/core/resolver (#35804)
This new directory combines code from the following locations:
- src/core/ext/filters/client_channel/resolver
- src/core/lib/resolver

Closes #35804

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35804 from markdroth:client_channel_resolver_reorg2 30660e6b00
PiperOrigin-RevId: 604665835
2024-02-06 09:07:56 -08:00
Mark D. Roth 148f59c15a [reorg] move LB policy code to src/core/load_balancing (#35786)
This new directory combines code from the following locations:
- src/core/ext/filters/client_channel/lb_policy
- src/core/lib/load_balancing

Closes #35786

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35786 from markdroth:client_channel_resolver_reorg 98554efb98
PiperOrigin-RevId: 604351832
2024-02-05 10:05:35 -08:00
Yijie Ma 77ad5a786e [CSM O11Y] CSM Service Label Plumbing from LB Policies to CallAttemptTracer (#35210)
<!--

If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the appropriate
lang label.

-->

Closes #35210

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35210 from yijiem:csm-service-label 6a6a7d1774
PiperOrigin-RevId: 597641393
2024-01-11 13:37:44 -08:00
Mark D. Roth aa326c947c [xDS] fix aggregate cluster design (#35313)
Implements the aggregate cluster changes described in gRFC A75 (https://github.com/grpc/proposal/pull/405).

Closes #35313

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35313 from markdroth:xds_aggregate_cluster_fix 85cfd70c59
PiperOrigin-RevId: 597326624
2024-01-10 12:40:56 -08:00
Mark D. Roth c7101d0867 [xDS] move CDS and EDS watchers into xds resolver (#35011)
Implements gRFC A74 (https://github.com/grpc/proposal/pull/404).

Closes #35011

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35011 from markdroth:xds_watchers_in_xds_resolver a39f71f37f
PiperOrigin-RevId: 595134549
2024-01-02 09:20:06 -08:00
Mark D. Roth 5f41fde4f2 [xDS] add test for mTLS for aggregate clusters (#34927)
Closes #34927

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/34927 from markdroth:xds_mtls_aggregate_cluster_fix 6aa956997f
PiperOrigin-RevId: 582838553
2023-11-15 16:14:04 -08:00
Mark D. Roth e8f2e27b2c
[memory test] reland: add benchmarks with xDS enabled (#34820)
Relands #34785, which was reverted in #34818.

The first commit is the revert. The second commit removes the gtest
dependency from the xds_server library, which should address the
testonly problem internally.
2023-10-31 12:56:40 -07:00
Mark D. Roth b1901eb732
Revert "[memory test] add benchmarks with xDS enabled" (#34818)
Reverts grpc/grpc#34785
2023-10-27 15:25:43 -07:00
Mark D. Roth e0a5190a5e
[memory test] add benchmarks with xDS enabled (#34785)
Initial results:

```
default: call/client: 15038.38208
default: call/server: 12873.64608
default: call/xds_client: 15850.04544
default: call/xds_server: 13177.15968
default: channel/client: 27747.9424
default: channel/server: 19939.328
default: channel/xds_client: 45272.2688
default: channel/xds_server: 20805.632
```

b/297028827
2023-10-27 13:52:05 -07:00
Mark D. Roth 066d5dd2bb
[xDS e2e tests] move xDS bootstrap and resource helpers into their own library (#34773)
This paves the way for reusing these helpers in a standalone xDS server
for memory usage tests.
2023-10-24 12:33:00 -07:00
Mark D. Roth 30a24a37d1
[e2e tests] refactor code for determining local IP address (#34769) 2023-10-23 17:27:21 -07:00
Mark D. Roth bb6a6faa69
[SSA] support multiple addresses per endpoint (#34472) 2023-10-06 15:23:33 -07:00
Mark D. Roth 24f995538c
[xDS] add support for multiple addresses per endpoint (#34506)
Co-authored-by: markdroth <markdroth@users.noreply.github.com>
2023-09-29 09:46:45 -07:00
Mark D. Roth 7a06614f95
[resolver and LB policy APIs] reland: change address list to support multiple addresses per endpoint (#34531)
Re-land #33567, which was reverted in #34527.

First commit is a pure revert, second commit is a small fix needed to
avoid breaking internal callers.
2023-09-28 15:40:05 -07:00
Mark D. Roth 41f26de3b6
Revert "[resolver and LB policy APIs] change address list to support multiple addresses per endpoint" (#34527)
Reverts grpc/grpc#33567 due to import problems.
2023-09-28 12:15:35 -07:00
Mark D. Roth fd2e8c9462
[resolver and LB policy APIs] change address list to support multiple addresses per endpoint (#33567)
More changes as part of the dualstack design:
- Change resolver and LB policy APIs to support multiple addresses per
endpoint. Specifically, replace `ServerAddress` with
`EndpointAddresses`, which encodes more than one address. Per-address
channel args are retained at the same level, so they are now
per-endpoint. For now, `EndpointAddress` provides a single-address ctor
and a single-address accessor for backward compatibility, so
`ServerAdress` is an alias for `EndpointAddresses`; eventually, this
alias and the single-address methods will be removed.
- Add an `EndpointAddressSet` class, which represents an unordered set
of addresses to be used as a map key. This will be used in a number of
LB policies that need to store per-endpoint state.
- Change the LB policy API's `ChannelControlHelper::CreateSubchannel()`
method to take the address and per-endpoint channel args as separate
parameters, so that we don't need to construct a legacy `ServerAddress`
object as we create a new subchannel for each address in the endpoint.
- Change pick_first to flatten the address list.
- Change ring_hash to use `EndpointAddressSet` as the key for its
endpoint map, and to use the first address of the endpoint as the hash
key.
- Change WRR to use `EndpointAddressSet` as the key for its endpoint
weight map.

Note that support for multiple addresses per endpoint is guarded in RR
by the existing `round_robin_delegate_to_pick_fist` experiment and in
WRR by the existing `wrr_delegate_to_pick_first` experiment.

This PR does *not* include support for multiple addresses per endpoint
for the outlier_detection or xds_override_host LB policies; those will
come in subsequent PRs.
2023-09-28 09:32:36 -07:00
Eugene Ostroukhov 2f78fffa37
[xds ssa] Remove environment variable protection for stateful affinity (#34435) 2023-09-22 13:06:51 -07:00
Craig Tiller 86b931c354
[work-serializer] Dispatch on run experiment (relanding) (#34372)
Reverts grpc/grpc#34371
2023-09-15 12:46:04 -07:00
Craig Tiller d589caa679
Revert "[work-serializer] Dispatch on run experiment" (#34371)
Reverts grpc/grpc#34274

(needs some changes internally)
2023-09-15 09:10:31 -07:00
Craig Tiller 1705470950
[work-serializer] Dispatch on run experiment (#34274)
Co-authored-by: ctiller <ctiller@users.noreply.github.com>
Co-authored-by: Mark D. Roth <roth@google.com>
2023-09-15 07:49:42 -07:00
Eugene Ostroukhov 77f80f3de5
[ssa test] Test TTL attribute on cookie (#34326) 2023-09-13 14:09:44 -07:00
Eugene Ostroukhov 59bab7f27f
[ssa test] Add test for per-route SSA configuration (#34313) 2023-09-12 21:02:13 -07:00
Eugene Ostroukhov 3d1f242abe
[Session Affinity] Update validation and add a test case (#34277) 2023-09-07 17:41:56 -07:00
Mark D. Roth a315171880
[ring_hash] delegate to pick_first as per dualstack design (#34244)
Rolls forward the changes from #33093 and some from #33568, which were
rolled back in #33718.
2023-09-07 09:39:35 -07:00
Mark D. Roth 6dea42c874
[XdsClient] replace e2e test with unit test (#34258)
This should address one of the failures we're seeing in #34224.

The test failure is caused by the changes in timing triggering a race
condition. In the code at head, we delay sending out the subscription
for the first CDS watch until we've already seen the other two CDS
watches, because the previous send_message op has not yet completed, and
by the time it does, we've seen all 3 watches, so we can send a
subscription for all 3 at the same time. With the WorkSerializer change,
the send_message op is complete by the time we see the first CDS watch,
so we subscribe to only that resource, and then later add the other two.
The result is that we'll NACK twice with two different messages, the
first one including only the error about the first resource, and the
second one including all three.

I suspect this same race condition would have been triggered eventually
by the EventEngine migration anyway; the current test basically depends
on the single-thread timing of the iomgr approach. So I'm addressing it
by replacing the e2e test with a unit test that covers the same cases
without the timing issue.
2023-09-06 09:58:35 -07:00
Mark D. Roth 1c54662866
[xDS] improve RPC failure status message when aggregate cluster graph has no leaf clusters (#34201)
Old message:
new_cluster_1: UNAVAILABLE: errors validating xds_cluster_resolver LB
policy config: [field:discoveryMechanisms error:must be non-empty]

New message:
new_cluster_1: FAILED_PRECONDITION: aggregate cluster graph has no leaf
clusters
2023-08-30 12:07:52 -07:00
Mohan Li ab024624da
[pick_first] de-experiment pick first (#34054)
De-experiment pick first since we have both affinity and randomness E2E
test running successfully.

---------

Co-authored-by: Yash Tibrewal <yashkt@google.com>
2023-08-17 09:54:55 -07:00
Yousuk Seung 4acb7d38b9
[xds] Apply the slowdown factor only once to LRS load reporting period (#34042)
<!--

If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the
appropriate
lang label.

-->
2023-08-11 22:43:36 +00:00
Luwei Ge a5f1121982
[xDS] Remove filter name from GenerateServiceConfig (#33915)
We decided to not populate `policy_name` with the HTTP filter name in
xDS case. So removing it from `GenerateServiceConfig`. This will be
consistent across languages. The gRFC
[PR](https://github.com/grpc/proposal/pull/346) has been updated.
2023-08-08 12:23:52 -07:00
Vignesh Babu 0616c8b838
[xds] Regex fix in test (#33981) 2023-08-03 13:14:21 -07:00
Mark D. Roth 083bbee480
[LB policies] revert changes for dualstack design (#33718)
This reverts the following PRs: #32692 #33087 #33093 #33427 #33568

These changes seem to have introduced some flaky crashes. Reverting
while I investigate.
2023-07-14 15:59:42 -07:00
Eugene Ostroukhov e0bc8a2c85
[xDS LB] xDS pick first support (#33540) 2023-06-30 14:06:08 -07:00
Mark D. Roth f09357ccb4
[ring hash] delegate to pick_first instead of creating subchannels directly (#33093)
More work on the dualstack backend design:
- Change ring_hash policy to delegate to pick_first instead of creating
subchannels directly.
- Note that, as mentioned in the WIP gRFC, because we lazily create the
pick_first child policies, so there's no need to swap over to a new list
as an atomic whole. As a result, we don't use the endpoint_list library
in this policy; instead, we just update a map in-place.
- Remove now-unused subchannel_list library.
2023-06-28 06:49:53 -07:00
Eugene Ostroukhov 6451beba8e
Revert "Revert "Revert "Revert "[xDS LB] Override cluster with value … (#33424)
Previous attempt: #33416

This reverts commit 19460ea82f.
2023-06-14 13:42:59 -07:00
Mark D. Roth f34a39af74
[xDS e2e tests] add 1K RPCs to try to work around statistical problem (#33409) 2023-06-14 10:58:15 -07:00
Craig Tiller 19460ea82f
Revert "Revert "Revert "[xDS LB] Override cluster with value from cookie"" (#33379)" (#33416)
Reverts grpc/grpc#33388

Breaks import
2023-06-13 07:23:46 -07:00
Eugene Ostroukhov 57bb6fb65c
Revert "Revert "[xDS LB] Override cluster with value from cookie"" (#33388)
Reapplying #32973
2023-06-12 16:12:17 -07:00
Eugene Ostroukhov 6f685274a1
Revert "[xDS LB] Override cluster with value from cookie (#32973)" (#33379)
This reverts #32973 that causes breakages in internal CI.
2023-06-08 16:50:47 -07:00
Eugene Ostroukhov 8fe3472e53
[xDS LB] Override cluster with value from cookie (#32973) 2023-06-08 13:52:13 -07:00
Yousuk Seung c03cd744b2
[WRR] Prefer application_utilization to cpu_utilization (#33355)
<!--

If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the
appropriate
lang label.

-->
2023-06-07 17:38:29 -07:00
apolcyn 889412c416
[Rls] de-experimentalize RLS in XDS (#33290)
Integration tests are passing, so we should be ready to
de-experimentalize.

Related: internal bug b/265209578
2023-06-02 12:26:45 -07:00
Luwei Ge d1c0dc58cc
[Audit Logging] xDS e2e test for audit logging. (#33252)
Added tests involve:
1. Checking the # of logger invocations with multiple RBACs in the
chain.
2. Verifying content in audit context with action and audit condition
permutations.
3. Confirm custom logger and built-in logger configurations are working.
4. Confirm the feature is protected by the environment variable.

---------

Co-authored-by: rockspore <rockspore@users.noreply.github.com>
2023-05-26 12:43:37 -07:00
Mark D. Roth 52d687ad42
[xDS] second attempt: clean up cert provider factory and registry APIs (#33249)
Original was #33226, reverted in #33248.
2023-05-25 12:59:21 -07:00