Commit Graph

14034 Commits

Author SHA1 Message Date
Eugene Ostroukhov 8ea3b417cc [csds] Make grpc_dump_xds_configs return ClientStatusResponse (#35612)
Closes #35612

PiperOrigin-RevId: 602443727
2024-01-29 11:14:23 -08:00
AJ Heller 71fa68f7fb [EventEngine] Add QueryExtension Interfaces for *SupportsFd (#35648)
This PR:
* adds FD extensions to the public headers
* Adds the query extension interface to EventEngine, Listener, and Endpoint, via a new `Extensible` interface
* Refactors the PosixEventEngine to use the Extensible interface.

Closes #35648

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35648 from drfloob:posix-ee-query-interface 7cae28e0b3
PiperOrigin-RevId: 601794970
2024-01-26 10:03:18 -08:00
Stanley Cheung 48961e8a2c [PSM Interop] Add a payload to xds interop client when sending RPCs (#35545)
When testing CSM Observability, we discovered that the c++ xds interop client is not sending any payload with the `UnaryCall` RPCs so most of the metrics will have a value of 0.

Adding a payload to the xds interop client here.

We need this fix so that we can verify that the metrics are recording the right number of bytes being sent / received. So we need a non-trivial payload to be sent with the `UnaryCall` RPC between the xds interop client and server.

Closes #35545

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35545 from stanley-cheung:xds-client-payload 11be4e6f4f
PiperOrigin-RevId: 601596246
2024-01-25 16:12:45 -08:00
AJ Heller ba2f74a84c [EventEngine] Dump all thread stacks when the thread pool is wedged
This should help us debug rare flakes where the thread pool waits indefinitely

PiperOrigin-RevId: 601546451
2024-01-25 13:25:36 -08:00
Craig Tiller ab795b0adc Revert "[EventEngine] Enable the EventEngine DNS Resolver on Posix (#35573)" (#35667)
This reverts commit 6318e9e7e9.

<!--

If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the appropriate
lang label.

-->

Closes #35667

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35667 from ctiller:a 032999b51e
PiperOrigin-RevId: 601495207
2024-01-25 10:46:47 -08:00
Yash Tibrewal 76c45b98d1 [otel] Return absl::Status as a return from BuildAndRegisterGlobal (#35659)
Just to be future-proof, I'm amending the `void` return status of `BuildAndRegisterGlobal` in `OpenTelemetryPluginBuilder` to absl::Status.

This will be backported to 1.61 as well.

Closes #35659

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35659 from yashykt:UpdateOtelApiToAddStatus 07d3f41b8a
PiperOrigin-RevId: 601458408
2024-01-25 08:40:16 -08:00
Craig Tiller 19ad49d169 [bazel] disable fuzzer on mac (#35662)
Closes #35662

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35662 from ctiller:nononono 4f7d2fb893
PiperOrigin-RevId: 601448944
2024-01-25 08:03:46 -08:00
Yash Tibrewal 984daf98d7 [CSM o11y] Re-experimentalize CSM OTel Plugin Option (#35660)
We are no longer sure about this API, so re-experimentalizing it.

This PR will be backported to 1.61 as well.

Closes #35660

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35660 from yashykt:ReexperimentalizeCsmPluginOption 4f114a54d9
PiperOrigin-RevId: 601378856
2024-01-25 01:57:24 -08:00
Yijie Ma 6318e9e7e9 [EventEngine] Enable the EventEngine DNS Resolver on Posix (#35573)
<!--

If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the appropriate
lang label.

-->

Closes #35573

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35573 from yijiem:enable-oss-ee-dns-posix-real 017b99312f
PiperOrigin-RevId: 601245249
2024-01-24 15:04:27 -08:00
Craig Tiller 870a66d9a0 [chaotic-good] Connection setup & test suites (#35650)
Closes #35650

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35650 from ctiller:shush-tsan fd55ea1be3
PiperOrigin-RevId: 601221780
2024-01-24 13:46:08 -08:00
apolcyn 7d5b53e2a7 [testing] minor logging improvement in http cli tests (#35654)
as title

Closes #35654

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35654 from apolcyn:fix_logs eb1fbcd7a5
PiperOrigin-RevId: 601198258
2024-01-24 12:26:44 -08:00
Yash Tibrewal 4cea5d0cba [xDS Interop] Use XdsServerBuilder when testing CSM Observability (#35634)
[prod:grpc/core/master/linux/grpc_xds_k8s_lb](https://fusion2.corp.google.com/ci/kokoro/prod:grpc%2Fcore%2Fmaster%2Flinux%2Fgrpc_xds_k8s_lb/activity/6c2577f0-965b-4f22-ac2a-5454a0f77444/log)

Closes #35634

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35634 from yashykt:CsmInteropTrial a8213cd10a
PiperOrigin-RevId: 601010836
2024-01-23 22:33:57 -08:00
Craig Tiller 6b6ce36f74 [fuzzing] Allow server fuzzer to take multiple connection requests at the same time (#35645)
Closes #35645

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35645 from ctiller:shush-multipass e63e1e5ddb
PiperOrigin-RevId: 600998039
2024-01-23 21:19:49 -08:00
Yash Tibrewal f51f14e70c [otel] Re-structure otel_plugin_test initialization (#35631)
Closes #35631

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35631 from yashykt:UpdateOtelPluginTest ba1a9cecc2
PiperOrigin-RevId: 600950962
2024-01-23 17:11:09 -08:00
Yijie Ma 16b71d91d8 [CSM O11Y] Fix issue when CSM optional labels are present in server metrics (#35633)
<!--

If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the appropriate
lang label.

-->

Closes #35633

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35633 from yijiem:labels-injector-patch 6876243943
PiperOrigin-RevId: 600931754
2024-01-23 15:55:49 -08:00
Craig Tiller 7507e3b644 [fuzzing] Teach server_fuzzer about the chaotic good protocol (#35643)
No explicit testing yet, just some building blocks

Closes #35643

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35643 from ctiller:shush-chaotic-protocol 48dcaa0632
PiperOrigin-RevId: 600928555
2024-01-23 15:43:34 -08:00
Craig Tiller 19f1b39829 [fuzzer] Eliminate use of mock_endpoint in server fuzzer (#35625)
Instead, connect a real client, which gives us the ability to exercise the listener portion of the code too.

Closes #35625

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35625 from ctiller:shush-connect b8f78993c6
PiperOrigin-RevId: 600836565
2024-01-23 10:40:03 -08:00
Esun Kim af95eafb34 [CI] Using clang-17 for clang tidy & format (#35595)
Clang-17 is now better at formatting. Clang-16 used to do some questionable formatting.

Closes #35595

PiperOrigin-RevId: 600829055
2024-01-23 10:19:25 -08:00
Esun Kim 1a20f21b6b Fixed multi-line comments warning (#35351)
Adding more stuff on top of https://github.com/grpc/grpc/pull/35127

Closes #35351

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35351 from veblush:fix-line 1b25e5df49
PiperOrigin-RevId: 600810979
2024-01-23 09:20:33 -08:00
Craig Tiller 562bc9b363 [fuzzing] better fallback rng (#35621)
`ProtoBitGen` provides a random number generator that returns values directly from fuzzer selected values, which allows us to test-into random selection algorithms deterministically.

Since the list of values provided by the fuzzer is limited, we need a fallback implementation. Previously we'd used something that was very correlated, and some of the distribution algorithms get into a very slow convergence mode when we do that (so we repeatedly return the same value for billions of iterations and cause timeouts in fuzzers).

Instead, when we run out of fuzzer supplied values, seed an mt19937 generator with the fuzzer selected values and use that from there on. Said generator will then produce values deterministically (for a given fuzzer input), but with a better distribution to allow convergence for fiddly algorithms.

Closes #35621

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35621 from ctiller:cg-timeout 6c9ef9cac5
PiperOrigin-RevId: 600607424
2024-01-22 16:45:31 -08:00
Craig Tiller e93f2dfbef [fuzzer] Separate scheduling from schedule creation for network input (#35618)
Closes #35618

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35618 from ctiller:shush ed7afaa4e8
PiperOrigin-RevId: 600557159
2024-01-22 13:43:52 -08:00
Yijie Ma 5bf0971972 [EventEngine] Fix issues found when enabling `event_engine_dns` experiment in OSS (#35530)
Using `AF_UNSPEC` for both IPv4 and IPv6 queries does not work in all cases. Specifically, for `localhost:<>`, c-ares only returns the IPv6 record i.e. `::1`.

<!--

If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the appropriate
lang label.

-->

Closes #35530

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35530 from yijiem:enable-oss-ee-dns-posix 452b5a2d81
PiperOrigin-RevId: 599989537
2024-01-19 18:42:26 -08:00
David Chamberlin 5b724c09c5 [tls] Add copy constructor for TlsCredentialsOptions (#35499)
<!--

If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the appropriate
lang label.

-->

Closes #35499

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35499 from dawidcha:cred_opts_copy_constr 330165930f
PiperOrigin-RevId: 599977221
2024-01-19 17:21:50 -08:00
Mark D. Roth b440474655 [XdsClient] Clean up mechanism used to store XdsServer in map keys.
For background, see #32826 and b/276944116.

PiperOrigin-RevId: 599876499
2024-01-19 12:45:01 -08:00
Craig Tiller 1751f1043e [transport] Add a transport test suite for promise based transports (#35476)
Implemented for inproc & chaotic-good

Closes #35476

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35476 from ctiller:v3-svall 5358538d54
PiperOrigin-RevId: 599701775
2024-01-18 20:09:21 -08:00
Craig Tiller 67d6b8ea74 [promise] Context improvements (#35592)
A few improvements to the promise context system (more coming)

Allow subclassed contexts:
If we have multiple different kinds of a base context, allow `GetContext<Derived>()` to mean `down_cast<Derived*>(GetContext<Base>())` everywhere for brevity.

Allow custom context lookup:
For a base context type, allow customization of how that context is looked up.

These two together allow:
1. normalization of activity lookup and context lookup to the same syntax (so we can write `GetContext<Activity>()` everywhere now
2. Party & Activity to share a context, so that anywhere we need to do a party specific operation we can write `GetContext<Party>()->...` and safely know that it's the current activity *and* it's a party.

Closes #35592

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35592 from ctiller:contextual-types 37ef948a36
PiperOrigin-RevId: 599651708
2024-01-18 16:03:46 -08:00
Craig Tiller 98472179fb [promises] Add an observer type (#35552)
We've got a few situations coming up with promises that will want a "broadcast new value to everywhere" situation.

Closes #35552

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35552 from ctiller:obs 30fd697ae3
PiperOrigin-RevId: 599609399
2024-01-18 13:35:49 -08:00
Esun Kim bc04417440 [Clean-up] Fix Abseil-usage (#35590)
Some minor fixes for https://github.com/grpc/grpc/pull/35589

Closes #35590

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35590 from veblush:abseil-fix a89b800739
PiperOrigin-RevId: 599524969
2024-01-18 09:09:35 -08:00
Craig Tiller eb17c1f7de [filters] Optimize construction/destruction (#35591)
Only call constructors when absolutely necessary (empty trivially constructible types don't need construction!!)
Similarly for destructors, if the destructor is trivial it means C++ will do no work destructing it... let's not even do the virtual function call to get there.

(also fix a bug where we weren't calling this stuff anyway, and add a test that would have caught that)

Closes #35591

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35591 from ctiller:filter-min 2933152d61
PiperOrigin-RevId: 599521371
2024-01-18 08:55:04 -08:00
Craig Tiller 584c0c0c98 [call-v3] Filter executor (#35533)
A call execution environment for the V3 runtime.

The `CallFilters` class will ultimately be a (private) member of `CallSpine`, and the `StackBuilder` component will be used by a channel when all of the filters it needs are known to allow the call spine to start processing a call.

This is accompanied by a reasonably extensive test suite.

I expect to fine tune semantics, implementation, and tests over the coming weeks/months as we iterate to bring up the rest of the pieces.

Closes #35533

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35533 from ctiller:filters 689c7b527b
PiperOrigin-RevId: 599220150
2024-01-17 10:39:37 -08:00
Craig Tiller c77ba326e9 [gprpp] Implement down_cast - a debug-time checked down cast (#35548)
Closes #35548

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35548 from ctiller:down_cast cd7501dadd
PiperOrigin-RevId: 599176414
2024-01-17 08:09:02 -08:00
Yash Tibrewal 8231340aef [CSM] Add support for GCE resources (#35371)
Add support for GCE resources in CSM Observability.

Additionally, fix a bug where we were not adding the remote workload's canonical service label for unknown resource types.

Also, if zone and region are both specified, zone takes precedence.

Closes #35371

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35371 from yashykt:GceSupportToCsm e3064d8c3c
PiperOrigin-RevId: 597989825
2024-01-12 16:30:16 -08:00
Craig Tiller 9f6789e2b2 [chttp2] Rework settings management to be a c++ type (#35449)
Closes #35449

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35449 from ctiller:h2-settings f67e542a8b
PiperOrigin-RevId: 597970333
2024-01-12 15:21:05 -08:00
Craig Tiller 8a5f659c66 [chaotic-good] Fix fuzzer bug (#35537)
We were getting errors due to insane amounts of padding: enforce limits, fix b/319533934.

Closes #35537

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35537 from ctiller:fff 9f5f31ef27
PiperOrigin-RevId: 597899598
2024-01-12 11:30:22 -08:00
ericsalo a6823b6e97 Fix deprecation warnings from upb (#35510)
Replace "upb:collections" with "upb:message"
Replace "upb/collections" with "upb/message"
Replace "upb:upb" with "upb:mem" and/or "upb:base"
Replace "upb/upb.hpp" with "upb/mem/arena.hpp" and/or "upb/base/status.hpp"

Closes #35510

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35510 from ericsalo:master 112392a050
PiperOrigin-RevId: 597872730
2024-01-12 10:15:56 -08:00
Craig Tiller 02c10df298 [api-fuzzer] Fix inproc channels (#35538)
Closes #35538

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35538 from ctiller:fff2 52b69968a0
PiperOrigin-RevId: 597848114
2024-01-12 08:43:34 -08:00
Yash Tibrewal c5a8e5af64 [OTel] Add back server_selector that got deleted by a merge fiasco (#35532)
It looks like this ended up getting deleted in https://github.com/grpc/grpc/pull/34350 probably when merging.

Also, the `Init` method in the otel test library is getting unwieldy. I'm going to send out a follow-up PR to convert this into a builder instead.

Closes #35532

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35532 from yashykt:OTelPluginBuilderFix 372bf26338
PiperOrigin-RevId: 597846622
2024-01-12 08:33:17 -08:00
Yash Tibrewal acc1ad1b2b [CSM] De-experimentalize CSM OTel Plugin Option (#35526)
Closes #35526

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35526 from yashykt:DeexperimentalizeCsmPluginOption 441eccb857
PiperOrigin-RevId: 597675383
2024-01-11 15:55:26 -08:00
Yash Tibrewal 5880e05e9d [OTel] Fix clang-tidy caused by merge fiasco (#35529)
Closes #35529

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35529 from yashykt:FixMasterClangTidy eb44716ffa
PiperOrigin-RevId: 597660074
2024-01-11 14:50:54 -08:00
Yijie Ma 77ad5a786e [CSM O11Y] CSM Service Label Plumbing from LB Policies to CallAttemptTracer (#35210)
<!--

If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the appropriate
lang label.

-->

Closes #35210

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35210 from yijiem:csm-service-label 6a6a7d1774
PiperOrigin-RevId: 597641393
2024-01-11 13:37:44 -08:00
Yash Tibrewal d2cc24f189 [OTel] De-experimentalize API (#35509)
Closes #35509

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35509 from yashykt:DeeexperimentalizeOTelPlugin f5edf94f2c
PiperOrigin-RevId: 597613307
2024-01-11 11:44:57 -08:00
Yash Tibrewal 6741e2064d [xds interop] Use new CSM Observability APIs (#35508)
Closes #35508

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35508 from yashykt:XdsInteropNewCsm 09d408c082
PiperOrigin-RevId: 597588033
2024-01-11 10:15:04 -08:00
Mark D. Roth a446df61f4 [xDS] remove unnecessary string from XdsConfig struct (#35503)
I realized that this field wasn't actually necessary, since the string is already present in the map key.

Closes #35503

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35503 from markdroth:xds_config_remove_cluster_name 94d5edc133
PiperOrigin-RevId: 597375018
2024-01-10 15:42:43 -08:00
Mark D. Roth aa326c947c [xDS] fix aggregate cluster design (#35313)
Implements the aggregate cluster changes described in gRFC A75 (https://github.com/grpc/proposal/pull/405).

Closes #35313

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35313 from markdroth:xds_aggregate_cluster_fix 85cfd70c59
PiperOrigin-RevId: 597326624
2024-01-10 12:40:56 -08:00
Mark D. Roth 6a4b5ccea3 [SSA] change xds_override_host policy to manage subchannels based on last-used time rather than EDS health state (#35397)
Part of the work needed for in-progress gRFC A75 (https://github.com/grpc/proposal/pull/405).

Closes #35397

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35397 from markdroth:xds_ssa_subchannel_management_revamp 8902deafad
PiperOrigin-RevId: 597288930
2024-01-10 10:28:49 -08:00
Mark D. Roth 20e5b2c4bb [xDS] read connection idle timeout from CDS resource (#35395)
Part of the work needed for in-progress gRFC A75 (https://github.com/grpc/proposal/pull/405).

Closes #35395

PiperOrigin-RevId: 597064473
2024-01-09 15:41:22 -08:00
Eugene Ostroukhov e73b76a7da [Test] Fix bug in waiting for the Orca OOB report (#35467)
Make sure there is no unnecessary delays when there are multiple reports in the queue.

This change also adds a test for the custom LB policy.

Closes #35467

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35467 from eugeneo:tasks/orca-test-timeout-316026521 4aab50a118
PiperOrigin-RevId: 597007131
2024-01-09 12:21:45 -08:00
Yash Tibrewal 5888738c5a [OTel] Add a PluginOption API (#35434)
Closes #35434

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35434 from yashykt:OTelPluginOption 1db870bed4
PiperOrigin-RevId: 596966190
2024-01-09 18:28:43 +00:00
Mark D. Roth 0df505d9db [api_fuzzer] Cap the number of comma-delimited strings in the target URI to 1K.
PiperOrigin-RevId: 596739224
2024-01-08 16:54:43 -08:00
Craig Tiller 86522af60d [chaotic-good] Client & server transport (#35400)
Adapts work from https://github.com/grpc/grpc/pull/34728 and previous changes from @nanahpang, implements new v3 filter/transport interface, and brings up the core of the chaotic good transport.

The next change will bring a more complete test suite (for this and inproc).

Closes #35400

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35400 from ctiller:v3-server 737ca5431a
PiperOrigin-RevId: 596621152
2024-01-08 09:46:16 -08:00