Commit Graph

2947 Commits

Author SHA1 Message Date
veblush f76f2493ab Automated change: Fix sanity tests 2024-04-09 19:31:39 +00:00
Craig Tiller b0cf42d86e [clang-format] Remove requirement that port_platform.h is at the top (#36281)
Closes #36281

PiperOrigin-RevId: 623176865
2024-04-09 08:58:34 -07:00
Craig Tiller b361574a0b Fix fuzzer builds internally for core e2e tests
The build layout I chose in https://github.com/grpc/grpc/pull/36198 causes problems for Google's internal build systems. This change fixes things so that link order is correct - unfortunately at the cost of adding 90-some C++ compilations.

I've made a new .cc file that is as minimal as possible to reduce the cost of those compilations down as far as I can.

PiperOrigin-RevId: 623012695
2024-04-08 18:50:44 -07:00
Yash Tibrewal 70839a9b19 [OTel C++] Add experimental optional locality label available to client per-attempt metrics (#36254)
As per https://github.com/grpc/proposal/pull/419, the experimental optional label `grpc.lb.locality` is added to the follow per-call metrics -
* grpc.client.attempt.duration
* grpc.client.attempt.sent_total_compressed_message_size
* grpc.client.attempt.rcvd_total_compressed_message_size

Closes #36254

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36254 from yashykt:OTelOptionalLabelsOnPerCall c5390c99a1
PiperOrigin-RevId: 622973959
2024-04-08 15:52:59 -07:00
Craig Tiller f45b9a8f3d [build] Deduplicate compilation of core e2e tests (#36198)
Closes #36198

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36198 from ctiller:e2edup 38e1d82c1d
PiperOrigin-RevId: 620955836
2024-04-01 14:44:01 -07:00
Yijie Ma c54c69dcdd [Metrics] New OpenTelemetry Plugin Implementation of Stats Plugin (#36070)
<!--

If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the appropriate
lang label.

-->

Closes #36070

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36070 from yijiem:grpc-metrics 72653727b1
PiperOrigin-RevId: 618529035
2024-03-23 19:37:26 -07:00
Yash Tibrewal 041c283ccc [experiments] Fix initialization order fiasco (#36168)
Internal ref : b/324720620

Closes #36168

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36168 from yashykt:FixStaticOrderInitFiasco 14f2a89503
PiperOrigin-RevId: 618328195
2024-03-22 17:32:17 -07:00
Yijie Ma 2d88de979e [test] Use std::regex_search in no_logging test (#36131)
<!--

If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the appropriate
lang label.

-->

Closes #36131

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36131 from yijiem:no-logging f3f638176f
PiperOrigin-RevId: 617630514
2024-03-20 14:47:48 -07:00
Yash Tibrewal bb9f4466e1 [api-fuzzer] Remove ping support (#36146)
Pings were removed from the Core API in https://github.com/grpc/proposal/pull/203

Closes #36146

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36146 from yashykt:DisablePingFromApiFuzzer e9a783aecb
PiperOrigin-RevId: 617014710
2024-03-18 18:43:26 -07:00
Yash Tibrewal 424e95ccbb [Max Message Limits] Improve logging to determine client/server (#36103)
Will help debugging errors like #35805

Closes #36103

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36103 from yashykt:HelpDebugMessageSizeLimits 5507a257f3
PiperOrigin-RevId: 615564790
2024-03-13 15:36:00 -07:00
Yijie Ma 88585c43e9 [test] Allow channel-level "Sending goaway" log in no_logging test (#36100)
<!--

If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the appropriate
lang label.

-->

Closes #36100

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36100 from yijiem:no-logging 970a640475
PiperOrigin-RevId: 615299752
2024-03-12 23:30:13 -07:00
Mark D. Roth 0213523907 [build] move channel out of grpc_base (#35924)
This adds the following new targets:
- `channel`: A virtual interface for a channel.
- `legacy_channel`: A channel implementation that supports the filter stack and call v2.
- `channel_create`: A standalone function to create a channel.
- `server_interface`: A base class with a few accessor methods used in surface/call.cc.
- `server`: The actual server implementation.
- `api_trace`, `call_tracer`, `server_call_tracer_filter`, `call_finalization`: These were split out of `grpc_base` to avoid various dependency problems.
- `compression`: This is a combination of the previously existing `compression_internal` target and the compression code that was part of `grpc_base`.

Closes #35924

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35924 from markdroth:channel_interface 94a7fffddb
PiperOrigin-RevId: 612512438
2024-03-04 11:07:17 -08:00
AJ Heller fc412cd390 [test] Add comment about workaround (re #35969) (#35971)
Followup from #35969

This also fixes a few scenarios we had not caught earlier.

Closes #35971

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35971 from drfloob:doc 32de2d7033
PiperOrigin-RevId: 609509072
2024-02-22 14:58:27 -08:00
AJ Heller ab8a756b1d [test] Fix use-after-free in http proxy fixture (#35968)
See MSAN failure here: https://source.cloud.google.com/results/invocations/f4240bd5-960a-4736-95ba-b9408979f8e0/targets/%2F%2Ftest%2Fcore%2Fend2end:cancel_after_invoke_test@poller%3Depoll1/log

Closes #35968

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35968 from drfloob:fix-http_fixture-proxy-unref-early 5c432aff07
PiperOrigin-RevId: 609494949
2024-02-22 14:17:11 -08:00
AJ Heller 6fb93e79c2 [test] Decrease ping timeout on CancelAfterInvoke tests (#35969)
Supersedes #35967

Closes #35969

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35969 from drfloob:land/35967 30a6947343
PiperOrigin-RevId: 609471457
2024-02-22 13:05:39 -08:00
Yijie Ma 9db40fa845 [EventEngine] Enable the EventEngine DNS Resolver on Posix (#35573)
<!--

If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the appropriate
lang label.

-->

Closes #35573

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35573 from yijiem:enable-oss-ee-dns-posix-real 1df91d1d84
PiperOrigin-RevId: 609193851
2024-02-21 18:35:54 -08:00
youyuanwu a58f83c947 [EventEngine] Support AF_UNIX for windows (#34801)
#22285
Unix domain socket has been added to windows: [AF_UNIX comes to Windows](https://devblogs.microsoft.com/commandline/af_unix-comes-to-windows/)

Golang net pkg has adopted this long ago:
https://go-review.googlesource.com/c/go/+/125456
https://go-review.googlesource.com/c/sys/+/132555
grpc-go already support this.

AF_UNIX on windows is seamlessly integrated with winsock API.
The modification needed are:
* Set the right address family AF_UNIX depending on the address config, instead of using AF_INET6 all the time.
* Ignore socket options for tcp.

Closes #34801

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/34801 from youyuanwu:dev 3d9b5c097b
PiperOrigin-RevId: 607083164
2024-02-14 13:15:57 -08:00
Mark D. Roth 17d9e20ff1 [slice] use absl base64 APIs instead of slice base64 APIs (#35851)
Closes #35851

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35851 from markdroth:base64 6caf83d3da
PiperOrigin-RevId: 606759013
2024-02-13 15:05:50 -08:00
Mark D. Roth 41606054c2 [load_file] remove grpc_load_file() in favor of grpc_core::LoadFile() (#35857)
Closes #35857

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35857 from markdroth:highlander 7b7d95aaa9
PiperOrigin-RevId: 605742734
2024-02-09 15:16:24 -08:00
Craig Tiller d1cb0c0874 [chaotic-good] Bring up core e2e tests (#35663)
Closes #35663

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35663 from ctiller:shush-e2e 0c1f82ed71
PiperOrigin-RevId: 605717663
2024-02-09 13:34:46 -08:00
Mark D. Roth 21cb320080 [reorg] move service config code to src/core/service_config (#35843)
Closes #35843

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35843 from markdroth:client_channel_reorg4 0c50ada6f9
PiperOrigin-RevId: 605466874
2024-02-08 16:58:20 -08:00
Mark D. Roth 10e83973e7 [reorg] move resolver code to src/core/resolver (#35804)
This new directory combines code from the following locations:
- src/core/ext/filters/client_channel/resolver
- src/core/lib/resolver

Closes #35804

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35804 from markdroth:client_channel_resolver_reorg2 30660e6b00
PiperOrigin-RevId: 604665835
2024-02-06 09:07:56 -08:00
Mark D. Roth 148f59c15a [reorg] move LB policy code to src/core/load_balancing (#35786)
This new directory combines code from the following locations:
- src/core/ext/filters/client_channel/lb_policy
- src/core/lib/load_balancing

Closes #35786

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35786 from markdroth:client_channel_resolver_reorg 98554efb98
PiperOrigin-RevId: 604351832
2024-02-05 10:05:35 -08:00
AJ Heller 71fa68f7fb [EventEngine] Add QueryExtension Interfaces for *SupportsFd (#35648)
This PR:
* adds FD extensions to the public headers
* Adds the query extension interface to EventEngine, Listener, and Endpoint, via a new `Extensible` interface
* Refactors the PosixEventEngine to use the Extensible interface.

Closes #35648

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35648 from drfloob:posix-ee-query-interface 7cae28e0b3
PiperOrigin-RevId: 601794970
2024-01-26 10:03:18 -08:00
Craig Tiller 19ad49d169 [bazel] disable fuzzer on mac (#35662)
Closes #35662

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35662 from ctiller:nononono 4f7d2fb893
PiperOrigin-RevId: 601448944
2024-01-25 08:03:46 -08:00
Craig Tiller 870a66d9a0 [chaotic-good] Connection setup & test suites (#35650)
Closes #35650

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35650 from ctiller:shush-tsan fd55ea1be3
PiperOrigin-RevId: 601221780
2024-01-24 13:46:08 -08:00
Craig Tiller 6b6ce36f74 [fuzzing] Allow server fuzzer to take multiple connection requests at the same time (#35645)
Closes #35645

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35645 from ctiller:shush-multipass e63e1e5ddb
PiperOrigin-RevId: 600998039
2024-01-23 21:19:49 -08:00
Craig Tiller 7507e3b644 [fuzzing] Teach server_fuzzer about the chaotic good protocol (#35643)
No explicit testing yet, just some building blocks

Closes #35643

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35643 from ctiller:shush-chaotic-protocol 48dcaa0632
PiperOrigin-RevId: 600928555
2024-01-23 15:43:34 -08:00
Craig Tiller 19f1b39829 [fuzzer] Eliminate use of mock_endpoint in server fuzzer (#35625)
Instead, connect a real client, which gives us the ability to exercise the listener portion of the code too.

Closes #35625

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35625 from ctiller:shush-connect b8f78993c6
PiperOrigin-RevId: 600836565
2024-01-23 10:40:03 -08:00
Craig Tiller e93f2dfbef [fuzzer] Separate scheduling from schedule creation for network input (#35618)
Closes #35618

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35618 from ctiller:shush ed7afaa4e8
PiperOrigin-RevId: 600557159
2024-01-22 13:43:52 -08:00
Yijie Ma 5bf0971972 [EventEngine] Fix issues found when enabling `event_engine_dns` experiment in OSS (#35530)
Using `AF_UNSPEC` for both IPv4 and IPv6 queries does not work in all cases. Specifically, for `localhost:<>`, c-ares only returns the IPv6 record i.e. `::1`.

<!--

If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the appropriate
lang label.

-->

Closes #35530

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35530 from yijiem:enable-oss-ee-dns-posix 452b5a2d81
PiperOrigin-RevId: 599989537
2024-01-19 18:42:26 -08:00
Craig Tiller 02c10df298 [api-fuzzer] Fix inproc channels (#35538)
Closes #35538

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35538 from ctiller:fff2 52b69968a0
PiperOrigin-RevId: 597848114
2024-01-12 08:43:34 -08:00
Yijie Ma 77ad5a786e [CSM O11Y] CSM Service Label Plumbing from LB Policies to CallAttemptTracer (#35210)
<!--

If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the appropriate
lang label.

-->

Closes #35210

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35210 from yijiem:csm-service-label 6a6a7d1774
PiperOrigin-RevId: 597641393
2024-01-11 13:37:44 -08:00
Yash Tibrewal 5888738c5a [OTel] Add a PluginOption API (#35434)
Closes #35434

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35434 from yashykt:OTelPluginOption 1db870bed4
PiperOrigin-RevId: 596966190
2024-01-09 18:28:43 +00:00
Mark D. Roth 0df505d9db [api_fuzzer] Cap the number of comma-delimited strings in the target URI to 1K.
PiperOrigin-RevId: 596739224
2024-01-08 16:54:43 -08:00
Craig Tiller fede9051e1 [call-v3] Begin migration of channel_init to templates.
Eventually for call-v3 we're going to want to have registration of filters generate the appropriate glue into the channel runtime to execute a call.

Begin that process now and gradually by introducing the new syntax and allowing a piecemeal migration to it - by the time we're done converting filters to the v3 APIs we'll also have the registration piece done.

PiperOrigin-RevId: 596013927
2024-01-05 09:34:36 -08:00
Craig Tiller 03403724f5 [fuzzing] Add ability for api-fuzzer to create inproc channels (#35277)
Closes #35277

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35277 from ctiller:inproc-fuzzing 853c11f144
PiperOrigin-RevId: 590420030
2023-12-12 19:55:27 -08:00
Gregory Cooke 12c0058bf2 [Security - Testing] OpenSSL 1.0.2 and 1.1.1 CI Testing (#35236)
Enable OpenSSL 1.0.2 tests and add a container for 1.1.1 so that it is tested during portability testing as well.

Closes #35236

PiperOrigin-RevId: 590345568
2023-12-12 14:52:43 -08:00
Craig Tiller ff63ad9413 [call-v3] Convert message compression filter (#35269)
Closes #35269

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35269 from ctiller:cg-compress 0304821f07
PiperOrigin-RevId: 590010564
2023-12-11 17:58:00 -08:00
Craig Tiller a7e90045db [promises] Fix fuzzer found uncancelable call bug in client promise code (#34909)
Closes #34909

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/34909 from ctiller:lambda-mcfly 92f759d763
PiperOrigin-RevId: 589989462
2023-12-11 16:35:55 -08:00
Craig Tiller 2b1d06f037 [fuzzing] Add ability to listen to ports to api-fuzzer (#35273)
Unsure when this got dropped!

Closes #35273

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35273 from ctiller:server-port be94e30a06
PiperOrigin-RevId: 589919718
2023-12-11 13:04:19 -08:00
Craig Tiller 9eadf42dbf [promises] Add an API to access new style filters (#35200)
Will be used during this transition time to run 5-pipe style filters somewhat more natively. Once everything is getting closer to 5-pipes, we'll drop this method and have the channel stack understand how to create an interception-map that can be reused per-call, instead of creating the interception-map every time a call is created.

Closes #35200

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35200 from ctiller:cg-channel-filter-api 2fc11dd273
PiperOrigin-RevId: 587940947
2023-12-04 22:12:37 -08:00
Craig Tiller 501b895736 [fuzzing-heck] Fix a bug that comes up with promises + work serializer dispatch (#35196)
b/310341170

I'm kind of proud of our testing for finding this

Closes #35196

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35196 from ctiller:fuzz-no 28c95606f9
PiperOrigin-RevId: 587798657
2023-12-04 12:06:19 -08:00
AJ Heller a215eb6717 [test] Sleep between server restarts in retry_transparent_max_concurrent_streams (#35149)
This hack temporarily quiets the flaky test report for a known race.

This is the only end2end test that shuts down & restarts a server in the same test execution. The PosixEventEngine's Listener implementation asynchronously shuts down listening ports after Listener destruction. Some changes can possibly be made here to only proceed in server restart after the `on_shutdown` callback is called, ensuring all ports are closed before proceeding.

Closes #35149

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35149 from drfloob:hack/max_concurrent_fix_for_posix_ee_listener 9a7b7b53dd
PiperOrigin-RevId: 586471281
2023-11-29 15:38:03 -08:00
Craig Tiller f4219440c6
[server] Fix fuzzing detected memory leak (#34908) 2023-11-09 10:07:45 -08:00
Vignesh Babu 77e9fe49aa
Revert "[security] Support a list of SSL ECs" (#34900)
Breaks internal tests
2023-11-08 14:05:02 -08:00
Mark D. Roth 8a000f45f8
[grpclb and fake resolver] clean up e2e tests and simplify fake resolver (#34887)
Changes to fake resolver:
- Add `WaitForReresolutionRequest()` method to fake resolver response
generator to allow tests to tell when re-resolution has been requested.
- Change fake resolver response generator API to have only one mechanism
for injecting results, regardless of whether the result is an error or
whether it's triggered by a re-resolution.

Changes to grpclb_end2end_test:
- Change balancer interface such that instead of setting a list of
responses with fixed delays, the test can control exactly when each
response is set.
- Change balancer impl to always send the initial LB response, as
expected by the grpclb protocol.
- Change balancer impl to always read load reports, even if load
reporting is not expected to be enabled. (The latter case will still
cause the test to fail.) Reads are done in a different thread than
writes.
- Allow each test to directly control how many backends and balancers
are started and the client load reporting interval, so that (a) we don't
waste resources starting servers we don't need and (b) there is no need
to arbitrarily split tests across different test classes.
- Add timeouts to `WaitForAllBackends()` functionality, so that tests
will fail with a useful error rather than timing out.
- Improved ergonomics of various helper functions in the test framework.

In the process of making these changes, I found a couple of bugs:
- A bug in pick_first, which I fixed in #34885.
- A bug in grpclb, in which we were using the wrong condition to decide
whether to propagate a re-resolution request from the child policy,
which I've fixed in this PR. (This bug probably originated way back in
#18344.)

This should address a lot of the flakes seen in grpclb_e2e_test
recently.
2023-11-06 14:42:43 -08:00
Luwei Ge eaa72802a0
[security] Support a list of SSL ECs (#34867)
Addresses #23235
2023-11-03 14:17:28 -07:00
AJ Heller c85cdfe656
[test] Fix http_proxy end2end test fixture (#34838)
Built upon @Vignesh2208 's work in #33156

This adds ref counting to the http_proxy fixture object, fixing test
flakes identified by the introduction of EventEngine listeners. Proxy
objects were either being deleted twice, or sometimes not at all,
resulting in two different sorts of flakes.
2023-11-01 12:13:28 -07:00
Yash Tibrewal 609e96446a
[http2] Only run http2_stats test if http2_stats_fix experiment is enabled (#34833) 2023-10-31 10:26:11 -07:00