Commit Graph

8140 Commits

Author SHA1 Message Date
Vignesh Babu 087b74f739
Revert "Revert "[interop] Add absl dependency to interop server"" #33828 (#33830)
Reverts https://github.com/grpc/grpc/pull/33676

The serve method needs to be called with args. The previous attempt did
not change the signature in commands.py which lead to errors such as
https://screenshot.googleplex.com/6wZVER9ZETMGAmA.

The fix is in
4e211d0291
2023-07-24 12:02:02 -07:00
Paulo Castello da Costa 457fd65b93
[Test] Fix load test configuration generation for `node`. (#33821)
Scenarios for language `node` specify the server language as `node`
(instead of leaving it blank), so a flag must be added to
`--allow_server_language=node`.

Scenarios for language `node_purejs` differ in name and in scenario
settings, but otherwise run on identical clients and servers. This
change treats `node_purejs` as `node` for the purpose of generating load
test configurations.
2023-07-24 00:29:37 -07:00
Vignesh Babu f85b7c79ee
[experiments] Fix processing of platform specific test tags (#33749)
Also adds a unit test: experiments_tag_test which should fail if the
appropriate tags are not set for it.
2023-07-21 16:06:01 -07:00
apolcyn d4cbb7a96d
[ruby] fix crash when prefork/postfork is used without previously using grpc (#33788)
Should fix https://github.com/grpc/grpc/issues/33787
2023-07-20 11:26:02 -07:00
AJ Heller 2c81c5619e
[deps] Remove libuv dependency (#33748) 2023-07-19 10:56:11 -07:00
Craig Tiller 0853003d84
[python] It should be an error to have unformatted code (#33752)
`black` doesn't generate an error for the `--diff` command line
argument, but does for `--check` - so that's the one we should use in
sanity checks.
2023-07-18 13:46:07 -07:00
Richard Belleville d714ae72e7
[dependency] Restrict cython to less than 3.X (#33738)
This should resolve breakage on master caused by the jump to Cython
3.0.0 this morning.
2023-07-18 12:56:16 -07:00
Sergii Tkachenko d779808e01
[PSM Interop] Fix PyYAML Cython build / Upgrade PyYAML to 6.0 (#33745)
Upgrades PyYAML from 5.4.1 to 6.0 to address cython build issue:
https://github.com/yaml/pyyaml/issues/601.

Changelog: https://github.com/yaml/pyyaml/blob/master/CHANGES
2023-07-17 18:40:54 -07:00
AJ Heller 107e4dc0b6
[deps] Upgrade google/benchmark dependency (#33736)
Some command line arguments have changed, and the public docs no longer
describe how to use the 1-year-old version.
2023-07-17 15:45:57 -07:00
github-actions[bot] c96f3dce4e
Automated fix for refs/heads/master (#33710)
PanCakes to the rescue!

We noticed that our 'sanity' test was going to fail, but we think we can
fix that automatically, so we put together this PR to do just that!

If you'd like to opt-out of these PR's, add yourself to NO_AUTOFIX_USERS
in .github/workflows/pr-auto-fix.yaml

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2023-07-14 22:01:09 +00:00
AJ Heller c81bdf7a83
[Protobuf] Upgrade third_party/protobuf to 23.4 (#33695)
This was done manually due to a problem with
`tools/distrib/python/make_grpcio_tools.py`. ~I fixed it in this PR
(depends on cl/547979185), so there is a fair chance this upgrade will
work normally for the next release.~ The fix may be problematic for
upgrading protobuf on older release branches, so the improvement will be
worked on separately. CC @jtattermusch

This also updates the UPB dep to the latest commit on the 23.x branch.
2023-07-14 12:04:40 -07:00
AJ Heller 5e3fe08f37
[BoringSSL] Update third_party/boringssl-with-bazel (#33690)
Change was created by the release automation script. See go/grpc-release
2023-07-13 15:38:12 -07:00
apolcyn 82fb0b1676
[ruby fork support] Revert "[ruby fork support] skip new fork tests since they're flaky at introduction #33664" (#33666)
Followup to https://github.com/grpc/grpc/pull/33664 after fixing bugs causing test flakes
2023-07-13 15:04:40 -07:00
Vignesh Babu c9eeca9681
Revert "[interop] Add absl dependency to interop server" (#33676)
Reverts grpc/grpc#33647
2023-07-12 16:21:46 -07:00
apolcyn b32d94de05
[ruby fork support] skip new fork tests since they're flaky at introduction (#33664)
These tests have been flaky since they were added in
https://github.com/grpc/grpc/pull/33430
2023-07-12 12:02:11 -07:00
Richard Belleville eb7b72ea5b
[PSM Interop] Loosen affinity test assertions (#33648)
As part of the dualstack backend designs, subchannels will be created
lazily. Therefore, instead of asserting that there is 1 READY subchannel
and `n - 1` IDLE subchannels, we just assert that there is 1 READY
subchannel.
2023-07-12 11:26:01 -07:00
Vignesh Babu fa95c4c439
[interop] Add absl dependency to interop server (#33647)
<!--

If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the
appropriate
lang label.

-->
2023-07-11 15:02:17 -07:00
apolcyn fe11ec9bcb
[ruby] experimental client side fork support (#33430)
Adds experimental fork support to gRPC/Ruby

Works towards https://github.com/grpc/grpc/issues/8798 (see caveats for why this wasn't marked fixed yet)
Works towards https://github.com/grpc/grpc/issues/33578 (see caveats for why this wasn't marked fixed yet)

This leverages existing `pthread_atfork` based C-core support for
forking that python/php use, but there's a bit extra involved mainly
because gRPC/Ruby has additional background threads.

New tests under `src/ruby/end2end` show example usage.

Based on https://github.com/grpc/grpc/pull/33495

Caveats:
- Bidi streams are not yet supported (bidi streams spawn background
threads which are not yet fork safe)
- Servers not supported
- Only linux supported
2023-07-10 13:34:10 -07:00
Vignesh Babu 974798a427
[tracing] Fix flakiness in tcp_posix_test (#33639)
tcp_posix_test is incorrectly assuming that all endpoint_writes with
timestamps enabled will be successfully traced. Remove the timestamps
checking related tests to prevent flakes when the test is enabled
internally.
2023-07-10 12:59:57 -07:00
nanahpang 0cc9d16e9c
[chaotic-good] Implement a promise-based endpoint for chaotic-good transport to read & write to EventEngine::Endpoint. (#33257)
This PR is continuing the work of prototyping in
https://github.com/grpc/grpc/pull/31592, and the design doc is at
[link](https://docs.google.com/document/d/1vRy0yse-d1heLQRmLPo_0figsTPXJAnNN84tBCAne_s/edit?pli=1&resourcekey=0-JvUPdq0LaZq8gMkgT9Pzlw#heading=h.qgvc5vr55ytg).

<!--

If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the
appropriate
lang label.

-->
2023-07-06 17:18:48 -07:00
Yash Tibrewal 9984f1bd5b
Revert "Revert "Revert "Revert "[HTTP2] Fix inconsistencies in keepalive configuration ( #33428)" (#33512)"" (#33601)
Reverts grpc/grpc#33599

Needs to be cherry-picked
2023-07-06 15:13:12 -07:00
Sergii Tkachenko 8101a808ec
[PSM Interop] Set canonical server via canonical-* tag, not commit sha (#33588)
While this is less transparent, it makes it significantly easier to
deliver images with latest security patches.
2023-07-05 17:17:44 -07:00
Craig Tiller c5bb43ab61
[chttp2] Eliminate grpc_chttp2_stream_map (#33503)
No need for a bespoke type anymore... and a step along the path to
C++ification.

---------

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2023-07-05 15:07:31 -07:00
Yash Tibrewal 523d9a2135
Revert "Revert "Revert "[HTTP2] Fix inconsistencies in keepalive configuration ( #33428)" (#33512)" (#33599)
Reverts grpc/grpc#33586

Looks like this has to be cherry-picked
2023-07-04 19:25:17 -07:00
Sergii Tkachenko e9446ca20c
[PSM Interop] Bump URL Map canonical server from 1.40.0 to 1.56.0 (#33587)
Similar to https://github.com/grpc/grpc/pull/33542.

Note that there's a ticket to automatically use the one specified in the
--server_image_canonical flag, but for now we just hardcode.

ref b/261911148, b/282106799.
2023-06-30 17:08:59 -07:00
Yash Tibrewal 8d9164fc16
Revert "Revert "[HTTP2] Fix inconsistencies in keepalive configuration ( #33428)" (#33512) (#33586)
Along with an experiment this time

<!--

If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the
appropriate
lang label.

-->
2023-06-30 16:27:30 -07:00
Eugene Ostroukhov e0bc8a2c85
[xDS LB] xDS pick first support (#33540) 2023-06-30 14:06:08 -07:00
Vignesh Babu bf3ffcf600
[experiments] A unit test for the experiments framework (#33334)
Adds a test for the experiments codegen. It updates the codegen to parse
test_experiments.yaml and test_experiments_rollouts.yaml files and
generate test_experiments.h and test_experiments.cc files along with an
experiments_test.cc file. The experiments test verifies the returned
value of IsExperimentEnabled with the expected value.
2023-06-29 12:21:11 -07:00
Yash Tibrewal 98417f3bd0
Revert "Revert "[otel] Add bazel dependency"" (#33560)
Reverts grpc/grpc#33559
2023-06-27 12:28:47 -07:00
Mark D. Roth 017153a0c5
Revert "[otel] Add bazel dependency" (#33559)
Reverts grpc/grpc#33548
2023-06-27 10:34:45 -07:00
Yash Tibrewal 875b7fdcff
[otel] Add bazel dependency (#33548)
Add bazel dependency on opentelemetry-cpp.
<!--

If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the
appropriate
lang label.

-->
2023-06-27 00:47:01 -07:00
Sergii Tkachenko 2878e85ef5
[PSM Interop] Bump the canonical server from v1.48.1 to v1.56.0 (#33542)
### From:

* Closest tag: `v1.48.1`
* Branch: https://github.com/grpc/grpc-java/commits/v1.48.x
* Commit: grpc/grpc-java@d56f8fbe1d
* Image:
gcr.io/grpc-testing/xds-interop/java-server:d56f8fbe1d2822bc4f91515dd471ad49493fc385


### To:

* Closest tag: `v1.56.0`
* Branch: https://github.com/grpc/grpc-java/commits/v1.56.x
* Commit: grpc/grpc-java@558b5b0bfa
* Image:
gcr.io/grpc-testing/xds-interop/java-server:558b5b0bfac8e21755c223063274a779b3898afe
2023-06-26 14:29:58 -07:00
github-actions[bot] dde1291a97
Automated fix for refs/heads/master (#33546)
PanCakes to the rescue!

We noticed that our 'sanity' test was going to fail, but we think we can
fix that automatically, so we put together this PR to do just that!

If you'd like to opt-out of these PR's, add yourself to NO_AUTOFIX_USERS
in .github/workflows/pr-auto-fix.yaml

Co-authored-by: HannahShiSFB <HannahShiSFB@users.noreply.github.com>
2023-06-26 12:46:19 -07:00
alto-ruby 059144abd6
[Ruby] add aarch64 linux build (#33079)
This adds pre-built library for aarch64 linux, will help improve the
install speed and avoid building environment issues at customer side.

@apolcyn @jtattermusch Can you help build and push the new rake compiler
image?
Will update the tag and hash after the image is available

Manually tested locally:
```
uname -a
Linux u20 5.15.49-linuxkit #1 SMP PREEMPT Tue Sep 13 07:51:32 UTC 2022 aarch64 aarch64 aarch64 GNU/Linux
```
```
time gem install /work/ruby/grpc/pkg/grpc-1.56.0.dev-aarch64-linux.gem
Successfully installed grpc-1.56.0.dev-aarch64-linux
Parsing documentation for grpc-1.56.0.dev-aarch64-linux
Installing ri documentation for grpc-1.56.0.dev-aarch64-linux
Done installing documentation for grpc after 0 seconds
1 gem installed

real	0m22.794s
user	0m17.268s
sys	0m5.156s
```
```
ruby greeter_server.rb &
[1] 319
ruby greeter_client.rb
"Greeting: Hello world"
```

Fixes:
https://github.com/grpc/grpc/issues/31855
https://github.com/grpc/grpc/issues/29489
2023-06-25 13:00:40 -07:00
Eugene Ostroukhov 297db35a58
[third_party] update @envoy-api (#33539)
Updates to head, required to support xDS configuration for the
pick-first
2023-06-23 15:38:33 -07:00
Yash Tibrewal ba878c804d
Revert "[HTTP2] Fix inconsistencies in keepalive configuration (#33428)" (#33512)
This reverts commit e107ff5e99.




<!--

If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the
appropriate
lang label.

-->
2023-06-21 15:49:35 -07:00
Vignesh Babu cd4ff81b3f
Revert "Revert "[resource quota] Fix bugs in iomgr and event engine endpoint interactions with resource quota"" (#33499)
Reverts grpc/grpc#33417

Deadlock
https://fusion2.corp.google.com/invocations/99834386-79ff-4707-86eb-52e604774ea9/details
fixed in the c9a1bdc3dc commit.
2023-06-21 09:36:44 -07:00
Yash Tibrewal f8dd1a05dc
[http2] Rename configuration_test to ping_configuration_test (#33500)
<!--

If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the
appropriate
lang label.

-->
2023-06-20 14:19:59 -07:00
Sergii Tkachenko fb9e927322
[PSM Interop] Outlier detection: use native cpp server since v1.57.x (#33135)
Related:
- CPP implementation: be99673d06,
d55431995c.
- Feature request in go: https://github.com/grpc/grpc-go/issues/6288
- Feature request in python: https://github.com/grpc/grpc/issues/33134

FYI @eugeneo @zasweq

---------

Co-authored-by: Eugene Ostroukhov <eostroukhov@gmail.com>
2023-06-20 13:32:53 -07:00
Yash Tibrewal e107ff5e99
[HTTP2] Fix inconsistencies in keepalive configuration (#33428)
Noticed some inconsistencies in our keepalive configuration - 

* Earlier, even if keepalive pings were disabled, we would be scheduling
keepalive pings at an interval of INT_MAX ms.
* We were not using `g_default_client_keepalive_permit_without_calls` /
`g_default_server_keepalive_permit_without_calls`. They are both false
by default but they can be overridden in
`grpc_chttp2_config_default_keepalive_args`.

<!--

If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the
appropriate
lang label.

-->
2023-06-17 17:15:47 -07:00
Eugene Ostroukhov d55431995c
[interop] Implement "hostname" for RPC behavior (#33446)
This enables outlier detection test. See #33135
2023-06-16 11:14:01 -07:00
Craig Tiller 1bfcca66fe
Revert "[testing] Adjust port selection algorithm on RBE" (#33448)
Reverts grpc/grpc#33429
2023-06-14 14:51:23 -07:00
Sergii Tkachenko 181a24f546
[PSM Interop] Pip freeze breaks down into requirements and their deps (#33426)
This is a no-op, just reordering `requirements.lock`.

By providing `-r requirements.txt` to `pip freeze` it's able to break up
dependencies required via `requirements.txt`, and sub-dependencies
installed to satisfy them.
2023-06-14 13:15:29 -07:00
Craig Tiller 969c228934
[testing] Adjust port selection algorithm on RBE (#33429)
I've got a hypothesis that we're losing isolation between test shards
right now for "some reason".

This is a change to reflect test sharding in the port distribution that
we use, in an attempt to alleviate that.

---------

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2023-06-14 12:21:59 -07:00
Craig Tiller 80dbe90c18
Revert "[resource quota] Fix bugs in iomgr and event engine endpoint interactions with resource quota" (#33417)
Reverts grpc/grpc#33375

Breaks import
2023-06-13 08:11:53 -07:00
Craig Tiller 7e6606f5a6
[windows] Add a check for too long path names (#33418)
We should probably cap this so that our customers have a chance of
cloning the repository.

---------

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2023-06-13 07:41:06 -07:00
Vignesh Babu e6c1b13aed
[resource quota] Fix bugs in iomgr and event engine endpoint interactions with resource quota (#33375)
The following bugs are fixed:

* Missing ExecCtx in event engine endpoints and listeners
* Ref counting issue with iomgr endpoint which causes crashes in
overloaded situations

The PR includes a test which triggers these bugs by simulating an
overloaded system.
2023-06-12 16:20:08 -07:00
apolcyn 017786f151
[c-ares] Upgrade c-ares dependency to 1.19.1 (#33392)
WIP
2023-06-12 13:38:20 -07:00
Esun Kim 076f719286
Revert "Revert "[Test] Dedup vs2019 test in the Windows test" (#33393)" (#33394) 2023-06-09 16:10:29 -07:00
Sergii Tkachenko de6ed9ba9f
[Python] Migrate from yapf to black (#33138)
- Switched  from yapf to black
- Reconfigure isort for black
- Resolve black/pylint idiosyncrasies 

Note: I used `--experimental-string-processing` because black was
producing "implicit string concatenation", similar to what described
here: https://github.com/psf/black/issues/1837. While currently this
feature is experimental, it will be enabled by default:
https://github.com/psf/black/issues/2188. After running black with the
new string processing so that the generated code merges these `"hello" "
world"` strings concatenations, then I removed
`--experimental-string-processing` for stability, and regenerated the
code again.

To the reviewer: don't even try to open "Files Changed" tab 😄 It's
better to review commit-by-commit, and ignore `run black and isort`.
2023-06-09 15:08:55 -07:00