Commit Graph

579 Commits

Author SHA1 Message Date
Jan Tattermusch 785b85426c
Upgrade abseil to 20230125.2 (#32733)
To get a fix for
https://github.com/abseil/abseil-cpp/issues/1419#issuecomment-1485282854
2023-03-28 20:38:56 +02:00
Yousuk Seung 16c03db9ac
Revert "Revert "WRR: Support EPS" (#32723)" (#32725)
This reverts commit 7bd9267f32.




<!--

If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the
appropriate
lang label.

-->
2023-03-27 16:49:50 -07:00
Stanley Cheung 39780ba2b5
Update third_party/boringssl-with-bazel (#32721)
Change was created by the release automation script. See
go/grpc-release.
2023-03-27 15:11:48 -07:00
Esun Kim 7bd9267f32
Revert "WRR: Support EPS" (#32723)
Reverts grpc/grpc#32657
2023-03-27 11:53:17 -07:00
Yousuk Seung 4429066516
WRR: Support EPS (#32657)
<!--

If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the
appropriate
lang label.

-->
2023-03-24 16:23:51 -07:00
Jan Tattermusch a27b86fb95
Switch linux RBE to ubuntu18.04, get rid of rbe_autoconfig (#32559)
First step in the modernization of our RBE stack (see
go/rbe-tech-debt-notes).

- Get rid of the deprecated rbe_autoconfig and start using
[rbe_configs_gen](https://github.com/bazelbuild/bazel-toolchains#rbe_configs_gen---cli-tool-to-generate-configs)
+ check in the generated toolchain configs.
- Switch from marketplace.gcr.io/google/rbe-ubuntu16-04 to
marketplace.gcr.io/google/rbe-ubuntu18-04 (this image is still not owned
by us, but at least it's newer and demonstrates how a switch to a newer
docker image is done).
- provide script for generating the linux RBE toolchain configs.
- cleanup RBE configuration in the bazelrc files used for remote build
2023-03-10 09:34:53 +01:00
Jan Tattermusch 9c45499dfc
Update third_party/boringssl-with-bazel (#32492)
Upgrade boringssl to the latest "master-with-bazel"
- use the `'USE_HEADERMAP' => 'NO'` fix for ObjC
- update the key for asm optimizations on mac/apple in python's setup.py

This PR depends on monterey fixes here:
https://github.com/grpc/grpc/pull/32493 and the boringssl's build
simplification
https://boringssl-review.googlesource.com/c/boringssl/+/56465.

---------

Co-authored-by: Hannah Shi <hannahshisfb@gmail.com>
2023-03-02 10:18:00 +01:00
Jan Tattermusch b3d68001f7
Fix nits in boringssl and protobuf update instructions (#32484)
Point to authoritative info.
2023-02-28 20:30:33 +01:00
Jan Tattermusch 83fa0fa297
Update boringssl-with-bazel readme (#32084)
The docs change is extracted from
https://github.com/grpc/grpc/pull/31869 and
https://github.com/grpc/grpc/pull/31938.
The actual upgrade of boringssl is in progress, but in the meantime we
can at least make sure the instructions are up-to-date.
I'll also update the internal counterpart (cl/501499368)

Co-authored-by: Hannah Shi <hannahshisfb@gmail.com>
2023-02-27 16:17:59 +01:00
Esun Kim ea4204bb51
Update boringssl to grpc-202302 (#32353)
This is to address
[CVE-2023-0286](https://www.trellix.com/en-us/about/newsroom/stories/research/cve-2023-0286-the-openssl-who-cried-severity-high.html)
2023-02-13 10:27:46 -08:00
Esun Kim 905df9dee6
Update Abseil to 20230125 (#32139)
* Update abseil to 1.20230125.0

* Fix IWYU
2023-01-30 16:20:24 -08:00
Mark D. Roth 1c4b138322
Revert "Revert "WRR: xDS LB policy support (#32067)" (#32180)" (#32182)
This reverts commit d7193a28c4.
2023-01-23 14:14:32 -08:00
Alisha Nanda d7193a28c4
Revert "WRR: xDS LB policy support (#32067)" (#32180)
This reverts commit 8a024963f5.
2023-01-23 09:16:11 -08:00
Mark D. Roth 8a024963f5
WRR: xDS LB policy support (#32067)
* WRR: port StaticStrideScheduler to OSS

* WIP

* Automated change: Fix sanity tests

* fix build

* remove unused aliases

* fix another type mismatch

* remove unnecessary include

* move benchmarks to their own file, and don't run it on windows

* Automated change: Fix sanity tests

* add OOB reporting

* generate_projects

* clang-format

* add config parser test

* clang-tidy and minimize lock contention

* add config defaults

* add oob_reporting_period config field and add basic test

* Automated change: Fix sanity tests

* fix test

* change test to use basic RR

* WIP: started exposing peer address to LB policy API

* first WRR test passing!

* small cleanup

* port RR fix to WRR

* test helper refactoring

* more test helper refactoring

* WIP: trying to fix test to have the right weights

* more WIP -- need to make pickers DualRefCounted

* fix timer ref handling and get tests working

* clang-format

* iwyu and generate_projects

* fix build

* add test for OOB reporting

* keep only READY subchannels in the picker

* add file missed in a previous commit

* fix sanity

* iwyu

* add weight expiration period

* add tests for weight update period and OOB reporting period

* Automated change: Fix sanity tests

* lower bound for timer interval

* consistently apply grpc_test_slowdown_factor()

* cache time in test

* add blackout_period tests

* avoid some unnecessary copies

* clang-format

* add field to config test

* simplify orca watcher tracking

* attempt to fix build

* iwyu

* generate_projects

* update xds proto dependency

* add xDS LB policy entry to registry

* add "_experimental" suffix to policy name

* update LB policy name and remove debug log

* add env var protection

* generate_projects

* gen_upb_api

* WRR: update tests to cover qps plumbing

* WIP

* Automated change: Fix sanity tests

* more WIP

* basic WRR e2e test working

* add OOB test

* add xDS WRR e2e test

* clang-format

* fix sanity

* ignore duplicate addresses

* Automated change: Fix sanity tests

* add new tracer to doc/environment_variables.md

* retain scheduler state across pickers

* Automated change: Fix sanity tests

* use separate mutexes for scheduler and timer

* sort addresses to avoid index churn

* remove fetch_sub for wrap around in RR case

Co-authored-by: markdroth <markdroth@users.noreply.github.com>
2023-01-23 08:08:52 -08:00
Jan Tattermusch dafcb3b6e6
Update third_party/protobuf to 3.21.12 (#32136)
* Update third_party/protobuf

* run tools/distrib/python/make_grpcio_tools.py

* update build_handwritten.yaml

* regenerate projects

* manual change: fix tools/distrib/check_protobuf_pod_version.sh to accept new-style protobuf tags

* improve comments for check_protobuf_pod_version hack
2023-01-20 21:18:53 +01:00
apolcyn c1a4b526fb
[ruby]: add pre-compiled binaries for ruby 3.2; drop them for ruby 2.6 (#32089)
[ruby]: add pre-compiled binaries for ruby 3.2; drop them for ruby 2.6
2023-01-19 15:32:26 -08:00
Mark D. Roth b0d7131ab3
WRR: plumb qps in backend metric data (#31903)
* plumb qps in backend metric data

* pull in updated orca proto

* gen_upb_api

* Automated change: Fix sanity tests

Co-authored-by: markdroth <markdroth@users.noreply.github.com>
2023-01-09 10:07:58 -08:00
Hannah Shi d8a52a5043
build multi arc ios unit test for m1 (#31930) 2023-01-05 16:27:21 -08:00
Jan Tattermusch 7ab2a95979
More cleanup after moving testing docker images to artifact registry. (#31689)
* remove DOCKERHUB_ORGANIZATION from prepare_build_interop_rc

* switch rubys rake-compiler-dock to .current_version files

* remove DOCKERHUB_ORGANIZATION from prepare_build_linux_rc

* fix .current_version files for third_party/rake-compiler-dock

* fix: avoid .current_version files named */.current_version

* fix: avoid pushing in local only mode
2022-11-24 14:09:16 +01:00
Jan Tattermusch ed39634fe8
Move docker images from dockerhub to artifact registry. (#31591)
* push_testing_images.sh overhaul

* simplify dockerize scripts

* update run_in_docker.sh script

* add generated .current_version files for docker images

* improvements to push_testing_images.sh

* add local-only check for push_testing_images.sh to sanity tests

* fix sanity check

* add expiration for TRANSFER_FROM_DOCKERHUB

* add a readme file
2022-11-17 10:12:39 +01:00
Esun Kim 3d104df977
Upgrade re2 to 2022-04-01 (#31665) 2022-11-15 23:46:57 -08:00
Mikhail Lappo 5ec616e6a8
Update zlib (#31356) (#31357)
To mitigate CVE-2022-37434
2022-10-21 13:14:43 -07:00
Mark D. Roth a2b473ec89
address_sorting BUILD: remove license comment breaking import (#31145) 2022-09-27 15:51:21 -07:00
Richard Belleville e609ad88c9
Update protobuf to v21.6 (#31015)
* Update third_party/protobuf

* run tools/distrib/python/make_grpcio_tools.py

* update build_handwritten.yaml

* regenerate projects

* Update Python version floor
2022-09-16 17:50:52 -07:00
AJ Heller d547df2872
Upgrade google/benchmark to v1.7.0 (#30902)
* Upgrade google/benchmark to v1.7.0

Reason: Setup and Teardown methods are nice to have for multithreaded
tests.

* use version tag

* Revert "use version tag"

This reverts commit 6c7f5e0d35.
2022-09-08 18:01:56 -07:00
3405691582 e55be30ffb
Start to buildfix OpenBSD. (#30557)
Some minor changes for the platform:

* `sys/types.h` needs to be included before `netinet/tcp.h` to provide
  definitions for `u_int8_t`, `u_int16_t`, and `u_int32_t`.

* cares has an OpenBSD-specific config header but is not actually hooked
  up in cares.BUILD. Remedy that.
2022-08-18 09:09:40 -07:00
Akash Patel 1b8f7b1c79
fix python detection on windows (#30302) 2022-08-17 10:27:34 -07:00
apolcyn 27a9bde44b
Update protobuf submodule to 3.21.5 (#30548)
* Update third_party/protobuf

* run tools/distrib/python/make_grpcio_tools.py

* regenerate protos for ruby, php

* update build_handwritten.yaml

* regenerate projects
2022-08-10 09:39:26 -07:00
Yash Tibrewal 41ec08c69a
Update third_party/protobuf to 3.21.4 (#30377)
* Update third_party/protobuf

* run tools/distrib/python/make_grpcio_tools.py

* regenerate protos for ruby, php

* update build_handwritten.yaml

* regenerate projects

* Build - Use :well_known_type_protos instead of :well_known_protos

* Fix target

* Update upb

* Update Python for Protobuf 4.21 (#140)

* Update protobuf dependency on grpcio-tools

* Off by one

* Drop python 3.6 support

* Try upgrading pip

* And in the other script

* Try to figure out if we're compatible with abi3

* See what we've already got installed

* Update the requirements.txt file I didn't know existed

* And here too

* See what's installed

* Let's try that again

* Remove

* Try to confirm version

* Let me see the generated code

* Fix non-Bazel test runner

* Work for all test directories

* Regenerate example protos

* Clean up

* Generate .pyi files

* Fix type checking and linting

* Exclude pyi files from isort

* Upgrade to 3.21.4

* Update iwyu to get around messy protobuf IWYU rules

Co-authored-by: Richard Belleville <gnossen@gmail.com>
2022-08-04 09:39:49 -07:00
Mark D. Roth 90cc6fa706
xDS: upgrade the commit of the xDS protos that we're using (#30318)
* xDS: upgrade the commit of the xDS protos that we're using

* Automated change: Fix sanity tests

Co-authored-by: markdroth <markdroth@users.noreply.github.com>
2022-07-18 13:37:34 -07:00
Johnny Shields 34a66d9496
Ruby: Fixes for x64-mingw-ucrt (PR to master) (#30081)
* Fix Ruby Windows 3.1 (x64-mingw-ucrt) build:
- Pass-in CPPFLAG "GPR_WINDOWS_UCRT" to pickup correct shared object file.
- Rename intermediate legacy SO file from 'grpc_c.64.ruby' to 'grpc_c.64-msvcrt.ruby' for clarity.
- Add artifact target for Ruby x64-mingw-ucrt

* Change CPP flag name
2022-07-14 12:45:40 -07:00
Esun Kim 27509c345c
Update upb to 20220621 (#30156)
* Update third_party/upb to e4635f223e7d36dfbea3b722a4ca4807a7e882e2

* Update grpc_deps

* Update src/upb/gen_build_yaml.py

* Regen projects

* Gen_upb_api

* Fix missing json files

* Fix missing textformat

* Fix missing upb/arena

* Sanitize

* Fix missing port_def

* Fix missing array.h
2022-07-01 13:57:56 -07:00
Esun Kim dcf9612186
Upgrade Abseil to LTS 20220623.0 (#30155)
* Upgrade abseil to 20220623.0

* Fix subchannel include
2022-06-30 16:02:08 -07:00
Jan Tattermusch 99a7f2995e
Update win RBE docker image to VS2019 (#30072)
* update win RBE docker image and toolchain

and switch win RBE builds to using it.

* delete legacy RBE windows toolchain config

* switching to VS2019 makes #28544 obsolete
2022-06-28 10:23:12 +02:00
Ara Nguyen f3fc1d4d75
use constraints value in @platforms instead of @bazel_tools//platforms https://github.com/bazelbuild/bazel/issues/8622 (#29842) 2022-06-07 12:40:54 -07:00
Johnny Shields 6d33399d1e
Support for prebuilt Ruby binary on x64-mingw-ucrt platform (#29684)
* Add support for prebuilt Ruby binary on x64-mingw-ucrt platform, which is used for Windows RubyInstaller Ruby 3.1+
2022-05-31 08:33:27 -07:00
apolcyn 409db4dd0f
Upgrade ruby rake-compiler-dock images (#29304)
* Upgrade rake-compiler-dock images
2022-05-11 13:58:45 -07:00
Nicolas Noble fcb8157f25
Upgrading zlib. (#29531)
CVE-2018-25032
2022-05-02 09:24:10 -07:00
Hannah Shi bf4ab9654b
[Objc] iOS ThirdPartyTest suite for libuv dns test, add additional dns test (#28573)
* add ios libuv additional dns test

* rewrite LibuvGetAddrInfoTests.m

* comment address

* clang format
2022-04-18 12:33:56 -07:00
Esun Kim 83e8f5f0ba
Workaround for the unaligned malloc (#29380) 2022-04-12 16:25:34 -07:00
Esun Kim 696e4928e6
Upgrade upb to the latest (#29333)
* Update upb manual

* Update third_party/upb to bef53686ec702607971bd3ea4d4fefd80c6cc6e8

* Update grpc_deps.bzl

* Update doc

* Regen UPB

* Update by review

* New upb files
2022-04-07 10:28:16 -07:00
donnadionne 08451810a6
Upgrading envoyproxy/data-plane-api (#29264)
* Upgrading envoyproxy/data-plane-api

* Updating bazel/grpc_deps.bzl

* add sha

* update check_submodule.sh

* add generated files
2022-03-31 10:28:26 -07:00
Lidi Zheng af97f15b9d
[xDS Proto] Update Bazel build dependencies for Envoy API (#29219)
* [xDS Proto] Update Bazel build dependencies for Envoy API

* Remove protoc_gen_validate as a submodule

* Remove unused import
2022-03-29 11:18:39 -07:00
Hannah Shi 842ed8d461
[Objc] add ios libuv timer test (#28570)
* use libuv in third party

* add ios libuv timer test

* name changed correspondingly

* format code

* address comment

* remove libuv patch
2022-03-16 13:03:01 -07:00
Esun Kim c08eb4fc41
Attempt to upgrade upb to the latest (#28685)
* Update upb to 85e5c76cb7098d3ae35e575d652a40568013a149

* Update grpc_deps.bzl

* Run gen_upb_api.sh

* Run run_sed.sh

* Update gen_build_yaml.py

* Regen projects

* Reformat

* Update third_party/upb to 950cdab42fe4c83f06a2be46eeb46a068e486eed

* Update third_party/upb to c9c31241bc39922b092a81d192badd50cccaaa81

* Update src/upb/gen_build_yaml.py

* Generated projects

* Update third_party/upb to a02d92e0257a35f11d4a58b6a932506cbdbb2f29

* Patching in https://github.com/protocolbuffers/upb/pull/524

* Sed script went too far.

Co-authored-by: Nicolas 'Pixel' Noble <nicolas@nobis-crew.org>
2022-02-28 09:26:31 -08:00
Akash Patel 75841dd1e0
fix python command on windows (#28895) 2022-02-18 10:52:03 -08:00
AJ Heller 40f2c2a4e1
Upgrade googletest (#27287) 2022-02-14 18:16:05 -08:00
Richard Belleville b8ee9ac7e5
Make Compatible with Bazel 5.0 (#28683)
* See what happens when we remove NDK

* Try to debug Kokoro in a super hacky way

* And echo the external IP too

* Attempt to fix NDK installation

* And actually run the portion of the code I need to test out

* Clean up

* Actually test against bazel 5

* Put export in proper file

* Make android an optional dependency

* Escape paths for Windows

* Revert switch to Bazel 5.0
2022-02-11 15:18:18 -08:00
AJ Heller fc6f0277a4
Update third_party/protobuf to v3.19.4 (#28842)
* Update third_party/protobuf

* run tools/distrib/python/make_grpcio_tools.py

* update build_handwritten.yaml

* regenerate projects
2022-02-11 08:47:36 -08:00
Jan Tattermusch 1cf6b085f2
Enable ccache in more builds (#28665)
* upgrade ruby docker images to debian 11

* upgrade C# docker images to debian11

* update sanity dockerimage to debian11

* upgrade cxx interop to debian11

* add ccache to python interop images

* enable use of ccache for interop tests

* adjust run_tests.py to new docker images

* add ccache to rake-compiler-dock docker images

* improve prepare_ccache_symlinks_rc

* enable use of ccache in rake-compiler-dock docker containers

* add ccache support for python_manylinux2014_aarch64

* add ccache support for python_linux_armv7

* deduplicate python3.9 install

* ccache for crosscompiled darwin gems is broken

* fix bash -l resetting of PATH in grpc_artifact_python_linux_armv7
2022-01-25 20:37:04 +01:00