Commit Graph

13299 Commits

Author SHA1 Message Date
AJ Heller d14020d4e2
[EventEngine] Add invalid handle types to the public API (#32202)
* [EventEngine] Add invalid handle types to the public API

* Automated change: Fix sanity tests

* add definition for static constexpr members

* sanitize

Co-authored-by: drfloob <drfloob@users.noreply.github.com>
2023-01-25 17:31:34 -08:00
AJ Heller 6a92df7237
[fixup] EventEngine Nomenclature (#32195) 2023-01-25 09:31:46 -08:00
Vignesh Babu fa5a6c42a6
[EventEngine] Modify iomgr to allow creation of posix event engine listeners and server side endpoints (#31928)
* [WIP] EventEngine iomgr endpoint shims

* [WIP] EventEngine::Endpoint iomgr shims for the PosixEventEngine

* Util functions to help with posix engine listener implementation

* sanity

* update comments in posix_engine_listener_utils.h

* review comments

* iwyu

* revert prev commit

* iwyu

* update build

* update

* regenerate projects

* regenerate projects

* minor fixes

* update BUILD

* sanity

* update build

* regenerate projects

* fix unused parameter

* sanity

* update

* sanity

* regenerate_projects

* remove unused variable

* start

* update

* regenerate_projects

* sanity

* update

* fixes

* update

* regenerate_projects

* update

* fix sanity and msan failure

* more fixes

* build failure

* update

* fix

* sanity

* fixes

* update

* regenerate projects

* fix sanity

* review comments

* An EventEngine subclass to be implemented by all posix based event engines

* sanity

* comments

* update

* review comments

* re-word

* fix

* update

* review comments

* regenerate projects

* syntax fix

* add lock free event benchmark

* releasable mutex lock

* fix build isue

* update

* start

* regenerate projects

* update

* fix

* windows build

* update

* windows portability issue

* update

* update

* update

* update

* format

* update

* update

* start

* Update tcp server interface to move on_accept_cb to create method

* update

* start

* update

* update

* update

* update

* update

* update

* update

* update

* sanity

* update

* update

* update

* windows build

* fix msan

* fix sanity

* regenerate projects

* update

* iwyu

* Fix resolved address length related bugs in tcp_socket_utils and listener_utils

* iwyu

* cleanup

* cleanup src/core/lib/event_engine/tcp_socket_utils.cc

* iwyu

* fix

* regenerate projects

* fix sanity

* re-write endpoint shim

* more re-write

* cleanup

* update

* regenerate projects

* review comments

* build issue

* more build issue fixes plus adding event_engine_trace

* even more build issue fixes

* iwyu

* add static_cast

* fix sanity

* update

* update

* sanity

* fix

* Fix

* Review comments

* fix

* iwyu

* fix build issue

Co-authored-by: AJ Heller <hork@google.com>
2023-01-24 18:22:14 -08:00
Mark D. Roth 1c4b138322
Revert "Revert "WRR: xDS LB policy support (#32067)" (#32180)" (#32182)
This reverts commit d7193a28c4.
2023-01-23 14:14:32 -08:00
Esun Kim 564a46291b
Update all Debian:9 to Debian:10 (#31917) 2023-01-23 10:50:41 -08:00
Yash Tibrewal e7f1e9d1d9
Gcp Observability Logging: Fix the log name and resource for the gcp logging API (#32148)
* Gcp Observability Logging: Fix the log name and resource for the gcp logging API

* iwyu

* Fix build

* Fix build - try 2
2023-01-23 10:01:16 -08:00
Alisha Nanda d7193a28c4
Revert "WRR: xDS LB policy support (#32067)" (#32180)
This reverts commit 8a024963f5.
2023-01-23 09:16:11 -08:00
Mark D. Roth 8a024963f5
WRR: xDS LB policy support (#32067)
* WRR: port StaticStrideScheduler to OSS

* WIP

* Automated change: Fix sanity tests

* fix build

* remove unused aliases

* fix another type mismatch

* remove unnecessary include

* move benchmarks to their own file, and don't run it on windows

* Automated change: Fix sanity tests

* add OOB reporting

* generate_projects

* clang-format

* add config parser test

* clang-tidy and minimize lock contention

* add config defaults

* add oob_reporting_period config field and add basic test

* Automated change: Fix sanity tests

* fix test

* change test to use basic RR

* WIP: started exposing peer address to LB policy API

* first WRR test passing!

* small cleanup

* port RR fix to WRR

* test helper refactoring

* more test helper refactoring

* WIP: trying to fix test to have the right weights

* more WIP -- need to make pickers DualRefCounted

* fix timer ref handling and get tests working

* clang-format

* iwyu and generate_projects

* fix build

* add test for OOB reporting

* keep only READY subchannels in the picker

* add file missed in a previous commit

* fix sanity

* iwyu

* add weight expiration period

* add tests for weight update period and OOB reporting period

* Automated change: Fix sanity tests

* lower bound for timer interval

* consistently apply grpc_test_slowdown_factor()

* cache time in test

* add blackout_period tests

* avoid some unnecessary copies

* clang-format

* add field to config test

* simplify orca watcher tracking

* attempt to fix build

* iwyu

* generate_projects

* update xds proto dependency

* add xDS LB policy entry to registry

* add "_experimental" suffix to policy name

* update LB policy name and remove debug log

* add env var protection

* generate_projects

* gen_upb_api

* WRR: update tests to cover qps plumbing

* WIP

* Automated change: Fix sanity tests

* more WIP

* basic WRR e2e test working

* add OOB test

* add xDS WRR e2e test

* clang-format

* fix sanity

* ignore duplicate addresses

* Automated change: Fix sanity tests

* add new tracer to doc/environment_variables.md

* retain scheduler state across pickers

* Automated change: Fix sanity tests

* use separate mutexes for scheduler and timer

* sort addresses to avoid index churn

* remove fetch_sub for wrap around in RR case

Co-authored-by: markdroth <markdroth@users.noreply.github.com>
2023-01-23 08:08:52 -08:00
Vignesh Babu 8cf04e9a54
[EventEngine] Modify iomgr to allow creation of event engine clients and client side endpoints (#31661)
* [WIP] EventEngine iomgr endpoint shims

* [WIP] EventEngine::Endpoint iomgr shims for the PosixEventEngine

* Util functions to help with posix engine listener implementation

* sanity

* update comments in posix_engine_listener_utils.h

* review comments

* iwyu

* revert prev commit

* iwyu

* update build

* update

* regenerate projects

* regenerate projects

* minor fixes

* update BUILD

* sanity

* update build

* regenerate projects

* fix unused parameter

* sanity

* update

* sanity

* regenerate_projects

* remove unused variable

* start

* update

* regenerate_projects

* sanity

* update

* fixes

* update

* regenerate_projects

* update

* fix sanity and msan failure

* more fixes

* build failure

* update

* fix

* sanity

* fixes

* update

* regenerate projects

* fix sanity

* review comments

* An EventEngine subclass to be implemented by all posix based event engines

* sanity

* comments

* update

* review comments

* re-word

* fix

* update

* review comments

* regenerate projects

* syntax fix

* add lock free event benchmark

* releasable mutex lock

* fix build isue

* update

* start

* regenerate projects

* update

* fix

* windows build

* update

* windows portability issue

* update

* update

* update

* update

* format

* update

* update

* update

* update

* update

* fix sanity

* regenerate projects

* update

* iwyu

* Fix resolved address length related bugs in tcp_socket_utils and listener_utils

* iwyu

* cleanup src/core/lib/event_engine/tcp_socket_utils.cc

* iwyu

* fix

* regenerate projects

* fix sanity

* re-write endpoint shim

* more re-write

* cleanup

* update

* review comments

* build issue

* more build issue fixes plus adding event_engine_trace

* even more build issue fixes

* iwyu

* add static_cast

* update

* remove redundant code

* update

* deduplicate

* iwyu

* Fix review comments and regenerate_projects

* sanity

* review comments

* fix include guards

Co-authored-by: AJ Heller <hork@google.com>
2023-01-20 19:47:14 -08:00
Alisha Nanda 2f0734272d
Added insecure connections counter (#32115)
* Fetch PR 32088 and rebase

* update build file

* update BUILD

* format stats data

* address iwyu error

* Automated change: Fix sanity tests

Co-authored-by: ananda1066 <ananda1066@users.noreply.github.com>
2023-01-20 17:03:18 -08:00
Yash Tibrewal aa11978541
Fix include guards for src/ and test/ directories (#32167)
* Fix include guards for src/ and test/ directories

* Sanity

* Update new files
2023-01-20 16:27:27 -08:00
Jan Tattermusch dafcb3b6e6
Update third_party/protobuf to 3.21.12 (#32136)
* Update third_party/protobuf

* run tools/distrib/python/make_grpcio_tools.py

* update build_handwritten.yaml

* regenerate projects

* manual change: fix tools/distrib/check_protobuf_pod_version.sh to accept new-style protobuf tags

* improve comments for check_protobuf_pod_version hack
2023-01-20 21:18:53 +01:00
Mark D. Roth 76c82265b4
WRR: implement WRR LB policy (#31904)
* WRR: port StaticStrideScheduler to OSS

* WIP

* Automated change: Fix sanity tests

* fix build

* remove unused aliases

* fix another type mismatch

* remove unnecessary include

* move benchmarks to their own file, and don't run it on windows

* Automated change: Fix sanity tests

* add OOB reporting

* generate_projects

* clang-format

* add config parser test

* clang-tidy and minimize lock contention

* add config defaults

* add oob_reporting_period config field and add basic test

* Automated change: Fix sanity tests

* fix test

* change test to use basic RR

* WIP: started exposing peer address to LB policy API

* first WRR test passing!

* small cleanup

* port RR fix to WRR

* test helper refactoring

* more test helper refactoring

* WIP: trying to fix test to have the right weights

* more WIP -- need to make pickers DualRefCounted

* fix timer ref handling and get tests working

* clang-format

* iwyu and generate_projects

* fix build

* add test for OOB reporting

* keep only READY subchannels in the picker

* add file missed in a previous commit

* fix sanity

* iwyu

* add weight expiration period

* add tests for weight update period and OOB reporting period

* Automated change: Fix sanity tests

* lower bound for timer interval

* consistently apply grpc_test_slowdown_factor()

* cache time in test

* add blackout_period tests

* avoid some unnecessary copies

* clang-format

* add field to config test

* simplify orca watcher tracking

* attempt to fix build

* iwyu

* generate_projects

* add "_experimental" suffix to policy name

* WRR: update tests to cover qps plumbing

* WIP

* more WIP

* basic WRR e2e test working

* add OOB test

* fix sanity

* ignore duplicate addresses

* Automated change: Fix sanity tests

* add new tracer to doc/environment_variables.md

* retain scheduler state across pickers

* Automated change: Fix sanity tests

* use separate mutexes for scheduler and timer

* sort addresses to avoid index churn

* remove fetch_sub for wrap around in RR case

Co-authored-by: markdroth <markdroth@users.noreply.github.com>
2023-01-20 12:14:37 -08:00
apolcyn 86d314a4cb
Undrop ruby 2.6 (#32160) 2023-01-20 09:36:59 -08:00
apolcyn c1a4b526fb
[ruby]: add pre-compiled binaries for ruby 3.2; drop them for ruby 2.6 (#32089)
[ruby]: add pre-compiled binaries for ruby 3.2; drop them for ruby 2.6
2023-01-19 15:32:26 -08:00
Yash Tibrewal 82c697e2e7
CI: Move kokoro bazel output root to /tmpfs/bazel (#32156) 2023-01-19 15:32:04 -08:00
Clément Jean 7d957f0132
guard multiprocessing code with main (#31992)
* guard multiprocessing code with main

* Update fix_build_deps.py

* merge last master

* add back execution bit to fix_build_deps

Co-authored-by: AJ Heller <hork@google.com>
2023-01-18 13:13:49 -08:00
Jan Tattermusch 7fd808f6f9
Bump version to 1.53.0-dev (on master branch) (#32086)
* bump version to 1.53.0-dev

* regenerate projects
2023-01-18 09:41:33 +01:00
Doug Fawley d26d68ab82
Add v1.50-1.52 releases of grpc-go to interop matrix (#31700) 2023-01-16 15:46:35 +01:00
Sergii Tkachenko 80e5f98139
xds interop: Fix buildscripts not continuing on a failed test suite (#32093)
Apparently there's a difference between bash 3 and bash 4. OSX comes with bash 3 out-of-box, so for whoever wrote this logic it "worked on my machine".
2023-01-13 18:13:09 -08:00
Hannah Shi 24e158d5a9
[ObjC]cf event engine timer (#31140)
* Apple CF event engine using posix timer

* update generated files

* fix ci errors
2023-01-12 16:31:58 -08:00
AJ Heller 6778bb3501
[EventEngine] Relandx3: WindowsEventEngine client (#32031)
* Reland x3: "WindowsEventEngine Client implementation (#31848)"

This reverts commit 2c4d55b3a1.

* fixes

* fix

* fix
2023-01-12 10:26:17 -08:00
Vignesh Babu 5680a9b57b
Move posix event engine to new interface defined in #31816 (#31890)
* update

* regenerate projects

* fix sanity

* review comments

* An EventEngine subclass to be implemented by all posix based event engines

* sanity

* comments

* update

* review comments

* re-word

* fix

* update

* review comments

* regenerate projects

* syntax fix

* add lock free event benchmark

* releasable mutex lock

* fix build isue

* update

* start

* regenerate projects

* update

* fix

* windows build

* update

* windows portability issue

* update

* update

* format

* update

* update

* update

* fix sanity

* regenerate projects

* update

* iwyu

* fix

* fix

* update

* fix sanity

* review comments
2023-01-12 09:41:55 -08:00
Craig Tiller 63d1edd0f2
[hpack] Add stat for received hpack metadata size (#32062)
* [hpack] Add stat for received hpack metadata size

* Automated change: Fix sanity tests

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2023-01-12 09:35:39 -08:00
Jan Tattermusch 1e32218bb8
Bump core version to 30.0.0 for upcoming release (#32083)
* bump C-core version

* regenerate projects
2023-01-12 15:21:52 +01:00
Larry Safran ad1069b499
add v1.52.0 java release to client matrix (#32080) 2023-01-11 16:22:40 -08:00
Craig Tiller 2caa79feeb
[promises] Multi-producer, single-consumer for inter-activity comms (#31927)
* mpsc

* Automated change: Fix sanity tests

* tests

* tests

* Automated change: Fix sanity tests

* comments

* comments

* fix

* Automated change: Fix sanity tests

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2023-01-11 15:11:17 -08:00
Eugene Ostroukhov 73fa138432
xds_override_host LB: move XdsOverrideHostLbConfig to `internal` namespace (#32061) 2023-01-11 10:29:38 -08:00
Craig Tiller 67f364e23e
[cleanup] Eliminate usage of GRPC_ASSERT(false...); (#31757)
* crash function

* progress

* fix

* fix

* Automated change: Fix sanity tests

* fix

* fix

* fix

* fixes

* Automated change: Fix sanity tests

* fix

* Automated change: Fix sanity tests

* fix

* fix

* use cpp attr

* Automated change: Fix sanity tests

* fix

* fix

* fix

* fix

* fix

* fix

* add exclusion

* fix

* typo

* fix

* fmt

* Update tcp_socket_utils.cc

* Automated change: Fix sanity tests

* fix

* revert php changes

* Automated change: Fix sanity tests

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2023-01-11 08:50:32 -08:00
Anuj Dhungana 175c2c946a
doc: npm package of grpc migrate (#32041) 2023-01-10 13:57:25 -08:00
Craig Tiller 9beb72836e
[event_engine+promises] EventEngine based wakeup scheduler for activities (#31926)
* eliminate activity* from wakers

* event engine activity wakeup scheduler

* Automated change: Fix sanity tests

* add exec ctx

* fix

* iwyu

* fix

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2023-01-09 14:02:26 -08:00
Mark D. Roth b0d7131ab3
WRR: plumb qps in backend metric data (#31903)
* plumb qps in backend metric data

* pull in updated orca proto

* gen_upb_api

* Automated change: Fix sanity tests

Co-authored-by: markdroth <markdroth@users.noreply.github.com>
2023-01-09 10:07:58 -08:00
Sergii Tkachenko 70d0b21e2a
xDS interop: resume circuit_breaking test (#32038)
This is a test running on legacy (GCE) framework.
It was disabled in https://github.com/grpc/grpc/pull/27355,
however it has never been ported to the new framework.
Unless there were reasons I'm not aware of, disabling it is a mistake.
Other languages still run it:

- Java: cc5378453f/buildscripts/kokoro/xds_v3.sh (L21)
- Go: f2fbb0e07e/test/kokoro/xds.sh (L30)
- Node: 81083bd229/packages/grpc-js-xds/scripts/xds.sh (L57)
2023-01-06 14:49:01 -08:00
Yash Tibrewal ad6f6c49e0
Logging filter (#31755)
* initial

* Intermediate

* Another try

* Try multiple necesary pulls

* Filter works other than client half close

* Fixes

* Add a cancelled RPC test

* Handle trailer only responses

* Tests for disabled logging and truncated payloads

* Fix authority and peer

* Add TODOs and asserts for half-close

* Fix tests for half-close and cancel

* 2d748fcb1cf45cac62729b8346ad15e6abc79e97

* Fix sanity checks

* Strict bazel build

* Fix package

* IWYU

* Fix cmake

* Explicit cast to string

* Size casts

* Fix Arena leak and disable macos build for now

* Reviewer comments
2023-01-06 13:52:15 -08:00
Andres Beltran 2904ee8fd3
Add support for systemd socket activation (#31667)
* Revert "Revert "Add support for systemd socket activation (#30485)" (#31617)"

This reverts commit 867dc6cae2.

* Add checks to unix tests

* Ran generate_projects.sh and fixed styling in test

* Fix variable in unit test

* Use reinterpret_cast in test

* Rebase and fix sanity failures
2023-01-05 14:53:33 -08:00
AJ Heller 2c4d55b3a1
Revert "Revert "Revert "WindowsEventEngine Client implementation (#31848)" (#… (#31974)" (#32018)
This reverts commit aa83319695.
2023-01-04 11:09:56 -08:00
Cheng-Yu Chung 384982ba2b
[bugfixes] Fix vulnerability for PyYAML (#32007)
* Fix vulnerability for PyYAML

* Fix

* Revert "Fix"

This reverts commit 5522d226c4.

* Fix

* Fix format
2023-01-04 12:33:29 +08:00
Hannah Shi 2adab34f7a
fix generate_projects.sh fails in M1 Mac (#31868)
* fix generate_projects.sh fails in m1 mac

* use buildozer-linux-arm64 for linux arm64 platform
2023-01-03 18:05:19 -08:00
Vignesh Babu 8bd2b9af52
Fixing some bugs in posix event engine (#31785)
* update

* regenerate projects

* fix sanity

* review comments

* fix

* add lock free event benchmark

* releasable mutex lock

* fix build isue

* update

* fix sanity

* regenerate projects

* iwyu

* Fix resolved address length related bugs in tcp_socket_utils and listener_utils

* iwyu

* cleanup src/core/lib/event_engine/tcp_socket_utils.cc

* update endpoint to use SliceCast into MutableSlice

* fix sanity

* review comments

* review comments

* regenerate projects

* fix

* rm unused variable

* fix
2023-01-03 17:52:46 -08:00
AJ Heller aa83319695
Revert "Revert "WindowsEventEngine Client implementation (#31848)" (#… (#31974)
* Revert "Revert "WindowsEventEngine Client implementation (#31848)" (#31965)"

This reverts commit 58ad7950a0.

* iwyu & visibility
2023-01-03 16:21:29 -08:00
apolcyn e31cfaaa11
Update default ruby version to 3.2 on linux/macos ruby builds (#32010)
* Use ruby 3.2 in macos and linux ruby builds
2023-01-03 13:38:15 -08:00
Craig Tiller a414dbd304
Revert #30349 (#32008)
* [iwyu] Add missing #include <type_traits> to fix build breakage with LLVM after e0a66116fc

* Update IWYU mappings

New libc++ doesn't provide <type_traits> when including <utility>.

* run iwyu

Co-authored-by: alexfh <alexfh@google.com>
2023-01-03 11:00:30 -08:00
Vignesh Babu dafbe7031b
Separate out tsi_alts_frame_protector into its own target (#31988)
* start

* update
2023-01-03 09:46:16 -08:00
apolcyn b27d726342
testing: bump timeout of grpc/abseil-head build job from 4 to 5 hours (#31936) 2023-01-03 12:35:44 +01:00
Yijie Ma f99b8b5bc4
Convert c-style comments to C++-style comments (#31923)
* baseline

* fix clang-tidy

* manually revert these files

* manually fixup at eof

* revert 2 more files

* change check_deprecated_grpc++.py

* change end2end_defs.include template

* fix check_include_guards.py

* untrack tools/distrib/python/convert_cstyle_comments_to_cpp.py

not yet ready to be submitted

* fix

yapf check_include_guards.py
remove a space...

* fix version.cc.template

* fix version_info.h.template
2022-12-22 23:01:53 -08:00
AJ Heller 58ad7950a0
Revert "WindowsEventEngine Client implementation (#31848)" (#31965)
This reverts commit bc7e155e55.
2022-12-22 09:59:14 -08:00
Eugene Ostroukhov 878a0ea6c2
xds_override_host LB: add basic support for overriding host (#31954)
This is same as #31819 but with fixed compilation issues.
2022-12-21 16:30:09 -08:00
Craig Tiller feb43c589f
[iwyu] Better error checking in script (#31957)
* fff

* fff

* fff

* better error checking

* x

* fix

* Automated change: Fix sanity tests

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2022-12-21 14:55:33 -08:00
Esun Kim 25b0aff1e7
Update a sanity docker image to run iwyu (#31956)
* Added missing iwyu deps

* Docker image pushed
2022-12-21 14:54:38 -08:00
Eugene Ostroukhov 60e4aea792
Revert "xds_override_host LB: add basic support for overriding host (#31819)" (#31948)
This reverts commit a500a74c6c.
2022-12-21 10:02:39 -08:00