Commit Graph

51839 Commits

Author SHA1 Message Date
Hannah Shi 96d27634a4
run ios CronetTest from bazel (#30439) 2022-09-27 14:49:38 -07:00
Yash Tibrewal 6d249c0af2
Observability: Experimental arg to disable client side tracing (#31093)
* Observability: Experimental arg to disable client side tracing

* Fix IWYU

* Reviewer comments

* Reviewer comments

* Reviewer comment: Move experimental/internal arg to different file

* Fix build

* Fix IWYU
2022-09-27 14:34:07 -07:00
Aaron Pham ac585b8f7e
chore: add bazel support for M1 mac (#31127)
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
2022-09-27 13:17:35 -07:00
Jan Tattermusch 2f5d06af6b
make legacy artifact -> packages -> distribtest jobs a noop (#31124) 2022-09-27 21:01:32 +02:00
AJ Heller 298ace8cbf
small test cleanup (#31137) 2022-09-27 11:51:23 -07:00
Cheng-Yu Chung 409f558871
Revert "Revert "Remove `grpc_codegen` (#30960)" (#31099)" (#31100)
This reverts commit eada386921.
2022-09-27 11:24:35 -04:00
Craig Tiller 5fa37322cb
Revert "Tcp endpoint implementation for posix event engine. (#30559)" (#31134)
This reverts commit a81391d931.
2022-09-27 08:04:25 -07:00
Vignesh Babu a81391d931
Tcp endpoint implementation for posix event engine. (#30559)
* convert tcp code to use EndpointConfig and successfully compile

* regenerate projects

* copying config options used in tcp code to create map backed endpoint configs

* fix minor build issues

* fix includes in tests

* fix some build and test issues

* modifying endpoint config definition from an abstract interface into a concrete implementation

* minor fix

* add unreachable code check

* fix some windows build issues

* fix more build issues

* adding a grpc_tcp_generic_options struct to hold data extracted from EndpointConfig

* code formatting

* simplify

* fix sanity checks

* fix windows issues

* some posix fixes

* disabling copy constructor and copy assignment operator for endpoint config

* fix sanity checks

* fix syntax error

* fix weird formatting

* fix build issue

* fix review comments

* reverting un-necessary change

* remove channel args completely from windows impl since it is unused

* remove unused headers and fix usage in endpoint_pair_windows.cc

* cleanup

* cleanup

* fix some initialization issues

* re-generate projects

* removing some files

* fix ifdef for windows

* fixing windows build issue due to improper header file include

* Automated change: Fix sanity tests

* start

* regenerate-projects

* delete some files

* Automated change: Fix sanity tests

* review comments

* add comment

* Migrating posix event pollers to use new event poller interface

* add inline attributes

* Automated change: Fix sanity tests

* remove ref/unref from iomgr engine closure ad add custom closure types internal to the pollers

* updating time util usage

* use unique_ptrs

* update comments

* Automated change: Fix sanity tests

* review comments

* rename GetPointer to GetVoidPointer

* cleanup

* Automated change: Fix sanity tests

* removing EndpointConfig::Get method

* Automated change: Fix sanity tests

* cleanup

* fix

* fix review comments

* review comments

* cleanup

* update comments

* fix

* cleanup

* update comments

* Automated change: Fix sanity tests

* fix misleading comments

* bug fixes

* fix

* fix

* start

* revert some changes for bug fixes and allow spurious wakeups for poll based poller

* sanity

* fix

* minor fix

* review comments

* fix

* comment

* remove re-defined function

* fix review comments

* fix windows iocp build issue due to removed function

* change Milliseconds return type

* fix

* remove header

* regenerate projects

* fix sanity

* code

* fix sanity

* Automated change: Fix sanity tests

* Forking iomgr internal_errqueue defines for posix event engine

* fix BUILD file

* start

* update

* sanity

* regenerate_projects

* Automated change: Fix sanity tests

* rename

* review comments

* update

* review comments

* Automated change: Fix sanity tests

* fix BUILD

* add no_windows tag

* update

* update

* remove debug statements

* delete unused file

* update

* fix BUILD

* regenerate projects

* update

* fix BUILD

* update

* fix sanity

* minor additions to configure global tcp user timeout values

* cleanup

* fix build syntax

* change poller interface for posix event engine

* update event_poller_posix_test

* fix build error

* Add more unit tests

* windows ee changes

* update comment

* remove unused deps

* fix msan issue

* add comment

* adding rcv lowat helper to socket wrapper

* fix sanity

* review comments

* fix build issue

* fix sanity

* sanity

* iwyu sanity

* fix build

* merge conflict

* Automated change: Fix sanity tests

* update

* update

* fix macos issue

* fix msan and ubsan issues

* fix macos build issue

* comment

* review comments

* remove unused dep

* sanity

* merge conflict

* mac build issue

* regenerate projects

* build issue

* fix typo and address review comments

* review comments

* Automated change: Fix sanity tests

* update

* merge issue

* review comments

* sanity

* fix build issue

* review comments

* add thread annotations

* MacOS fix unused variable

* add some more #endif comments

* missing semi-colon

* Automated change: Fix sanity tests

* sanity

* sanity again

* sanity try again

* sanity again

* try again with no_include pragma

Co-authored-by: Vignesh2208 <Vignesh2208@users.noreply.github.com>
2022-09-26 17:20:17 -07:00
Esun Kim ba8af0157b
Removing GRPC_ERROR_[REF|UNREF|IS_NONE] (#31089)
* Removing GRPC_ERROR_[REF|UNREF|IS_NONE]

* Clean up unnecessary error

* Fix

* Fix2

* Remove unnecessary else
2022-09-26 15:41:12 -07:00
Jan Tattermusch 255083b724
Fix ruby builds on kokoro macos monterey images (#31125)
* cocoapods is not a ruby dependency

* fix rvm update on macos monterey images
2022-09-26 20:46:54 +02:00
Mark D. Roth cdb7d2c93c
XdsEndpointResourceType: use ValidationErrors and add unit tests (#31076)
* simplify XdsResourceType::Decode() API

* fix xds_client_test

* xDS endpoint: use ValidationErrors to improve error messages

* fix sanity

* add xDS endpoint resource type test

* clang-format

* remove a couple of now-unnecessary end2end tests

* generate_projects

* fix xds_csds_e2e_test

* iwyu

* attempt to avoid ubsan failure
2022-09-26 11:45:56 -07:00
AmosWang 1aac7caba9
upgrade gevent version in bazel requirement (#31115) 2022-09-26 11:10:27 -07:00
Craig Tiller ebc4f236b6
[slice] Remove _internal variants of APIs (#30953)
* [slice] Remove _internal variants of APIs

* Automated change: Fix sanity tests

* fix

* reduce bloat

* fixes

* Automated change: Fix sanity tests

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2022-09-24 13:34:00 -07:00
Craig Tiller 93fdc99756
[init] Remove grpc_register_plugin (#30929)
* [init] Remove grpc_register_plugin

* Automated change: Fix sanity tests

* fix

* Automated change: Fix sanity tests

* fix

* fix

* fix

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2022-09-23 22:08:01 -07:00
Craig Tiller 1402e974f7
[promises] End-to-end flow control for a series of pipes (#30994)
* fix promise

* prototype

* progress

* implement new api

* Revert "fix promise"

This reverts commit ded85e7d19.

* Revert "Revert "fix promise""

This reverts commit c2acef1958.

* progress

* done

* Automated change: Fix sanity tests

* fix

* fix

* fix

* Automated change: Fix sanity tests

* comments

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2022-09-23 14:04:33 -07:00
Richard Belleville 3439cc29d2
Update xds-protos package to pull in new generated code (#31113) 2022-09-23 13:41:25 -07:00
Craig Tiller 32b43f0168
[channel_args] Make channel_stack_init take new type (#30841)
* [channel_args] Make channel_stack_init take new type

* fix

* fix

* fix

* Automated change: Fix sanity tests

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2022-09-23 11:31:03 -07:00
Yousuk Seung 747d016397
Support --max_recv_msg_size in grpc cli (#31106)
* Support --max_recv_msg_size in grpc cli

* Comment typos fixed in tests.
2022-09-23 11:13:29 -07:00
Craig Tiller 62c2260dc2
[experiments] Add a placeholder experiment to test rollout monitoring (#31110) 2022-09-23 11:03:32 -07:00
AJ Heller fb14fdf0e0
Performant thread-safe Work Queue (#30821)
* WorkQueue

* weaken the large obj stress test for Windows; documentation

* update comment

* Add WorkQueue microbenchmark. Results below ...

------------------------------------------------------------------------------------------
Benchmark                                Time             CPU   Iterations UserCounters...
------------------------------------------------------------------------------------------
BM_WorkQueueIntptrPopFront/1           297 ns          297 ns      2343500 items_per_second=3.3679M/s
BM_WorkQueueIntptrPopFront/8          7022 ns         7020 ns        99356 items_per_second=1.13956M/s
BM_WorkQueueIntptrPopFront/64        59606 ns        59590 ns        11770 items_per_second=1074k/s
BM_WorkQueueIntptrPopFront/512      477867 ns       477748 ns         1469 items_per_second=1071.7k/s
BM_WorkQueueIntptrPopFront/4096    3815786 ns      3814925 ns          184 items_per_second=1073.68k/s
I0902 19:05:22.138022069      12 test_config.cc:194]         TestEnvironment ends
================================================================================

* use int64_t for times. 0 performance change

------------------------------------------------------------------------------------------
Benchmark                                Time             CPU   Iterations UserCounters...
------------------------------------------------------------------------------------------
BM_WorkQueueIntptrPopFront/1           277 ns          277 ns      2450292 items_per_second=3.60967M/s
BM_WorkQueueIntptrPopFront/8          6718 ns         6716 ns       105497 items_per_second=1.19126M/s
BM_WorkQueueIntptrPopFront/64        56428 ns        56401 ns        12268 items_per_second=1.13474M/s
BM_WorkQueueIntptrPopFront/512      458953 ns       458817 ns         1550 items_per_second=1.11591M/s
BM_WorkQueueIntptrPopFront/4096    3686357 ns      3685120 ns          191 items_per_second=1.1115M/s
I0902 19:25:31.549382949      12 test_config.cc:194]         TestEnvironment ends
================================================================================

* add PopBack tests: same performance profile exactly

* use Mutex instead of Spinlock

It's safer, and so far equally performant in benchmarks of opt builds

* add deque test for comparison. It is faster on all tests.

* Add sparsely-populated multi-threaded benchmarks.

* fix

* fix

* refactor to help thread safety analysis

* Specialize WorkQueue for Closure*s and AnyInvocables

* remove unused callback storage

* add single-threaded benchmark for closure vs invocable

* sanitize

* missing include

* move bm_work_queue to microbenchmarks so it isn't exported

* s/workqueue/work_queue/g

* use nullptr instead of optionals for popped closures

* reviewer test suggestion

* private things are private

* add a work_queue fuzzer

Ran for 10 minutes @ 42 jobs @ 42 workers. Zero failures.

Checked in a selection of 100 good seeds after merging the thousands of
results.

* fix

* fix header guards

* nuke the corpora

* feedback

* sanitize

* Timestamp::Now

* fix

* fuzzers do not work on windows

* windows does not like multithreaded benchmark tests
2022-09-23 09:10:42 -07:00
Craig Tiller f15ba1ffc7
[tls] Remove support for pthread tls (#31040)
* [tls] Remove support for pthread tls

* fix

* fix

* fix

* lalala

* fix

* fix

* Clean up deployment target changes

* additional clean up of deployment target

* regen podspec for updated file list

* remove destination change

* deployment target override for ios cpp test

* fix?

* fix run test script & update test destination

* [tls] Remove support for pthread tls

* fix

* fix

* fix

* lalala

* fix

* fix

* Clean up deployment target changes

* additional clean up of deployment target

* regen podspec for updated file list

* remove destination change

* deployment target override for ios cpp test

* fix?

* fix run test script & update test destination

* merge

* fix

* final script fix for proper destination & target

* more deployment target fix

* objc ios test script fix

Co-authored-by: Denny C. Dai <dennycd@me.com>
Co-authored-by: dennycd <dennycd@google.com>
2022-09-22 22:39:35 -07:00
Craig Tiller c3714eced8
fix test (#31101) 2022-09-22 20:22:13 -07:00
Craig Tiller e8df8185e5
[debug] Fine grained event tracing (#31105)
* moved-from-stats

* [debug] Fine grained event tracing

* Automated change: Fix sanity tests

* comment

* Automated change: Fix sanity tests

* iwyu

* Automated change: Fix sanity tests

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2022-09-22 16:32:10 -07:00
Esun Kim 945fe012ed
Silient unused-but-set-parameter in case of Contexts = {} (#30456)
* Silient  in case of Contexts = {}

* Fix
2022-09-22 16:14:57 -07:00
Esun Kim f8de7c8c68
Clean up supported bazel versions (#31067)
* Bazel versions

* Generate docs
2022-09-22 16:14:00 -07:00
Richard Belleville cfe7d52d98
Update install_all_python_modules.sh to account for new packages (#31103)
* Update install_all_python_modules.sh to account for new packages

* Improve readability
2022-09-22 13:56:47 -07:00
Vignesh Babu 2e78e92dbe
Rename RETURN_IF_ERROR to RETURN_IF_NOT_OK (#31102)
* Add utils to create and prepare socket for tcp client

* Automated change: Fix sanity tests

* review comments

* review comments

* regenerate projects

* regenerate projects

* review comments

* comments

* fix typo

* remove unused parameter

* review comments

* remove static

* re-introduce forward declaration

* Rename RETURN_IF_ERROR to RETURN_IF_NOT_OK

Co-authored-by: Vignesh2208 <Vignesh2208@users.noreply.github.com>
2022-09-22 11:38:10 -07:00
Esun Kim 097729125a
Removed a unnecessary function (#31090) 2022-09-22 10:07:08 -07:00
Vignesh Babu 9bcee18e1a
Add utils to create and prepare socket for tcp client (#31009)
* Add utils to create and prepare socket for tcp client

* Automated change: Fix sanity tests

* review comments

* review comments

* regenerate projects

* regenerate projects

* review comments

* comments

* fix typo

* remove unused parameter

* review comments

* remove static

* re-introduce forward declaration

Co-authored-by: Vignesh2208 <Vignesh2208@users.noreply.github.com>
2022-09-22 09:14:50 -07:00
Cheng-Yu Chung eada386921
Revert "Remove `grpc_codegen` (#30960)" (#31099)
This reverts commit b2d64efa3a.
2022-09-22 08:29:43 -07:00
Craig Tiller 6575ae70b3
[slice] Ensure result is used in test (#31083)
Otherwise sufficiently good compilers may inline SumSlice, see that the result is not used, and discard the code that triggers the crash entirely.
2022-09-21 17:22:22 -07:00
Craig Tiller 52f21adb01
[experiments] Mark periodic resource quota reclamation unbroken (#31088) 2022-09-21 17:04:43 -07:00
Cheng-Yu Chung b2d64efa3a
Remove `grpc_codegen` (#30960)
* Remove `grpc_codegen`

* Update
2022-09-21 18:16:34 -04:00
Mark D. Roth e5a602fd6d
outlier detection LB policy: add ctors to work around internal toolchain bug (#31091) 2022-09-21 15:09:53 -07:00
Mark D. Roth b340c37305
XdsClient: simplify XdsResourceType::Decode() API (#31070)
* simplify XdsResourceType::Decode() API

* fix xds_client_test

* fix sanity
2022-09-21 13:41:11 -07:00
Craig Tiller bd6fab6ab7
[iomgr] Reduce the size of combiner_test (#31085) 2022-09-21 13:21:16 -07:00
Zach Reyes f7c3f4df3a
Add v1.49.0 release of grpc-go to interop matrix (#30719) 2022-09-21 10:47:32 -07:00
AJ Heller c0e5e35c7a
Replace EventEngine::Promise with grpc_core::Notification (#31027)
* Replace EventEngine::Promise with grpc_core::Notification

* remove promise.h

* Automated change: Fix sanity tests

* fix windows

* fix iocp

* fix client_test

Co-authored-by: drfloob <drfloob@users.noreply.github.com>
2022-09-21 09:30:23 -07:00
Craig Tiller 1ad4bbe78e
Revert "fix (#31018)" (#31021)
This reverts commit 2e6f268dfc.
2022-09-21 09:28:28 -07:00
Mark D. Roth dccd69283f
LB policies: fix use of declarative JSON API (#31072) 2022-09-21 05:38:20 -07:00
Jan Tattermusch 7df6ae60d5
make objc kokoro jobs compatible with macos monterey image (#31065) 2022-09-21 13:08:51 +02:00
Craig Tiller 4420d11ee0
[event-engine] Throttle thread starting in thread pool (#31075)
* [event_engine] Thread pool thread start throttling

* add test

* review feedback

* fix
2022-09-20 19:45:25 -07:00
Craig Tiller da08fe1d6f
Revert "Remove `include/grpcpp/impl/codegen/client_callback.h` (#31005)" (#31079)
This reverts commit 17f515744f.
2022-09-20 18:58:54 -07:00
Yash Tibrewal 9cff4d2d28
OpenCensus Plugin: Add missing measure and views for started RPCs (#31034) 2022-09-20 17:26:16 -07:00
Richard Belleville 2d94537854
Re-add compiler protos to grpcio-tools (#31062)
* Re-add compiler protos to grpcio-tools

* Attempt to break

* Actually fail if protobufs don't compile. Who wrote this thing?

* Add an actual test

* yapf

* Whoops. Thought I'd reverted you

* Please the formatter gods
2022-09-20 17:15:39 -07:00
Esun Kim ffbd468982
Clean-up (#31073) 2022-09-20 17:04:31 -07:00
CZRZ 90f4f33536
Remove unnecessary gflags related comments in makefile (#28846)
Signed-off-by: chengzrz <czrzrichard@gmail.com>

Signed-off-by: chengzrz <czrzrichard@gmail.com>
2022-09-20 16:04:12 -07:00
Denny C. Dai 343cb0ecb9
Setting C++ lang dialect for objc cocoapod specs (#31071) 2022-09-20 13:59:17 -07:00
Esun Kim fa6571634c
Upmerge changes from 1.49.x (#31068)
* Bump 1.49.x branch to 1.49.0.pre1 (#30615)

* bump version to 1.49.0-pre1

* regenerate projects

* [backport][v1.49.x] forkable fixes (#30646)

* Fix forkable globals (#30608)

* Fix forkable repeated registration (#30642)

This fixes a bug that could occur on repeated grpc initialization (after a complete shutdown)

Fixes #30640

* Drop support for ruby 2.5 (#30699) (#30762)

* Drop ruby 2.5 support

* Backport: "stabilize the C2P resolver URI scheme" to v1.49.x (#30654)

* stabilize the C2P resolver URI scheme

* Bump 1.49 branch to 1.49.0.pre2 (#30786)

* bump version to 1.49.0-pre2

* regenerate projects

* Update protobuf on ancillary packages (#30795) (#30805)

* Bump release version on 1.49 to 1.49.0.pre3 (#30814)

* bump version to 1.49.0-pre3

* regenerate projects

* xDS interop: enable pod log collection in the buildscripts (#30735) (#30856)

- Enables pod log collection in all PSM interop jobs implemented in https://github.com/grpc/grpc/pull/30594.
- Associate test suite runs with their own log file, so it's displayed on "Target Log" tab

* xDS interop: buildscripts: fix run_test return status (#30768) (#30875)

To capture the return status of the test in run_test the last command must be the call to the test itself.
This removes `set +x`, which makes the run_test always return success, and not propagate the test status.

I can't find it, but this exact error bit us before. Looks like it leaked to other scripts.
The good thing is if the test was executed, it's failure would still be picked up from the result xml.

However, if the test framework didn't start in the first place, the result will be false positive.
Example: https://source.cloud.google.com/results/invocations/98d3e679-ec8a-40bd-9f36-88179747b0d6/targets

```
/home/kbuilder/.pyenv/versions/k8s_xds_test_runner/bin/python3: Error while finding module specification for 'tests.authz_test' (ModuleNotFoundError: No module named 'tests')
+ set +x
Failed test suites: 0

[ID: 3548168] Command finished after 625 secs, exit value: 0
```

* client_channel: fix crash when cancelling a watch after SHUTDOWN (#30885) (#30928)

* Support Python 3.11 (#30818) (#30944)

* Support Python 3.11

* Update build images for 3.11

* Whoopsie

* The architecture of this thing is garbage

* Silence ownership warning

* Account for change in git behavior

* Fix directory

* I am in great pain

* Update Windows and arm linux

* Agh

* Clean up

* Bump 1.49 branch to 1.49.0 (#30974)

* bump version to 1.49.0

* regenerate projects

* Update protobuf to v21.6 on 1.49.x (#31028)

* Update third_party/protobuf

* run tools/distrib/python/make_grpcio_tools.py

* update build_handwritten.yaml

* regenerate projects

* Bump v1.49.x to v1.49.1 (#31037)

* bump version to 1.49.1

* regenerate projects

* Automated change: Fix sanity tests

Co-authored-by: gnossen <gnossen@users.noreply.github.com>

* Fix ruby windows ucrt build (#31053)

Co-authored-by: apolcyn <apolcyn@google.com>
Co-authored-by: AJ Heller <hork@google.com>
Co-authored-by: Richard Belleville <rbellevi@google.com>
Co-authored-by: Sergii Tkachenko <sergiitk@google.com>
Co-authored-by: Mark D. Roth <roth@google.com>
Co-authored-by: Richard Belleville <gnossen@gmail.com>
Co-authored-by: gnossen <gnossen@users.noreply.github.com>
2022-09-20 12:03:33 -07:00
Mark D. Roth 07df5ff9c7
json_object_loader: refactor ErrorList into its own library (#31049)
* json_object_loader: refactor ErrorList into its own library

* fix observability_config_test

* generate_projects

* iwyu
2022-09-20 11:58:27 -07:00