Commit Graph

13071 Commits

Author SHA1 Message Date
Hannah Shi 96d27634a4
run ios CronetTest from bazel (#30439) 2022-09-27 14:49:38 -07:00
Aaron Pham ac585b8f7e
chore: add bazel support for M1 mac (#31127)
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
2022-09-27 13:17:35 -07:00
Jan Tattermusch 2f5d06af6b
make legacy artifact -> packages -> distribtest jobs a noop (#31124) 2022-09-27 21:01:32 +02:00
Craig Tiller 5fa37322cb
Revert "Tcp endpoint implementation for posix event engine. (#30559)" (#31134)
This reverts commit a81391d931.
2022-09-27 08:04:25 -07:00
Vignesh Babu a81391d931
Tcp endpoint implementation for posix event engine. (#30559)
* convert tcp code to use EndpointConfig and successfully compile

* regenerate projects

* copying config options used in tcp code to create map backed endpoint configs

* fix minor build issues

* fix includes in tests

* fix some build and test issues

* modifying endpoint config definition from an abstract interface into a concrete implementation

* minor fix

* add unreachable code check

* fix some windows build issues

* fix more build issues

* adding a grpc_tcp_generic_options struct to hold data extracted from EndpointConfig

* code formatting

* simplify

* fix sanity checks

* fix windows issues

* some posix fixes

* disabling copy constructor and copy assignment operator for endpoint config

* fix sanity checks

* fix syntax error

* fix weird formatting

* fix build issue

* fix review comments

* reverting un-necessary change

* remove channel args completely from windows impl since it is unused

* remove unused headers and fix usage in endpoint_pair_windows.cc

* cleanup

* cleanup

* fix some initialization issues

* re-generate projects

* removing some files

* fix ifdef for windows

* fixing windows build issue due to improper header file include

* Automated change: Fix sanity tests

* start

* regenerate-projects

* delete some files

* Automated change: Fix sanity tests

* review comments

* add comment

* Migrating posix event pollers to use new event poller interface

* add inline attributes

* Automated change: Fix sanity tests

* remove ref/unref from iomgr engine closure ad add custom closure types internal to the pollers

* updating time util usage

* use unique_ptrs

* update comments

* Automated change: Fix sanity tests

* review comments

* rename GetPointer to GetVoidPointer

* cleanup

* Automated change: Fix sanity tests

* removing EndpointConfig::Get method

* Automated change: Fix sanity tests

* cleanup

* fix

* fix review comments

* review comments

* cleanup

* update comments

* fix

* cleanup

* update comments

* Automated change: Fix sanity tests

* fix misleading comments

* bug fixes

* fix

* fix

* start

* revert some changes for bug fixes and allow spurious wakeups for poll based poller

* sanity

* fix

* minor fix

* review comments

* fix

* comment

* remove re-defined function

* fix review comments

* fix windows iocp build issue due to removed function

* change Milliseconds return type

* fix

* remove header

* regenerate projects

* fix sanity

* code

* fix sanity

* Automated change: Fix sanity tests

* Forking iomgr internal_errqueue defines for posix event engine

* fix BUILD file

* start

* update

* sanity

* regenerate_projects

* Automated change: Fix sanity tests

* rename

* review comments

* update

* review comments

* Automated change: Fix sanity tests

* fix BUILD

* add no_windows tag

* update

* update

* remove debug statements

* delete unused file

* update

* fix BUILD

* regenerate projects

* update

* fix BUILD

* update

* fix sanity

* minor additions to configure global tcp user timeout values

* cleanup

* fix build syntax

* change poller interface for posix event engine

* update event_poller_posix_test

* fix build error

* Add more unit tests

* windows ee changes

* update comment

* remove unused deps

* fix msan issue

* add comment

* adding rcv lowat helper to socket wrapper

* fix sanity

* review comments

* fix build issue

* fix sanity

* sanity

* iwyu sanity

* fix build

* merge conflict

* Automated change: Fix sanity tests

* update

* update

* fix macos issue

* fix msan and ubsan issues

* fix macos build issue

* comment

* review comments

* remove unused dep

* sanity

* merge conflict

* mac build issue

* regenerate projects

* build issue

* fix typo and address review comments

* review comments

* Automated change: Fix sanity tests

* update

* merge issue

* review comments

* sanity

* fix build issue

* review comments

* add thread annotations

* MacOS fix unused variable

* add some more #endif comments

* missing semi-colon

* Automated change: Fix sanity tests

* sanity

* sanity again

* sanity try again

* sanity again

* try again with no_include pragma

Co-authored-by: Vignesh2208 <Vignesh2208@users.noreply.github.com>
2022-09-26 17:20:17 -07:00
Jan Tattermusch 255083b724
Fix ruby builds on kokoro macos monterey images (#31125)
* cocoapods is not a ruby dependency

* fix rvm update on macos monterey images
2022-09-26 20:46:54 +02:00
Mark D. Roth cdb7d2c93c
XdsEndpointResourceType: use ValidationErrors and add unit tests (#31076)
* simplify XdsResourceType::Decode() API

* fix xds_client_test

* xDS endpoint: use ValidationErrors to improve error messages

* fix sanity

* add xDS endpoint resource type test

* clang-format

* remove a couple of now-unnecessary end2end tests

* generate_projects

* fix xds_csds_e2e_test

* iwyu

* attempt to avoid ubsan failure
2022-09-26 11:45:56 -07:00
Craig Tiller ebc4f236b6
[slice] Remove _internal variants of APIs (#30953)
* [slice] Remove _internal variants of APIs

* Automated change: Fix sanity tests

* fix

* reduce bloat

* fixes

* Automated change: Fix sanity tests

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2022-09-24 13:34:00 -07:00
Richard Belleville 3439cc29d2
Update xds-protos package to pull in new generated code (#31113) 2022-09-23 13:41:25 -07:00
AJ Heller fb14fdf0e0
Performant thread-safe Work Queue (#30821)
* WorkQueue

* weaken the large obj stress test for Windows; documentation

* update comment

* Add WorkQueue microbenchmark. Results below ...

------------------------------------------------------------------------------------------
Benchmark                                Time             CPU   Iterations UserCounters...
------------------------------------------------------------------------------------------
BM_WorkQueueIntptrPopFront/1           297 ns          297 ns      2343500 items_per_second=3.3679M/s
BM_WorkQueueIntptrPopFront/8          7022 ns         7020 ns        99356 items_per_second=1.13956M/s
BM_WorkQueueIntptrPopFront/64        59606 ns        59590 ns        11770 items_per_second=1074k/s
BM_WorkQueueIntptrPopFront/512      477867 ns       477748 ns         1469 items_per_second=1071.7k/s
BM_WorkQueueIntptrPopFront/4096    3815786 ns      3814925 ns          184 items_per_second=1073.68k/s
I0902 19:05:22.138022069      12 test_config.cc:194]         TestEnvironment ends
================================================================================

* use int64_t for times. 0 performance change

------------------------------------------------------------------------------------------
Benchmark                                Time             CPU   Iterations UserCounters...
------------------------------------------------------------------------------------------
BM_WorkQueueIntptrPopFront/1           277 ns          277 ns      2450292 items_per_second=3.60967M/s
BM_WorkQueueIntptrPopFront/8          6718 ns         6716 ns       105497 items_per_second=1.19126M/s
BM_WorkQueueIntptrPopFront/64        56428 ns        56401 ns        12268 items_per_second=1.13474M/s
BM_WorkQueueIntptrPopFront/512      458953 ns       458817 ns         1550 items_per_second=1.11591M/s
BM_WorkQueueIntptrPopFront/4096    3686357 ns      3685120 ns          191 items_per_second=1.1115M/s
I0902 19:25:31.549382949      12 test_config.cc:194]         TestEnvironment ends
================================================================================

* add PopBack tests: same performance profile exactly

* use Mutex instead of Spinlock

It's safer, and so far equally performant in benchmarks of opt builds

* add deque test for comparison. It is faster on all tests.

* Add sparsely-populated multi-threaded benchmarks.

* fix

* fix

* refactor to help thread safety analysis

* Specialize WorkQueue for Closure*s and AnyInvocables

* remove unused callback storage

* add single-threaded benchmark for closure vs invocable

* sanitize

* missing include

* move bm_work_queue to microbenchmarks so it isn't exported

* s/workqueue/work_queue/g

* use nullptr instead of optionals for popped closures

* reviewer test suggestion

* private things are private

* add a work_queue fuzzer

Ran for 10 minutes @ 42 jobs @ 42 workers. Zero failures.

Checked in a selection of 100 good seeds after merging the thousands of
results.

* fix

* fix header guards

* nuke the corpora

* feedback

* sanitize

* Timestamp::Now

* fix

* fuzzers do not work on windows

* windows does not like multithreaded benchmark tests
2022-09-23 09:10:42 -07:00
Craig Tiller f15ba1ffc7
[tls] Remove support for pthread tls (#31040)
* [tls] Remove support for pthread tls

* fix

* fix

* fix

* lalala

* fix

* fix

* Clean up deployment target changes

* additional clean up of deployment target

* regen podspec for updated file list

* remove destination change

* deployment target override for ios cpp test

* fix?

* fix run test script & update test destination

* [tls] Remove support for pthread tls

* fix

* fix

* fix

* lalala

* fix

* fix

* Clean up deployment target changes

* additional clean up of deployment target

* regen podspec for updated file list

* remove destination change

* deployment target override for ios cpp test

* fix?

* fix run test script & update test destination

* merge

* fix

* final script fix for proper destination & target

* more deployment target fix

* objc ios test script fix

Co-authored-by: Denny C. Dai <dennycd@me.com>
Co-authored-by: dennycd <dennycd@google.com>
2022-09-22 22:39:35 -07:00
Craig Tiller e8df8185e5
[debug] Fine grained event tracing (#31105)
* moved-from-stats

* [debug] Fine grained event tracing

* Automated change: Fix sanity tests

* comment

* Automated change: Fix sanity tests

* iwyu

* Automated change: Fix sanity tests

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2022-09-22 16:32:10 -07:00
Richard Belleville cfe7d52d98
Update install_all_python_modules.sh to account for new packages (#31103)
* Update install_all_python_modules.sh to account for new packages

* Improve readability
2022-09-22 13:56:47 -07:00
Zach Reyes f7c3f4df3a
Add v1.49.0 release of grpc-go to interop matrix (#30719) 2022-09-21 10:47:32 -07:00
AJ Heller c0e5e35c7a
Replace EventEngine::Promise with grpc_core::Notification (#31027)
* Replace EventEngine::Promise with grpc_core::Notification

* remove promise.h

* Automated change: Fix sanity tests

* fix windows

* fix iocp

* fix client_test

Co-authored-by: drfloob <drfloob@users.noreply.github.com>
2022-09-21 09:30:23 -07:00
Jan Tattermusch 7df6ae60d5
make objc kokoro jobs compatible with macos monterey image (#31065) 2022-09-21 13:08:51 +02:00
Richard Belleville 2d94537854
Re-add compiler protos to grpcio-tools (#31062)
* Re-add compiler protos to grpcio-tools

* Attempt to break

* Actually fail if protobufs don't compile. Who wrote this thing?

* Add an actual test

* yapf

* Whoops. Thought I'd reverted you

* Please the formatter gods
2022-09-20 17:15:39 -07:00
Mark D. Roth 07df5ff9c7
json_object_loader: refactor ErrorList into its own library (#31049)
* json_object_loader: refactor ErrorList into its own library

* fix observability_config_test

* generate_projects

* iwyu
2022-09-20 11:58:27 -07:00
Esun Kim 78c7cda232
Bump core version to 28.0.0 for upcoming release (#31069)
* bump C-core version

* regenerate projects
2022-09-20 11:41:58 -07:00
Craig Tiller bc4f98bb36
[build] Fixes for experiments, poll strategy (#31042)
* fix

* Automated change: Fix sanity tests

* fix

* fix

* fix

* fix

* fix

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2022-09-19 22:19:36 -07:00
sanjaypujare 731a7d6fd1
xds-interop-tests: we have decided to enable cross-lang testing for master and latest only (currently 1.49) (#30794) 2022-09-18 09:19:44 -07:00
Richard Belleville e609ad88c9
Update protobuf to v21.6 (#31015)
* Update third_party/protobuf

* run tools/distrib/python/make_grpcio_tools.py

* update build_handwritten.yaml

* regenerate projects

* Update Python version floor
2022-09-16 17:50:52 -07:00
Cheng-Yu Chung 7cb9731e4e
Remove `include/grpcpp/impl/codegen/call_op_set_interface.h` (#30990) 2022-09-16 17:27:50 -04:00
Craig Tiller 6b95573959
Reland: [event_engine] Thread pool that can handle deletion in a callback (#30996)
* Revert "Revert "Revert "Revert "[event_engine] Thread pool that can handle deletion in a callback"" (#30973)" (#30995)"

This reverts commit dffdd8c083.

* tp-fix

* signalling fix

* better test

* use grpc_core::Notification

* comment

* Automated change: Fix sanity tests

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2022-09-16 10:37:57 -07:00
Craig Tiller d9ac89a441
[gprpp] absl::Notification polyfill (#31008)
* [gprpp] absl::Notification polyfill

* grpc_core::

* Automated change: Fix sanity tests

* fix

* Automated change: Fix sanity tests

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2022-09-16 09:06:03 -07:00
Paulo Castello da Costa 5d2767601a
Delete obsolete import. (#31010) 2022-09-16 02:04:06 -07:00
Richard Belleville ead11434c3
Update release verification script for now PyPi API (#31007) 2022-09-15 15:42:16 -07:00
Zach Reyes e21199e7c9
xDS Interop: Add GoLang master to supported Outlier Detection versions (#30855) 2022-09-15 13:22:50 -07:00
apolcyn dfb432c217
Revert "Revert "[chttp2] fix stream leak with queued flow control update and absence of writes (#30907)" (#30991)" (#30992)
* properly synchronize grpc_iomgr_count_objects_for_testing

* Revert "Revert "[chttp2] fix stream leak with queued flow control update and absence of writes (#30907)" (#30991)"

This reverts commit 0f2a0f5fc9.
2022-09-15 08:23:21 -07:00
Craig Tiller dffdd8c083
Revert "Revert "Revert "[event_engine] Thread pool that can handle deletion in a callback"" (#30973)" (#30995)
This reverts commit fed749d100.
2022-09-14 16:45:26 -07:00
Esun Kim 888c64adf8
Fix the build script (#30985)
* Fix the build script

* Set -ex

* fix unsigned int comparisons and ignored return values

Co-authored-by: AJ Heller <hork@google.com>
2022-09-14 15:36:55 -07:00
Craig Tiller 0f2a0f5fc9
Revert "[chttp2] fix stream leak with queued flow control update and absence of writes (#30907)" (#30991)
This reverts commit d8f98fb1a7.
2022-09-14 14:44:04 -07:00
Craig Tiller c2ab8c99bb
Revert "Revert "[c++] Move environment functions to C++ (#30937)" (#30986)" (#30988)
This reverts commit 96264e07b8.
2022-09-14 14:19:36 -07:00
Cheng-Yu Chung d07224a13e
Remove `include/grpcpp/impl/codegen/call_hook.h` (#30967) 2022-09-14 16:44:30 -04:00
apolcyn d8f98fb1a7
[chttp2] fix stream leak with queued flow control update and absence of writes (#30907)
fix stream leak caused by queued flow control update and absence of writes
2022-09-14 13:06:08 -07:00
Craig Tiller 96264e07b8
Revert "[c++] Move environment functions to C++ (#30937)" (#30986)
This reverts commit 74c0d6fe3f.
2022-09-14 11:53:55 -07:00
Craig Tiller 74c0d6fe3f
[c++] Move environment functions to C++ (#30937)
* [gprpp] Move env to C++

* move headers/impl

* Automated change: Fix sanity tests

* fix

* fix

* Automated change: Fix sanity tests

* Update http_proxy.cc

* fix

* fix

* rename

* fix merge

* fix

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2022-09-14 10:17:15 -07:00
Craig Tiller fed749d100
Revert "Revert "[event_engine] Thread pool that can handle deletion in a callback"" (#30973)
* Revert "Revert "[event_engine] Thread pool that can handle deletion in a callback (#30763)" (#30972)"

This reverts commit ccc787a020.

* Update thread_pool.cc
2022-09-14 10:16:52 -07:00
Richard Belleville b15097142a
Silence dubious ownership issue in bazel image (#30963)
* Silence dubious ownership issue in bazel image

* Agh. It's a template
2022-09-13 16:42:21 -07:00
Richard Belleville ccc787a020
Revert "[event_engine] Thread pool that can handle deletion in a callback (#30763)" (#30972)
This reverts commit f6e1cf1dc0.
2022-09-13 15:28:20 -07:00
Craig Tiller 92f58c18a8
Revert "Revert "[chttp2] Improve huffman decode efficiency (#30479)" (#30961)" (#30962)
This reverts commit bfea47093f.
2022-09-13 13:16:55 -07:00
Craig Tiller f6e1cf1dc0
[event_engine] Thread pool that can handle deletion in a callback (#30763)
* [event_engine] Thread pool that can handle deletion in a callback

* missed file

* Automated change: Fix sanity tests

* simplify memory model

* review feedback

* Automated change: Fix sanity tests

* detect-stuckness

* fix

* Automated change: Fix sanity tests

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2022-09-13 13:12:44 -07:00
Craig Tiller bfea47093f
Revert "[chttp2] Improve huffman decode efficiency (#30479)" (#30961)
This reverts commit 6c7f921f5f.
2022-09-13 12:25:00 -07:00
Craig Tiller 6c7f921f5f
[chttp2] Improve huffman decode efficiency (#30479)
* first pass

* refinement #1

* fix

* opt

* 8

* 5

* 12

* 10

* 7

* 15

* 8n

* 15n

* 8a

* 16a

* 15a

* 5a

* 10a

* 7a

* 9a

* 12a

* 11a

* 11b

* 11b

* 7b

* 15b

* 14b

* 15i

* 12i

* 9i

* 10i

* finalizing

* fix

* tweak

* 10f

* 10r

* 7r

* 8r

* 9r

* more

* rething

* 8s

* 9s

* 12s

* 7s

* 10s

* 15snr

* 12snr

* 14snr

* 13snr

* 11snr

* 20snr

* 15snr

* 18snr

* 7snr

* 7snr

* 12-2stp

* 13-2stp

* 14-2stp

* 11-2stp

* 10-2stp

* 9-3stp

* 8-3stp

* 7-3stp

* 8-2stp

* auto-tune1

* max-depth-2

* max-depth-3

* max-depth-2

* abbreviate

* working version

* add benchmark

* fix

* fix

* fix

* ditch define

* better code layout

* static fns

* fix

* Automated change: Fix sanity tests

* Automated change: Fix sanity tests

* fix build

* cleanup and comment

* fmt

* fix

* test

* tag

* Automated change: Fix sanity tests

* fix

* fix

* Automated change: Fix sanity tests

* fix

* fix

* fix

* progress

* progress

* progress

* fixes

* cleanup

* Automated change: Fix sanity tests

* Automated change: Fix sanity tests

* fmt

* fix

* fix

* Automated change: Fix sanity tests

* fix

* fix

* Automated change: Fix sanity tests

* fix

* add encode/decode fuzzer

* Automated change: Fix sanity tests

* fix

* fix

* fix

* review feedback

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2022-09-13 11:24:35 -07:00
Vignesh Babu 0867393bde
Revert "Revert "Forking tcp socket utils for posix event engine endpoints"" (#30949)
* Revert "Revert "Forking tcp socket utils for posix event engine endpoints (#30383)" (#30946)"

This reverts commit 33b3fe89c4.

* make test robust
2022-09-12 18:21:01 -07:00
Mark D. Roth aaec373a10
Second attempt: XdsClient: add unit test and fix watcher notification bugs (#30943)
* Revert "Revert "XdsClient: add unit test and fix watcher notification bugs (#30823)" (#30942)"

This reverts commit 6d2c4a8314.

* use GRPC_CUSTOM_JSONUTIL macro for JsonPrintOptions
2022-09-12 16:58:31 -07:00
Richard Belleville 33b3fe89c4
Revert "Forking tcp socket utils for posix event engine endpoints (#30383)" (#30946)
This reverts commit d7cce32449.
2022-09-12 14:29:56 -07:00
AJ Heller 95aeffcb73
Pythonify extract_metadata_from_bazel_xml.py (#30796)
Autodetected improvements.
2022-09-12 13:07:02 -07:00
Richard Belleville 6d2c4a8314
Revert "XdsClient: add unit test and fix watcher notification bugs (#30823)" (#30942)
This reverts commit bcd8c991e6.
2022-09-12 12:01:42 -07:00
Richard Belleville 2297249e47
Support Python 3.11 (#30818)
* Support Python 3.11

* Update build images for 3.11

* Whoopsie

* The architecture of this thing is garbage

* Silence ownership warning

* Account for change in git behavior

* Fix directory

* I am in great pain

* Update Windows and arm linux

* Agh

* Clean up
2022-09-12 11:53:44 -07:00