Commit Graph

3952 Commits

Author SHA1 Message Date
Vitaly Buka 9d0922e02d
Init grpc_completion_queue_attributes::cq_shutdown_cb (#27864) 2021-11-01 10:07:09 -07:00
Lidi Zheng b8c38d6bc2
Support generic_xds_config in GCE/GKE frameworks and Python tests (#27796)
* Support generic_xds_config in GCE/GKE frameworks and Python tests

* Correct the field name

* Address comments

* Add new CSDS support to the regular K8s testcases

* Support incomplete CSDS config dump
2021-10-25 16:41:48 -07:00
Lidi Zheng 96995801d8
[Aio] Allow sequential types for server interceptors (#27818) 2021-10-25 13:09:06 -07:00
Yash Tibrewal 6d2a641dd3
XdsChannelStackModifier class (#27744)
* ChannelStackModifier class

* Regenerate projects

* clang-tidy

* Use CoreConfiguration for inserting xDS HTTP filters

* New lines

* Move test to test/core/xds

* Add comment for placement of filter stack

* Fix sanity

* Fix memory leak - destroy builder

* Don't build xds_channel_stack_modifier_test on non-bazel build systems due to census dependency
2021-10-22 13:48:30 -07:00
Richard Belleville eab375e439
Reland #27275 (#27806)
* Revert "Revert "Add Aspects to Bazel py_proto_library and py_grpc_library Rules (#27275)" (#27805)"

This reverts commit 763e5b40f3.

* Remove unused symbol
2021-10-21 19:37:24 -07:00
Craig Tiller 763e5b40f3
Revert "Add Aspects to Bazel py_proto_library and py_grpc_library Rules (#27275)" (#27805)
This reverts commit e970b8fa03.
2021-10-21 17:05:56 -07:00
Michael Beardsworth e970b8fa03
Add Aspects to Bazel py_proto_library and py_grpc_library Rules (#27275)
* Move most py_proto_library macro contents into rule.

The _generate_pb2_src rule is modified to produce a valid PyInfo
provider that can be depended on directly by py_.* rules.

Keeping an intermediate macro in place increases complexity without
adding any real value.

* Move most py_grpc_library macro contents into rule.

The _generate_pb2_grpc_src rule is modified to produce a valid PyInfo
provider that can be depended on directly by py_.* rules.

Keeping an intermediate macro in place increase complexity without
adding any real value.

* Remove support for additional plugins.

It's easy enough for a plugin author to build their own rule-set, and it
adds extra unused (and untested) complexity to gRPC's rules.

* Rewrite py_proto_library to track dependencies correctly.

* Propagate Python dependencies of py_grpc_library rules to consumers.

* Add test coverage for aspect-based py_proto_library.
2021-10-21 15:56:37 -07:00
Ming-Chuan 34fdb542b5
Add resolver for binder transport URI scheme (#27529)
This commit is part of the effort to create binder channel with
GRPC_CLIENT_CHANNEL type.

The resolver will be used during name resolution, and the result will
later be used to identify the corresponding endpoint binder in
SubchannelConnector.

Besides the unit test, this change is tested with other changes locally
end to end on real device.
2021-10-21 14:17:19 +08:00
Richard Belleville 25d8458721
Migrate remaining tests with protobuf dep to Python 3 only (#27776)
* Migrate remaining tests with protobuf dep to Python 3 only

* Move Bazel Python repo tests off of Python 2

* Format
2021-10-20 09:13:52 -07:00
Mark D. Roth 5406048ef6
Second attempt: RLS LB policy (#27748)
* allow connectivity state watching to work on lame channels

* Revert "Revert RLS LB policy (#27738)"

This reverts commit 4567af504e.

* fix build

* fix lame_client_test
2021-10-20 07:41:07 -07:00
Lidi Zheng 09a55f26f9
[Aio] Add add_done_callback/done/cancelled methods to ServicerContext (#27767)
* WIP

* [Aio] Add add_done_callback method to aio server

* Allow >20 public methods on the ServicerContext class
2021-10-19 12:25:18 -07:00
Lidi Zheng 2bc96189b9
[Aio] Correct the typing of input Metadata (#27768) 2021-10-19 11:30:50 -07:00
Bolutife Ogunsola c1d4e96433
Address leak when using request stream interceptors (#25449) (#27571)
* Don't leave pending tasks on the asyncio queue

The results of these pending tasks are not needed, leaving them
on the queue grows the size of the queue until the call completes.

This fix slows the growth of the memory in the test example.

* Address 'leaking' Futures from cygrpc

Cancelling unneeded Tasks is not sufficient as this leaves behind
cancelled Futures in the cygrpc layer, which still occupy memory.

Instead, avoid creating unneeded tasks in the first place.

* Address review comments

1. Ignore unused return values
2. Fix formatting
2021-10-18 17:55:43 -07:00
clayg fbf9801b83
Catch InteralError in _write (#27240) 2021-10-15 14:06:07 -07:00
Mark D. Roth 4567af504e
Revert RLS LB policy (#27738)
* Revert "fix for import of RLS LB policy (#27736)"

This reverts commit cfcdbc4905.

* Revert "RLS LB policy (#27352)"

This reverts commit 84ddc3289f.
2021-10-15 12:19:57 -07:00
Mark D. Roth 84ddc3289f
RLS LB policy (#27352)
* RLS LB policy plugin implementation

* Add RLS tests

* rls proto

* generate projects

* Fix adding child policy to child policy map

* Fix issues

* Fix rotten bits

* generate projects

* revert some strange changes

* First batch of fixes

* second batch of addressing comments

* generate_projects

* 3rd batch of fixes

* generate_projects

* 4th batch of fix

* 5th batch of fix

* Empty commit to trigger github PR refresh

* Add insecure build

* generate project

* update picker in a closure

* mutex guard audit

* clang-format

* Aggregate child policy states

* Use OrphanablePtr on ChildPolicyOwner()

* More fixes on comments

* Remove include grpc_security from rls.cc

* Fix key builder test as we removed RlsFindPathFromMetadata

* Update rls proto and add multi-target capability

* code changes to fix build after merging master

* remove support for insecure builds

* WIP

* add LB policy metadata lookup API

* add API for creating errors from C++ strings

* add missing include

* finished updating JSON parsing

* use DualRefCounted<> for ChildPolicyWrapper

* use grpc_core::Mutex instead of std::recursive_mutex

* add lock annotations and fix some lifetime issues

* misc fixes and cleanups

* simplify child policy connectivity state machine

* add comment about hopping into ExecCtx

* don't use wait_for_ready for RLS call

* extraKeys and constantKeys support

* add lock annotations for child policy wrapper connectivity state

* improve logging

* simplify child policy state handling in picker

* use C++ style comments

* remove rls_config.proto, since it's not needed

* rewrote tests

* fix build

* fix copyright headers

* clang-format

* appease clang-tidy

* remove unnecessary dependendency on grpclb

* buildifier

* remove illegal term

* remove unneeded includes

* clang-format

* fix clang-tidy

* fix build

* move class declaration into .cc file

* clang-tidy again

* fix build

* reorganize code and misc cleanups

* clang-format

* fixed a bunch of asan bugs; still some left to debug

* fix asan problem

* make test work on IPv6-only machines

* move LRU list handling into Cache::Entry and add lock annotations

* use preincrement instead of postincrement

* fix deadlock

* add more FIXMEs

* clean up backoff timer logic

* fix build from merge

* clang-format

* fix include path to work on podspec builds

* clean up picker logic

* clang-format

* update rls.proto

* populate reason and stale_header_data in RLS request

* improve logging and fix some cache size type issues

* clang-format

* add some TODOs about a better way to expose channel creds to LB policies

* centralize handling of channel's default authority

* clang-format

* fix backup poller bug

* fix handling of call creds and authority on RLS channel

* remove unused params

* clang-format

* use two-phase update for child policies to avoid deadlocks

* fix portability issue

* minor cleanups

* update for change in grpc_error_get_status() API

* change test to store RLS requests and responses in proto form

* clang-format

* account for test slowdown factor in timeouts

* fix tsan failure and channelz linkage

* fix RLS authority death test

* fix test to not try to connect to target that should not work

* remove illegal term

* cacheSizeBytes is a required field

* add missing BUILD deps from merge

* add tests for connectivity state reporting and fix bugs found

* fix BUILD sanity

* buildifier

* fix BUILD package path

* fix bugs related to child policy lifetime and updates

* remove unnecessary srand() call from grpc_init()

* add test for two cache entries with the same target

* update rls.proto

* change RLS config parser test to support GRPC_ERROR_IS_ABSEIL_STATUS

* update upb codegen

* fix include path

* add env var guard

* avoid duplication of CountedService

* generate_projects

* add upper limit for cacheSizeBytes

* fix build

Co-authored-by: Muxi Yan <mxyan@google.com>
2021-10-14 12:53:46 -07:00
Craig Tiller 98cf516874
Remove old backwards compatibility cronet compression workaround code (#27701) 2021-10-14 10:36:37 -07:00
Mark D. Roth 464c1c519f
switch to new location of TypedStruct and ORCA protos (#27700)
* upgrade xDS protos

* switch to new location of TypedStruct and ORCA protos

* fix sanity
2021-10-14 10:28:24 -07:00
Mark D. Roth e20dfeccc2
upgrade xDS protos (#27698) 2021-10-13 13:27:47 -07:00
Craig Tiller 1df153f46e
Revert "Revert "Move metadata setting into HPackParser (#27545)" (#27686)" (#27688)
This reverts commit 0e6ced8800.
2021-10-13 08:44:57 -07:00
Yash Tibrewal a61bb8ed5b
Move service config code out of client_client to its own target (#27683)
* Move service config code out of client_client to its own target

* Fix BUILD

* Fix test build

* Reviewer comments

* Reviewer comments

* Automated change: Fix sanity tests

* Reviewer comments

Co-authored-by: yashykt <yashykt@users.noreply.github.com>
2021-10-12 18:30:04 -07:00
Craig Tiller 0e6ced8800
Revert "Move metadata setting into HPackParser (#27545)" (#27686)
This reverts commit f57a1f7d8c.
2021-10-12 08:37:00 -07:00
Craig Tiller f57a1f7d8c
Move metadata setting into HPackParser (#27545)
* forward work to get append, limit check into parser

* further cleanup

* Remove chttp2_incoming_metadata_buffer

* Automated change: Fix sanity tests

* test fixes

* fix cronet

* wip

* first pass mementofication of parsing - libgrpc compiles

* further progress

* fixes

* fixes

* fix leak

* Automated change: Fix sanity tests

* fix leak

* fixes

* x

* improve fuzzer

* init ordering fix

* Update hpack_parser_table.h

* Update hpack_parser.cc

* Update hpack_parser_test.cc

* Automated change: Fix sanity tests

* Automated change: Fix sanity tests

* i mustache myself why

* fix use after free

* Automated change: Fix sanity tests

* fixes

* eliminate second pass metadata - this is an illegal frame and fails the new parser

* handle mementos with size > sizeof(intptr_t)

* fix

* add parsed metadata header, test

* Automated change: Fix sanity tests

* fix includes

* phase1

* phase2

* Update parsed_metadata.h

* fix

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2021-10-11 20:02:29 -07:00
Craig Tiller a21ad2c296
Remove UDP code - it's unused (#27595)
* Remove UDP code - its unused

* Automated change: Fix sanity tests

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2021-10-11 13:34:47 -07:00
Craig Tiller 8d8d07139c
Fix client idle filter (#27611)
* trial-change

* compiles!

* added test

* Automated change: Fix sanity tests

* update test

* Fix slow idle detection

* review feedback

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2021-10-08 12:30:11 -07:00
Lidi Zheng 60028a82a9
[Aio] Resolve deprecated warnings from asyncio (#27635)
* [Aio] Resolve deprecated warnings from asyncio

* Use 3.6 compatible way of creating an asyncio.Task
2021-10-07 14:05:09 -07:00
Richard Belleville 7aa43b7a55
Create Bazel gevent test harness (#27507)
* WIP

* Add gevent test suite run under Bazel.

* Fix things up

* Yapf

* Fix up Bazel files

* Make py_grpc_test fancier

* Attempt to fix Windows RBE

* Attempt to kick GitHub

* Fix Python 2 runs

* Yet more fixes

* And the patch file too

* I am an idiot

* Mark gevent tests flaky

* Try to make rules_python more tolerant

* Typo

* Exclude reconnect test from gevent

* Remove unnecessary parts of patch

* Buildifier

* You saw nothing

* isort

* Move py_grpc_test to an internal-only file

* Review comments

* More reviewer comments

* Review
2021-10-07 14:04:34 -07:00
Craig Tiller 79ef60f079
Integrate new resource quota, event engine (#27522)
* Integrate new resource quota, event engine

* Automated change: Fix sanity tests

* windows fix

* Automated change: Fix sanity tests

* Update memory_allocator.h

* Automated change: Fix sanity tests

* first round review feedback

* review feedback

* Automated change: Fix sanity tests

* Update memory_quota.h

* get re-export formatted right

* Automated change: Fix sanity tests

* Update memory_allocator.cc

* Automated change: Fix sanity tests

* MemoryOwner has-a MemoryAllocator

* using fix

* review feedback

* Automated change: Fix sanity tests

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2021-10-06 13:29:57 -07:00
Mark D. Roth 3d83dd3776
Revert "Revert "centralize handling of channel's default authority (#27544)" (#27616)" (#27624)
This reverts commit 0611f7c9b3.
2021-10-06 12:15:55 -07:00
Esun Kim c56b0a3818
Update boringssl to the latest (#27606)
* Change boringssl branch name

* update submodule boringssl-with-bazel with origin/main-with-bazel

* update boringssl dependency to main-with-bazel commit SHA

* regenerate files

* Increment podspec version

* generate boringssl prefix headers
2021-10-06 10:02:23 -07:00
Mark D. Roth 0611f7c9b3
Revert "centralize handling of channel's default authority (#27544)" (#27616)
This reverts commit f0a0628795.
2021-10-06 07:26:24 -07:00
Mark D. Roth f0a0628795
centralize handling of channel's default authority (#27544)
* centralize handling of channel's default authority

* clang-format

* fix backup poller bug
2021-10-05 11:57:47 -07:00
Lidi Zheng 9978223a26
Add python_requires field to grpcio-* packages (#27495)
- Update the run_tests.py and distrib tests
- Update the Dockerfiles
2021-09-28 16:01:15 -07:00
Craig Tiller 9b41852a5e
channel_init --> core configuration system (#27347)
* channel_init --> core configuration system

* Automated change: Fix sanity tests

* fixes

* Automated change: Fix sanity tests

* fixes

* fixes

* eliminate grpc_base_c

* fixes

* fix merge

* fix

* update visibility

* fixes

* feedback

* fix comment

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2021-09-27 15:51:39 -07:00
Craig Tiller c5172a134e
Fix build splitup for slices. (#27452)
* slice fix

* fix properlike

* Automated change: Fix sanity tests

* actually delete the code that should have been

* fixes

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2021-09-23 16:36:43 -07:00
Craig Tiller d4407b7e59
Revert "Remove dead code (#27445)" (#27461)
This reverts commit 693f36f91c.
2021-09-23 12:56:40 -07:00
Craig Tiller 693f36f91c
Remove dead code (#27445)
* Remove dead code

* Automated change: Fix sanity tests

* build

* remove benchmark

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2021-09-22 19:31:00 -07:00
Craig Tiller ffa4365031
Separate common slice utilities, grpc_error into their own libraries (#27441)
* make error independent

* Separate grpc_error into its own library

This is forward work to move Closure, ExecCtx into their own libraries
in order to make use of them in the activity code for resource quota
wakeups.

* Automated change: Fix sanity tests

* fixes

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2021-09-22 18:44:32 -07:00
Craig Tiller 66ae0d5c08
Separate slices from mdelems for static data (#27372)
* separate slice into internal and refcount, with refcount being the minimal base

* python3-ize

* Separate slices from mdelems for static data

Allows us to separate a small build target with just some slice basics
from the rest, which will help simplify the build system longer term.

Additionally exposed an opportunity to eliminate an init function.

Should also help in a month or so when it's time to jettison interned
metadata and slices entirely.

* fix compilation

* Automated change: Fix sanity tests

* fixes

* fixfixfix

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2021-09-21 17:27:54 -07:00
Lidi Zheng 46257815d4
Disable grpc_status_test for sync stack in Python2 (#27374)
* Disable grpc_status_test for sync stack in Python2

* Correct the if condition

* Also disable this test in Bazel

* Clean the unnecessary import in BUILD
2021-09-17 10:30:09 -07:00
donnadionne b9a997be11
Bump version to v1.41.0 (#27370)
* Bump version to v1.41.x

* Regenerate projects
2021-09-16 16:32:34 -07:00
Esun Kim 91e0660638
Replace grpc_error* with grpc_error_handle (#27311)
* Replace grpc_error* with grpc_error_handle

* fix python build
2021-09-14 16:33:42 -07:00
francisco souza f97bf05169
python: fix type annotation for the _metadata field (#27251)
It's a tuple with many MetadatumType's.

Co-authored-by: francisco souza <fsouza@users.noreply.github.com>
2021-09-13 16:25:53 -07:00
Richard Belleville 33dd1bd8d0
Report Python xDS Interop Client SIGINTs (#27289) 2021-09-08 14:46:48 -07:00
Craig Tiller 999e36fb31
Rebuild HPACK encoder table as C++ (#27192)
* Rebuild HPACK encoder table as C++

* move comment

* incguards

* build

* Automated change: Fix sanity tests

* c++ initialization ftw

* Automated change: Fix sanity tests

* Add missing header

* Add missing header

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2021-09-02 14:55:33 -07:00
Lidi Zheng 4729f6fdcc
Add Python 3.10 drop 3.5 (#26074)
* Add Python 3.10.0rc1 binary wheels

* Drop Python 3.5 artifacts

* Document the drop of 3.5

* Fix the wrong pip pointer

* Update manylinux2014 to a newer version, remove 3.5 distribtest

* Update manylinux aarch64 to see if the absl error go away

* Use the preferred alias

* Allow different wheel library to produce different tag order

* Remove unused shell var and log produced wheels

* Use copy instead of move

* Make bash happy about the wildcard

* Upgrade the debian image to use 3.5+ Python

* Polish the comments for the Dockerfiles
2021-09-02 14:02:53 -07:00
Richard Belleville 8fca16ac45
Add compute_engine_channel_credentials to __all__ (#27235)
* Add compute_engine_channel_credentials to __all__

* And amend api_test
2021-09-02 13:25:15 -07:00
donnadionne 022cff7d7c
update submodule boringssl-with-bazel with origin/master-with-bazel (#27208)
* update submodule boringssl-with-bazel with origin/master-with-bazel

* update boringssl dependency to master-with-bazel commit SHA

* regenerate files

* Updated sha256

* generate boringssl prefix headers

* Increment podspec version
2021-09-01 11:25:01 -07:00
AJ Heller 2d16865693
Delete libuv-iomgr implementation and GRPC_UV build option (#27188)
This has been unmaintained for years, last supported in gRPC-core v1.24.
2021-08-31 13:26:43 -07:00
Ashitha Santhosh 1ff1f8fd83
SDK authorization filter. (#26468)
* SDK authorization filter.
2021-08-25 21:00:35 -07:00