Commit Graph

338 Commits

Author SHA1 Message Date
Craig Tiller 220cf641fa
Reland slice metadata conversion (#28210)
* Fix status code for resource exhaustion

* Revert "Revert "Move a bunch of slice typed metadata to new system (#28107)" (#28208)"

This reverts commit 7717587063.

* fix test
2021-11-29 13:38:40 -08:00
Craig Tiller 7717587063
Revert "Move a bunch of slice typed metadata to new system (#28107)" (#28208)
This reverts commit 4d432aefc1.
2021-11-24 15:41:36 -08:00
Craig Tiller 4d432aefc1
Move a bunch of slice typed metadata to new system (#28107)
* Eliminate most of grpc_message metadata handling

* Eliminate most of host metadata handling

* Remove more callouts without fixing code

* fiiixes

* typo

* Automated change: Fix sanity tests

* try-shrink

* Automated change: Fix sanity tests

* size tweaks

* less tricks

* deunique

* commonize

* commonize

* Automated change: Fix sanity tests

* size tuning, fixes

* Automated change: Fix sanity tests

* fix

* size tuning, fixes

* remove constexpr

* fix

* reuse code

* fix

* tweak code

* more tweaks

* tell no lies

* fixes

* fixes

* Automated change: Fix sanity tests

* fix

* fix

* fix

* fix

* fix?

* fix binder

* fix

* fix

* fixes

* Automated change: Fix sanity tests

* fix

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2021-11-24 11:42:37 -08:00
Craig Tiller 46eae022cc
Remove unused override, and the static metadata that supports it (#28193) 2021-11-23 12:12:21 -08:00
Craig Tiller 86d119c8ab
Remove unused constants (#28156) 2021-11-19 17:27:39 -08:00
Esun Kim 9be7ca5a55
Roll-forward #27780 (#27951)
* Revert "Revert #27780 (#27944)"

This reverts commit 26e7560309.

* Fix google_api_upbdefs
2021-11-18 18:18:02 -08:00
Craig Tiller 0794973bbb
Reland user-agent metadata work (#28109)
* Revert "Revert "user-agent metadata trait, also: grpc_core::Slice is born (#27770)" (#28108)"

This reverts commit 89d08dad9d.

* will it blend

* will it blend

* will it blend

* lcnagfmt

* sanity

* will it blend

* sleep @ end

* will it blend

* Revert "sleep @ end"

This reverts commit d6bca8ed3d.

* review feedback

* review feedback
2021-11-17 19:04:29 -08:00
Jan Tattermusch 89d08dad9d
Revert "user-agent metadata trait, also: grpc_core::Slice is born (#27770)" (#28108)
This reverts commit 7a40f50cb3.
2021-11-16 06:52:52 -08:00
Craig Tiller 7a40f50cb3
user-agent metadata trait, also: grpc_core::Slice is born (#27770)
* new slice api

* storage-classes

* Automated change: Fix sanity tests

* tweaks

* refinement

* refinement

* compiles

* Automated change: Fix sanity tests

* better impl

* convert to gtest

* clean

* fixes

* tests

* Automated change: Fix sanity tests

* more-tests

* clarity

* comments

* comments

* fixes

* comment-updates

* fixes

* spam

* Automated change: Fix sanity tests

* move transport size into transport!

* mebbefix

* fix type

* x

* fix-merge

* review feedback

* review feedback

* Automated change: Fix sanity tests

* meh

* review feedback

* fix

* resolve compile issue

* Remove slice_refcount dependency on RefCounted

* update init for channel_data in http_client_filter

* Automated change: Fix sanity tests

* remove banned function

* fixes

* will it blend

* will it blend

* fix refcount init

* fix

* fix comment

* Fix typo in bloat script

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2021-11-15 19:16:20 -08:00
Richard Belleville 59693b7e72
Migrate Infrastructure Scripts to Python 3 (#27135)
* Run 2to3 on tools directory

* Delete github_stats_tracking

* Re-run 2to3

* Remove unused script

* Remove unused script

* Remove unused line count utility

* Yapf. Isort

* Remove accidentally included file

* Migrate tools/distrib directory to python 3

* Remove unnecessary shebang

* Restore line_count directory

* Immediately convert subprocess.check_output output to string

* Take care of Python 2 shebangs

* Invoke scripts using a Python 3 interpreter

* Yapf. Isort

* Try installing Python 3 first

* See if we have any Python 3 versions installed

* Add Python 3.7 to Windows path

* Try adding a symlink

* Try to symlink differently

* Install six for Python 3

* Run run_interop_tests with python 3

* Try installing six in python3.7 explicitly

* Revert "Try installing six in python3.7 explicitly"

This reverts commit 2cf60d72f3.

* And debug some more

* Fix issue with jobset.py

* Add debug for CI failure

* Revert microbenchmark changes
2021-11-08 08:55:25 -08:00
Esun Kim 26e7560309
Revert #27780 (#27944) 2021-11-04 15:00:37 -07:00
Esun Kim d5971f49c4
Added a new bazel-integrated upb code-gen (#27780)
* Added bazel-integrated upb code-gen

* Update by review

* Update by review
2021-11-01 12:51:48 -07:00
Craig Tiller 1c885232e0
Reland te:trailers metadata change with fixes (#27785)
* Revert "Revert "Reland te: trailers metadata stuff (#27781)" (#27784)"

This reverts commit 33fff40174.

* in which i admit defeat for this round
2021-10-20 14:50:33 -07:00
Craig Tiller 33fff40174
Revert "Reland te: trailers metadata stuff (#27781)" (#27784)
This reverts commit f5ab108ab7.
2021-10-20 11:21:53 -07:00
Craig Tiller f5ab108ab7
Reland te: trailers metadata stuff (#27781)
* Revert "Revert "Add metadata specialization for te: trailers (#27714)" (#27779)"

This reverts commit 2039cec4b1.

* safer register call
2021-10-20 08:29:58 -07:00
Mark D. Roth 5406048ef6
Second attempt: RLS LB policy (#27748)
* allow connectivity state watching to work on lame channels

* Revert "Revert RLS LB policy (#27738)"

This reverts commit 4567af504e.

* fix build

* fix lame_client_test
2021-10-20 07:41:07 -07:00
Craig Tiller 2039cec4b1
Revert "Add metadata specialization for te: trailers (#27714)" (#27779)
This reverts commit 7cded136b7.
2021-10-19 16:42:41 -07:00
Craig Tiller 7cded136b7
Add metadata specialization for te: trailers (#27714)
* Add metadata specialization for te: trailers

* remove the evidence

* update microbenchmarks

* ugh

* cleanup copy code

* fix asan detected leak

* Automated change: Fix sanity tests

* append

* binder

* fix-cronet

* inproc fixes

* Automated change: Fix sanity tests

* convert cronet encode path

* fixes

* fixes

* review feedback

* Automated change: Fix sanity tests

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2021-10-19 16:23:22 -07:00
Mark D. Roth 4567af504e
Revert RLS LB policy (#27738)
* Revert "fix for import of RLS LB policy (#27736)"

This reverts commit cfcdbc4905.

* Revert "RLS LB policy (#27352)"

This reverts commit 84ddc3289f.
2021-10-15 12:19:57 -07:00
Mark D. Roth 84ddc3289f
RLS LB policy (#27352)
* RLS LB policy plugin implementation

* Add RLS tests

* rls proto

* generate projects

* Fix adding child policy to child policy map

* Fix issues

* Fix rotten bits

* generate projects

* revert some strange changes

* First batch of fixes

* second batch of addressing comments

* generate_projects

* 3rd batch of fixes

* generate_projects

* 4th batch of fix

* 5th batch of fix

* Empty commit to trigger github PR refresh

* Add insecure build

* generate project

* update picker in a closure

* mutex guard audit

* clang-format

* Aggregate child policy states

* Use OrphanablePtr on ChildPolicyOwner()

* More fixes on comments

* Remove include grpc_security from rls.cc

* Fix key builder test as we removed RlsFindPathFromMetadata

* Update rls proto and add multi-target capability

* code changes to fix build after merging master

* remove support for insecure builds

* WIP

* add LB policy metadata lookup API

* add API for creating errors from C++ strings

* add missing include

* finished updating JSON parsing

* use DualRefCounted<> for ChildPolicyWrapper

* use grpc_core::Mutex instead of std::recursive_mutex

* add lock annotations and fix some lifetime issues

* misc fixes and cleanups

* simplify child policy connectivity state machine

* add comment about hopping into ExecCtx

* don't use wait_for_ready for RLS call

* extraKeys and constantKeys support

* add lock annotations for child policy wrapper connectivity state

* improve logging

* simplify child policy state handling in picker

* use C++ style comments

* remove rls_config.proto, since it's not needed

* rewrote tests

* fix build

* fix copyright headers

* clang-format

* appease clang-tidy

* remove unnecessary dependendency on grpclb

* buildifier

* remove illegal term

* remove unneeded includes

* clang-format

* fix clang-tidy

* fix build

* move class declaration into .cc file

* clang-tidy again

* fix build

* reorganize code and misc cleanups

* clang-format

* fixed a bunch of asan bugs; still some left to debug

* fix asan problem

* make test work on IPv6-only machines

* move LRU list handling into Cache::Entry and add lock annotations

* use preincrement instead of postincrement

* fix deadlock

* add more FIXMEs

* clean up backoff timer logic

* fix build from merge

* clang-format

* fix include path to work on podspec builds

* clean up picker logic

* clang-format

* update rls.proto

* populate reason and stale_header_data in RLS request

* improve logging and fix some cache size type issues

* clang-format

* add some TODOs about a better way to expose channel creds to LB policies

* centralize handling of channel's default authority

* clang-format

* fix backup poller bug

* fix handling of call creds and authority on RLS channel

* remove unused params

* clang-format

* use two-phase update for child policies to avoid deadlocks

* fix portability issue

* minor cleanups

* update for change in grpc_error_get_status() API

* change test to store RLS requests and responses in proto form

* clang-format

* account for test slowdown factor in timeouts

* fix tsan failure and channelz linkage

* fix RLS authority death test

* fix test to not try to connect to target that should not work

* remove illegal term

* cacheSizeBytes is a required field

* add missing BUILD deps from merge

* add tests for connectivity state reporting and fix bugs found

* fix BUILD sanity

* buildifier

* fix BUILD package path

* fix bugs related to child policy lifetime and updates

* remove unnecessary srand() call from grpc_init()

* add test for two cache entries with the same target

* update rls.proto

* change RLS config parser test to support GRPC_ERROR_IS_ABSEIL_STATUS

* update upb codegen

* fix include path

* add env var guard

* avoid duplication of CountedService

* generate_projects

* add upper limit for cacheSizeBytes

* fix build

Co-authored-by: Muxi Yan <mxyan@google.com>
2021-10-14 12:53:46 -07:00
Mark D. Roth 464c1c519f
switch to new location of TypedStruct and ORCA protos (#27700)
* upgrade xDS protos

* switch to new location of TypedStruct and ORCA protos

* fix sanity
2021-10-14 10:28:24 -07:00
Mark D. Roth e20dfeccc2
upgrade xDS protos (#27698) 2021-10-13 13:27:47 -07:00
Craig Tiller 66ae0d5c08
Separate slices from mdelems for static data (#27372)
* separate slice into internal and refcount, with refcount being the minimal base

* python3-ize

* Separate slices from mdelems for static data

Allows us to separate a small build target with just some slice basics
from the rest, which will help simplify the build system longer term.

Additionally exposed an opportunity to eliminate an init function.

Should also help in a month or so when it's time to jettison interned
metadata and slices entirely.

* fix compilation

* Automated change: Fix sanity tests

* fixes

* fixfixfix

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2021-09-21 17:27:54 -07:00
Craig Tiller b669a3c521
Revert "Revert "a useful change (#27381)" (#27411)" (#27412)
This reverts commit 5db17f7350.
2021-09-20 19:46:11 -07:00
Craig Tiller 5db17f7350
Revert "a useful change (#27381)" (#27411)
This reverts commit 11eb70bbed.
2021-09-20 10:30:59 -07:00
Craig Tiller 11eb70bbed
a useful change (#27381)
* a useful change

* Automated change: Fix sanity tests

* fix

* fix

* QsortCompare it is!

* Automated change: Fix sanity tests

* add missing header

* clang-tidy fixes, tweak clamp to only use <

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2021-09-20 09:13:27 -07:00
Richard Belleville b6cc72f129
Revert "Run 2to3 on tools directory (#26002)" (#27183)
This reverts commit a388361c3d.
2021-08-30 12:29:20 -07:00
Richard Belleville a388361c3d
Run 2to3 on tools directory (#26002)
* Run 2to3 on tools directory

* Delete github_stats_tracking

* Re-run 2to3

* Remove unused script

* Remove unused script

* Remove unused line count utility

* Yapf. Isort

* Remove accidentally included file

* Restore line_count directory
2021-08-30 09:25:21 -07:00
Craig Tiller 46bd552337
Eliminate gen_legal_metadata_characters (#27019)
* eliminate gen_legal_metadata_characters

* Automated change: Fix sanity tests

* Automated change: Fix sanity tests

* fix

* Automated change: Fix sanity tests

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2021-08-17 20:09:16 -07:00
Craig Tiller 5dc93179eb
modernize % encoding routines (#27018)
* modernize % encoding routines

* Automated change: Fix sanity tests

* use bitset

* Automated change: Fix sanity tests

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2021-08-16 11:16:18 -07:00
Craig Tiller 78c98a9b06
Switch template metaprogramming helper for promises library (#26908)
* Switch template metaprogramming helper for promises library

* py3
2021-08-09 13:21:51 -07:00
Lidi Zheng 826da6136c
Update Envoy API to the latest version (2021-07-30) (#26848)
* Bump Envoy and related submodules to latest

* Update googleapis

* Update scripts for newer submodules

* Update udpa

* Add opentelemetry

* Python changes for opentelemetry

* Update udpa in check_submodules

* Add opentelemetry to check submodules

* Regenerate upb files

* Add new proto dependency to upb-gen and Bazel

* Regenerate project
2021-08-03 09:12:42 -07:00
Lidi Zheng 2231c2ba77
Introduce Python import sorting to our sanity test suite (#26768)
* Add isort_code.sh to sanity tests

* Run tools/distrib/isort_code.sh

* Fine tune the import order for relative imports

* Make pylint and project generation happy

* Fix a few corner cases

* Use --check instead of --diff

* The import order impacts test result somehow

* Make isort print diff and check output at the same time

* Let tools/run_tests/python_utils be firstparty library

* Run isort against latest HEAD
2021-07-26 12:31:21 -07:00
Craig Tiller 987d526efc
Eliminate some HPACK LUTs (#26657)
Eliminate HPACK table

Use a switch statement instead of a table lookup for the first byte of
HPACK parsing.

This will lead the way to some other improvements down the track (I have
a substantial overhaul here planned), but this is a necessary first
step.
2021-07-14 12:16:17 -07:00
Craig Tiller 95dfe09710
Use constexpr to simplify HPACK parser (#26656)
* Use constexpr to simplify HPACK parser

This is a bit of a trial balloon: in C++11, generate at load time a
table that used to be hard coded. In C++14, generate that same table at
compile time, but eliminate the code generator.

Should this work out, I'd like to expand the technique so that we can
eliminate some of the confusing tables in this file by keeping the code
that generates them *in the same place* as the code that consumes them.

* comments

* c++14 fixes
2021-07-12 15:04:07 -07:00
Ashitha Santhosh 64d1894314
Revert: gRPC SDK Authorization Policy Proto. (#26508) 2021-06-18 22:05:26 -07:00
Lidi Zheng 1040fbdb9f
Re-introduce "CSDS Implementation" (#25762)
* Revert "Revert "CSDS Implementation (#25038)" (#25745)"

This reverts commit 98fd4e1e36.

* Add xDS special Bazel build rules

* Add 2 todos to remove the added rules
2021-03-22 13:14:19 -07:00
Lidi Zheng 98fd4e1e36
Revert "CSDS Implementation (#25038)" (#25745)
This reverts commit 27de24a38e.
2021-03-17 13:38:47 -07:00
Lidi Zheng 27de24a38e
CSDS Implementation (#25038)
* Implement the xDS Config Dump as CSDS in Core

* Revemp the logic of caching and constructing the CSDS response
* Unref created error
* Fix the clang tidy complains
* Resolve comments about symbol preload and Json ctor
* Improve readability
* Merge with new ADS logic && support latest CSDS
  * Refactor the version/client_status logic
  * Add support for error_state
  * Add support for does_not_exist
  * Add node information
* Fix the ProtoBuf segfault
* More test cases
* Refactor to use bytes as cache instead of JSON

* Apply clang_tidy's suggestion

* Resolve reviewer's requests

* Tiny stylish fixes && make sanity test happy

* WIP: still working on the matchers

* Update a bunch of matchers

* Improve readability a bit

* Resolve reviewer's suggestions

* Resolve reviewer's comments

* Make Unpack fail fast
2021-03-16 18:17:37 -07:00
Lidi Zheng 3b067c9f3f
Implement xDS client-side fault injection filter (#24354)
* Refactored with dynamic filters
* Error-tolerance tuned
* Fix leak of grpc_error and data race of canceller
* Adopt the latest xDS HTTP filter framework
* Fix fault injection tests' conflict with router filter
* Test alternative setup (override, no-override) without copy
* Refactor file strcutures of fault injection filter
* Rewrite the Json parsing/assembling logic again
* Added logic for aborting streaming RPC && resolve comments
2021-03-09 13:14:21 -08:00
Mark D. Roth 49601b0334
Second attempt: xDS HTTP filter support for gRPC client (#25558)
* Revert "Revert "xDS HTTP filter support for gRPC client" (#25555)"

This reverts commit 785a128a83.

* xds_end2end_test: avoid flakes from lingering shutdown from previous test
2021-02-26 07:39:27 -08:00
donnadionne 785a128a83
Revert "xDS HTTP filter support for gRPC client" (#25555)
This reverts commit b2164837a8.
2021-02-25 07:18:43 -08:00
Mark D. Roth b2164837a8 xDS HTTP filter support for gRPC client 2021-02-11 13:17:15 -08:00
Lidi Zheng 2161ed080d Update the udpa protos that moved around 2021-02-04 17:00:44 -08:00
Donna Dionne 19b1f8f64a Cds Watchers changes to support aggregate Cds and Logical DNS Cds. 2021-01-21 14:25:50 -08:00
Ashitha Santhosh ab9b01a03b
Merge pull request #25129 from ashithasantosh/sdk_authorization_policy
gRPC SDK Authorization Policy proto.
2021-01-14 10:14:01 -08:00
Sergii Tkachenko 8db79e2e71 Python style: reformat code using updated yapf 0.30.0 2021-01-13 14:00:12 -05:00
Ashitha Santhosh 1602a02a82 Add gRPC SDK Authorization Policy proto. 2021-01-13 00:43:43 -08:00
Esun Kim e4014fc2a6 Updated gen_upb_api.sh 2020-12-21 10:12:51 -08:00
Esun Kim 7d7c3cc711 Update by review 2020-12-02 13:42:53 -08:00