* Fix status code for resource exhaustion
* Revert "Revert "Move a bunch of slice typed metadata to new system (#28107)" (#28208)"
This reverts commit 7717587063.
* fix test
* Revert "Revert "user-agent metadata trait, also: grpc_core::Slice is born (#27770)" (#28108)"
This reverts commit 89d08dad9d.
* will it blend
* will it blend
* will it blend
* lcnagfmt
* sanity
* will it blend
* sleep @ end
* will it blend
* Revert "sleep @ end"
This reverts commit d6bca8ed3d.
* review feedback
* review feedback
* Run 2to3 on tools directory
* Delete github_stats_tracking
* Re-run 2to3
* Remove unused script
* Remove unused script
* Remove unused line count utility
* Yapf. Isort
* Remove accidentally included file
* Migrate tools/distrib directory to python 3
* Remove unnecessary shebang
* Restore line_count directory
* Immediately convert subprocess.check_output output to string
* Take care of Python 2 shebangs
* Invoke scripts using a Python 3 interpreter
* Yapf. Isort
* Try installing Python 3 first
* See if we have any Python 3 versions installed
* Add Python 3.7 to Windows path
* Try adding a symlink
* Try to symlink differently
* Install six for Python 3
* Run run_interop_tests with python 3
* Try installing six in python3.7 explicitly
* Revert "Try installing six in python3.7 explicitly"
This reverts commit 2cf60d72f3.
* And debug some more
* Fix issue with jobset.py
* Add debug for CI failure
* Revert microbenchmark changes
* RLS LB policy plugin implementation
* Add RLS tests
* rls proto
* generate projects
* Fix adding child policy to child policy map
* Fix issues
* Fix rotten bits
* generate projects
* revert some strange changes
* First batch of fixes
* second batch of addressing comments
* generate_projects
* 3rd batch of fixes
* generate_projects
* 4th batch of fix
* 5th batch of fix
* Empty commit to trigger github PR refresh
* Add insecure build
* generate project
* update picker in a closure
* mutex guard audit
* clang-format
* Aggregate child policy states
* Use OrphanablePtr on ChildPolicyOwner()
* More fixes on comments
* Remove include grpc_security from rls.cc
* Fix key builder test as we removed RlsFindPathFromMetadata
* Update rls proto and add multi-target capability
* code changes to fix build after merging master
* remove support for insecure builds
* WIP
* add LB policy metadata lookup API
* add API for creating errors from C++ strings
* add missing include
* finished updating JSON parsing
* use DualRefCounted<> for ChildPolicyWrapper
* use grpc_core::Mutex instead of std::recursive_mutex
* add lock annotations and fix some lifetime issues
* misc fixes and cleanups
* simplify child policy connectivity state machine
* add comment about hopping into ExecCtx
* don't use wait_for_ready for RLS call
* extraKeys and constantKeys support
* add lock annotations for child policy wrapper connectivity state
* improve logging
* simplify child policy state handling in picker
* use C++ style comments
* remove rls_config.proto, since it's not needed
* rewrote tests
* fix build
* fix copyright headers
* clang-format
* appease clang-tidy
* remove unnecessary dependendency on grpclb
* buildifier
* remove illegal term
* remove unneeded includes
* clang-format
* fix clang-tidy
* fix build
* move class declaration into .cc file
* clang-tidy again
* fix build
* reorganize code and misc cleanups
* clang-format
* fixed a bunch of asan bugs; still some left to debug
* fix asan problem
* make test work on IPv6-only machines
* move LRU list handling into Cache::Entry and add lock annotations
* use preincrement instead of postincrement
* fix deadlock
* add more FIXMEs
* clean up backoff timer logic
* fix build from merge
* clang-format
* fix include path to work on podspec builds
* clean up picker logic
* clang-format
* update rls.proto
* populate reason and stale_header_data in RLS request
* improve logging and fix some cache size type issues
* clang-format
* add some TODOs about a better way to expose channel creds to LB policies
* centralize handling of channel's default authority
* clang-format
* fix backup poller bug
* fix handling of call creds and authority on RLS channel
* remove unused params
* clang-format
* use two-phase update for child policies to avoid deadlocks
* fix portability issue
* minor cleanups
* update for change in grpc_error_get_status() API
* change test to store RLS requests and responses in proto form
* clang-format
* account for test slowdown factor in timeouts
* fix tsan failure and channelz linkage
* fix RLS authority death test
* fix test to not try to connect to target that should not work
* remove illegal term
* cacheSizeBytes is a required field
* add missing BUILD deps from merge
* add tests for connectivity state reporting and fix bugs found
* fix BUILD sanity
* buildifier
* fix BUILD package path
* fix bugs related to child policy lifetime and updates
* remove unnecessary srand() call from grpc_init()
* add test for two cache entries with the same target
* update rls.proto
* change RLS config parser test to support GRPC_ERROR_IS_ABSEIL_STATUS
* update upb codegen
* fix include path
* add env var guard
* avoid duplication of CountedService
* generate_projects
* add upper limit for cacheSizeBytes
* fix build
Co-authored-by: Muxi Yan <mxyan@google.com>
* separate slice into internal and refcount, with refcount being the minimal base
* python3-ize
* Separate slices from mdelems for static data
Allows us to separate a small build target with just some slice basics
from the rest, which will help simplify the build system longer term.
Additionally exposed an opportunity to eliminate an init function.
Should also help in a month or so when it's time to jettison interned
metadata and slices entirely.
* fix compilation
* Automated change: Fix sanity tests
* fixes
* fixfixfix
Co-authored-by: ctiller <ctiller@users.noreply.github.com>
* Bump Envoy and related submodules to latest
* Update googleapis
* Update scripts for newer submodules
* Update udpa
* Add opentelemetry
* Python changes for opentelemetry
* Update udpa in check_submodules
* Add opentelemetry to check submodules
* Regenerate upb files
* Add new proto dependency to upb-gen and Bazel
* Regenerate project
* Add isort_code.sh to sanity tests
* Run tools/distrib/isort_code.sh
* Fine tune the import order for relative imports
* Make pylint and project generation happy
* Fix a few corner cases
* Use --check instead of --diff
* The import order impacts test result somehow
* Make isort print diff and check output at the same time
* Let tools/run_tests/python_utils be firstparty library
* Run isort against latest HEAD
Eliminate HPACK table
Use a switch statement instead of a table lookup for the first byte of
HPACK parsing.
This will lead the way to some other improvements down the track (I have
a substantial overhaul here planned), but this is a necessary first
step.
* Use constexpr to simplify HPACK parser
This is a bit of a trial balloon: in C++11, generate at load time a
table that used to be hard coded. In C++14, generate that same table at
compile time, but eliminate the code generator.
Should this work out, I'd like to expand the technique so that we can
eliminate some of the confusing tables in this file by keeping the code
that generates them *in the same place* as the code that consumes them.
* comments
* c++14 fixes
* Implement the xDS Config Dump as CSDS in Core
* Revemp the logic of caching and constructing the CSDS response
* Unref created error
* Fix the clang tidy complains
* Resolve comments about symbol preload and Json ctor
* Improve readability
* Merge with new ADS logic && support latest CSDS
* Refactor the version/client_status logic
* Add support for error_state
* Add support for does_not_exist
* Add node information
* Fix the ProtoBuf segfault
* More test cases
* Refactor to use bytes as cache instead of JSON
* Apply clang_tidy's suggestion
* Resolve reviewer's requests
* Tiny stylish fixes && make sanity test happy
* WIP: still working on the matchers
* Update a bunch of matchers
* Improve readability a bit
* Resolve reviewer's suggestions
* Resolve reviewer's comments
* Make Unpack fail fast
* Refactored with dynamic filters
* Error-tolerance tuned
* Fix leak of grpc_error and data race of canceller
* Adopt the latest xDS HTTP filter framework
* Fix fault injection tests' conflict with router filter
* Test alternative setup (override, no-override) without copy
* Refactor file strcutures of fault injection filter
* Rewrite the Json parsing/assembling logic again
* Added logic for aborting streaming RPC && resolve comments
* Revert "Revert "xDS HTTP filter support for gRPC client" (#25555)"
This reverts commit 785a128a83.
* xds_end2end_test: avoid flakes from lingering shutdown from previous test
Merge remote-tracking branch 'upstream/master' into celeval_constructor
Iterate through policies and store CEL condition
git push origin ce Merge remote-tracking branch 'upstream/master' into celeval_constructor
added class and constructor comments for CelEvaluationEngine
CelEvaluationEngine class and constructor
Added cel_evaluation_engine to build
ran clang_tidy_code.sh
updated BUILD with cel_evaluation_engine
added rbac.upb.h/cc to BUILD
Rewrote cel_evaluation_engine code with upb conventions
removed unnecessary deps for cel_evaluation_engine in BUILD
Cel_evaluation_engine uses arenas to manage condition pointers
Used upb::Arena instead of upb_arena*, added temp_arena to save memory
Added action_allow_ member variable
Added fullstops to comments
initialize action_allow_
changed variable names to be more clear
removed unnecessary headers
correctly initialize the action_allow_ variable
changed constructor to use an initializer list
ran clang_tidy and clang_format scripts
added cel_engine_test
resolved merge conflicts and rewrote cel engine constructor to use new map api
changed variables to be more clear and made them const
changed syntax.upb.h so the build succeeds
changed syntax.upb.h so the build succeeds
changed version of upb in bazel_dpes
regenerated upb files with new upb version
added TODO regarding two-policy design
modified CelEvaluationEngine and tests to use two rbac policies
Made test messages consistent and added a case with too many policies
changed name from cel_engine to authorization_engine
fixed merge issues and updated authorization engine to v3 rbac
made constructor public and added namespace grpc_core
1) Statically pre-compute static slice indices to save some ALU ops when linking
batched metadata.
2) Change some asserts to debug_asserts since they can provably not be triggered
with the current implementation.
3) Save some slice comparison cycles inside CH2 parsing.
In several cases, we create grpc mdelem structures using known-static
metadata inputs. Furthermore, in several cases we create a slice on
the heap (e.g. grpc_slice_from_copied_buffer) where we know we are
transferring refcount ownership. In several cases, then, we can:
1) Avoid unnecessary ref/unref operations that are no-ops (for static
slices) or superfluous (if we're transferring ownership).
2) Avoid unnecessarily comprehensive calls to grpc_slice_eq (since
they'd only be called with static or interned slice arguments,
which by construction would have equal refcounts if they were
in fact equal.
3) Avoid unnecessary checks to see if a slice is interned (when we
know that they are).
To avoid polluting the internal API, we introduce the notion of
strongly-typed grpc_slice objects. We draw a distinction between
Internal (interned and static-storage) slices and Extern (inline and
non-statically allocated). We introduce overloads to
grpc_mdelem_create() and grpc_mdelem_from_slices() for the fastpath
cases identified above based on these slice types.
From the programmer's point of view, though, nothing changes - they
need only use grpc_mdelem_create() and grpc_mdelem_from_slices() as
before, and the appropriate fastpath will be picked based on type
inference. If no special knowledge exists for the slice type (i.e. we
pass in generic grpc_slice objects), the slowpath method will still
always return correct behaviour.
This is good for:
- Roughly 1-3% reduction in CPU time for several unary/streaming
ping pong fullstack microbenchmarks.
- Reduction of about 15-20% in CPU time for some hpack parser
microbenchmarks.
- 10-12% reduction of CPU time for metadata microbenchmarks involving
interned slice comparisons.
All these changes need to go together to make sense
- changes to use new version of upb in bazel
- allowing includes in build target option
- script for generating c code (upb) for protos
- generated code for example protos
- adding changes for non-bazel builds
- change sanity tests to ignore the generated files.
protos
All these changes need to go together to make sense
- changes to use new version of upb in bazel
- allowing includes in build target option
- script for generating c code for protos
- generated code for example build
- adding changes for non-bazel builds
- change sanity tests to ignore the generated files.
Esun Kim