Commit Graph

287 Commits

Author SHA1 Message Date
Lidi Zheng 1040fbdb9f
Re-introduce "CSDS Implementation" (#25762)
* Revert "Revert "CSDS Implementation (#25038)" (#25745)"

This reverts commit 98fd4e1e36.

* Add xDS special Bazel build rules

* Add 2 todos to remove the added rules
2021-03-22 13:14:19 -07:00
Lidi Zheng 98fd4e1e36
Revert "CSDS Implementation (#25038)" (#25745)
This reverts commit 27de24a38e.
2021-03-17 13:38:47 -07:00
Lidi Zheng 27de24a38e
CSDS Implementation (#25038)
* Implement the xDS Config Dump as CSDS in Core

* Revemp the logic of caching and constructing the CSDS response
* Unref created error
* Fix the clang tidy complains
* Resolve comments about symbol preload and Json ctor
* Improve readability
* Merge with new ADS logic && support latest CSDS
  * Refactor the version/client_status logic
  * Add support for error_state
  * Add support for does_not_exist
  * Add node information
* Fix the ProtoBuf segfault
* More test cases
* Refactor to use bytes as cache instead of JSON

* Apply clang_tidy's suggestion

* Resolve reviewer's requests

* Tiny stylish fixes && make sanity test happy

* WIP: still working on the matchers

* Update a bunch of matchers

* Improve readability a bit

* Resolve reviewer's suggestions

* Resolve reviewer's comments

* Make Unpack fail fast
2021-03-16 18:17:37 -07:00
Lidi Zheng 3b067c9f3f
Implement xDS client-side fault injection filter (#24354)
* Refactored with dynamic filters
* Error-tolerance tuned
* Fix leak of grpc_error and data race of canceller
* Adopt the latest xDS HTTP filter framework
* Fix fault injection tests' conflict with router filter
* Test alternative setup (override, no-override) without copy
* Refactor file strcutures of fault injection filter
* Rewrite the Json parsing/assembling logic again
* Added logic for aborting streaming RPC && resolve comments
2021-03-09 13:14:21 -08:00
Mark D. Roth 49601b0334
Second attempt: xDS HTTP filter support for gRPC client (#25558)
* Revert "Revert "xDS HTTP filter support for gRPC client" (#25555)"

This reverts commit 785a128a83.

* xds_end2end_test: avoid flakes from lingering shutdown from previous test
2021-02-26 07:39:27 -08:00
donnadionne 785a128a83
Revert "xDS HTTP filter support for gRPC client" (#25555)
This reverts commit b2164837a8.
2021-02-25 07:18:43 -08:00
Mark D. Roth b2164837a8 xDS HTTP filter support for gRPC client 2021-02-11 13:17:15 -08:00
Lidi Zheng 2161ed080d Update the udpa protos that moved around 2021-02-04 17:00:44 -08:00
Donna Dionne 19b1f8f64a Cds Watchers changes to support aggregate Cds and Logical DNS Cds. 2021-01-21 14:25:50 -08:00
Ashitha Santhosh ab9b01a03b
Merge pull request #25129 from ashithasantosh/sdk_authorization_policy
gRPC SDK Authorization Policy proto.
2021-01-14 10:14:01 -08:00
Sergii Tkachenko 8db79e2e71 Python style: reformat code using updated yapf 0.30.0 2021-01-13 14:00:12 -05:00
Ashitha Santhosh 1602a02a82 Add gRPC SDK Authorization Policy proto. 2021-01-13 00:43:43 -08:00
Esun Kim e4014fc2a6 Updated gen_upb_api.sh 2020-12-21 10:12:51 -08:00
Esun Kim 7d7c3cc711 Update by review 2020-12-02 13:42:53 -08:00
Yash Tibrewal 2a56886513 Update submodules: envoy-api, protoc-gen-validate, udpa 2020-09-29 12:03:40 -07:00
Ashitha Santhosh c50a33f0d1 Import checked.proto for CEL-based Authorization Engine. 2020-09-15 03:48:07 -07:00
Jan Tattermusch 54613cc45b upb codegen: put .upbdefs.h and .upbdefs.c files in a different directory 2020-08-20 18:17:20 +02:00
Yash Tibrewal 9e4ceb041e Move the proto to third_party 2020-08-12 14:21:14 -07:00
Yash Tibrewal 8da364d741 Add proto for MeshCA MeshCertificateService 2020-08-07 11:02:38 -07:00
Michael Wang d0311ac81a Basic class definition and constructor Cel eval engine
Merge remote-tracking branch 'upstream/master' into celeval_constructor

Iterate through policies and store CEL condition

git push origin ce Merge remote-tracking branch 'upstream/master' into celeval_constructor

added class and constructor comments for CelEvaluationEngine

CelEvaluationEngine class and constructor

Added cel_evaluation_engine to build

ran clang_tidy_code.sh

updated BUILD with cel_evaluation_engine

added rbac.upb.h/cc to BUILD

Rewrote cel_evaluation_engine code with upb conventions

removed unnecessary deps for cel_evaluation_engine in BUILD

Cel_evaluation_engine uses arenas to manage condition pointers

Used upb::Arena instead of upb_arena*, added temp_arena to save memory

Added action_allow_ member variable

Added fullstops to comments

initialize action_allow_

changed variable names to be more clear

removed unnecessary headers

correctly initialize the action_allow_ variable

changed constructor to use an initializer list

ran clang_tidy and clang_format scripts

added cel_engine_test

resolved merge conflicts and rewrote cel engine constructor to use new map api

changed variables to be more clear and made them const

changed syntax.upb.h so the build succeeds

changed syntax.upb.h so the build succeeds

changed version of upb in bazel_dpes

regenerated upb files with new upb version

added TODO regarding two-policy design

modified CelEvaluationEngine and tests to use two rbac policies

Made test messages consistent and added a case with too many policies

changed name from cel_engine to authorization_engine

fixed merge issues and updated authorization engine to v3 rbac

made constructor public and added namespace grpc_core
2020-08-04 18:23:21 +00:00
Mark D. Roth 181781387d xDS v3 support 2020-07-23 10:20:55 -07:00
Mark D. Roth a0f08b1b06 Revert "Merge pull request #21941 from markdroth/xds_logging"
This reverts commit 37d5d93983, reversing
changes made to 0e8190a3c9.
2020-07-17 10:19:28 -07:00
Mark D. Roth c1d7180daa Use upb textformat library to dump out raw xDS proto messages 2020-06-24 10:10:03 -07:00
michaelywg 308fbc6ade Added rbac.proto and dependencies, then ran gen_upb_api.sh 2020-06-11 15:47:23 -07:00
Mark D. Roth b95843c1b7 xds: Add support for send_all_clusters field in LRS. 2020-05-04 11:16:52 -07:00
Esun Kim 1adb077815 Changed gen_upb_api.h to use root BUILD file 2020-03-03 14:23:16 -08:00
Mark D. Roth b68f813e54 xds: send envoy.lb.does_not_support_overprovisioning client capability 2020-02-26 08:23:27 -08:00
Mark D. Roth ac7c1d20ed Add LDS/RDS into XdsClient 2020-01-24 08:05:02 -08:00
Esun Kim 7be821ac47 Yapf all target python sources 2020-01-03 09:25:10 -08:00
Esun Kim 3a519a0b64 Replaced grpc_core New & Delete with C++ new & delete 2019-11-06 17:52:26 -08:00
Juanli Shen 0a736c0d37 Update envoy API to HEAD 2019-10-24 17:21:26 -07:00
Juanli Shen 2dc42a1e02 Use ADS 2019-10-04 17:02:52 -07:00
Esun Kim fa7bdff69f Fix buffer-overflow in grpc_static_mdelem_for_static_strings 2019-08-30 10:55:27 -07:00
Mark D. Roth 21c5424477 Add API for accessing per-call backend metric data in LB policies. 2019-08-28 12:48:06 -07:00
Arjun Roy 2767accc1b Fixed init-order-fiasco for static slice table.
Fixes init-order bug affecting https://github.com/grpc/grpc/issues/19819 which
was first exposed by this commit:
857375a142
2019-08-27 16:29:12 -07:00
Esun Kim 7ec6e8a4de Remove nanopb from gRPC 2019-08-21 13:31:21 -07:00
Juanli Shen 45dd8be442 Use LRS in xds policy 2019-08-09 12:08:32 -07:00
Juanli Shen 79f191114e Use EDS in xds LB policy 2019-08-07 22:43:45 -07:00
Esun Kim 6ddfb384c1 Added google/protobuf/descriptor.upb.* to google_api_upb 2019-08-07 12:08:06 -07:00
Esun Kim f88bd06ee5 Go UPB! 2019-08-06 15:27:14 -07:00
Nicolas Noble d7167c4a7d
Merge pull request #19786 from nicolasnoble/bazel-version-force
Few fixes to the gen_upb_api.sh script.
2019-08-02 16:25:13 -07:00
Arjun Roy b437dc0c68 Reduce instruction count for CH2 metadata ops.
1) Statically pre-compute static slice indices to save some ALU ops when linking
batched metadata.
2) Change some asserts to debug_asserts since they can provably not be triggered
with the current implementation.
3) Save some slice comparison cycles inside CH2 parsing.
2019-08-01 14:29:22 -07:00
Arjun Roy 557446a11e Added specializations for grpc_mdelem_create.
In several cases, we create grpc mdelem structures using known-static
metadata inputs. Furthermore, in several cases we create a slice on
the heap (e.g. grpc_slice_from_copied_buffer) where we know we are
transferring refcount ownership. In several cases, then, we can:

1) Avoid unnecessary ref/unref operations that are no-ops (for static
   slices) or superfluous (if we're transferring ownership).
2) Avoid unnecessarily comprehensive calls to grpc_slice_eq (since
   they'd only be called with static or interned slice arguments,
   which by construction would have equal refcounts if they were
   in fact equal.
3) Avoid unnecessary checks to see if a slice is interned (when we
   know that they are).

To avoid polluting the internal API, we introduce the notion of
strongly-typed grpc_slice objects. We draw a distinction between
Internal (interned and static-storage) slices and Extern (inline and
non-statically allocated). We introduce overloads to
grpc_mdelem_create() and grpc_mdelem_from_slices() for the fastpath
cases identified above based on these slice types.

From the programmer's point of view, though, nothing changes - they
need only use grpc_mdelem_create() and grpc_mdelem_from_slices() as
before, and the appropriate fastpath will be picked based on type
inference. If no special knowledge exists for the slice type (i.e. we
pass in generic grpc_slice objects), the slowpath method will still
always return correct behaviour.

This is good for:
- Roughly 1-3% reduction in CPU time for several unary/streaming
  ping pong fullstack microbenchmarks.
- Reduction of about 15-20% in CPU time for some hpack parser
  microbenchmarks.
- 10-12% reduction of CPU time for metadata microbenchmarks involving
  interned slice comparisons.
2019-07-30 17:07:44 -07:00
Nicolas "Pixel" Noble d38178a27d Few fixes to the gen_upb_api.sh script. 2019-07-30 02:53:15 +02:00
Esun Kim ceb12c6be5
Merge pull request #19777 from veblush/upb-check
Add check_upb_output to sanity test
2019-07-29 15:17:49 -07:00
Esun Kim e45c5f021b Add check_upb_output to sanity test 2019-07-29 10:06:49 -07:00
Mark D. Roth db3d8be647 Add MetadataInterface abstraction to LB policy API. 2019-07-27 12:35:38 -07:00
Esun Kim 7e367da22a Added more protos to upb 2019-07-26 09:11:46 -07:00
Nicolas "Pixel" Noble 5495a7f7b1 Merge branch 'master' of https://github.com/grpc/grpc into moiz-upb 2019-07-16 19:07:24 +02:00
Arjun Roy b1d73a01f1 Removed duplicate static table from hpack table. Removed an or instruction for
every usage of static grpc metadata. Inlined hpack table lookups for static
metadata.

This leads to faster hpack parser creation:
BM_HpackParserInitDestroy 5.32µs ± 1% 0.06µs ± 1%  -98.91% (p=0.000 n=18+19)

And slightly faster parsing:
BM_HpackParserParseHeader<RepresentativeClientInitialMetadata, OnInitialHeader>
456ns ± 1%              435ns ± 1%   -4.74%        (p=0.000 n=18+19)
BM_HpackParserParseHeader<MoreRepresentativeClientInitialMetadata,
OnInitialHeader>
1.06µs ± 2%             1.04µs ± 2%   -1.82%        (p=0.000 n=19+20)

It also yields a slight (0.5 - 1.0 microsecond) reduction in CPU time for
fullstack unary pingpong:
BM_UnaryPingPong<TCP, NoOpMutator, NoOpMutator>/0/512
[polls/iter:3.0001                         ]            23.9µs ± 2%
23.0µs ± 1%  -3.63%          (p=0.002 n=6+6)
BM_UnaryPingPong<TCP, NoOpMutator, NoOpMutator>/0/32768
[polls/iter:3.00015                        ]            35.1µs ± 1%
34.2µs ± 1%  -2.57%          (p=0.036 n=5+3)
BM_UnaryPingPong<MinTCP, NoOpMutator, NoOpMutator>/8/0
[polls/iter:3.00011                        ]            21.7µs ± 3%
21.2µs ± 2%  -2.44%          (p=0.017 n=6+5)
2019-07-15 15:39:31 -07:00