Commit Graph

2153 Commits

Author SHA1 Message Date
Craig Tiller 5bfe38f10f
Revert "Convert client_auth_filter to promises (#28767)" (#28951)
This reverts commit cadce47ebf.
2022-02-23 08:17:37 -08:00
Craig Tiller cadce47ebf
Convert client_auth_filter to promises (#28767)
* Convert client_auth_filter to promises

* outline

* x

* x

* fixes

* remove-spam

* Async sequences over iterators

* iterate

* better semantics

* fix

* clang-tidy

* Automated change: Fix sanity tests

* Automated change: Fix sanity tests

* fixes

* fixes

* Automated change: Fix sanity tests

* progress

* progress

* progress

* progress

* progress

* credentials_test passes

* Automated change: Fix sanity tests

* Make Activity more of an interface

Move mutex and wakeup logic into a new class between PromiseActivity<>
and Activity (so that the sharing can persist), but make Activity closer
to a pure interface, so that whilst we migrate code we can implement
better fakes without forcing allocation.

* fixes

* Automated change: Fix sanity tests

* fixes

* test the awful failure

* x

* fix typo

* fix race

* 4.9

* x

* review feedback

* review feedback

* Automated change: Fix sanity tests

* review feedback

* replace AuthMetadataContext interface with GetRequestMetadataArgs struct

* Automated change: Fix sanity tests

* fix merge

* Fixup

* fix

* fix

* fix

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
Co-authored-by: Mark D. Roth <roth@google.com>
2022-02-23 07:36:19 -08:00
AJ Heller 85189b24bb
Reintroduce: Avoid fully qualifying namespaces (and add check) (#28917)
Based on a handful of https://abseil.io/tips, it's generally advised to
only fully-qualify namespaces when in a `using` statement, or when it's
otherwise required for compilation. In all other cases, the general
recommendation is to not fully-qualify.

This change fixes most `grpc.*` namespace uses. There are potential
challenges in trying to make blanket changes to non-gRPC namespace uses,
such as `::testing`, since there is also a `grpc::testing` namespace.
2022-02-18 16:18:54 -08:00
AJ Heller e72a5fe5dd
Revert "Avoid fully qualifying namespaces (and add check) (#28901)" (#28916)
This reverts commit fc7314c701.
2022-02-17 17:56:19 -08:00
AJ Heller fc7314c701
Avoid fully qualifying namespaces (and add check) (#28901)
Based on a handful of https://abseil.io/tips, it's generally advised to
only fully-qualify namespaces when in a `using` statement, or when it's
otherwise required for compilation. In all other cases, the general
recommendation is to not fully-qualify.

This change fixes most `grpc.*` namespace uses. There are potential
challenges in trying to make blanket changes to non-gRPC namespace uses,
such as `::testing`, since there is also a `grpc::testing` namespace.
2022-02-17 16:23:25 -08:00
donnadionne 431d314eca
Calculate clock difference and accomodate it to deflake timeout tests (#28868)
* Calculating clock difference and accomendate it to deflake timeout
tests.

* Fixing according to code review comments.
2022-02-17 15:40:55 -08:00
Oscar Shi e0a5c310cd
Increase number of threads in stress test (#28821)
In some configurations 100 threads is not enough to cause exhaustion failing tests.

Increase number of threads to ensure resource exhaustion
2022-02-15 13:58:01 +01:00
AJ Heller 40f2c2a4e1
Upgrade googletest (#27287) 2022-02-14 18:16:05 -08:00
Ashitha Santhosh 1fee3d72be
remove SDK term from gRPC authz (#28843) 2022-02-14 11:44:15 -08:00
donnadionne 1baca37a9e
Applying aggregate ringhash policy (#28861)
* Applying aggregate ringhash policy

* Fixing according to code review comments.

* typo
2022-02-11 16:16:26 -08:00
Ashitha Santhosh 55e4af1b14
Revert "Revert "Update to rbac policy struct and end2end authz test. (#27074)" (#28552)" (#28620)
This reverts commit 8ca42ec6f8.
2022-02-11 14:03:24 -08:00
yihuaz b458db9246
Eliminate gRPC insecure build (#25586)
* force submit

* fix test error

* remove is_client from local tsi and its callsites

* fix too_many_pings_test

* add missing dep
2022-02-10 11:17:18 -08:00
scwhittle 14169dd0c5
[issue #28771] Fix pick_first policy to clear selected_ when deleting subchannel_list_ (#28824)
* [issue #28771] Fix pick_first policy to clear selected_ when promoting
a pending subchannel list to the active subchannel list when all
subchannels have been attempted and are in an error state.

* address comments

* revert idle_filter
2022-02-10 09:48:57 -08:00
Craig Tiller 9bcabbac32
Disable flaky test (#28776) 2022-02-02 23:12:35 -08:00
donnadionne 55db347396
LRS changes for federation (#28504)
* Passing xds server object instead of just a string name

* Adding xds server to policy

* Refactor ToJson

* Using XdsServer for load reporting

* code review comments

* fixing code review comments

* Taking care of lifetime of the XdsServer key

* code review comments

* Fixing channel_state storage and re-run tests (1 assert hit)

* Checking for server in the bootstrap file

* Adding LRS test

* adding a bootstrap file ToJson and parse test

* fixing code review comments

* fixing code review comments.

* fixing test

* break out the federation lrs test

* Fixed last bit of code review comments

* fixing error message to be more precise
2022-01-21 10:55:58 -08:00
Mark D. Roth 86c483a678
fix priority policy to cancel failover timer on IDLE and add test (#28563) 2022-01-20 13:34:58 -08:00
Craig Tiller 74a2cb6e2b
Revert "Revert "Reland slice changes (#28601)" (#28615)" (#28624)
* Revert "Revert "Reland slice changes (#28601)" (#28615)"

This reverts commit 939bbfc336.

* Change random seed back to nanoseconds
2022-01-20 12:47:33 -08:00
Mark D. Roth 9f090a3a32
xds: accept SelfConfigSource for RDS and EDS ConfigSources (#28618) 2022-01-19 13:57:15 -08:00
Craig Tiller 939bbfc336
Revert "Reland slice changes (#28601)" (#28615)
This reverts commit b33e0d40af.
2022-01-19 09:40:19 -08:00
Vignesh Babu aeea02fab8
TLS Session Keys export for GRPC C++ (#26812)
* Adding TLS Key export logic to core and c++ wrappers

* Adding and end2end cpp tls key export test and updating broken test due to interface changes

* regenerate projects

* updating tls key export core logic with addition of APIs to grpc_security.h

* undoing changes to tls_security_connector_test

* regenerate projects

* changing the logging format enum name as per GRFC comments

* regenerate projects

* removing some commented code

* updating changes as per review comments

* adding GRPCAPI annotations to functions defined in grpc_security.h

* regenerate projects

* fixed some code styling issues

* removing grpc_security.h include from tls_credentials_options.h

* updating files as per review comments

* minor fixes

* moving some code around

* removing key log format from tls session key log config and converting it to a simple string

* regenerate projects

* fixing mistakes in recent merge with master

* regenerate projects

* regenerate projects

* fixing some distrib and snity errors

* fixing formatting errors

* fixing more sanity checks and raising supported openssl versions to 1.1.1

* updating min supported openssl version to 1.1.1

* updating min supported openssl version in tls_key_export_test

* updating test to fix incorrect vector initialization

* updating as per latest comments

* fixing sanity checks

* addressing review comments

* fixing sanity checks

* fixed c++ comment style

* Automated change: Fix sanity tests

* fixing review comments

Co-authored-by: Vignesh2208 <Vignesh2208@users.noreply.github.com>
2022-01-18 14:44:55 -08:00
Mark D. Roth 0fb47cd886
pick_first: make TRANSIENT_FAILURE sticky (#28571) 2022-01-18 13:25:07 -08:00
Craig Tiller b33e0d40af
Reland slice changes (#28601)
* Revert "Revert "Eliminate slice interning (#28363)" (#28598)"

This reverts commit 03bf699600.

* fix?
2022-01-18 09:29:49 -08:00
Jan Tattermusch 03bf699600
Revert "Eliminate slice interning (#28363)" (#28598)
This reverts commit 6703186b7a.
2022-01-18 15:44:18 +01:00
Yash Tibrewal 1caa3e8cfd
xDS: Rbac filter updates (#28568) 2022-01-18 04:46:40 -08:00
Craig Tiller 6703186b7a
Eliminate slice interning (#28363)
Eliminate slice interning, and structures in slices to support it.
Reduces grpc_slice_refcount from 40 bytes (+ a required 8 bytes elsewhere) to 16 bytes.
Removes a pointer dereference for every slice ref/unref.

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2022-01-17 20:25:45 -08:00
Mark D. Roth e59dcd5c87
xds: force-enable retries in xds_end2end_test to make things work internally (#28564) 2022-01-14 09:34:39 -08:00
Mark D. Roth 227d65367c
rls: fix various bugs in adaptive throttling code (#28477)
* rls: fix adaptive throttling window size

* clang-format

* fix adaptive throttling logic and fix FailedRlsRequestWithoutDefaultTarget test
2022-01-14 08:21:32 -08:00
Jan Tattermusch 8ca42ec6f8
Revert "Update to rbac policy struct and end2end authz test. (#27074)" (#28552)
This reverts commit b64167a034.
2022-01-14 07:45:38 -08:00
Ashitha Santhosh b64167a034
Update to rbac policy struct and end2end authz test. (#27074)
* Empty principals checks for authenticated connection

* fix sanity check

* clang-format

* principals:[] will result in ANY

* minor correction after merging

* clang-format

* formatting

* clang-format

* remove unnecessary header

* Remove unnecessary target from BUILD
2022-01-12 15:13:35 -08:00
Yash Tibrewal 114d388389
Fix xDS client for multiple watchers (#28521)
* Fix XdsClient for multiple watchers

* Reviewer comment
2022-01-12 11:41:06 -08:00
Mark D. Roth 5c30de312b
xds: handle percent-encoding in new-style resource names (#28515) 2022-01-12 10:09:06 -08:00
Yash Tibrewal 9ffbc2d360
XdsEnd2EndTest : Use a queue to save resource updates (#28467)
* Fix XdsClient for multiple watchers on the same resource

* xds_end2end_test: Don't use XdsCredentials for XdsRbacNackTests

* Use separate response states for EDS and RDS resources

* Reviewer comments

* Reviewer comments

* Reviewer comments

* Remove blank link

* Reviewer comments
2022-01-11 15:02:12 -08:00
Mark D. Roth bccd1c7c22
xds_e2e_test: clean up WaitForAllBackends() and add timeout (#28514)
* xds_e2e_test: clean up WaitForAllBackends() and add timeout

* fix drop tests

* restore resetting of backend counters

* fix BalancerRestart test

* fix StressTest

* clang-format
2022-01-11 08:37:54 -08:00
Ashitha Santhosh d884a7a440
RBAC Authorization Engine updates based on A41 proposal (#27754)
* RBAC Authorization Engine updates based on A41 proposal

* Fix headermatcher

* Support for metadata

* Maps host to :authority, special-case te and :method

* removes method hardcoding

* add factory methods

* minor formatting

* fix test

* formatting

* remove constructors

* fix api usage

* ReqServerName test in xds_end2end_test

* Initialize bool
2022-01-10 19:52:20 -08:00
Craig Tiller 55b365f287
Move resolver code around a little (#27846)
* move service config

* service config should not depend on grpc_base

* move resolver, deal with fallout

* Automated change: Fix sanity tests

* fix

* fix

* Automated change: Fix sanity tests

* review feedback

* review feedback

* fix

* Automated change: Fix sanity tests

* fix

* Automated change: Fix sanity tests

* fix vis

* fix

* fixes

* Automated change: Fix sanity tests

* visibility

* fix

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2022-01-06 10:53:40 -08:00
donnadionne e520fc137c
xDS Federation: bootstrap and xds_resolver changes (#27938)
* xDS Federation: bootstrap and xds_resolver changes

* code review fixes

* fixing code review comments

* fixing code review comments

* fixing code review comments

* code review comments

* fixing code review comments

* First very basic test to make sure parsing and reconstruction work as
expected.

* clean up

* fixing logic error about authority

* fixing resource type parsing

* fixing code review comments

* simplify parsing!

* Parsing method signature update

* fixing code review comments

* clean up

* working progress for the test with generated bootstrap

* reorg the bootstrap file

* fixing tests

* Adding more to test authorities

* Added a test and it passes

* addressing code review comments

* code review comments to make parser cleaner and more efficient

* Merge in authority prefixes

* fixing sanity error and xds boostrap test error

* small fix

* Release all tests that pass; reduce scope for DeadUpdate

* Updated test strcuture and how to pass in the index for balancers to be
used as xds server uri and authority xds server uri

* code review comments

* code review fixes

* code review comment

* Making test structure changes

* fixing code review comments

* fixing code review comments

* Fixing test regression

* fixing bootstrap tests

* cleanup files

* enabling localhost:xxx for xds server; updated server tests and will fix
one more NameExpected test with testsetup.

* Finally removing fake reolsver for xds server

* Fixing bootstrap tests

* Rewrite builder

* Fixing code review comments

* fixing code review comments

* Fixing all tests to use Setup again

* fixing small sanity error

* Found the source of xds server nack test faiure and fix added

* small code review fixes

* Remove fake resolver! YAY!

* Fixing according to code review comments

* Setup plugin in bootfile

* Added more tests.

* Adding server test

* fixing a regression

* regression

* sanity fix

* fixing code review

* fixing code review comments

* Re-combine SecurityNaming tests.

* Add Rds new resource type and new tests

* Added PercentEncode test

* fixing code review comments

* refactor test a bit more

* fixing code review comments

* fixing according to code review comments

* fixing code review comments

* fixing code review comments
2022-01-05 11:13:11 -08:00
Yash Tibrewal f00432c70b
xds_end2end_test: Set skip cancelled check (#28453)
* xds_end2end_test: Set skip cancelled check

* Reviewer comment
2021-12-29 18:15:19 -08:00
Yash Tibrewal ab6d7b4f02
Reland rbac filter (#28442)
* Revert "Revert "xDS: Add support for RBAC HTTP filter (#28309)" (#28441)"

This reverts commit 7aae5c66df.

* Fix internal clang errors

* Ipv6 addresses are 128 bit
2021-12-29 11:42:58 -08:00
Craig Tiller 7aae5c66df
Revert "xDS: Add support for RBAC HTTP filter (#28309)" (#28441)
This reverts commit 6ea8214879.
2021-12-28 20:11:38 +00:00
Yash Tibrewal 6ea8214879
xDS: Add support for RBAC HTTP filter (#28309)
* xDS: ADD RBAC HTTP filter support

* sanity, upb regenerate files

* Revert PerChannelArg changes

* Reviewer comments

* Reviewer comments

* Reviewer comments

* Remove unnecessary header

* Fix sanity

* Add RBAC service config parsing tests

* Don't make a copy of the metadata batch

* Revert expr_proto changes

* Some more tests

* Reviewer comments

* Reviewer comments

* No metadata changes needed

* Fix leak of DynamicXdsServerConfigSelectorProvider

* Fix deadlock issues

* Fix test compilation
2021-12-23 20:46:36 -08:00
Ashitha Santhosh 4fd524cd17
Fix presence match in authorization. (#28269)
* Fix presence match in authorization.

* Remove header

* Add test

* fix regex to include whitespace characters
2021-12-13 14:53:05 -08:00
Mark D. Roth f76f2653c3
Second attempt: XdsClient: remove resource-type-specific methods from XdsClient API (#28305)
* Revert "Revert "XdsClient: remove resource-type-specific methods from XdsClient API (#28231)" (#28301)"

This reverts commit 3e779b68fe.

* remove global resource type registry
2021-12-08 09:24:33 -08:00
Mark D. Roth 3e779b68fe
Revert "XdsClient: remove resource-type-specific methods from XdsClient API (#28231)" (#28301)
This reverts commit 747a5d8c24.
2021-12-07 15:46:10 -08:00
Mark D. Roth 747a5d8c24
XdsClient: remove resource-type-specific methods from XdsClient API (#28231)
* WIP

* introduce XdsResourceType API and change Listener parsing to use it

* converted RouteConfig parsing

* convert cluster and endpoint parsing

* cleanup

* clang-format

* attempt to work around compiler problems

* move XdsResourceType to its own file, and move endpoint code out of XdsApi

* move cluster parsing to its own file

* move route config parsing to its own file

* move listener parsing to its own file

* clang-format

* minor cleanup

* plumbed XdsResourceType throughout XdsClient

* a bit of cleanup

* more cleanup

* construct full resource names before calling XdsApi::CreateAdsRequest()

* remove some unneeded code

* clean up includes and have XdsResourceType initialize the upb symtab

* more cleanup of unnecessary code

* more cleanup

* update comment

* clang-format

* add missing virtual dtor

* fix build

* remove resource-type-specific methods from XdsClient API

* have each resource type register itself upon instantiation

* remove comment

* add missing virtual dtor

* clang-format
2021-12-07 11:00:08 -08:00
Mark D. Roth ac70281e74
improve RPC status messages when name resolution fails (#28091)
* improve RPC status messages when name resolution fails

* fix tests

* clang-format

* fix rls_end2end_test

* fix build

* fix service_config_end2end_test

* fix too_many_pings_test

* fix tests

* fix client_channel to propagate resolution_note to LB policy

* improve comment
2021-12-07 09:05:38 -08:00
Mark D. Roth 81a4ea444e
xds: remove legacy EDS-only workflow (#28274)
* xds: remove legacy EDS-only workflow

* remove unused constants

* remove unused data member
2021-12-06 08:00:15 -08:00
Yash Tibrewal 3689072979
xDS: Add graceful shutdown for old connections on listener resource update (#28154)
* xDS: Add graceful shutdown for old connections on listener resource update

* Add TODOs for review

* Reviewer comments

* Reviewer comments

* Fix merge

* Fix comment

* s/GRPC_ARG_DRAIN_GRACE_TIME_MS/GRPC_ARG_SERVER_CONFIG_CHANGE_DRAIN_GRACE_TIME_MS
2021-12-02 15:42:47 -08:00
Mark D. Roth 9722651156
xds_end2end_test: remove num_balancers from test suite and remove dependence on fake resolver for xDS channel (#28245)
* xds_end2end_test: remove num_balancers from test suite

* fix clang-tidy
2021-12-01 16:28:15 -08:00
Yash Tibrewal 25446c468a
Support RDS updates on the server (#27851)
* Port changes from #27388

* Reviewer comments

* Fix resource timeout issue

* Cleanup

* Fix clang-tidy

* Revert benchmark

* Restructure

* clang-tidy

* Automated change: Fix sanity tests

* Partial commit

* Reviewer comments

* Fixes

* Reviewer comments

* Reviewer comments

* Reviewer comments

* Reviewer comments

* clang-format

* Fix FaultInjection tests

* clang-tidy

Co-authored-by: yashykt <yashykt@users.noreply.github.com>
2021-11-18 16:10:31 -08:00
Esun Kim 11f440775b
Fix xds_end2end_test dyld (#28133) 2021-11-18 16:08:41 -08:00
Mark D. Roth 1050eedb87
enable clang-tidy readability-static-definition-in-anonymous-namespace check (#28033) 2021-11-17 07:48:43 -08:00
Mark D. Roth fff84402ac
Second attempt: XdsClient: fix resource timeout behavior (#28088)
* Revert "Revert "XdsClient: fix resource timeout behavior (#27860)" (#28032)"

This reverts commit 817eed0928.

* use the right status code enum
2021-11-15 14:53:15 -08:00
Craig Tiller a629c9a03e
Reland resource quota work (#28017)
* Check if memory owner available prior to polling it

The transport may drop the memory owner during its destruction sequence

* tcp_fix

* Revert "Revert "New resource quota integration (#27643)" (#28014)"

This reverts commit 0ea2c37263.

* clang-format

* fix-path

* fix
2021-11-15 08:23:51 -08:00
Craig Tiller 817eed0928
Revert "XdsClient: fix resource timeout behavior (#27860)" (#28032)
This reverts commit 7fdb40d8b8.
2021-11-12 19:58:16 -08:00
Mark D. Roth 7fdb40d8b8
XdsClient: fix resource timeout behavior (#27860)
* XdsClient: fix resource timeout behavior

* fix clang-tidy

* more clang-tidy fixes

* yet more clang-tidy
2021-11-12 12:17:45 -08:00
Craig Tiller 0ea2c37263
Revert "New resource quota integration (#27643)" (#28014)
This reverts commit 39f0877d25.
2021-11-11 14:26:19 -08:00
Craig Tiller 39f0877d25
New resource quota integration (#27643)
* new resource quota integration

* Automated change: Fix sanity tests

* fix

* fix

* fixes

* fixes

* fixes

* Automated change: Fix sanity tests

* fixes

* fixes

* Automated change: Fix sanity tests

* fixes

* fix

* fixes

* windows-fix

* fixes

* fixes

* fix

* fix-asan

* banned

* banned

* fixes

* clang-tidy-fix

* Automated change: Fix sanity tests

* fix-cronet

* review feedback

* review feedback

* Automated change: Fix sanity tests

* fixes

* bug fix

* fixes

* compile fix

* exclude megabyte size payloads from 1byte tests

* windows fix

* start moving ios

* keep moving windows

* Get windows compilation working.

* Automated change: Fix sanity tests

* better

* fixes

* remove slice buffer from memory_allocator.h

* Revert "remove slice buffer from memory_allocator.h"

This reverts commit 234a63b6e3.

* ugh

* #fixtests

* pthread tls fixes

* Automated change: Fix sanity tests

* fixfixfix

* xxx

* add reset

* review feedback

* fix

* fix

* fixes

* fix

* mac progress

* cpp-impl-of

* rename ptr

* Automated change: Fix sanity tests

* memory-owner-is-a-memory-allocator

* fixes

* fix

* fix from prod

* fix

* Fix issue leading to bad pointers being returned on Windows.

* Automated change: Fix sanity tests

* fix multislice bug

* argh

* hyrums law fixes

* hyrums law fixes

* clang-format

* hyrums law fixes

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2021-11-10 22:53:04 -08:00
ZhenLian 2e14f6fa70
Support Custom Post-handshake Verification in TlsCredentials (#25631)
* custom verification refactoring - post-handshake verification
2021-11-10 13:45:52 -08:00
Yash Tibrewal 00e8a2d085
xds_end2end_test: Only start backends when needed (#27911)
* xds_end2end_test: Fix flakiness on WaitForLdsNack

* xds_end2end_test: Only start the server when we want

* Revert WaitForNack changes

* Fixes

* Fix CsdsShortAdsTimeoutTest

* Fix sanity
2021-11-04 14:35:25 -07:00
Mark D. Roth 3bde8305cc
Second attempt: Don't build RLS on mobile (#27878)
* Revert "revert changes to not build RLS on mobile (#27875)"

This reverts commit 73ad459c51.

* new approach using multiple select() calls
2021-11-01 10:26:01 -07:00
Mark D. Roth 73ad459c51
revert changes to not build RLS on mobile (#27875)
* Revert "fix mobile builds to properly exclude xDS (#27855)"

This reverts commit 64d026e234.

* Revert "don't build RLS on mobile (#27838)"

This reverts commit 2a69c525b8.
2021-10-29 12:00:00 -07:00
Mark D. Roth 08c7b254bd
provide more slack in RLS end2end tests (#27842) 2021-10-27 15:49:54 -07:00
Mark D. Roth 2a69c525b8
don't build RLS on mobile (#27838)
* don't build RLS on mobile

* buildifier

* fix build rules to avoid "Illegal ambiguous match" bazel error
2021-10-27 15:46:00 -07:00
Mark D. Roth 06b5061d35
change CSDS to populate new generic_xds_configs field (#27794) 2021-10-26 13:24:27 -07:00
Yash Tibrewal 9ac9a013fc
Update RDS parsing for use on servers (#27715)
* Update RDS parsing for use on servers

* Unused variable

* Reviewer comments

* Automated change: Fix sanity tests

* Fix tests

* Reviewer comments

* Reviewer comments

* clang-tidy

* Reviewer comments

* Fix test

* Reviewer comments

* Reviewer comments

Co-authored-by: yashykt <yashykt@users.noreply.github.com>
2021-10-26 00:32:35 -07:00
Mark D. Roth 5406048ef6
Second attempt: RLS LB policy (#27748)
* allow connectivity state watching to work on lame channels

* Revert "Revert RLS LB policy (#27738)"

This reverts commit 4567af504e.

* fix build

* fix lame_client_test
2021-10-20 07:41:07 -07:00
Craig Tiller f1f3ac7f02
Revert "Revert "allow connectivity state watching to work on lame channels (#27747)" (#27766)" (#27774)
This reverts commit 790ce5c97f.
2021-10-19 15:28:31 -07:00
Craig Tiller 790ce5c97f
Revert "allow connectivity state watching to work on lame channels (#27747)" (#27766)
This reverts commit a2bd7b8440.
2021-10-19 08:16:29 -07:00
Mark D. Roth d6a0c0b8b5
xds_end2end_test: allow setting Listener to use in SetRouteConfiguration() (#27756)
* xds_end2end_test: allow setting Listener to use in SetRouteConfiguration()

* clang-format
2021-10-19 07:36:54 -07:00
Mark D. Roth a2bd7b8440
allow connectivity state watching to work on lame channels (#27747)
* allow connectivity state watching to work on lame channels

* fix build

* fix lame_client_test
2021-10-18 14:47:47 -07:00
Mark D. Roth 4567af504e
Revert RLS LB policy (#27738)
* Revert "fix for import of RLS LB policy (#27736)"

This reverts commit cfcdbc4905.

* Revert "RLS LB policy (#27352)"

This reverts commit 84ddc3289f.
2021-10-15 12:19:57 -07:00
Mark D. Roth cfcdbc4905
fix for import of RLS LB policy (#27736) 2021-10-15 07:56:48 -07:00
Mark D. Roth 84ddc3289f
RLS LB policy (#27352)
* RLS LB policy plugin implementation

* Add RLS tests

* rls proto

* generate projects

* Fix adding child policy to child policy map

* Fix issues

* Fix rotten bits

* generate projects

* revert some strange changes

* First batch of fixes

* second batch of addressing comments

* generate_projects

* 3rd batch of fixes

* generate_projects

* 4th batch of fix

* 5th batch of fix

* Empty commit to trigger github PR refresh

* Add insecure build

* generate project

* update picker in a closure

* mutex guard audit

* clang-format

* Aggregate child policy states

* Use OrphanablePtr on ChildPolicyOwner()

* More fixes on comments

* Remove include grpc_security from rls.cc

* Fix key builder test as we removed RlsFindPathFromMetadata

* Update rls proto and add multi-target capability

* code changes to fix build after merging master

* remove support for insecure builds

* WIP

* add LB policy metadata lookup API

* add API for creating errors from C++ strings

* add missing include

* finished updating JSON parsing

* use DualRefCounted<> for ChildPolicyWrapper

* use grpc_core::Mutex instead of std::recursive_mutex

* add lock annotations and fix some lifetime issues

* misc fixes and cleanups

* simplify child policy connectivity state machine

* add comment about hopping into ExecCtx

* don't use wait_for_ready for RLS call

* extraKeys and constantKeys support

* add lock annotations for child policy wrapper connectivity state

* improve logging

* simplify child policy state handling in picker

* use C++ style comments

* remove rls_config.proto, since it's not needed

* rewrote tests

* fix build

* fix copyright headers

* clang-format

* appease clang-tidy

* remove unnecessary dependendency on grpclb

* buildifier

* remove illegal term

* remove unneeded includes

* clang-format

* fix clang-tidy

* fix build

* move class declaration into .cc file

* clang-tidy again

* fix build

* reorganize code and misc cleanups

* clang-format

* fixed a bunch of asan bugs; still some left to debug

* fix asan problem

* make test work on IPv6-only machines

* move LRU list handling into Cache::Entry and add lock annotations

* use preincrement instead of postincrement

* fix deadlock

* add more FIXMEs

* clean up backoff timer logic

* fix build from merge

* clang-format

* fix include path to work on podspec builds

* clean up picker logic

* clang-format

* update rls.proto

* populate reason and stale_header_data in RLS request

* improve logging and fix some cache size type issues

* clang-format

* add some TODOs about a better way to expose channel creds to LB policies

* centralize handling of channel's default authority

* clang-format

* fix backup poller bug

* fix handling of call creds and authority on RLS channel

* remove unused params

* clang-format

* use two-phase update for child policies to avoid deadlocks

* fix portability issue

* minor cleanups

* update for change in grpc_error_get_status() API

* change test to store RLS requests and responses in proto form

* clang-format

* account for test slowdown factor in timeouts

* fix tsan failure and channelz linkage

* fix RLS authority death test

* fix test to not try to connect to target that should not work

* remove illegal term

* cacheSizeBytes is a required field

* add missing BUILD deps from merge

* add tests for connectivity state reporting and fix bugs found

* fix BUILD sanity

* buildifier

* fix BUILD package path

* fix bugs related to child policy lifetime and updates

* remove unnecessary srand() call from grpc_init()

* add test for two cache entries with the same target

* update rls.proto

* change RLS config parser test to support GRPC_ERROR_IS_ABSEIL_STATUS

* update upb codegen

* fix include path

* add env var guard

* avoid duplication of CountedService

* generate_projects

* add upper limit for cacheSizeBytes

* fix build

Co-authored-by: Muxi Yan <mxyan@google.com>
2021-10-14 12:53:46 -07:00
Mark D. Roth 6339b03ada
Revert "Revert "start splitting up xds_end2end_test (#27702)" (#27725)" (#27726)
This reverts commit 9177c5324a.
2021-10-14 11:41:45 -07:00
Mark D. Roth 464c1c519f
switch to new location of TypedStruct and ORCA protos (#27700)
* upgrade xDS protos

* switch to new location of TypedStruct and ORCA protos

* fix sanity
2021-10-14 10:28:24 -07:00
Mark D. Roth 9177c5324a
Revert "start splitting up xds_end2end_test (#27702)" (#27725)
This reverts commit 69a978231b.
2021-10-14 08:47:17 -07:00
Mark D. Roth 69a978231b
start splitting up xds_end2end_test (#27702)
* move test xDS server to its own library

* move xds_end2end_test to xds subdir, and refactor CountedService

* add TODO

* fix sanity
2021-10-13 13:11:53 -07:00
Mark D. Roth 2b813d2bff
grpclb: implement subchannel caching (#27657)
* grpclb: implement subchannel caching

* code review changes

* fix clang tidy

* code review changes
2021-10-13 12:14:00 -07:00
Yash Tibrewal a61bb8ed5b
Move service config code out of client_client to its own target (#27683)
* Move service config code out of client_client to its own target

* Fix BUILD

* Fix test build

* Reviewer comments

* Reviewer comments

* Automated change: Fix sanity tests

* Reviewer comments

Co-authored-by: yashykt <yashykt@users.noreply.github.com>
2021-10-12 18:30:04 -07:00
Ashitha Santhosh a1db97be90
Revert "Revert "Revert "Revert "File watcher authorization policy provider implementation"" (#27605)" (#27644)" (#27645)
This reverts commit b8e01f73a0.
2021-10-08 15:52:13 -07:00
AJ Heller b8e01f73a0
Revert "Revert "Revert "File watcher authorization policy provider implementation"" (#27605)" (#27644)
This reverts commit 5912aedb2b.
2021-10-07 16:45:30 -07:00
Esun Kim ec4c61a3ae
Various fixes on absl::Status migration (#27557) 2021-10-07 14:49:59 -07:00
Ashitha Santhosh 5912aedb2b
Revert "Revert "File watcher authorization policy provider implementation"" (#27605)
* Revert "Revert "File watcher authorization policy provider implementation (#26779)" (#27591)"

This reverts commit a1ad74f7af.

* Remove license comment
2021-10-07 09:54:23 -07:00
Mark D. Roth 3d83dd3776
Revert "Revert "centralize handling of channel's default authority (#27544)" (#27616)" (#27624)
This reverts commit 0611f7c9b3.
2021-10-06 12:15:55 -07:00
Mark D. Roth 0611f7c9b3
Revert "centralize handling of channel's default authority (#27544)" (#27616)
This reverts commit f0a0628795.
2021-10-06 07:26:24 -07:00
Mark D. Roth f0a0628795
centralize handling of channel's default authority (#27544)
* centralize handling of channel's default authority

* clang-format

* fix backup poller bug
2021-10-05 11:57:47 -07:00
Craig Tiller a1ad74f7af
Revert "File watcher authorization policy provider implementation (#26779)" (#27591)
This reverts commit 310accd077.
2021-10-05 09:12:08 -07:00
Craig Tiller ce0aba2577
Increase XDS e2e shard count (#27582)
Provides a 40% reduction in runtime for this test internally, and since
it's on our critical path for landing changes, this is significant.
2021-10-04 17:22:13 -07:00
Ashitha Santhosh 310accd077
File watcher authorization policy provider implementation (#26779)
* File watcher authorization policy provider implementation
2021-10-04 14:22:45 -07:00
Lidi Zheng 9cd68439a2
Change the time-getting logic in xds test to what ExecCtx does (#27467) 2021-09-24 10:38:15 -07:00
Esun Kim 47586fb36e
Revert "Revert "Added absl::Status support to error_utils (#27358)" (#27418)" (#27429)
This reverts commit 933676c56c.
2021-09-21 14:42:57 -07:00
Craig Tiller 933676c56c
Revert "Added absl::Status support to error_utils (#27358)" (#27418)
This reverts commit 9b3f75d322.
2021-09-21 09:48:57 -07:00
Esun Kim 9b3f75d322
Added absl::Status support to error_utils (#27358) 2021-09-20 18:03:17 -07:00
donnadionne 1caff522e4
Adding a log to make logging in WaitForAllBackends and WaitForBackend (#27365)
consistent.

This helps to uncover a case where the first test in the shard seems to
encounter longer delays than the rest of the tests.
2021-09-20 11:48:43 -07:00
Alisha Nanda 0a502d8f2e
Fix race between Read and ServerContext::IsCancelled in Sync API (#27056)
* Fix OOM issues in qps tests

* Add more verbose logging.

* Fix clang error

* Fix race between IsCancelled and Read

* Fix build errors from using bool in C code
2021-09-15 12:29:02 -07:00
donnadionne 10f218030d
xDS retry policy should be considered present even if there are no supported policies in retry_on (#27315)
* Ensure that per route retry policy (even when there are no supported
retry_on statuses) still takes precedence over virtual host level retry
policy.

Added a test to guard this case.

* Taking care of code review comments and removing unnecessary block
2021-09-13 18:13:45 -07:00
Lidi Zheng 15184d30ad
Fix injected abort error not recorded after injected delay (#27215)
* Fix injected abort error not recorded after injected delay

* Add the abort after delay test case

* Make Kokoro's clang_format happy
2021-09-13 09:52:20 -07:00
Mark D. Roth 3763be8796
add APIs for creating errors from C++ strings (#27310)
* add API for creating errors from C++ strings

* add missing include
2021-09-10 16:15:36 -07:00
Yash Tibrewal 72171a3326
De-experimentalize XdsCredentials and XdsServerCredentials API (#26544)
* De-experimentalize XdsCredentials and XdsServerCredentials API

* Use GRPC_DEPRECATED
2021-09-09 16:22:25 -07:00
Yash Tibrewal 362aff3458
xDS: Remove environmental variable guard for security (#27290) 2021-09-09 16:21:15 -07:00
Yash Tibrewal 7fd731f704
De-experimentalize XdsServerBuilder (#27296) 2021-09-09 14:14:11 -07:00
Yash Tibrewal 1a009cf4e5
xDS Security: Use new way to fetch certificate provider plugin instance config (#27264)
* xDS Security: Use new way to fetch certificate provider plugin instance
config

* Reviewer comments

* Additional fields to NACK

* Move NACKing tests for tls_certificates and tls_certificate_sds_securet_configs to client-side
2021-09-09 14:12:05 -07:00
Yash Tibrewal f26c107651
xDS server serving status: Use a struct to allow more fields to be added in the future (#27242) 2021-09-09 11:11:57 -07:00
Mark D. Roth f3497eb790
implement improved xDS NACK semantics (#27276)
* refactor xDS response parsing

* fix build

* implement improved xDS NACK semantics

* fix clang-tidy

* fix test
2021-09-08 13:37:07 -07:00
Craig Tiller ea389c00c2
Adjust include order per style guide (#27175)
Introduce clang-format configuration to sort includes closer to our rules.
2021-09-08 12:14:44 -07:00
Mark D. Roth e468b00c56
refactor xDS response parsing (#27272)
* refactor xDS response parsing

* fix build
2021-09-08 07:57:38 -07:00
Mark D. Roth 2cb3831c83
fix retry code to fail batches instead of creating attempt if previously cancelled from surface (#27217)
* fix retry code to fail batches instead of creating attempt if previously cancelled from surface

* add xDS end2end tests covering the FI use-case that triggered the bug

* fix memory leak
2021-09-01 15:47:16 -07:00
Yash Tibrewal cb2e7554a3
xds_end2end_test: Fix UnsupportedHttpFilter test on aws (#27147) 2021-08-27 09:10:46 -07:00
Ta-Wei Tu 7dd1da9e1d
Break after WriteLast() (#27165)
If we continue to call `Write()` after `WriteLast()` (because client is still sending us data and thus `Read()` returns `true`), the buffer might become invalid and results in assertion failures.
2021-08-27 23:19:12 +08:00
Ashitha Santhosh 1ff1f8fd83
SDK authorization filter. (#26468)
* SDK authorization filter.
2021-08-25 21:00:35 -07:00
donnadionne 59fc02a94b
Enabling Ring Hash by default. (#27111) 2021-08-24 19:59:19 -07:00
Yash Tibrewal 6da9eb9f3f
xDS PSM Server Security: Nack unsupported require_sni and ocsp_staple_policy values (#26878)
* Nack unsupported require_sni and ocsp_staple_policy values

* Reviewer comments
2021-08-24 10:50:08 -07:00
AJ Heller 6ba4a92af7
Reorder gtest/gmock headers (#27070)
Per the style guide (or spirit thereof), these should be placed with the other external/third_party headers.
2021-08-20 14:36:08 -07:00
Yash Tibrewal 0ab13090e7
Nack certain unsupported fields in CertificateValidationContext (#26880)
* Nack certain unsupported fields in CertificateValidationContext

* Regenerate projects

* Reviewer comments
2021-08-20 11:03:20 -07:00
AJ Heller d10617edb5
Move resource_user ownership into chttp2 transport/server/connector v2 (#27032)
Reintroducing PR #26643, which was reverted in #27029

Fixed a memory leak and added a test that would have caught it (ASAN build): ca0c8c4
2021-08-18 17:04:00 -07:00
AJ Heller 4ba3fa52b9
Clean up some instances of -Wunreachable-code-aggressive (#27054)
See https://github.com/grpc/grpc/pull/26951 for why this is not enforced
at the moment.
2021-08-18 11:43:46 -07:00
AJ Heller 58f8f5ced3
Add macro to define gtest death_test_style using an older syntax (#27042)
This temporarily unblocks a related gtest upgrade. The ultimate goal is
to upgraade our gtest dependencies, but I don't have the cycles to
manage a potentially messy migration until at least next week. This PR
is coordinated with an internal change.
2021-08-17 17:12:28 -07:00
Yash Tibrewal 533f02642d
Disable xds_credentials_test on iOS (#27043) 2021-08-17 13:33:05 -07:00
Mark D. Roth 9f782faa34
xds: remove env var protection for retries (#27039) 2021-08-17 13:23:25 -07:00
Craig Tiller f5d3ed2db1
Revert "Move resource_user ownership into chttp2 transport/server/connector (#26643)" (#27029)
This reverts commit d1935a65a1. Will be rolled forward with a fix.
2021-08-16 19:07:39 -07:00
AJ Heller d1935a65a1
Move resource_user ownership into chttp2 transport/server/connector (#26643) 2021-08-16 13:50:56 -07:00
Tamir Duberstein 34338e5798
Avoid undefined behavior in pthread TLS (#26999)
See code commentary for an explanation.

Add an additional constructor to allow `log_linux.cc` to compile with
GPR_PTHREAD_TLS. Without it:

```
../../third_party/grpc/src/core/lib/gpr/log_linux.cc:78:33: error: no viable conversion from 'int' to 'grpc_core::PthreadTlsImpl<long>'
  static GPR_THREAD_LOCAL(long) tid = 0;
                                ^     ~
../../third_party/grpc/src/core/lib/gpr/tls.h:64:3: note: candidate constructor not viable: no known conversion from 'int' to 'const grpc_core::PthreadTlsImpl<long> &' for 1st argument
  PthreadTlsImpl(const PthreadTlsImpl&) = delete;
  ^
1 error generated.
```
2021-08-12 19:25:38 -07:00
Tamir Duberstein 00c03c55ff
Implement type safety for TLS (#26942)
* Implement type safety for TLS

This is mostly free when compiler support is available, but requires
careful templating when implemented using pthread.

Significantly slimmed the tls.h interface; it now only defines the "TLS
keyword" for each supported compiler, delegating enforcement of correct
usage (i.e. must be static) to the compiler itself.

Implemented implicit conversion for the pthread wrapper so it can be
used (mostly) the same as native support. Notable exception to this is
that static_cast<void*> is needed when printing a pointer stored in TLS
as %p.

* Use GPR_THREAD_LOCAL macros consistently
2021-08-11 13:24:26 -07:00
Yash Tibrewal 404e56b2d9
Fix XdsEnabledServerTest (#26956) 2021-08-10 10:47:12 -07:00
Tamir Duberstein 3fd01fabae
Add thread safety annotations (#26871)
This exposes a bug in clang, reported upstream as
https://bugs.llvm.org/show_bug.cgi?id=51368.

The clang bug is mitigated using a fake scoped lock; that allows the
current code to compile while also serving as a change detector to
prevent it from going stale; if the compiler bug is fixed, the compiler
will see an overlapping locking requirement, and reject this code, which
will prompt a human being to remove this workaround.
2021-08-06 15:44:22 -07:00
Yash Tibrewal 29d220a0ef
Re-experimentalize XdsServerBuilder (#26892)
* Re-experimentalize XdsServerBuilder

* Fix compiler issue
2021-08-06 10:29:28 -07:00
Tamir Duberstein 013e67a029
Deprecate grpc{_core,::internal}::WaitUntil (#26866)
It is not possible for such a function to be implemented in a way that
is understood by annotalysis. Mark it deprecated and replace instances
of its use with direct mutex/condvar usage.

Add a bunch of missing thread safety annotations while I'm here.
2021-08-04 17:03:25 -07:00
Mark D. Roth 639b7acdfa
fix client_lb_end2end_test ClientLbPickArgsTest.Basic flakiness (#26868)
* fix client_lb_end2end_test ClientLbPickArgsTest.Basic flakiness

* clang-format
2021-08-04 09:46:47 -07:00
Tamir Duberstein 86c9de8231
Remove `grpc_core::WaitUntilWith{Deadline,Timeout}` (#26867)
These functions are essentially unused.
2021-08-03 15:46:36 -07:00
Mark D. Roth 6b71ec3704
LB policy API improvements (#26481)
* LB policy API improvements

* clang-format

* fix build

* a bit more cleanup

* use absl::variant<> for pick result

* fix retry_lb_drop test

* clang-format

* fix grpclb_end2end_test

* fix xds_end2end_test

* try to make variant code a bit cleaner

* clang-format

* fix memory leak

* fix build

* clang-format

* fix error refcount bug

* remove PickResult factory functions

* clang-format

* add ctors to structs

* clang-format

* fix clang-tidy

* update comments

* move LB recv_trailing_metadata callback instead of copying it

* use Match() instead of providing PickResult::Handle()

* don't use Match() for now, since it breaks lock annotations

* update retry_lb_fail test
2021-08-03 09:43:07 -07:00
Yash Tibrewal 49b5f9c420
Fix XdsServerSecurityTest (#26817) 2021-07-29 13:28:18 -07:00
donnadionne 7eaf37bce5
Adding retry policy (#26566) 2021-07-23 12:17:17 -07:00
Mark D. Roth 3104a9964c
xds: validate that terminal filters come at the end of the filter chain (#26742)
* xds: validate that terminal filters come at the end of the filter chain

* clang-format
2021-07-23 08:06:58 -07:00
Craig Tiller 4f9828fe0b
Remove unnecessary include (#26761) 2021-07-22 16:19:34 -07:00
Mark D. Roth 246c57829b
xds: notify watchers when NACKing resource updates (#26757)
* xds: notify watchers when NACKing resource updates

* check status of failed RPC when waiting for NACK
2021-07-22 13:12:19 -07:00
Alisha Nanda d73ec07fe7
Fix small typo in xds end2end test (#26685) 2021-07-14 10:53:55 -07:00
Yash Tibrewal c52005c161
NACK xDS updates when certificate provider instace names are unrecognized (#26614)
* NACK xDS updates when certificate provider instace names are unrecognized

* Reviewer comments

* Reviewer comments

* Clang format

* Fix compilation error
2021-07-11 22:10:04 -07:00
Lidi Zheng b97693d572
[Roll Forward] Tighten the error tolerance requirement by 100x (#26626)
* Tighten the error tolerance requirement by 10x

* Make it 5 sigma instead of 4.5

* Rewrap comments

* Loosen the max concurrent requests in certain test cases
2021-07-09 10:33:43 -07:00
Yash Tibrewal 670a26caef
NACK xDS updates where transport_name is not recognized (#26612)
* NACK xDS updates where transport_name is not recognized

* Reviewer comments
2021-07-08 11:14:07 -07:00
Yash Tibrewal 095045b31c
NACK xds updates requiring client certificates when no validation certificate provider instance is mentioned (#26613) 2021-07-07 13:22:13 -07:00
Yash Tibrewal 8b5fbf8900
TLS Security Connector: Add an always-fail-handshaker when certificates are not ready (#26561)
* TLS Security Connector: Add an always-fail-handshaker when certificates are not ready

* Reviewer suggestion

* Add test
2021-07-04 12:39:29 -07:00
Yash Tibrewal 4daedf0233
De-experimentalize XdsServerBuilder (#26543) 2021-07-02 19:33:15 -07:00
Lidi Zheng f198fb5b41
Revert "Tighten the error tolerance requirement by 100x (#26588)" (#26593)
This reverts commit f835f3f97c.
2021-07-01 16:21:12 -07:00
Lidi Zheng f835f3f97c
Tighten the error tolerance requirement by 100x (#26588)
* Tighten the error tolerance requirement by 10x

* Make it 5 sigma instead of 4.5

* Rewrap comments
2021-07-01 09:48:37 -07:00
Mark D. Roth 34bf26357b
Change StringMatcher to not support case-insensitive regex matching. (#26567)
* Change StringMatcher to not support case-insensitive regex matching.

* clang-format

* clang-tidy
2021-06-29 09:25:33 -07:00
Vijay Pai cf1ebf7387
Remove experimental tag from uses of the C++ callback API (#26495) 2021-06-20 00:47:07 -07:00
Yash Tibrewal bb994526ba
Fix ExistingRpcsOnResourceDeletion flakiness (#26419) 2021-06-02 18:48:44 -07:00
Lidi Zheng a36199b2d5
Tighten the probability requirement from 99.99% to 99.995% (#26408)
* Tighten the probability requirement from 99.99% to 99.995%

* Update comments
2021-06-01 17:14:00 -07:00
Vijay Pai 8aff94558c
De-experimentalize C++ callback API (#25728)
* De-experimentalize callback API

* Make FromServerContext based on ServerContextBase

* Fix lambda

* Fix headers

* De-experimentalize tests

* clang-format

* Fix consistency checks

* wip

* Fix const-ness of callback client read RPC requests

* Fix golden file

* Give full route_guide callback client example with Hold etc

* Complete example route-guide server

* De-experimentalize test services

* Omit unneeded using

* Remove some uses of non-experimental macro from test

* clang-format examples

* De-experimentalize async stub calls

* Remove experimental namespace use in qps, ubms

* De-experimentalize alarms, generic stubs, and context allocator

* De-experimentalize SetContextAllocator

* clang-format

* Fix conflicts

* Leave obsolete API in place until users can be migrated
2021-05-27 14:55:25 -07:00
Vijay Pai 153bc67f45
Revert "Revert "Revert "Revert "Expose trailers-only response status through C++ callback API"" (#26365)" (#26375)" (#26379)
This reverts commit 259a74c0e6.
2021-05-27 09:16:05 -07:00