Commit Graph

2358 Commits

Author SHA1 Message Date
Mark D. Roth 86e282ba97
weighted_target and RLS: delegate to child picker on error (#29870)
* weighted_target and RLS: delegate to child picker on error

* fix RLS bug that caused us to always use the last target in the list

* add comment
2022-06-22 15:21:18 -07:00
Mark D. Roth 7976501534
client_lb_end2end_test: refactor connection injectors and add test for sticky TF (#29993)
* refactor connection injectors in client_lb_end2end_test and add test for sticky TF

* Automated change: Fix sanity tests

Co-authored-by: markdroth <markdroth@users.noreply.github.com>
2022-06-22 12:22:00 -07:00
Mark D. Roth d0c8b29ef7
xDS: fix endpoint weight defaults (#30079)
* xDS: fix endpoint weight defaults

* add test

* NACK endpoint weight of 0

* iwyu

* clang-tidy

* fix test
2022-06-22 11:54:38 -07:00
Yash Tibrewal 78d6d71af3
Call: Send cancel op down the stack even when no ops are sent (#30004)
* Call: Send cancel op down the stack even when no ops are sent

* Add test
2022-06-14 13:10:41 -07:00
Mark D. Roth f12972506e
xDS: improve status messages from xDS resolver failures (#29975) 2022-06-14 10:25:30 -07:00
Mark D. Roth a78df68e96
xds: add "ignore_resource_deletion" server feature (#29633)
* xds: add "ignore_resource_deletion" server feature

* add logging

* clang-format

* fix build

* fix build

* add test for LDS resource deletion on gRPC server

* clang-format
2022-06-13 14:01:13 -07:00
Craig Tiller 6532084a06
[iwyu] rest of the owl: src/core/lib - excluding iomgr, gpr (#29735)
* [iwyu] rest of the owl: src/core/lib - excluding iomgr, gpr

* Automated change: Fix sanity tests

* Automated change: Fix sanity tests

* fixes

* Automated change: Fix sanity tests

* fix

* fix

* fixes

* fixes

* x

* fix?

* fix

* windows fix

* fix

* fix

* fix

* fix

* fix

* debug

* fix

* fix

* Revert "debug"

This reverts commit c99b8d12dd.

* Automated change: Fix sanity tests

* fix?

* Automated change: Fix sanity tests

* header cleanup

* fix

* fix

* fix

* Automated change: Fix sanity tests

* fix

* fix

* Automated change: Fix sanity tests

* Automated change: Fix sanity tests

* fixes

* Automated change: Fix sanity tests

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2022-06-10 13:04:47 -07:00
Vignesh Babu f8eedac1fc
Changing grpc_tcp_client_vtable to include TCP cancel connect method (#29968)
* Changing grpc_tcp_client_vtable to include TCP cancel connect method

* fix unused parameter error

* update

* fix sanity checks
2022-06-10 10:56:34 -07:00
Mark D. Roth 8459824e5c
xds end2end tests: change tests to check all RPC failure statuses (#29974)
* WIP

* fix from merge

* fix ring_hash tests

* ring hash: fix picker propagation bug in xds_cluster_manager policy

* fix build

* clang-format (ish)

* fix build

* more conversion

* remove old CheckRpcSendFailure

* fix SendRpcsAndCountFailuresWithMessage

* fix WaitForBackend

* clang-format

* revert xds resolver change, that will be done in a separate PR

* Automated change: Fix sanity tests

Co-authored-by: markdroth <markdroth@users.noreply.github.com>
2022-06-09 18:00:38 -07:00
Mark D. Roth 470a3066c7
ring hash: fix picker propagation bug in xds_cluster_manager policy (#29959)
* ring hash: fix picker propagation bug in xds_cluster_manager policy

* fix build

* fix build
2022-06-09 10:08:47 -07:00
Esun Kim c6ebbb5482
Added GRPC_ERROR_IS_NONE (#29937)
* Added GRPC_ERROR_IS_NONE

* Convert

* Fix
2022-06-08 11:24:20 -07:00
Mark D. Roth 27e6a094e1
client_lb_end2end_test: fix broken tests (#29938)
* client_lb_end2end_test: fix broken tests

* poll polling engine gives slightly different error messages
2022-06-07 15:32:47 -07:00
Mark D. Roth 35b7d88654
client_lb_end2end_test: explicitly check failed RPC statuses (#29906)
* client_lb_end2end_test: explicitly check failed RPC statuses

* appease clang-tidy

* fix memory leak
2022-06-07 11:59:08 -07:00
Mark D. Roth 17bec49af7
rls: use UNAVAILABLE instead of status from RLS RPC, and test failure statuses (#29893)
* rls: use UNAVAILABLE instead of status from RLS RPC, and test failure statuses

* simplify
2022-06-07 10:11:20 -07:00
Mark D. Roth e49c61cd2f
client_channel: remove synchronous subchannel connectivity state API (#28339)
* ring_hash: don't recreate ring when individual subchannel states change

* client_channel: remove synchronous subchannel connectivity state API

* change subchannel list to automatically start watching all subchannels

* use a separate loop to start watches, so list size is logged correctly

* fix RR to re-resolve on IDLE again

* fix ring_hash to delay promoting new subchannel list

* fix pick_first to wait for all subchannels to report state

* clean up SubchannelList API

* fix unused argument error

* fix another unused argument error

* clang-format

* fix RR to not re-resolve on initial IDLE state

* also don't re-resolve in initial TF state; same for ring_hash

* clang-format

* change RR and PF to initially report CONNECTING, and add second loop to priority policy

* simplify priority logic a bit

* fix grpclb to drop ref to stats object even if the subchannel call is never started

* fix memory leak in ring_hash

* fix tsan failure in grpclb code

* iwyu

* add missing BUILD deps

* update outlier_detection policy

* fix test

* fix pick_first to not report TF prematurely due to subchannel sharing

* fix test to not depend on timing
2022-06-03 06:44:26 -07:00
Anirudh Ramachandra e89883cc87
Revert "Revert "Fix grpc_sockaddr_to_uri to return URI encoded string. Also see #29323 which has a similar fix for unix abstract addresses."" (#29798)
Reverts #29639
2022-06-02 13:51:04 -07:00
Yash Tibrewal 5d9cdc8e67
xds: Add XdsLbRegistry (#29756)
* xds: Add XdsLbRegistry

* Formatting

* Fix upb output

* Allow alternative type url for Custom LBs

* Cleanup

* Reviewer comments

* Add some comments

* Reviewer comments

* Reviewer comments

* Reviewer comments

* Reviewer comments

* Fix tests

* Fix tests

* Fix tests

* Fix tests

* Fix tests

* Unused parameters

* Change to make TextFormat usage work internally

* Fix namespace qualification error
2022-06-01 14:08:26 -07:00
Mark D. Roth bbf56baa88
client_lb_end2end_test: fix flake in RoundRobinTest.UpdateInError (#29847) 2022-06-01 07:47:19 -07:00
Yash Tibrewal 8cdf8beafd
xDS: Workaround to get gRPC clients working with istio (#29841)
* xDS: Workaround to get gRPC clients working with istio

* Formatting

* Comment test too

* Disable instead of commenting
2022-05-31 17:39:46 -07:00
Mark D. Roth 65a7ce3f04
Revert "Revert "server: add method to expose authority seen by server (#29768)" (#29806)" (#29807)
This reverts commit e39e943529.
2022-05-31 14:15:45 -07:00
Mark D. Roth 64b422091e
xds_core_end2end_test: increase timeouts to fix flakiness (#29803)
* xds_core_end2end_test: increase timeouts to fix flakiness

* clang-format
2022-05-27 14:34:53 -07:00
Mark D. Roth a4cbeee699
xds_core_end2end_test: increase RPC timeout on BootstrapSource test (#29802) 2022-05-27 14:32:53 -07:00
Mark D. Roth 98c40ce772
client_lb_end2end_test: increase timeouts in RoundRobinTest.UpdateInError (#29805) 2022-05-27 08:11:23 -07:00
Mark D. Roth e39e943529
Revert "server: add method to expose authority seen by server (#29768)" (#29806)
This reverts commit 449adce98b.
2022-05-26 16:15:14 -07:00
Mark D. Roth 449adce98b
server: add method to expose authority seen by server (#29768)
* server: add method to expose authority seen by server

* Automated change: Fix sanity tests

Co-authored-by: markdroth <markdroth@users.noreply.github.com>
2022-05-26 15:03:28 -07:00
Mark D. Roth fb1cc54d41
xds_core_end2end_test: fix federation test timeouts (#29789) 2022-05-26 12:05:57 -07:00
Mark D. Roth 9c23d7999c
subchannel: fix reset backoff when connection attempt is in flight (#29792)
* subchannel: fix reset backoff when connection attempt is in flight

* clang-tidy
2022-05-26 10:53:11 -07:00
Mark D. Roth 3f5931294e
xds_cluster_type_end2end_test: fix flake in AggregateClusterTest.FallBackWithConnectivityChurn test (#29786) 2022-05-25 14:29:09 -07:00
Mark D. Roth 709dff9ca8
security and subchannel: implement UniqueTypeName and use it in these interfaces (#29709)
* implement UniqueTypeName API

* convert security code to use UniqueTypeName

* change subchannel data producer API to use UniqueTypeName

* sanitize

* add missing build dep

* fix credentials_test

* fix certificate_provider_store_test

* fix tls_security_connector_test

* attempt to fix windows build

* avoid unnecessary allocation

* work around MSVC 2017 bug

* sanity

* change factory to not be templated

* fix sanity

* fix bug in chttp2 connector that used server creds instead of channel creds

* add missing build dep

* simplify API
2022-05-23 09:45:20 -07:00
Nicolas Noble 667691c499
server: per-rpc backend metric reporting (#29621)
Users can now report per-rpc metrics from servers to clients.
2022-05-20 15:54:30 -07:00
Mark D. Roth a6d70b449b
xds: don't start resource timer after ADS stream restart if resource is already cached (#29668)
* xds: don't start resource timer after ADS stream restart

* fix sanity

* add missing build dep

* attempt to use std::move() for absl::Status

* sanity
2022-05-20 13:40:43 -07:00
apolcyn 965feb5726
xds: Remove aggregate and logical dns clusters env var guard (#29742)
* Remove aggregate and logical dns clusters env var guard
2022-05-20 12:45:33 -07:00
Mark D. Roth 5c9a971904
xds: use federation env var to guard new-style resource name parsing (#29725) 2022-05-18 15:03:01 -07:00
donnadionne 03cf989610
outlier detection: implement LB policy and xDS configuration (#29343)
* Initial skeleton for outlier detection

* fixing code review comments (modifying child policy)

* Skeleton and all tests passing except for 1

* small code review comments fix

* Adding the parsing of the policy in cds and put it in discovery
mechansim json format

* Parsing outlier detection json policy from parent

* Adding parsing of the updates

* Added Subchannel wrapper and watcher wrapper: and all states pass
through and all tests still pass

* added framework to do eject and uneject

* fixing code review comments

* restore a test

* fixing code review comments

* taking care of code review comments

* removing debug code and rebuild build files

* fixing according to code review comments

* fixing code review comments

* Adding address to subchannel map

* addressing code review comments

* adding call counter

* Refcount SubchannelState (in the map) and store them in Subcahnnel Wrapper

* fixing counterss

* Call counter and tracker skleton added

* Call counter

* addressing code review comments

* addressing code review comments

* Added CallCounter and timer

* fixing sanity; but more importantly: taking out timer temporarly as it
was causing test failures.

* sanity

* fixing according to code review comments

* addressing code review comments

* all algorithms implemented

* addressing code review comment about starting the timer

* protect private vars

* small fix

* Added one more corner case

* fixing EjectionTimer

* Fixing according to code review suggestions.

* fixing according to code reveiw comments

* taking care of code review comments

* fixing sanity issues

* Adding proto to tests

* First test

* Fixing according to code review comments

* Tests all working now

* fixing a crash

* fixing build files

* fixing sanity

* sanity

* Simplifying tests

* merge and update

* format

* sanity and format

* Fixing asan error

* fixing parsing logic and error handling

* 6 more tests done

* Added verifying unejection to tests

* Added all the tests

* fixing according to code review comments

* fixing asan and ubsan

* Fixing tests according to code review comments

* Added both algorithm tests

* added percentage enforcement tests

* fixing tsan error

* keeping debug, but fix warning

* remove debugs

* fixing IWYU and build errors after

* test comments change only but very important

* fixing code review comments

* one more refactorying of util function

* Removed debugs and added one more helper method

* one more logic fix

* Fixing last bit of code review comments and added disable tests

* fixing code review comments

* fixing IWYU

* sanity format

* protecting the feature with environment var:
registering policy and generating policy

* added a todo according to code review comments

* fixing a clang finding at import time

* build fix after synching to latest
2022-05-16 21:37:34 -07:00
Mark D. Roth 5fe979d756
xds: add tests for unknown authority (#29681)
* xds: add tests for unknown authority

* improve trace log message

* fix memory leak
2022-05-13 14:30:12 -07:00
Anirudh Ramachandra 440ae5684c
Revert "Fix grpc_sockaddr_to_uri to return URI encoded string. Also see #29323 which has a similar fix for unix abstract addresses." 2022-05-11 09:57:13 -07:00
Anirudh Ramachandra a31224fc10
Fix grpc_sockaddr_to_uri to return URI encoded string. Also see #29323 which has a similar fix for unix abstract addresses. (#29478) 2022-05-10 13:40:39 -07:00
Craig Tiller f56282a6b6
[iwyu] surface (#29588)
* [iwyu] chttp2

* fix

* Automated change: Fix sanity tests

* Automated change: Fix sanity tests

* Automated change: Fix sanity tests

* fwd file

* fix

* fix

* Automated change: Fix sanity tests

* fix

* [iwyu] surface

* [iwyu] surface

* add no-return hint

* fix

* Automated change: Fix sanity tests

* Add flag

* sweep1

* fixes

* speedup

* fixes

* fixes

* fixes

* fix

* fix

* fix

* fix

* Automated change: Fix sanity tests

* fix

* fic

* x

* ugh

* fix

* fix

* Automated change: Fix sanity tests

* fix?

* fixes

* fix

* Automated change: Fix sanity tests

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2022-05-09 15:56:44 -07:00
Craig Tiller 5918f98ecb
[iwyu] Enable nosafe_headers (#29591)
* Add flag

* sweep1

* fixes

* speedup

* fixes

* fix

* fix

* fix

* Automated change: Fix sanity tests

* fix

* fix

* fix

* fix

* fix

* fix

* fix

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2022-05-06 11:31:11 -07:00
Nicolas Noble 050e4c3abe
client_lb_end2end_test: add OrcaLoadReport into EchoRequest proto (#29505)
* Embed OrcaLoadReport into end2end test.

* Regenerate project files.

* Automated change: Fix sanity tests

* Address comments.

* Maybe fixing iOS test?

* Fixing grpc_tool_test.

* Derp.

* Automated change: Fix sanity tests

* Fixing internal import.

* Fixing Google3 build.

* iOS fix attempt: take 2.
2022-05-03 09:34:57 -07:00
Mark D. Roth 1cd6e69347
subchannel: report IDLE upon existing connection failure and after backoff interval (#29428)
* subchannel: report IDLE upon existing connection failure and after backoff interval

* rename AttemptToConnect() to RequestConnection()

* clang-format

* fix unused parameter warning

* fix subchannel to handle either TF or SHUTDOWN from transport

* fix handling of ConnectedSubchannel failure

* pass status up in IDLE state to communicate keepalive info

* update comment

* split pick_first and round_robin tests into their own test suites

* improve log message

* add test

* clang-format

* appease clang-tidy

* fix test to do a poor man's graceful shutdown to avoid spurious RPC failures

* simplify round_robin logic and fix test flakes

* fix grpclb bug
2022-04-26 15:25:47 -07:00
Mark D. Roth 0ba3c59672
xds_ring_hash_end2end_test: fix flake in ContinuesConnectingWithoutPicks (#29461) 2022-04-21 07:11:03 -07:00
Mark D. Roth be53d2ce3f
LB policy API: add OOB backend metric API (#29012)
* WIP: add OOB backend metric API for LB policies

* fix some includes

* minor fixes

* picking this up again...

* more WIP

* health checking: cancel stream if response message fails to parse

* basic structure in place, but still have synchronization issues to address

* ORCA: implement ORCA RPC service for OOB backend metric reporting

* fix unused parameter error

* gen_upb_api

* add missing build deps

* increase test timing fudge factor

* add missing copyright header

* fix build and locking problems

* clang-format

* document API

* buildifier

* add test, but doesn't build yet

* new test working, but broke existing test, and need to fix server API

* don't register as a generic service

* update test for new orca service registration API

* fix build

* sanitize

* report interval defaults to min interval

* add channel trace event on UNIMPLEMENTED

* don't regenerate the response proto unless something changed

* add missing build dep

* fix comment
2022-04-20 14:14:30 -07:00
Mark D. Roth 6bda57194a
xds_end2end_test_lib: use DEBUG_LOCATION to log where failures originate (#29444) 2022-04-19 15:36:15 -07:00
Mark D. Roth 8ba275291a
connection delay injector: fix tsan problems (#29445) 2022-04-19 15:35:47 -07:00
Yash Tibrewal 8e0da602a8
HTTP: Rename PUT allowing channel arg (#29438) 2022-04-19 01:51:57 -07:00
Alisha Nanda 6753812278
Fix ubsan failure (#29437) 2022-04-18 19:57:02 -07:00
Yash Tibrewal eb96f90eb7
HTTP: Use channel arg instead of global to allow PUT requests (#29411)
* HTTP: Use channel arg instead of global to allow PUT requests

* Reviewer comments

* Reviewer comments

* clang-format

* Repo manager - AJ

* Reviewer comments

* unused parameter
2022-04-18 16:34:55 -07:00
Mark D. Roth 9a363dbf96
xds_end2end_test: Move CDS and EDS tests to their own file (#29430)
* xds_end2end_test: Move CDS and EDS tests to their own file

* fix build
2022-04-18 15:11:32 -07:00
Mark D. Roth 93a90e3c55
xds_ring_hash_end2end_test: increase RPC timeout to avoid failures on ARM (#29431) 2022-04-18 11:45:50 -07:00
Mark D. Roth 50df29bdec
ring_hash: trigger internal connection attemps on only one subchannel at a time (#29422)
* ring_hash: trigger internal connection attemps on only one subchannel at a time

* simplify logic

* add test

* clang-format
2022-04-18 09:36:27 -07:00
Yash Tibrewal f28695351e
Prevent direct usage of absl::Mutex (#29424)
* Add script to prevent absl::Mutex occurrences

* sanity and fixes

* shellcheck
2022-04-17 23:56:18 -07:00
Mark D. Roth e7ddd7b436
xds_end2end_test: Move aggregate and logical_dns cluster tests to their own file (#29298)
* move some code around

* remove num_backends parameter from XdsEnd2endTest

* remove use_xds_enabled_server param from XdsEnd2endTest

* remove xds_resource_does_not_exist_timeout_ms param from XdsEnd2endTest

* remove client_load_reporting_interval_seconds param from XdsEnd2endTest

* start moving CreateAndStartBackends() into individual tests

* finish moving CreateAndStartBackends() into individual tests

* remove unused variable

* remove SetEdsResourceWithDelay

* fix test flake

* clang-tidy

* clang-format

* move test framework to its own library

* fix build

* clang-format

* fix windows build

* rename TestType to XdsTestType

* move BackendServiceImpl inside of BackendServerThread

* clang-format

* move AdminServerThread to CSDS test suite

* remove unnecessary deps

* move aggregate and logical_dns cluster tests to their own file

* split aggregate and logical_dns tests into separate suites

* clang-format

* re-add flaky tag

* clang-tidy and remove unnecessary dep
2022-04-16 18:08:53 -07:00
Mark D. Roth bea5911569
xds_end2end_test: Move ring_hash tests to their own file (#29287)
* move some code around

* remove num_backends parameter from XdsEnd2endTest

* remove use_xds_enabled_server param from XdsEnd2endTest

* remove xds_resource_does_not_exist_timeout_ms param from XdsEnd2endTest

* remove client_load_reporting_interval_seconds param from XdsEnd2endTest

* start moving CreateAndStartBackends() into individual tests

* finish moving CreateAndStartBackends() into individual tests

* remove unused variable

* remove SetEdsResourceWithDelay

* fix test flake

* clang-tidy

* clang-format

* move test framework to its own library

* fix build

* clang-format

* fix windows build

* rename TestType to XdsTestType

* move BackendServiceImpl inside of BackendServerThread

* clang-format

* move AdminServerThread to CSDS test suite

* move ring_hash tests to their own file

* generate_projects

* remove unnecessary deps

* re-add flaky tag

* clang-format
2022-04-16 14:42:43 -07:00
Mark D. Roth b9d904da58
xds_end2end_test: move routing tests to their own file (#29409)
* xds_end2end_test: move routing tests to their own file

* clang-format and clang-tidy

* re-add flaky label
2022-04-16 08:26:32 -07:00
Yash Tibrewal 136055b043
xds_end2end_test: Fix race in security tests (#29408)
* xds_end2end_test: Fix race in security tests

* Reviewer comments
2022-04-15 15:49:48 -07:00
Lidi Zheng e9ad1d5f47
Improve the MaxFault test case (#29407) 2022-04-15 13:51:31 -07:00
Mark D. Roth dd14f803c3
xds_end2end_test: move core XdsClient tests to their own file (#29403)
* xds_end2end_test: move core XdsClient tests to their own file

* clang-format

* also move SecureNaming and XdsStreamErrorPropagation tests
2022-04-15 12:33:50 -07:00
Mark D. Roth 55e870dae6
orca_service_end2end_test: increase flake timing tolerance (#29405) 2022-04-14 17:12:26 -07:00
Mark D. Roth 6273832210
priority and ring_hash LBs: fix interactions when using ring_hash under priority (#29332)
* refactor connection delay injection from client_lb_end2end_test

* fix build

* fix build on older compilers

* clang-format

* buildifier

* a bit of code cleanup

* start failover time whenever the child reports CONNECTING, and don't cancel when deactivating

* clang-format

* rewrite test

* simplify logic in priority policy

* clang-format

* switch to using a bit to indicate child healthiness

* fix reversed comment

* more changes in priority and ring_hash.

priority:
- go back to starting failover timer upon CONNECTING, but only if seen
  READY or IDLE more recently than TRANSIENT_FAILURE

ring_hash:
- don't flap back and forth between IDLE and CONNECTING; once we go
  CONNECTING, we stay there until either TF or READY
- after the first subchannel goes TF, we proactively start another
  subchannel connecting, just like we do after a second subchannel
  reports TF, to ensure that we don't stay in CONNECTING indefinitely if
  we aren't getting any new picks
- always return ring hash's picker, regardless of connectivity state
- update the subchannel connectivity state seen by the picker upon
  subchannel list creation
- start proactive subchannel connection attempt upon subchannel list
  creation if needed

* ring_hash: fix connectivity state seen by aggregation and picker

* fix obiwan error

* swap the order of ring_hash aggregation rules 3 and 4

* restore original test

* refactor connection injector QueuedAttempt code

* add test showing that ring_hash will continue connecting without picks

* clang-format

* don't actually need seen_failure_since_ready_ anymore

* fix TSAN problem

* address code review comments
2022-04-14 17:08:38 -07:00
Mark D. Roth 214e3f3622
xds_end2end_test: Move RLS tests to their own file (#29290)
* move some code around

* remove num_backends parameter from XdsEnd2endTest

* remove use_xds_enabled_server param from XdsEnd2endTest

* remove xds_resource_does_not_exist_timeout_ms param from XdsEnd2endTest

* remove client_load_reporting_interval_seconds param from XdsEnd2endTest

* start moving CreateAndStartBackends() into individual tests

* finish moving CreateAndStartBackends() into individual tests

* remove unused variable

* remove SetEdsResourceWithDelay

* fix test flake

* clang-tidy

* clang-format

* move test framework to its own library

* fix build

* clang-format

* fix windows build

* rename TestType to XdsTestType

* move BackendServiceImpl inside of BackendServerThread

* clang-format

* move AdminServerThread to CSDS test suite

* move RLS tests to their own file

* remove unnecessary deps

* generate_projects
2022-04-14 11:22:39 -07:00
Nicolas Noble e9cf2894da
Fixes a flake with the LoadReporter end2end test. (#29383)
* Fixes a flake with the LoadReporter end2end test.

I *believe* the test is wrong, based on the .proto description of the
LoadReporter.

The protocol described in src/proto/grpc/lb/v1/load_reporter.proto has
the ReportLoad rpc returns a stream of LoadReportResponse, which itself
has a repeated field of Load messages. The comment before it states:

"It is not strictly necessary to aggregate all entries into one entry
per <tag, user_id> tuple, although it is preferred to do so."

Debugging the issue shows we are in fact properly getting all 3 expected
load report types, just in two separate messages instead of a single
one.

This new test codepath will coalesce the load report responses, and also
addresses the fact the original test wasn't verifying that we were
getting the 3 expected types.

* Automated change: Fix sanity tests

* Renaming variables.

* ASSERT_ -> EXPECT_

* Automated change: Fix sanity tests
2022-04-14 09:41:52 -07:00
Mark D. Roth 55b0405c86
xds_end2end_test: Move fault injection tests into their own file (#29283)
* move some code around

* remove num_backends parameter from XdsEnd2endTest

* remove use_xds_enabled_server param from XdsEnd2endTest

* remove xds_resource_does_not_exist_timeout_ms param from XdsEnd2endTest

* remove client_load_reporting_interval_seconds param from XdsEnd2endTest

* start moving CreateAndStartBackends() into individual tests

* finish moving CreateAndStartBackends() into individual tests

* remove unused variable

* remove SetEdsResourceWithDelay

* fix test flake

* clang-tidy

* clang-format

* move test framework to its own library

* fix build

* clang-format

* fix windows build

* move fault injection tests to their own file

* rename TestType to XdsTestType

* move BackendServiceImpl inside of BackendServerThread

* clang-format

* generate_projects

* appease clang-tidy

* move AdminServerThread to CSDS test suite

* remove unnecessary deps

* generate_projects

* don't mark test as flaky
2022-04-14 08:32:33 -07:00
Yash Tibrewal 5e989cf78d
HTTP: Conditionally allow PUT requests (#29397)
* Maybe fix for PUT deprecation

* Guard PUT request accepting with a flag and add tests

* Reviewer comments

* Add fallthrough notation

* Reviewer comments

Co-authored-by: Craig Tiller <ctiller@google.com>
2022-04-13 17:47:07 -07:00
Nicolas Noble 5a3cd992b0
Fixing the build breakage from #29285 (#29399) 2022-04-13 16:58:08 -07:00
Mark D. Roth 61987ec3a2
xds_end2end_test: Move CSDS tests to their own file (#29285)
* move some code around

* remove num_backends parameter from XdsEnd2endTest

* remove use_xds_enabled_server param from XdsEnd2endTest

* remove xds_resource_does_not_exist_timeout_ms param from XdsEnd2endTest

* remove client_load_reporting_interval_seconds param from XdsEnd2endTest

* start moving CreateAndStartBackends() into individual tests

* finish moving CreateAndStartBackends() into individual tests

* remove unused variable

* remove SetEdsResourceWithDelay

* fix test flake

* clang-tidy

* clang-format

* move test framework to its own library

* fix build

* clang-format

* fix windows build

* rename TestType to XdsTestType

* move BackendServiceImpl inside of BackendServerThread

* clang-format

* move CSDS tests to their own file

* move AdminServerThread to CSDS test suite

* remove unnecessary deps

* generate_projects

* fix build

* remove flaky label
2022-04-13 13:12:39 -07:00
Mark D. Roth 9a12b0def8
cds LB: fix diamond dependency handling (#29384)
* cds LB: fix diamond dependency handling

* instead of explicitly checking for loops, just check for max depth
2022-04-13 13:01:57 -07:00
Mark D. Roth d61433ecb2
cds LB: handle loops in aggregate cluster graph (#29382)
* cds LB: handle loops in aggregate cluster graph

* clang-format
2022-04-12 14:45:35 -07:00
Mark D. Roth 3a4058c07a
xds_end2end_test: move test framework to its own library (#29279)
* move some code around

* remove num_backends parameter from XdsEnd2endTest

* remove use_xds_enabled_server param from XdsEnd2endTest

* remove xds_resource_does_not_exist_timeout_ms param from XdsEnd2endTest

* remove client_load_reporting_interval_seconds param from XdsEnd2endTest

* start moving CreateAndStartBackends() into individual tests

* finish moving CreateAndStartBackends() into individual tests

* remove unused variable

* remove SetEdsResourceWithDelay

* fix test flake

* clang-tidy

* clang-format

* move test framework to its own library

* fix build

* clang-format

* fix windows build

* rename TestType to XdsTestType

* move BackendServiceImpl inside of BackendServerThread

* clang-format

* move AdminServerThread to CSDS test suite

* remove unnecessary deps
2022-04-12 13:42:58 -07:00
Mark D. Roth 94c538cd55
Second attempt: Adding is_optional case to RLS (#29378)
* Revert "Revert "Adding is_optional case to RLS (#29259)" (#29299)"

This reverts commit a6419dde06.

* change test to check the full error message
2022-04-12 11:21:44 -07:00
Mark D. Roth 60c56f7d01
Second attempt: implement ORCA RPC service for OOB backend metric reporting (#29352)
* Revert "Revert "ORCA: implement ORCA RPC service for OOB backend metric reporting (#29215)" (#29351)"

This reverts commit 71b355624f.

* move ORCA service to its own BUILD rule
2022-04-08 11:12:18 -07:00
Mark D. Roth 71b355624f
Revert "ORCA: implement ORCA RPC service for OOB backend metric reporting (#29215)" (#29351)
This reverts commit 6d6380de58.
2022-04-07 16:15:49 -07:00
Mark D. Roth 0b79940496
connection delay injector: cap delay by deadline (#29349) 2022-04-07 15:12:59 -07:00
Mark D. Roth 6d6380de58
ORCA: implement ORCA RPC service for OOB backend metric reporting (#29215)
* ORCA: implement ORCA RPC service for OOB backend metric reporting

* fix unused parameter error

* gen_upb_api

* add missing build deps

* increase test timing fudge factor

* add missing copyright header

* buildifier

* don't register as a generic service

* report interval defaults to min interval

* don't regenerate the response proto unless something changed

* use INTERNAL for proto parsing failure

* use absl::Duration in public API
2022-04-07 07:58:58 -07:00
Mark D. Roth 1fd3850502
client_lb_end2end_test: refactor connection delay injection into its own library (#29320)
* refactor connection delay injection from client_lb_end2end_test

* fix build

* fix build on older compilers

* clang-format

* buildifier
2022-04-06 15:23:41 -07:00
Mark D. Roth 96c19e8c98
weighted_target and xds_cluster_manager: don't update picker while update is in flight (#29313)
* weighted_target and xds_cluster_manager: don't update picker while update is in flight

* include deactivation and new child creation

* add test
2022-04-06 15:00:07 -07:00
Mark D. Roth e145c068f2
xds_cluster_resolver LB: don't reuse child numbers that might still be deactivated (#29316)
* xds_cluster_resolver LB: don't reuse child numbers that might still be deactivated

* clang-format
2022-04-06 06:43:39 -07:00
Mark D. Roth caacc7e3aa
xds_end2end_test: fix env var name in RLS tests (#29291)
* xds_end2end_test: fix env var name in RLS tests

* add ScopedExperimentalEnvVar to ensure env vars get unset after tests

* clang-format
2022-04-04 14:27:17 -07:00
Mark D. Roth a6419dde06
Revert "Adding is_optional case to RLS (#29259)" (#29299)
This reverts commit 2fd632a4c1.
2022-04-02 20:27:01 -07:00
Mark D. Roth 217ab2a793
xds_end2end_test: make each individual test start the number of backends it needs (#29271)
* move some code around

* remove num_backends parameter from XdsEnd2endTest

* remove use_xds_enabled_server param from XdsEnd2endTest

* remove xds_resource_does_not_exist_timeout_ms param from XdsEnd2endTest

* remove client_load_reporting_interval_seconds param from XdsEnd2endTest

* start moving CreateAndStartBackends() into individual tests

* finish moving CreateAndStartBackends() into individual tests

* remove unused variable

* fix test flake

* clang-tidy

* clang-format
2022-04-02 09:26:57 -07:00
donnadionne 2fd632a4c1
Adding is_optional case to RLS (#29259)
* Adding is_optional case to RLS

* integrated with the updated envoy data-plane

* Fixing an old bug and adding test

* Use the same plugin map for ignore

* Remove ignore set

* Fixed another test.

* addressing code review comments.

* clean up!
2022-04-01 12:33:39 -07:00
Mark D. Roth 775362a2ce
client_lb_end2end_test: fix internal build problem (#29265) 2022-03-30 18:33:29 -07:00
AJ Heller 18a8f6aad9
Refactor end2end tests to exercise each EventEngine (#29202)
* Refactor end2end tests to exercise each EventEngine

* fix incorrect bazel_only exclusions

* Automated change: Fix sanity tests

* microbenchmark fix

* sanitize, fix iOS flub

* Automated change: Fix sanity tests

* iOS fix

* reviewer feedback

* first pass at excluding EventEngine test expansion

Also caught a few cases where we should not test pollers, but should
test all engines. And two cases where we likely shouldn't be testing
either product.

* end2end fuzzers to be fuzzed differently via EventEngine.

* sanitize

* reviewer feedback

* remove misleading comment

* reviewer feedback: comments

* EE test_init needs to play with our build system

* fix golden file test runner

Co-authored-by: drfloob <drfloob@users.noreply.github.com>
2022-03-30 15:43:05 -07:00
Mark D. Roth 3c4441fcdc
xds: add node ID to status messages generated by XdsClient (#29253)
* eliminate unnecessary templates

* add xDS node ID to error messages generated by XdsClient

* change test to verify that node ID is present in status message

* clang-format
2022-03-30 09:29:00 -07:00
Mark D. Roth ac8f0b690e
xds: NACK EDS resources with duplicate localities in the same priority (#29231) 2022-03-29 11:56:10 -07:00
Craig Tiller 44167bdfd3
Convert server load reporting to promises (#28927)
* Call finalization for promises

* Convert filter to a promise

* Call finalization for promises

* comment

* split out and test

* dont use promise_detail:: directly

* fix

* Automated change: Fix sanity tests

* fix

* ?

* fix

* finish it

* modernize

* fix-test

* fix

* Review feedback

* Automated change: Fix sanity tests

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2022-03-29 08:11:36 -07:00
Mark D. Roth 23c4541392
xds_cluster_resolver: fix bug that caused us to incorrectly reuse priority child names (#29174)
* xds_cluster_resolver: fix bug that caused us to incorrectly reuse priority child names

* add debug logging

* assign child policy names within each discovery mechanism, not across all mechanisms

* clang-format

* add tests

* don't need EDS service name or LOGICAL_DNS hostname in child name

* remove unnecessary code
2022-03-24 14:27:47 -07:00
Mark D. Roth eded3b6bc3
client channel: record call completion even if recv_trailing_metadata was not started (#29198)
* client channel: record call completion even if recv_trailing_metadata was not started

* add test

* add test for opencensus filter

* remove unnecessary code
2022-03-24 14:22:16 -07:00
donnadionne 6989129db5
xDS RLS implementation (#29200)
* Revert "Revert "Initial structure for RLS (#28750)" (#29189)"

This reverts commit 7ece810843.

* fixing 1 unused using
2022-03-23 21:23:34 -07:00
donnadionne 7ece810843
Revert "Initial structure for RLS (#28750)" (#29189)
This reverts commit 362e0b8706.
2022-03-22 13:41:21 -07:00
donnadionne 362e0b8706
Initial structure for RLS (#28750)
* Initial structure for RLS

* Adding and building the proto to parse the Any proto for the plugins

* re-org

* Parsing the plugin

* Parsing more into json

* Parsed proto to json

* small cleanup

* Adding prefix

* Added new rls_experimental policy

* build files

* Fixing according to code review comments

* code review comments

* Adding sym changes

* adding action name check

* fixing code review comments.

* fixing unused var error

* clean up

* fixing code review comments

* fixing code review comments

* fixing according to code review comments.

* Remove unnecessary include

* small fix

* generate more, hard-code less

* Moving to using absl::variant

* absl::string_view and absl::variant of vector of std::string are not
playing nice together.

* fixed variant

* Using absl::variant now

* Checkint used plugins

* Refactor Parsing code and separating out Parsing of the plugin

* Fixing code review comments

* code review comments

* fixing code review comments.

* Addressing code review comments

* First end-to-end test

* generated build files

* commit generated files via tools/codegen/core/gen_upb_api.sh

* Fixing rls policy parsing tests

* Restore checks for the test server

* Refactor rls_server

* added keys to rls request

* fixing small logic error

* Complete the test using all the keys

* Separating out RLS test and rls_server thread

* sanity errors

* generated build files

* Complete the rest of the tests and sanity cleanup

* fixing code review comments: using upb_JsonEncode now!

* fixing code review comments

* fixing code review comments

* Fixing code review comments

* misisng fix

* simplifying tests

* simplify tests 2

* Linking in the correct proto for rls_config

* restore metadata check

* Add disable test

* Fixing RLS test and removing environment var that is no longer necessary

* Fixing "Wrong type" type of tests after json parsing change to accept
STRING for number

* adding json_encode.h/c to src/upb/gen_build_yaml.py and generate
necessary files.

* Fixing un-used var error

* fixing sanity errors

* Fixing the upb encoding buffer

* Fixing code review comments.

* Adding nack test for unkonwn plugin proto

* Last bit of code review comments

* fixing unused variable
2022-03-22 11:29:52 -07:00
Mark D. Roth 7b4a5fcba5
xds: accept resources wrapped in a Resource message (#29090)
* xds: accept resources wrapped in a Resource message

* fix v2 proto name
2022-03-18 14:16:22 -07:00
AJ Heller 1f03bbab39
Small cleanup (#29125)
The std::moves of temporaries prevented copy elision, and the functions
were unused (one with a TODO to re-enable later)
2022-03-16 18:26:41 -07:00
Yash Tibrewal 01f333a1c1
TlsCredentials: Comparator implementation (#28940)
* TlsCredentials: Comparator implementation

* Update tests

* Redundant namespace qualifiers

* clang-tidy

* Reviewer comments

* Reviewer comments

* Reviewer comments

* Reviewer comments

* Reviewer comments

* Add generator for grpc_tls_credentials_options.h

* Generator script modifications for tests

* Sanity

* Reviewer comments

* Reviewer comments

* Add a common place to look-up known types

* Formatting

* Script fix

* Reviewer comments

* Add test mode to script

* Add to sanitize.sh

* clang-tidy

* Fix

* Fix

* Reviewer comments

* Clang-format
2022-03-16 17:11:07 -07:00
Yash Tibrewal 674e938de5
More tests prep for graceful goaway (#29076)
* More tests prep for graceful goaway

* clang-tidy
2022-03-10 16:55:09 -08:00
Yash Tibrewal c9946c2183
Channelz Service Test Shutdown (#29068) 2022-03-09 17:04:39 -08:00
Yash Tibrewal 68d721058c
Prepare async_end2end_test for graceful goaway (#29054) 2022-03-09 16:06:05 -08:00
apolcyn 00d587dc56
Include ADS stream error in XDS error updates (#29014)
* include ADS stream error in XDS error updates

* Re-package XDS client errors into unavailable rather than unknown, in XDS resolver
2022-03-07 10:32:36 -08:00
Craig Tiller fe913387da
Remove idempotent/cacheable requests (#28922)
* Remove idempotent/cacheable requests

* more cleanup

* bump core version

* fix

* fix

* fix

* review feedback

* fixes

* fix

* remove more

* objc

* fix

* fix

* fix

* scrub

* Modify XdsRbacTests

Co-authored-by: Yash Tibrewal <yashkt@google.com>
2022-03-04 14:42:59 -08:00
Craig Tiller 87acbadba1
Third attempt: Convert client_auth_filter to promises (#28968)
* wip

* Automated change: Fix sanity tests

* fixes

* progress

* progress

* grpc compiles

* Automated change: Fix sanity tests

* fixing tests

* x

* progress

* better code

* Automated change: Fix sanity tests

* progress

* progress

* windows fix

* Make Duration metadata trivial

* better message

* fix

* Automated change: Fix sanity tests

* fix

* fix

* fix

* fix

* Automated change: Fix sanity tests

* Automated change: Fix sanity tests

* fix

* progress

* fixes

* fix

* fix

* spam

* un-disable errantly disabled tests

* gain insight

* Automated change: Fix sanity tests

* fixes

* fixes

* fix

* debug

* tweak

* fix

* fix timeout

* fix comment

* fixes

* x

* better test

* tests

* Automated change: Fix sanity tests

* missed file

* fix

* x

* fix

* fix

* fix

* fix

* Automated change: Fix sanity tests

* fix

* merge

* Automated change: Fix sanity tests

* Revert "Revert "Revert "Revert "Convert client_auth_filter to promises (#28767)" (#28951)" (#28952)" (#28967)"

This reverts commit 0f73576b17.

* fix potential memory leak

* Fix behavior if >1 pending request

* fix

* fix nullptr access

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2022-03-01 12:45:19 -08:00
Craig Tiller 5fc3ff8203
grpc_millis -> Timestamp/Duration (#28119)
* wip

* Automated change: Fix sanity tests

* fixes

* progress

* progress

* grpc compiles

* Automated change: Fix sanity tests

* fixing tests

* x

* progress

* better code

* Automated change: Fix sanity tests

* progress

* progress

* windows fix

* Make Duration metadata trivial

* better message

* fix

* Automated change: Fix sanity tests

* fix

* fix

* fix

* fix

* Automated change: Fix sanity tests

* Automated change: Fix sanity tests

* fix

* progress

* fixes

* fix

* fix

* spam

* un-disable errantly disabled tests

* gain insight

* Automated change: Fix sanity tests

* fixes

* fixes

* fix

* debug

* tweak

* fix

* fix timeout

* fix comment

* fixes

* x

* better test

* tests

* Automated change: Fix sanity tests

* missed file

* fix

* x

* fix

* fix

* fix

* fix

* Automated change: Fix sanity tests

* fix

* merge

* Automated change: Fix sanity tests

* fix

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2022-03-01 11:34:05 -08:00
Esun Kim c08eb4fc41
Attempt to upgrade upb to the latest (#28685)
* Update upb to 85e5c76cb7098d3ae35e575d652a40568013a149

* Update grpc_deps.bzl

* Run gen_upb_api.sh

* Run run_sed.sh

* Update gen_build_yaml.py

* Regen projects

* Reformat

* Update third_party/upb to 950cdab42fe4c83f06a2be46eeb46a068e486eed

* Update third_party/upb to c9c31241bc39922b092a81d192badd50cccaaa81

* Update src/upb/gen_build_yaml.py

* Generated projects

* Update third_party/upb to a02d92e0257a35f11d4a58b6a932506cbdbb2f29

* Patching in https://github.com/protocolbuffers/upb/pull/524

* Sed script went too far.

Co-authored-by: Nicolas 'Pixel' Noble <nicolas@nobis-crew.org>
2022-02-28 09:26:31 -08:00
Craig Tiller 0f73576b17
Revert "Revert "Revert "Convert client_auth_filter to promises (#28767)" (#28951)" (#28952)" (#28967)
This reverts commit 235098de22.
2022-02-24 10:07:13 -08:00
Craig Tiller 16a3ce51ff
Service config parser to core configuration (#28883)
* Service config parser to core configuration

* x

* Automated change: Fix sanity tests

* finish

* Automated change: Fix sanity tests

* oops

* fix race

* Automated change: Fix sanity tests

* back out mutex

* refactor

* optimize

* Automated change: Fix sanity tests

* fix

* fix

* split out interface

* review feedback

* x

* fixes

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2022-02-23 20:50:03 -08:00
Craig Tiller 235098de22
Revert "Revert "Convert client_auth_filter to promises (#28767)" (#28951)" (#28952)
* Revert "Revert "Convert client_auth_filter to promises (#28767)" (#28951)"

This reverts commit 5bfe38f10f.

* fix
2022-02-23 15:00:34 -08:00
Craig Tiller 5bfe38f10f
Revert "Convert client_auth_filter to promises (#28767)" (#28951)
This reverts commit cadce47ebf.
2022-02-23 08:17:37 -08:00
Craig Tiller cadce47ebf
Convert client_auth_filter to promises (#28767)
* Convert client_auth_filter to promises

* outline

* x

* x

* fixes

* remove-spam

* Async sequences over iterators

* iterate

* better semantics

* fix

* clang-tidy

* Automated change: Fix sanity tests

* Automated change: Fix sanity tests

* fixes

* fixes

* Automated change: Fix sanity tests

* progress

* progress

* progress

* progress

* progress

* credentials_test passes

* Automated change: Fix sanity tests

* Make Activity more of an interface

Move mutex and wakeup logic into a new class between PromiseActivity<>
and Activity (so that the sharing can persist), but make Activity closer
to a pure interface, so that whilst we migrate code we can implement
better fakes without forcing allocation.

* fixes

* Automated change: Fix sanity tests

* fixes

* test the awful failure

* x

* fix typo

* fix race

* 4.9

* x

* review feedback

* review feedback

* Automated change: Fix sanity tests

* review feedback

* replace AuthMetadataContext interface with GetRequestMetadataArgs struct

* Automated change: Fix sanity tests

* fix merge

* Fixup

* fix

* fix

* fix

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
Co-authored-by: Mark D. Roth <roth@google.com>
2022-02-23 07:36:19 -08:00
AJ Heller 85189b24bb
Reintroduce: Avoid fully qualifying namespaces (and add check) (#28917)
Based on a handful of https://abseil.io/tips, it's generally advised to
only fully-qualify namespaces when in a `using` statement, or when it's
otherwise required for compilation. In all other cases, the general
recommendation is to not fully-qualify.

This change fixes most `grpc.*` namespace uses. There are potential
challenges in trying to make blanket changes to non-gRPC namespace uses,
such as `::testing`, since there is also a `grpc::testing` namespace.
2022-02-18 16:18:54 -08:00
AJ Heller e72a5fe5dd
Revert "Avoid fully qualifying namespaces (and add check) (#28901)" (#28916)
This reverts commit fc7314c701.
2022-02-17 17:56:19 -08:00
AJ Heller fc7314c701
Avoid fully qualifying namespaces (and add check) (#28901)
Based on a handful of https://abseil.io/tips, it's generally advised to
only fully-qualify namespaces when in a `using` statement, or when it's
otherwise required for compilation. In all other cases, the general
recommendation is to not fully-qualify.

This change fixes most `grpc.*` namespace uses. There are potential
challenges in trying to make blanket changes to non-gRPC namespace uses,
such as `::testing`, since there is also a `grpc::testing` namespace.
2022-02-17 16:23:25 -08:00
donnadionne 431d314eca
Calculate clock difference and accomodate it to deflake timeout tests (#28868)
* Calculating clock difference and accomendate it to deflake timeout
tests.

* Fixing according to code review comments.
2022-02-17 15:40:55 -08:00
Oscar Shi e0a5c310cd
Increase number of threads in stress test (#28821)
In some configurations 100 threads is not enough to cause exhaustion failing tests.

Increase number of threads to ensure resource exhaustion
2022-02-15 13:58:01 +01:00
AJ Heller 40f2c2a4e1
Upgrade googletest (#27287) 2022-02-14 18:16:05 -08:00
Ashitha Santhosh 1fee3d72be
remove SDK term from gRPC authz (#28843) 2022-02-14 11:44:15 -08:00
donnadionne 1baca37a9e
Applying aggregate ringhash policy (#28861)
* Applying aggregate ringhash policy

* Fixing according to code review comments.

* typo
2022-02-11 16:16:26 -08:00
Ashitha Santhosh 55e4af1b14
Revert "Revert "Update to rbac policy struct and end2end authz test. (#27074)" (#28552)" (#28620)
This reverts commit 8ca42ec6f8.
2022-02-11 14:03:24 -08:00
yihuaz b458db9246
Eliminate gRPC insecure build (#25586)
* force submit

* fix test error

* remove is_client from local tsi and its callsites

* fix too_many_pings_test

* add missing dep
2022-02-10 11:17:18 -08:00
scwhittle 14169dd0c5
[issue #28771] Fix pick_first policy to clear selected_ when deleting subchannel_list_ (#28824)
* [issue #28771] Fix pick_first policy to clear selected_ when promoting
a pending subchannel list to the active subchannel list when all
subchannels have been attempted and are in an error state.

* address comments

* revert idle_filter
2022-02-10 09:48:57 -08:00
Craig Tiller 9bcabbac32
Disable flaky test (#28776) 2022-02-02 23:12:35 -08:00
donnadionne 55db347396
LRS changes for federation (#28504)
* Passing xds server object instead of just a string name

* Adding xds server to policy

* Refactor ToJson

* Using XdsServer for load reporting

* code review comments

* fixing code review comments

* Taking care of lifetime of the XdsServer key

* code review comments

* Fixing channel_state storage and re-run tests (1 assert hit)

* Checking for server in the bootstrap file

* Adding LRS test

* adding a bootstrap file ToJson and parse test

* fixing code review comments

* fixing code review comments.

* fixing test

* break out the federation lrs test

* Fixed last bit of code review comments

* fixing error message to be more precise
2022-01-21 10:55:58 -08:00
Mark D. Roth 86c483a678
fix priority policy to cancel failover timer on IDLE and add test (#28563) 2022-01-20 13:34:58 -08:00
Craig Tiller 74a2cb6e2b
Revert "Revert "Reland slice changes (#28601)" (#28615)" (#28624)
* Revert "Revert "Reland slice changes (#28601)" (#28615)"

This reverts commit 939bbfc336.

* Change random seed back to nanoseconds
2022-01-20 12:47:33 -08:00
Mark D. Roth 9f090a3a32
xds: accept SelfConfigSource for RDS and EDS ConfigSources (#28618) 2022-01-19 13:57:15 -08:00
Craig Tiller 939bbfc336
Revert "Reland slice changes (#28601)" (#28615)
This reverts commit b33e0d40af.
2022-01-19 09:40:19 -08:00
Vignesh Babu aeea02fab8
TLS Session Keys export for GRPC C++ (#26812)
* Adding TLS Key export logic to core and c++ wrappers

* Adding and end2end cpp tls key export test and updating broken test due to interface changes

* regenerate projects

* updating tls key export core logic with addition of APIs to grpc_security.h

* undoing changes to tls_security_connector_test

* regenerate projects

* changing the logging format enum name as per GRFC comments

* regenerate projects

* removing some commented code

* updating changes as per review comments

* adding GRPCAPI annotations to functions defined in grpc_security.h

* regenerate projects

* fixed some code styling issues

* removing grpc_security.h include from tls_credentials_options.h

* updating files as per review comments

* minor fixes

* moving some code around

* removing key log format from tls session key log config and converting it to a simple string

* regenerate projects

* fixing mistakes in recent merge with master

* regenerate projects

* regenerate projects

* fixing some distrib and snity errors

* fixing formatting errors

* fixing more sanity checks and raising supported openssl versions to 1.1.1

* updating min supported openssl version to 1.1.1

* updating min supported openssl version in tls_key_export_test

* updating test to fix incorrect vector initialization

* updating as per latest comments

* fixing sanity checks

* addressing review comments

* fixing sanity checks

* fixed c++ comment style

* Automated change: Fix sanity tests

* fixing review comments

Co-authored-by: Vignesh2208 <Vignesh2208@users.noreply.github.com>
2022-01-18 14:44:55 -08:00
Mark D. Roth 0fb47cd886
pick_first: make TRANSIENT_FAILURE sticky (#28571) 2022-01-18 13:25:07 -08:00
Craig Tiller b33e0d40af
Reland slice changes (#28601)
* Revert "Revert "Eliminate slice interning (#28363)" (#28598)"

This reverts commit 03bf699600.

* fix?
2022-01-18 09:29:49 -08:00
Jan Tattermusch 03bf699600
Revert "Eliminate slice interning (#28363)" (#28598)
This reverts commit 6703186b7a.
2022-01-18 15:44:18 +01:00
Yash Tibrewal 1caa3e8cfd
xDS: Rbac filter updates (#28568) 2022-01-18 04:46:40 -08:00
Craig Tiller 6703186b7a
Eliminate slice interning (#28363)
Eliminate slice interning, and structures in slices to support it.
Reduces grpc_slice_refcount from 40 bytes (+ a required 8 bytes elsewhere) to 16 bytes.
Removes a pointer dereference for every slice ref/unref.

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2022-01-17 20:25:45 -08:00
Mark D. Roth e59dcd5c87
xds: force-enable retries in xds_end2end_test to make things work internally (#28564) 2022-01-14 09:34:39 -08:00
Mark D. Roth 227d65367c
rls: fix various bugs in adaptive throttling code (#28477)
* rls: fix adaptive throttling window size

* clang-format

* fix adaptive throttling logic and fix FailedRlsRequestWithoutDefaultTarget test
2022-01-14 08:21:32 -08:00
Jan Tattermusch 8ca42ec6f8
Revert "Update to rbac policy struct and end2end authz test. (#27074)" (#28552)
This reverts commit b64167a034.
2022-01-14 07:45:38 -08:00
Ashitha Santhosh b64167a034
Update to rbac policy struct and end2end authz test. (#27074)
* Empty principals checks for authenticated connection

* fix sanity check

* clang-format

* principals:[] will result in ANY

* minor correction after merging

* clang-format

* formatting

* clang-format

* remove unnecessary header

* Remove unnecessary target from BUILD
2022-01-12 15:13:35 -08:00
Yash Tibrewal 114d388389
Fix xDS client for multiple watchers (#28521)
* Fix XdsClient for multiple watchers

* Reviewer comment
2022-01-12 11:41:06 -08:00
Mark D. Roth 5c30de312b
xds: handle percent-encoding in new-style resource names (#28515) 2022-01-12 10:09:06 -08:00
Yash Tibrewal 9ffbc2d360
XdsEnd2EndTest : Use a queue to save resource updates (#28467)
* Fix XdsClient for multiple watchers on the same resource

* xds_end2end_test: Don't use XdsCredentials for XdsRbacNackTests

* Use separate response states for EDS and RDS resources

* Reviewer comments

* Reviewer comments

* Reviewer comments

* Remove blank link

* Reviewer comments
2022-01-11 15:02:12 -08:00
Mark D. Roth bccd1c7c22
xds_e2e_test: clean up WaitForAllBackends() and add timeout (#28514)
* xds_e2e_test: clean up WaitForAllBackends() and add timeout

* fix drop tests

* restore resetting of backend counters

* fix BalancerRestart test

* fix StressTest

* clang-format
2022-01-11 08:37:54 -08:00
Ashitha Santhosh d884a7a440
RBAC Authorization Engine updates based on A41 proposal (#27754)
* RBAC Authorization Engine updates based on A41 proposal

* Fix headermatcher

* Support for metadata

* Maps host to :authority, special-case te and :method

* removes method hardcoding

* add factory methods

* minor formatting

* fix test

* formatting

* remove constructors

* fix api usage

* ReqServerName test in xds_end2end_test

* Initialize bool
2022-01-10 19:52:20 -08:00
Craig Tiller 55b365f287
Move resolver code around a little (#27846)
* move service config

* service config should not depend on grpc_base

* move resolver, deal with fallout

* Automated change: Fix sanity tests

* fix

* fix

* Automated change: Fix sanity tests

* review feedback

* review feedback

* fix

* Automated change: Fix sanity tests

* fix

* Automated change: Fix sanity tests

* fix vis

* fix

* fixes

* Automated change: Fix sanity tests

* visibility

* fix

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2022-01-06 10:53:40 -08:00
donnadionne e520fc137c
xDS Federation: bootstrap and xds_resolver changes (#27938)
* xDS Federation: bootstrap and xds_resolver changes

* code review fixes

* fixing code review comments

* fixing code review comments

* fixing code review comments

* code review comments

* fixing code review comments

* First very basic test to make sure parsing and reconstruction work as
expected.

* clean up

* fixing logic error about authority

* fixing resource type parsing

* fixing code review comments

* simplify parsing!

* Parsing method signature update

* fixing code review comments

* clean up

* working progress for the test with generated bootstrap

* reorg the bootstrap file

* fixing tests

* Adding more to test authorities

* Added a test and it passes

* addressing code review comments

* code review comments to make parser cleaner and more efficient

* Merge in authority prefixes

* fixing sanity error and xds boostrap test error

* small fix

* Release all tests that pass; reduce scope for DeadUpdate

* Updated test strcuture and how to pass in the index for balancers to be
used as xds server uri and authority xds server uri

* code review comments

* code review fixes

* code review comment

* Making test structure changes

* fixing code review comments

* fixing code review comments

* Fixing test regression

* fixing bootstrap tests

* cleanup files

* enabling localhost:xxx for xds server; updated server tests and will fix
one more NameExpected test with testsetup.

* Finally removing fake reolsver for xds server

* Fixing bootstrap tests

* Rewrite builder

* Fixing code review comments

* fixing code review comments

* Fixing all tests to use Setup again

* fixing small sanity error

* Found the source of xds server nack test faiure and fix added

* small code review fixes

* Remove fake resolver! YAY!

* Fixing according to code review comments

* Setup plugin in bootfile

* Added more tests.

* Adding server test

* fixing a regression

* regression

* sanity fix

* fixing code review

* fixing code review comments

* Re-combine SecurityNaming tests.

* Add Rds new resource type and new tests

* Added PercentEncode test

* fixing code review comments

* refactor test a bit more

* fixing code review comments

* fixing according to code review comments

* fixing code review comments

* fixing code review comments
2022-01-05 11:13:11 -08:00
Yash Tibrewal f00432c70b
xds_end2end_test: Set skip cancelled check (#28453)
* xds_end2end_test: Set skip cancelled check

* Reviewer comment
2021-12-29 18:15:19 -08:00
Yash Tibrewal ab6d7b4f02
Reland rbac filter (#28442)
* Revert "Revert "xDS: Add support for RBAC HTTP filter (#28309)" (#28441)"

This reverts commit 7aae5c66df.

* Fix internal clang errors

* Ipv6 addresses are 128 bit
2021-12-29 11:42:58 -08:00
Craig Tiller 7aae5c66df
Revert "xDS: Add support for RBAC HTTP filter (#28309)" (#28441)
This reverts commit 6ea8214879.
2021-12-28 20:11:38 +00:00
Yash Tibrewal 6ea8214879
xDS: Add support for RBAC HTTP filter (#28309)
* xDS: ADD RBAC HTTP filter support

* sanity, upb regenerate files

* Revert PerChannelArg changes

* Reviewer comments

* Reviewer comments

* Reviewer comments

* Remove unnecessary header

* Fix sanity

* Add RBAC service config parsing tests

* Don't make a copy of the metadata batch

* Revert expr_proto changes

* Some more tests

* Reviewer comments

* Reviewer comments

* No metadata changes needed

* Fix leak of DynamicXdsServerConfigSelectorProvider

* Fix deadlock issues

* Fix test compilation
2021-12-23 20:46:36 -08:00
Ashitha Santhosh 4fd524cd17
Fix presence match in authorization. (#28269)
* Fix presence match in authorization.

* Remove header

* Add test

* fix regex to include whitespace characters
2021-12-13 14:53:05 -08:00
Mark D. Roth f76f2653c3
Second attempt: XdsClient: remove resource-type-specific methods from XdsClient API (#28305)
* Revert "Revert "XdsClient: remove resource-type-specific methods from XdsClient API (#28231)" (#28301)"

This reverts commit 3e779b68fe.

* remove global resource type registry
2021-12-08 09:24:33 -08:00
Mark D. Roth 3e779b68fe
Revert "XdsClient: remove resource-type-specific methods from XdsClient API (#28231)" (#28301)
This reverts commit 747a5d8c24.
2021-12-07 15:46:10 -08:00
Mark D. Roth 747a5d8c24
XdsClient: remove resource-type-specific methods from XdsClient API (#28231)
* WIP

* introduce XdsResourceType API and change Listener parsing to use it

* converted RouteConfig parsing

* convert cluster and endpoint parsing

* cleanup

* clang-format

* attempt to work around compiler problems

* move XdsResourceType to its own file, and move endpoint code out of XdsApi

* move cluster parsing to its own file

* move route config parsing to its own file

* move listener parsing to its own file

* clang-format

* minor cleanup

* plumbed XdsResourceType throughout XdsClient

* a bit of cleanup

* more cleanup

* construct full resource names before calling XdsApi::CreateAdsRequest()

* remove some unneeded code

* clean up includes and have XdsResourceType initialize the upb symtab

* more cleanup of unnecessary code

* more cleanup

* update comment

* clang-format

* add missing virtual dtor

* fix build

* remove resource-type-specific methods from XdsClient API

* have each resource type register itself upon instantiation

* remove comment

* add missing virtual dtor

* clang-format
2021-12-07 11:00:08 -08:00
Mark D. Roth ac70281e74
improve RPC status messages when name resolution fails (#28091)
* improve RPC status messages when name resolution fails

* fix tests

* clang-format

* fix rls_end2end_test

* fix build

* fix service_config_end2end_test

* fix too_many_pings_test

* fix tests

* fix client_channel to propagate resolution_note to LB policy

* improve comment
2021-12-07 09:05:38 -08:00
Mark D. Roth 81a4ea444e
xds: remove legacy EDS-only workflow (#28274)
* xds: remove legacy EDS-only workflow

* remove unused constants

* remove unused data member
2021-12-06 08:00:15 -08:00
Yash Tibrewal 3689072979
xDS: Add graceful shutdown for old connections on listener resource update (#28154)
* xDS: Add graceful shutdown for old connections on listener resource update

* Add TODOs for review

* Reviewer comments

* Reviewer comments

* Fix merge

* Fix comment

* s/GRPC_ARG_DRAIN_GRACE_TIME_MS/GRPC_ARG_SERVER_CONFIG_CHANGE_DRAIN_GRACE_TIME_MS
2021-12-02 15:42:47 -08:00
Mark D. Roth 9722651156
xds_end2end_test: remove num_balancers from test suite and remove dependence on fake resolver for xDS channel (#28245)
* xds_end2end_test: remove num_balancers from test suite

* fix clang-tidy
2021-12-01 16:28:15 -08:00
Yash Tibrewal 25446c468a
Support RDS updates on the server (#27851)
* Port changes from #27388

* Reviewer comments

* Fix resource timeout issue

* Cleanup

* Fix clang-tidy

* Revert benchmark

* Restructure

* clang-tidy

* Automated change: Fix sanity tests

* Partial commit

* Reviewer comments

* Fixes

* Reviewer comments

* Reviewer comments

* Reviewer comments

* Reviewer comments

* clang-format

* Fix FaultInjection tests

* clang-tidy

Co-authored-by: yashykt <yashykt@users.noreply.github.com>
2021-11-18 16:10:31 -08:00
Esun Kim 11f440775b
Fix xds_end2end_test dyld (#28133) 2021-11-18 16:08:41 -08:00
Mark D. Roth 1050eedb87
enable clang-tidy readability-static-definition-in-anonymous-namespace check (#28033) 2021-11-17 07:48:43 -08:00
Mark D. Roth fff84402ac
Second attempt: XdsClient: fix resource timeout behavior (#28088)
* Revert "Revert "XdsClient: fix resource timeout behavior (#27860)" (#28032)"

This reverts commit 817eed0928.

* use the right status code enum
2021-11-15 14:53:15 -08:00
Craig Tiller a629c9a03e
Reland resource quota work (#28017)
* Check if memory owner available prior to polling it

The transport may drop the memory owner during its destruction sequence

* tcp_fix

* Revert "Revert "New resource quota integration (#27643)" (#28014)"

This reverts commit 0ea2c37263.

* clang-format

* fix-path

* fix
2021-11-15 08:23:51 -08:00
Craig Tiller 817eed0928
Revert "XdsClient: fix resource timeout behavior (#27860)" (#28032)
This reverts commit 7fdb40d8b8.
2021-11-12 19:58:16 -08:00
Mark D. Roth 7fdb40d8b8
XdsClient: fix resource timeout behavior (#27860)
* XdsClient: fix resource timeout behavior

* fix clang-tidy

* more clang-tidy fixes

* yet more clang-tidy
2021-11-12 12:17:45 -08:00
Craig Tiller 0ea2c37263
Revert "New resource quota integration (#27643)" (#28014)
This reverts commit 39f0877d25.
2021-11-11 14:26:19 -08:00
Craig Tiller 39f0877d25
New resource quota integration (#27643)
* new resource quota integration

* Automated change: Fix sanity tests

* fix

* fix

* fixes

* fixes

* fixes

* Automated change: Fix sanity tests

* fixes

* fixes

* Automated change: Fix sanity tests

* fixes

* fix

* fixes

* windows-fix

* fixes

* fixes

* fix

* fix-asan

* banned

* banned

* fixes

* clang-tidy-fix

* Automated change: Fix sanity tests

* fix-cronet

* review feedback

* review feedback

* Automated change: Fix sanity tests

* fixes

* bug fix

* fixes

* compile fix

* exclude megabyte size payloads from 1byte tests

* windows fix

* start moving ios

* keep moving windows

* Get windows compilation working.

* Automated change: Fix sanity tests

* better

* fixes

* remove slice buffer from memory_allocator.h

* Revert "remove slice buffer from memory_allocator.h"

This reverts commit 234a63b6e3.

* ugh

* #fixtests

* pthread tls fixes

* Automated change: Fix sanity tests

* fixfixfix

* xxx

* add reset

* review feedback

* fix

* fix

* fixes

* fix

* mac progress

* cpp-impl-of

* rename ptr

* Automated change: Fix sanity tests

* memory-owner-is-a-memory-allocator

* fixes

* fix

* fix from prod

* fix

* Fix issue leading to bad pointers being returned on Windows.

* Automated change: Fix sanity tests

* fix multislice bug

* argh

* hyrums law fixes

* hyrums law fixes

* clang-format

* hyrums law fixes

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2021-11-10 22:53:04 -08:00
ZhenLian 2e14f6fa70
Support Custom Post-handshake Verification in TlsCredentials (#25631)
* custom verification refactoring - post-handshake verification
2021-11-10 13:45:52 -08:00
Yash Tibrewal 00e8a2d085
xds_end2end_test: Only start backends when needed (#27911)
* xds_end2end_test: Fix flakiness on WaitForLdsNack

* xds_end2end_test: Only start the server when we want

* Revert WaitForNack changes

* Fixes

* Fix CsdsShortAdsTimeoutTest

* Fix sanity
2021-11-04 14:35:25 -07:00
Mark D. Roth 3bde8305cc
Second attempt: Don't build RLS on mobile (#27878)
* Revert "revert changes to not build RLS on mobile (#27875)"

This reverts commit 73ad459c51.

* new approach using multiple select() calls
2021-11-01 10:26:01 -07:00
Mark D. Roth 73ad459c51
revert changes to not build RLS on mobile (#27875)
* Revert "fix mobile builds to properly exclude xDS (#27855)"

This reverts commit 64d026e234.

* Revert "don't build RLS on mobile (#27838)"

This reverts commit 2a69c525b8.
2021-10-29 12:00:00 -07:00
Mark D. Roth 08c7b254bd
provide more slack in RLS end2end tests (#27842) 2021-10-27 15:49:54 -07:00
Mark D. Roth 2a69c525b8
don't build RLS on mobile (#27838)
* don't build RLS on mobile

* buildifier

* fix build rules to avoid "Illegal ambiguous match" bazel error
2021-10-27 15:46:00 -07:00
Mark D. Roth 06b5061d35
change CSDS to populate new generic_xds_configs field (#27794) 2021-10-26 13:24:27 -07:00
Yash Tibrewal 9ac9a013fc
Update RDS parsing for use on servers (#27715)
* Update RDS parsing for use on servers

* Unused variable

* Reviewer comments

* Automated change: Fix sanity tests

* Fix tests

* Reviewer comments

* Reviewer comments

* clang-tidy

* Reviewer comments

* Fix test

* Reviewer comments

* Reviewer comments

Co-authored-by: yashykt <yashykt@users.noreply.github.com>
2021-10-26 00:32:35 -07:00
Mark D. Roth 5406048ef6
Second attempt: RLS LB policy (#27748)
* allow connectivity state watching to work on lame channels

* Revert "Revert RLS LB policy (#27738)"

This reverts commit 4567af504e.

* fix build

* fix lame_client_test
2021-10-20 07:41:07 -07:00
Craig Tiller f1f3ac7f02
Revert "Revert "allow connectivity state watching to work on lame channels (#27747)" (#27766)" (#27774)
This reverts commit 790ce5c97f.
2021-10-19 15:28:31 -07:00
Craig Tiller 790ce5c97f
Revert "allow connectivity state watching to work on lame channels (#27747)" (#27766)
This reverts commit a2bd7b8440.
2021-10-19 08:16:29 -07:00
Mark D. Roth d6a0c0b8b5
xds_end2end_test: allow setting Listener to use in SetRouteConfiguration() (#27756)
* xds_end2end_test: allow setting Listener to use in SetRouteConfiguration()

* clang-format
2021-10-19 07:36:54 -07:00
Mark D. Roth a2bd7b8440
allow connectivity state watching to work on lame channels (#27747)
* allow connectivity state watching to work on lame channels

* fix build

* fix lame_client_test
2021-10-18 14:47:47 -07:00
Mark D. Roth 4567af504e
Revert RLS LB policy (#27738)
* Revert "fix for import of RLS LB policy (#27736)"

This reverts commit cfcdbc4905.

* Revert "RLS LB policy (#27352)"

This reverts commit 84ddc3289f.
2021-10-15 12:19:57 -07:00
Mark D. Roth cfcdbc4905
fix for import of RLS LB policy (#27736) 2021-10-15 07:56:48 -07:00
Mark D. Roth 84ddc3289f
RLS LB policy (#27352)
* RLS LB policy plugin implementation

* Add RLS tests

* rls proto

* generate projects

* Fix adding child policy to child policy map

* Fix issues

* Fix rotten bits

* generate projects

* revert some strange changes

* First batch of fixes

* second batch of addressing comments

* generate_projects

* 3rd batch of fixes

* generate_projects

* 4th batch of fix

* 5th batch of fix

* Empty commit to trigger github PR refresh

* Add insecure build

* generate project

* update picker in a closure

* mutex guard audit

* clang-format

* Aggregate child policy states

* Use OrphanablePtr on ChildPolicyOwner()

* More fixes on comments

* Remove include grpc_security from rls.cc

* Fix key builder test as we removed RlsFindPathFromMetadata

* Update rls proto and add multi-target capability

* code changes to fix build after merging master

* remove support for insecure builds

* WIP

* add LB policy metadata lookup API

* add API for creating errors from C++ strings

* add missing include

* finished updating JSON parsing

* use DualRefCounted<> for ChildPolicyWrapper

* use grpc_core::Mutex instead of std::recursive_mutex

* add lock annotations and fix some lifetime issues

* misc fixes and cleanups

* simplify child policy connectivity state machine

* add comment about hopping into ExecCtx

* don't use wait_for_ready for RLS call

* extraKeys and constantKeys support

* add lock annotations for child policy wrapper connectivity state

* improve logging

* simplify child policy state handling in picker

* use C++ style comments

* remove rls_config.proto, since it's not needed

* rewrote tests

* fix build

* fix copyright headers

* clang-format

* appease clang-tidy

* remove unnecessary dependendency on grpclb

* buildifier

* remove illegal term

* remove unneeded includes

* clang-format

* fix clang-tidy

* fix build

* move class declaration into .cc file

* clang-tidy again

* fix build

* reorganize code and misc cleanups

* clang-format

* fixed a bunch of asan bugs; still some left to debug

* fix asan problem

* make test work on IPv6-only machines

* move LRU list handling into Cache::Entry and add lock annotations

* use preincrement instead of postincrement

* fix deadlock

* add more FIXMEs

* clean up backoff timer logic

* fix build from merge

* clang-format

* fix include path to work on podspec builds

* clean up picker logic

* clang-format

* update rls.proto

* populate reason and stale_header_data in RLS request

* improve logging and fix some cache size type issues

* clang-format

* add some TODOs about a better way to expose channel creds to LB policies

* centralize handling of channel's default authority

* clang-format

* fix backup poller bug

* fix handling of call creds and authority on RLS channel

* remove unused params

* clang-format

* use two-phase update for child policies to avoid deadlocks

* fix portability issue

* minor cleanups

* update for change in grpc_error_get_status() API

* change test to store RLS requests and responses in proto form

* clang-format

* account for test slowdown factor in timeouts

* fix tsan failure and channelz linkage

* fix RLS authority death test

* fix test to not try to connect to target that should not work

* remove illegal term

* cacheSizeBytes is a required field

* add missing BUILD deps from merge

* add tests for connectivity state reporting and fix bugs found

* fix BUILD sanity

* buildifier

* fix BUILD package path

* fix bugs related to child policy lifetime and updates

* remove unnecessary srand() call from grpc_init()

* add test for two cache entries with the same target

* update rls.proto

* change RLS config parser test to support GRPC_ERROR_IS_ABSEIL_STATUS

* update upb codegen

* fix include path

* add env var guard

* avoid duplication of CountedService

* generate_projects

* add upper limit for cacheSizeBytes

* fix build

Co-authored-by: Muxi Yan <mxyan@google.com>
2021-10-14 12:53:46 -07:00
Mark D. Roth 6339b03ada
Revert "Revert "start splitting up xds_end2end_test (#27702)" (#27725)" (#27726)
This reverts commit 9177c5324a.
2021-10-14 11:41:45 -07:00
Mark D. Roth 464c1c519f
switch to new location of TypedStruct and ORCA protos (#27700)
* upgrade xDS protos

* switch to new location of TypedStruct and ORCA protos

* fix sanity
2021-10-14 10:28:24 -07:00
Mark D. Roth 9177c5324a
Revert "start splitting up xds_end2end_test (#27702)" (#27725)
This reverts commit 69a978231b.
2021-10-14 08:47:17 -07:00
Mark D. Roth 69a978231b
start splitting up xds_end2end_test (#27702)
* move test xDS server to its own library

* move xds_end2end_test to xds subdir, and refactor CountedService

* add TODO

* fix sanity
2021-10-13 13:11:53 -07:00
Mark D. Roth 2b813d2bff
grpclb: implement subchannel caching (#27657)
* grpclb: implement subchannel caching

* code review changes

* fix clang tidy

* code review changes
2021-10-13 12:14:00 -07:00
Yash Tibrewal a61bb8ed5b
Move service config code out of client_client to its own target (#27683)
* Move service config code out of client_client to its own target

* Fix BUILD

* Fix test build

* Reviewer comments

* Reviewer comments

* Automated change: Fix sanity tests

* Reviewer comments

Co-authored-by: yashykt <yashykt@users.noreply.github.com>
2021-10-12 18:30:04 -07:00
Ashitha Santhosh a1db97be90
Revert "Revert "Revert "Revert "File watcher authorization policy provider implementation"" (#27605)" (#27644)" (#27645)
This reverts commit b8e01f73a0.
2021-10-08 15:52:13 -07:00
AJ Heller b8e01f73a0
Revert "Revert "Revert "File watcher authorization policy provider implementation"" (#27605)" (#27644)
This reverts commit 5912aedb2b.
2021-10-07 16:45:30 -07:00
Esun Kim ec4c61a3ae
Various fixes on absl::Status migration (#27557) 2021-10-07 14:49:59 -07:00
Ashitha Santhosh 5912aedb2b
Revert "Revert "File watcher authorization policy provider implementation"" (#27605)
* Revert "Revert "File watcher authorization policy provider implementation (#26779)" (#27591)"

This reverts commit a1ad74f7af.

* Remove license comment
2021-10-07 09:54:23 -07:00
Mark D. Roth 3d83dd3776
Revert "Revert "centralize handling of channel's default authority (#27544)" (#27616)" (#27624)
This reverts commit 0611f7c9b3.
2021-10-06 12:15:55 -07:00
Mark D. Roth 0611f7c9b3
Revert "centralize handling of channel's default authority (#27544)" (#27616)
This reverts commit f0a0628795.
2021-10-06 07:26:24 -07:00
Mark D. Roth f0a0628795
centralize handling of channel's default authority (#27544)
* centralize handling of channel's default authority

* clang-format

* fix backup poller bug
2021-10-05 11:57:47 -07:00
Craig Tiller a1ad74f7af
Revert "File watcher authorization policy provider implementation (#26779)" (#27591)
This reverts commit 310accd077.
2021-10-05 09:12:08 -07:00
Craig Tiller ce0aba2577
Increase XDS e2e shard count (#27582)
Provides a 40% reduction in runtime for this test internally, and since
it's on our critical path for landing changes, this is significant.
2021-10-04 17:22:13 -07:00
Ashitha Santhosh 310accd077
File watcher authorization policy provider implementation (#26779)
* File watcher authorization policy provider implementation
2021-10-04 14:22:45 -07:00
Lidi Zheng 9cd68439a2
Change the time-getting logic in xds test to what ExecCtx does (#27467) 2021-09-24 10:38:15 -07:00
Esun Kim 47586fb36e
Revert "Revert "Added absl::Status support to error_utils (#27358)" (#27418)" (#27429)
This reverts commit 933676c56c.
2021-09-21 14:42:57 -07:00
Craig Tiller 933676c56c
Revert "Added absl::Status support to error_utils (#27358)" (#27418)
This reverts commit 9b3f75d322.
2021-09-21 09:48:57 -07:00
Esun Kim 9b3f75d322
Added absl::Status support to error_utils (#27358) 2021-09-20 18:03:17 -07:00
donnadionne 1caff522e4
Adding a log to make logging in WaitForAllBackends and WaitForBackend (#27365)
consistent.

This helps to uncover a case where the first test in the shard seems to
encounter longer delays than the rest of the tests.
2021-09-20 11:48:43 -07:00
Alisha Nanda 0a502d8f2e
Fix race between Read and ServerContext::IsCancelled in Sync API (#27056)
* Fix OOM issues in qps tests

* Add more verbose logging.

* Fix clang error

* Fix race between IsCancelled and Read

* Fix build errors from using bool in C code
2021-09-15 12:29:02 -07:00
donnadionne 10f218030d
xDS retry policy should be considered present even if there are no supported policies in retry_on (#27315)
* Ensure that per route retry policy (even when there are no supported
retry_on statuses) still takes precedence over virtual host level retry
policy.

Added a test to guard this case.

* Taking care of code review comments and removing unnecessary block
2021-09-13 18:13:45 -07:00
Lidi Zheng 15184d30ad
Fix injected abort error not recorded after injected delay (#27215)
* Fix injected abort error not recorded after injected delay

* Add the abort after delay test case

* Make Kokoro's clang_format happy
2021-09-13 09:52:20 -07:00
Mark D. Roth 3763be8796
add APIs for creating errors from C++ strings (#27310)
* add API for creating errors from C++ strings

* add missing include
2021-09-10 16:15:36 -07:00
Yash Tibrewal 72171a3326
De-experimentalize XdsCredentials and XdsServerCredentials API (#26544)
* De-experimentalize XdsCredentials and XdsServerCredentials API

* Use GRPC_DEPRECATED
2021-09-09 16:22:25 -07:00
Yash Tibrewal 362aff3458
xDS: Remove environmental variable guard for security (#27290) 2021-09-09 16:21:15 -07:00
Yash Tibrewal 7fd731f704
De-experimentalize XdsServerBuilder (#27296) 2021-09-09 14:14:11 -07:00
Yash Tibrewal 1a009cf4e5
xDS Security: Use new way to fetch certificate provider plugin instance config (#27264)
* xDS Security: Use new way to fetch certificate provider plugin instance
config

* Reviewer comments

* Additional fields to NACK

* Move NACKing tests for tls_certificates and tls_certificate_sds_securet_configs to client-side
2021-09-09 14:12:05 -07:00
Yash Tibrewal f26c107651
xDS server serving status: Use a struct to allow more fields to be added in the future (#27242) 2021-09-09 11:11:57 -07:00
Mark D. Roth f3497eb790
implement improved xDS NACK semantics (#27276)
* refactor xDS response parsing

* fix build

* implement improved xDS NACK semantics

* fix clang-tidy

* fix test
2021-09-08 13:37:07 -07:00
Craig Tiller ea389c00c2
Adjust include order per style guide (#27175)
Introduce clang-format configuration to sort includes closer to our rules.
2021-09-08 12:14:44 -07:00
Mark D. Roth e468b00c56
refactor xDS response parsing (#27272)
* refactor xDS response parsing

* fix build
2021-09-08 07:57:38 -07:00
Mark D. Roth 2cb3831c83
fix retry code to fail batches instead of creating attempt if previously cancelled from surface (#27217)
* fix retry code to fail batches instead of creating attempt if previously cancelled from surface

* add xDS end2end tests covering the FI use-case that triggered the bug

* fix memory leak
2021-09-01 15:47:16 -07:00
Yash Tibrewal cb2e7554a3
xds_end2end_test: Fix UnsupportedHttpFilter test on aws (#27147) 2021-08-27 09:10:46 -07:00
Ta-Wei Tu 7dd1da9e1d
Break after WriteLast() (#27165)
If we continue to call `Write()` after `WriteLast()` (because client is still sending us data and thus `Read()` returns `true`), the buffer might become invalid and results in assertion failures.
2021-08-27 23:19:12 +08:00
Ashitha Santhosh 1ff1f8fd83
SDK authorization filter. (#26468)
* SDK authorization filter.
2021-08-25 21:00:35 -07:00
donnadionne 59fc02a94b
Enabling Ring Hash by default. (#27111) 2021-08-24 19:59:19 -07:00
Yash Tibrewal 6da9eb9f3f
xDS PSM Server Security: Nack unsupported require_sni and ocsp_staple_policy values (#26878)
* Nack unsupported require_sni and ocsp_staple_policy values

* Reviewer comments
2021-08-24 10:50:08 -07:00
AJ Heller 6ba4a92af7
Reorder gtest/gmock headers (#27070)
Per the style guide (or spirit thereof), these should be placed with the other external/third_party headers.
2021-08-20 14:36:08 -07:00
Yash Tibrewal 0ab13090e7
Nack certain unsupported fields in CertificateValidationContext (#26880)
* Nack certain unsupported fields in CertificateValidationContext

* Regenerate projects

* Reviewer comments
2021-08-20 11:03:20 -07:00
AJ Heller d10617edb5
Move resource_user ownership into chttp2 transport/server/connector v2 (#27032)
Reintroducing PR #26643, which was reverted in #27029

Fixed a memory leak and added a test that would have caught it (ASAN build): ca0c8c4
2021-08-18 17:04:00 -07:00
AJ Heller 4ba3fa52b9
Clean up some instances of -Wunreachable-code-aggressive (#27054)
See https://github.com/grpc/grpc/pull/26951 for why this is not enforced
at the moment.
2021-08-18 11:43:46 -07:00
AJ Heller 58f8f5ced3
Add macro to define gtest death_test_style using an older syntax (#27042)
This temporarily unblocks a related gtest upgrade. The ultimate goal is
to upgraade our gtest dependencies, but I don't have the cycles to
manage a potentially messy migration until at least next week. This PR
is coordinated with an internal change.
2021-08-17 17:12:28 -07:00
Yash Tibrewal 533f02642d
Disable xds_credentials_test on iOS (#27043) 2021-08-17 13:33:05 -07:00
Mark D. Roth 9f782faa34
xds: remove env var protection for retries (#27039) 2021-08-17 13:23:25 -07:00
Craig Tiller f5d3ed2db1
Revert "Move resource_user ownership into chttp2 transport/server/connector (#26643)" (#27029)
This reverts commit d1935a65a1. Will be rolled forward with a fix.
2021-08-16 19:07:39 -07:00
AJ Heller d1935a65a1
Move resource_user ownership into chttp2 transport/server/connector (#26643) 2021-08-16 13:50:56 -07:00
Tamir Duberstein 34338e5798
Avoid undefined behavior in pthread TLS (#26999)
See code commentary for an explanation.

Add an additional constructor to allow `log_linux.cc` to compile with
GPR_PTHREAD_TLS. Without it:

```
../../third_party/grpc/src/core/lib/gpr/log_linux.cc:78:33: error: no viable conversion from 'int' to 'grpc_core::PthreadTlsImpl<long>'
  static GPR_THREAD_LOCAL(long) tid = 0;
                                ^     ~
../../third_party/grpc/src/core/lib/gpr/tls.h:64:3: note: candidate constructor not viable: no known conversion from 'int' to 'const grpc_core::PthreadTlsImpl<long> &' for 1st argument
  PthreadTlsImpl(const PthreadTlsImpl&) = delete;
  ^
1 error generated.
```
2021-08-12 19:25:38 -07:00
Tamir Duberstein 00c03c55ff
Implement type safety for TLS (#26942)
* Implement type safety for TLS

This is mostly free when compiler support is available, but requires
careful templating when implemented using pthread.

Significantly slimmed the tls.h interface; it now only defines the "TLS
keyword" for each supported compiler, delegating enforcement of correct
usage (i.e. must be static) to the compiler itself.

Implemented implicit conversion for the pthread wrapper so it can be
used (mostly) the same as native support. Notable exception to this is
that static_cast<void*> is needed when printing a pointer stored in TLS
as %p.

* Use GPR_THREAD_LOCAL macros consistently
2021-08-11 13:24:26 -07:00
Yash Tibrewal 404e56b2d9
Fix XdsEnabledServerTest (#26956) 2021-08-10 10:47:12 -07:00
Tamir Duberstein 3fd01fabae
Add thread safety annotations (#26871)
This exposes a bug in clang, reported upstream as
https://bugs.llvm.org/show_bug.cgi?id=51368.

The clang bug is mitigated using a fake scoped lock; that allows the
current code to compile while also serving as a change detector to
prevent it from going stale; if the compiler bug is fixed, the compiler
will see an overlapping locking requirement, and reject this code, which
will prompt a human being to remove this workaround.
2021-08-06 15:44:22 -07:00
Yash Tibrewal 29d220a0ef
Re-experimentalize XdsServerBuilder (#26892)
* Re-experimentalize XdsServerBuilder

* Fix compiler issue
2021-08-06 10:29:28 -07:00
Tamir Duberstein 013e67a029
Deprecate grpc{_core,::internal}::WaitUntil (#26866)
It is not possible for such a function to be implemented in a way that
is understood by annotalysis. Mark it deprecated and replace instances
of its use with direct mutex/condvar usage.

Add a bunch of missing thread safety annotations while I'm here.
2021-08-04 17:03:25 -07:00
Mark D. Roth 639b7acdfa
fix client_lb_end2end_test ClientLbPickArgsTest.Basic flakiness (#26868)
* fix client_lb_end2end_test ClientLbPickArgsTest.Basic flakiness

* clang-format
2021-08-04 09:46:47 -07:00
Tamir Duberstein 86c9de8231
Remove `grpc_core::WaitUntilWith{Deadline,Timeout}` (#26867)
These functions are essentially unused.
2021-08-03 15:46:36 -07:00
Mark D. Roth 6b71ec3704
LB policy API improvements (#26481)
* LB policy API improvements

* clang-format

* fix build

* a bit more cleanup

* use absl::variant<> for pick result

* fix retry_lb_drop test

* clang-format

* fix grpclb_end2end_test

* fix xds_end2end_test

* try to make variant code a bit cleaner

* clang-format

* fix memory leak

* fix build

* clang-format

* fix error refcount bug

* remove PickResult factory functions

* clang-format

* add ctors to structs

* clang-format

* fix clang-tidy

* update comments

* move LB recv_trailing_metadata callback instead of copying it

* use Match() instead of providing PickResult::Handle()

* don't use Match() for now, since it breaks lock annotations

* update retry_lb_fail test
2021-08-03 09:43:07 -07:00
Yash Tibrewal 49b5f9c420
Fix XdsServerSecurityTest (#26817) 2021-07-29 13:28:18 -07:00
donnadionne 7eaf37bce5
Adding retry policy (#26566) 2021-07-23 12:17:17 -07:00
Mark D. Roth 3104a9964c
xds: validate that terminal filters come at the end of the filter chain (#26742)
* xds: validate that terminal filters come at the end of the filter chain

* clang-format
2021-07-23 08:06:58 -07:00
Craig Tiller 4f9828fe0b
Remove unnecessary include (#26761) 2021-07-22 16:19:34 -07:00
Mark D. Roth 246c57829b
xds: notify watchers when NACKing resource updates (#26757)
* xds: notify watchers when NACKing resource updates

* check status of failed RPC when waiting for NACK
2021-07-22 13:12:19 -07:00
Alisha Nanda d73ec07fe7
Fix small typo in xds end2end test (#26685) 2021-07-14 10:53:55 -07:00
Yash Tibrewal c52005c161
NACK xDS updates when certificate provider instace names are unrecognized (#26614)
* NACK xDS updates when certificate provider instace names are unrecognized

* Reviewer comments

* Reviewer comments

* Clang format

* Fix compilation error
2021-07-11 22:10:04 -07:00
Lidi Zheng b97693d572
[Roll Forward] Tighten the error tolerance requirement by 100x (#26626)
* Tighten the error tolerance requirement by 10x

* Make it 5 sigma instead of 4.5

* Rewrap comments

* Loosen the max concurrent requests in certain test cases
2021-07-09 10:33:43 -07:00
Yash Tibrewal 670a26caef
NACK xDS updates where transport_name is not recognized (#26612)
* NACK xDS updates where transport_name is not recognized

* Reviewer comments
2021-07-08 11:14:07 -07:00
Yash Tibrewal 095045b31c
NACK xds updates requiring client certificates when no validation certificate provider instance is mentioned (#26613) 2021-07-07 13:22:13 -07:00
Yash Tibrewal 8b5fbf8900
TLS Security Connector: Add an always-fail-handshaker when certificates are not ready (#26561)
* TLS Security Connector: Add an always-fail-handshaker when certificates are not ready

* Reviewer suggestion

* Add test
2021-07-04 12:39:29 -07:00
Yash Tibrewal 4daedf0233
De-experimentalize XdsServerBuilder (#26543) 2021-07-02 19:33:15 -07:00
Lidi Zheng f198fb5b41
Revert "Tighten the error tolerance requirement by 100x (#26588)" (#26593)
This reverts commit f835f3f97c.
2021-07-01 16:21:12 -07:00
Lidi Zheng f835f3f97c
Tighten the error tolerance requirement by 100x (#26588)
* Tighten the error tolerance requirement by 10x

* Make it 5 sigma instead of 4.5

* Rewrap comments
2021-07-01 09:48:37 -07:00
Mark D. Roth 34bf26357b
Change StringMatcher to not support case-insensitive regex matching. (#26567)
* Change StringMatcher to not support case-insensitive regex matching.

* clang-format

* clang-tidy
2021-06-29 09:25:33 -07:00