Commit Graph

5713 Commits

Author SHA1 Message Date
Craig Tiller 717732c044
[promises] Convert http server filter (#29273)
* Begin adding abstractions for capturing batches

* code written

* fixes

* fix

* fmt

* placate clang-tidy

* simplify

* fixes

* Automated change: Fix sanity tests

* annotate unrefs

* fix tsan

* fix

* Update subchannel.cc

* Update BUILD

* Update generate_tests.bzl

* Update BUILD

* Automated change: Fix sanity tests

* Promiseize

* Move server promise based filter to Flusher

* review feedback

* compiles!

* fix

* fix

* start server initial metadata emulation

* comment

* comment

* fix

* refactoring-to-fix-the-bugs

* fixfixfix

* fix

* support cancellation

* fuzzer

* fix

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2022-05-04 07:55:50 -07:00
Craig Tiller 18bec00b58
[channel-stack] Eliminate post-init in channel stack builder (#29514)
* Eliminate post-init in channel stack builder

We've had a post init function on channel stack builder for a very long
time, an it serves to run some code after initialization completes.

We need the functionality for a few things, but the function passed in
is intimately tied to the filter in use - we never vary it between
multiple functions for the same filter... which means it makes more
sense to locate this functionality as part of the filter interface.

* fix

* Automated change: Fix sanity tests

* fix

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2022-05-03 13:06:51 -07:00
Nicolas Noble 050e4c3abe
client_lb_end2end_test: add OrcaLoadReport into EchoRequest proto (#29505)
* Embed OrcaLoadReport into end2end test.

* Regenerate project files.

* Automated change: Fix sanity tests

* Address comments.

* Maybe fixing iOS test?

* Fixing grpc_tool_test.

* Derp.

* Automated change: Fix sanity tests

* Fixing internal import.

* Fixing Google3 build.

* iOS fix attempt: take 2.
2022-05-03 09:34:57 -07:00
Vignesh Babu b92f885756
Adding a max_frame_size argument to grpc_endpoint_write API to allow a sender to control frame sizes (#29526)
* adding a max_frame_size argument to grpc_endpoint_write API

* fix syntax error

* fix typo
2022-05-02 12:32:18 -07:00
Vignesh Babu 53e382729f
Adding a min progress size argument to grpc_endpoint_read to allow gRPC to use TCP optimizations on the read path (#29503)
* adding a min progress size argument to grpc_endpoint_read

* fix missing argument error

* adding a static_cast

* reverting changes in tcp_posix.cc

* add missing changes to CFStreamEndpointTests.mm
2022-04-28 13:23:09 -07:00
ZhenLian 2badafbc4d
Expose NoOpCertificateVerifier to C++ (#29322)
* expose NoOpCertificateVerifier to C+
2022-04-27 13:46:01 -07:00
Mark D. Roth 1cd6e69347
subchannel: report IDLE upon existing connection failure and after backoff interval (#29428)
* subchannel: report IDLE upon existing connection failure and after backoff interval

* rename AttemptToConnect() to RequestConnection()

* clang-format

* fix unused parameter warning

* fix subchannel to handle either TF or SHUTDOWN from transport

* fix handling of ConnectedSubchannel failure

* pass status up in IDLE state to communicate keepalive info

* update comment

* split pick_first and round_robin tests into their own test suites

* improve log message

* add test

* clang-format

* appease clang-tidy

* fix test to do a poor man's graceful shutdown to avoid spurious RPC failures

* simplify round_robin logic and fix test flakes

* fix grpclb bug
2022-04-26 15:25:47 -07:00
Craig Tiller d6e91959a7
mark test nomsan (#29488) 2022-04-22 14:22:40 -07:00
Mark D. Roth 0ba3c59672
xds_ring_hash_end2end_test: fix flake in ContinuesConnectingWithoutPicks (#29461) 2022-04-21 07:11:03 -07:00
Craig Tiller 8bb45aa3a6
Convert grpc_channel to C++ (#29266)
* begin

* tests

* fix

* http

* Filter fuzzer

* progress

* basics

* progress

* Automated change: Fix sanity tests

* Automated change: Fix sanity tests

* Revert "Revert "HTTP Client Filter --> promises (#29031)" (#29181)"

This reverts commit 6ee276f672.

* stuff

* debug

* minimal reproduction

* progress

* progress

* create call

* progress

* recv trailing metadata

* wakeups

* corpus

* Automated change: Fix sanity tests

* Automated change: Fix sanity tests

* debug

* fix state machine for c#

* Revert "minimal reproduction"

This reverts commit 4d02d2e730.

* Revert "debug"

This reverts commit 7960842f48.

* Revert "debug"

This reverts commit a6f224e4a1.

* no-logging

* initial-metadata

* Revert "Revert "debug""

This reverts commit 951844e857.

* Better int conversion

* debug

* Fix for Cronet

* Revert "debug"

This reverts commit 4d641c4281.

* Revert "Better int conversion"

This reverts commit 4001b957cb.

* Revert "Revert "Revert "debug"""

This reverts commit d135c61043.

* progress

* progress

* Automated change: Fix sanity tests

* fix, c++ize

* handle transport, use objects

* enable more stuffs

* remove placeholder

* contexts

* fix

* fix

* Automated change: Fix sanity tests

* Automated change: Fix sanity tests

* x

* exceptional toast

* include idle filters, time

* fix

* namespace

* fixes

* final info

* progress

* cleanup

* progress

* progress

* progress

* Automated change: Fix sanity tests

* Automated change: Fix sanity tests

* progress

* Set int

* Set int

* Automated change: Fix sanity tests

* fix

* fix

* fix

* fixes

* fixes

* fixes

* fixes

* fix

* fix race

* fix race

* mac fix

* review feedback

* getgetget

* fix ios

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
Co-authored-by: Jan Tattermusch <jtattermusch@google.com>
2022-04-20 20:55:15 -07:00
Mark D. Roth be53d2ce3f
LB policy API: add OOB backend metric API (#29012)
* WIP: add OOB backend metric API for LB policies

* fix some includes

* minor fixes

* picking this up again...

* more WIP

* health checking: cancel stream if response message fails to parse

* basic structure in place, but still have synchronization issues to address

* ORCA: implement ORCA RPC service for OOB backend metric reporting

* fix unused parameter error

* gen_upb_api

* add missing build deps

* increase test timing fudge factor

* add missing copyright header

* fix build and locking problems

* clang-format

* document API

* buildifier

* add test, but doesn't build yet

* new test working, but broke existing test, and need to fix server API

* don't register as a generic service

* update test for new orca service registration API

* fix build

* sanitize

* report interval defaults to min interval

* add channel trace event on UNIMPLEMENTED

* don't regenerate the response proto unless something changed

* add missing build dep

* fix comment
2022-04-20 14:14:30 -07:00
Mark D. Roth 6bda57194a
xds_end2end_test_lib: use DEBUG_LOCATION to log where failures originate (#29444) 2022-04-19 15:36:15 -07:00
Mark D. Roth 8ba275291a
connection delay injector: fix tsan problems (#29445) 2022-04-19 15:35:47 -07:00
Yash Tibrewal 8e0da602a8
HTTP: Rename PUT allowing channel arg (#29438) 2022-04-19 01:51:57 -07:00
Alisha Nanda 6753812278
Fix ubsan failure (#29437) 2022-04-18 19:57:02 -07:00
Yash Tibrewal eb96f90eb7
HTTP: Use channel arg instead of global to allow PUT requests (#29411)
* HTTP: Use channel arg instead of global to allow PUT requests

* Reviewer comments

* Reviewer comments

* clang-format

* Repo manager - AJ

* Reviewer comments

* unused parameter
2022-04-18 16:34:55 -07:00
Mark D. Roth 9a363dbf96
xds_end2end_test: Move CDS and EDS tests to their own file (#29430)
* xds_end2end_test: Move CDS and EDS tests to their own file

* fix build
2022-04-18 15:11:32 -07:00
Mark D. Roth 93a90e3c55
xds_ring_hash_end2end_test: increase RPC timeout to avoid failures on ARM (#29431) 2022-04-18 11:45:50 -07:00
Mark D. Roth 50df29bdec
ring_hash: trigger internal connection attemps on only one subchannel at a time (#29422)
* ring_hash: trigger internal connection attemps on only one subchannel at a time

* simplify logic

* add test

* clang-format
2022-04-18 09:36:27 -07:00
Yash Tibrewal f28695351e
Prevent direct usage of absl::Mutex (#29424)
* Add script to prevent absl::Mutex occurrences

* sanity and fixes

* shellcheck
2022-04-17 23:56:18 -07:00
Mark D. Roth e7ddd7b436
xds_end2end_test: Move aggregate and logical_dns cluster tests to their own file (#29298)
* move some code around

* remove num_backends parameter from XdsEnd2endTest

* remove use_xds_enabled_server param from XdsEnd2endTest

* remove xds_resource_does_not_exist_timeout_ms param from XdsEnd2endTest

* remove client_load_reporting_interval_seconds param from XdsEnd2endTest

* start moving CreateAndStartBackends() into individual tests

* finish moving CreateAndStartBackends() into individual tests

* remove unused variable

* remove SetEdsResourceWithDelay

* fix test flake

* clang-tidy

* clang-format

* move test framework to its own library

* fix build

* clang-format

* fix windows build

* rename TestType to XdsTestType

* move BackendServiceImpl inside of BackendServerThread

* clang-format

* move AdminServerThread to CSDS test suite

* remove unnecessary deps

* move aggregate and logical_dns cluster tests to their own file

* split aggregate and logical_dns tests into separate suites

* clang-format

* re-add flaky tag

* clang-tidy and remove unnecessary dep
2022-04-16 18:08:53 -07:00
Mark D. Roth bea5911569
xds_end2end_test: Move ring_hash tests to their own file (#29287)
* move some code around

* remove num_backends parameter from XdsEnd2endTest

* remove use_xds_enabled_server param from XdsEnd2endTest

* remove xds_resource_does_not_exist_timeout_ms param from XdsEnd2endTest

* remove client_load_reporting_interval_seconds param from XdsEnd2endTest

* start moving CreateAndStartBackends() into individual tests

* finish moving CreateAndStartBackends() into individual tests

* remove unused variable

* remove SetEdsResourceWithDelay

* fix test flake

* clang-tidy

* clang-format

* move test framework to its own library

* fix build

* clang-format

* fix windows build

* rename TestType to XdsTestType

* move BackendServiceImpl inside of BackendServerThread

* clang-format

* move AdminServerThread to CSDS test suite

* move ring_hash tests to their own file

* generate_projects

* remove unnecessary deps

* re-add flaky tag

* clang-format
2022-04-16 14:42:43 -07:00
Mark D. Roth b9d904da58
xds_end2end_test: move routing tests to their own file (#29409)
* xds_end2end_test: move routing tests to their own file

* clang-format and clang-tidy

* re-add flaky label
2022-04-16 08:26:32 -07:00
Yash Tibrewal 136055b043
xds_end2end_test: Fix race in security tests (#29408)
* xds_end2end_test: Fix race in security tests

* Reviewer comments
2022-04-15 15:49:48 -07:00
Yousuk Seung 371d231b53
Support unix socket in grpc_sockaddr_to_string (#29169)
* Support unix socket in grpc_sockaddr_to_string

* make it return statusor

* clang fix

* made grpc_sockaddr_to_string() to return statusor

* Let Chttp2ServerListener::Start crash

* test failure fixed

* api_fuzzer fixed

* comments addressed.

* more comments addressed

* comments addressed

* fix other broken builds
2022-04-15 14:56:15 -07:00
Lidi Zheng e9ad1d5f47
Improve the MaxFault test case (#29407) 2022-04-15 13:51:31 -07:00
Mark D. Roth dd14f803c3
xds_end2end_test: move core XdsClient tests to their own file (#29403)
* xds_end2end_test: move core XdsClient tests to their own file

* clang-format

* also move SecureNaming and XdsStreamErrorPropagation tests
2022-04-15 12:33:50 -07:00
Mark D. Roth 55e870dae6
orca_service_end2end_test: increase flake timing tolerance (#29405) 2022-04-14 17:12:26 -07:00
Mark D. Roth 6273832210
priority and ring_hash LBs: fix interactions when using ring_hash under priority (#29332)
* refactor connection delay injection from client_lb_end2end_test

* fix build

* fix build on older compilers

* clang-format

* buildifier

* a bit of code cleanup

* start failover time whenever the child reports CONNECTING, and don't cancel when deactivating

* clang-format

* rewrite test

* simplify logic in priority policy

* clang-format

* switch to using a bit to indicate child healthiness

* fix reversed comment

* more changes in priority and ring_hash.

priority:
- go back to starting failover timer upon CONNECTING, but only if seen
  READY or IDLE more recently than TRANSIENT_FAILURE

ring_hash:
- don't flap back and forth between IDLE and CONNECTING; once we go
  CONNECTING, we stay there until either TF or READY
- after the first subchannel goes TF, we proactively start another
  subchannel connecting, just like we do after a second subchannel
  reports TF, to ensure that we don't stay in CONNECTING indefinitely if
  we aren't getting any new picks
- always return ring hash's picker, regardless of connectivity state
- update the subchannel connectivity state seen by the picker upon
  subchannel list creation
- start proactive subchannel connection attempt upon subchannel list
  creation if needed

* ring_hash: fix connectivity state seen by aggregation and picker

* fix obiwan error

* swap the order of ring_hash aggregation rules 3 and 4

* restore original test

* refactor connection injector QueuedAttempt code

* add test showing that ring_hash will continue connecting without picks

* clang-format

* don't actually need seen_failure_since_ready_ anymore

* fix TSAN problem

* address code review comments
2022-04-14 17:08:38 -07:00
Mark D. Roth 214e3f3622
xds_end2end_test: Move RLS tests to their own file (#29290)
* move some code around

* remove num_backends parameter from XdsEnd2endTest

* remove use_xds_enabled_server param from XdsEnd2endTest

* remove xds_resource_does_not_exist_timeout_ms param from XdsEnd2endTest

* remove client_load_reporting_interval_seconds param from XdsEnd2endTest

* start moving CreateAndStartBackends() into individual tests

* finish moving CreateAndStartBackends() into individual tests

* remove unused variable

* remove SetEdsResourceWithDelay

* fix test flake

* clang-tidy

* clang-format

* move test framework to its own library

* fix build

* clang-format

* fix windows build

* rename TestType to XdsTestType

* move BackendServiceImpl inside of BackendServerThread

* clang-format

* move AdminServerThread to CSDS test suite

* move RLS tests to their own file

* remove unnecessary deps

* generate_projects
2022-04-14 11:22:39 -07:00
Nicolas Noble e9cf2894da
Fixes a flake with the LoadReporter end2end test. (#29383)
* Fixes a flake with the LoadReporter end2end test.

I *believe* the test is wrong, based on the .proto description of the
LoadReporter.

The protocol described in src/proto/grpc/lb/v1/load_reporter.proto has
the ReportLoad rpc returns a stream of LoadReportResponse, which itself
has a repeated field of Load messages. The comment before it states:

"It is not strictly necessary to aggregate all entries into one entry
per <tag, user_id> tuple, although it is preferred to do so."

Debugging the issue shows we are in fact properly getting all 3 expected
load report types, just in two separate messages instead of a single
one.

This new test codepath will coalesce the load report responses, and also
addresses the fact the original test wasn't verifying that we were
getting the 3 expected types.

* Automated change: Fix sanity tests

* Renaming variables.

* ASSERT_ -> EXPECT_

* Automated change: Fix sanity tests
2022-04-14 09:41:52 -07:00
Mark D. Roth 55b0405c86
xds_end2end_test: Move fault injection tests into their own file (#29283)
* move some code around

* remove num_backends parameter from XdsEnd2endTest

* remove use_xds_enabled_server param from XdsEnd2endTest

* remove xds_resource_does_not_exist_timeout_ms param from XdsEnd2endTest

* remove client_load_reporting_interval_seconds param from XdsEnd2endTest

* start moving CreateAndStartBackends() into individual tests

* finish moving CreateAndStartBackends() into individual tests

* remove unused variable

* remove SetEdsResourceWithDelay

* fix test flake

* clang-tidy

* clang-format

* move test framework to its own library

* fix build

* clang-format

* fix windows build

* move fault injection tests to their own file

* rename TestType to XdsTestType

* move BackendServiceImpl inside of BackendServerThread

* clang-format

* generate_projects

* appease clang-tidy

* move AdminServerThread to CSDS test suite

* remove unnecessary deps

* generate_projects

* don't mark test as flaky
2022-04-14 08:32:33 -07:00
Yash Tibrewal 5e989cf78d
HTTP: Conditionally allow PUT requests (#29397)
* Maybe fix for PUT deprecation

* Guard PUT request accepting with a flag and add tests

* Reviewer comments

* Add fallthrough notation

* Reviewer comments

Co-authored-by: Craig Tiller <ctiller@google.com>
2022-04-13 17:47:07 -07:00
Nicolas Noble 5a3cd992b0
Fixing the build breakage from #29285 (#29399) 2022-04-13 16:58:08 -07:00
Mark D. Roth 61987ec3a2
xds_end2end_test: Move CSDS tests to their own file (#29285)
* move some code around

* remove num_backends parameter from XdsEnd2endTest

* remove use_xds_enabled_server param from XdsEnd2endTest

* remove xds_resource_does_not_exist_timeout_ms param from XdsEnd2endTest

* remove client_load_reporting_interval_seconds param from XdsEnd2endTest

* start moving CreateAndStartBackends() into individual tests

* finish moving CreateAndStartBackends() into individual tests

* remove unused variable

* remove SetEdsResourceWithDelay

* fix test flake

* clang-tidy

* clang-format

* move test framework to its own library

* fix build

* clang-format

* fix windows build

* rename TestType to XdsTestType

* move BackendServiceImpl inside of BackendServerThread

* clang-format

* move CSDS tests to their own file

* move AdminServerThread to CSDS test suite

* remove unnecessary deps

* generate_projects

* fix build

* remove flaky label
2022-04-13 13:12:39 -07:00
Mark D. Roth 9a12b0def8
cds LB: fix diamond dependency handling (#29384)
* cds LB: fix diamond dependency handling

* instead of explicitly checking for loops, just check for max depth
2022-04-13 13:01:57 -07:00
Mark D. Roth d61433ecb2
cds LB: handle loops in aggregate cluster graph (#29382)
* cds LB: handle loops in aggregate cluster graph

* clang-format
2022-04-12 14:45:35 -07:00
Mark D. Roth 3a4058c07a
xds_end2end_test: move test framework to its own library (#29279)
* move some code around

* remove num_backends parameter from XdsEnd2endTest

* remove use_xds_enabled_server param from XdsEnd2endTest

* remove xds_resource_does_not_exist_timeout_ms param from XdsEnd2endTest

* remove client_load_reporting_interval_seconds param from XdsEnd2endTest

* start moving CreateAndStartBackends() into individual tests

* finish moving CreateAndStartBackends() into individual tests

* remove unused variable

* remove SetEdsResourceWithDelay

* fix test flake

* clang-tidy

* clang-format

* move test framework to its own library

* fix build

* clang-format

* fix windows build

* rename TestType to XdsTestType

* move BackendServiceImpl inside of BackendServerThread

* clang-format

* move AdminServerThread to CSDS test suite

* remove unnecessary deps
2022-04-12 13:42:58 -07:00
Mark D. Roth 94c538cd55
Second attempt: Adding is_optional case to RLS (#29378)
* Revert "Revert "Adding is_optional case to RLS (#29259)" (#29299)"

This reverts commit a6419dde06.

* change test to check the full error message
2022-04-12 11:21:44 -07:00
apolcyn 61b34dfaee
Improve logging in interop soak test and add a flag to limit QPS (#29357)
* Improve logging in interop soak test and add a flag to limit QPS
2022-04-08 21:16:23 -07:00
Mark D. Roth 60c56f7d01
Second attempt: implement ORCA RPC service for OOB backend metric reporting (#29352)
* Revert "Revert "ORCA: implement ORCA RPC service for OOB backend metric reporting (#29215)" (#29351)"

This reverts commit 71b355624f.

* move ORCA service to its own BUILD rule
2022-04-08 11:12:18 -07:00
Mark D. Roth 71b355624f
Revert "ORCA: implement ORCA RPC service for OOB backend metric reporting (#29215)" (#29351)
This reverts commit 6d6380de58.
2022-04-07 16:15:49 -07:00
Mark D. Roth 0b79940496
connection delay injector: cap delay by deadline (#29349) 2022-04-07 15:12:59 -07:00
Mark D. Roth 6d6380de58
ORCA: implement ORCA RPC service for OOB backend metric reporting (#29215)
* ORCA: implement ORCA RPC service for OOB backend metric reporting

* fix unused parameter error

* gen_upb_api

* add missing build deps

* increase test timing fudge factor

* add missing copyright header

* buildifier

* don't register as a generic service

* report interval defaults to min interval

* don't regenerate the response proto unless something changed

* use INTERNAL for proto parsing failure

* use absl::Duration in public API
2022-04-07 07:58:58 -07:00
Mark D. Roth 1fd3850502
client_lb_end2end_test: refactor connection delay injection into its own library (#29320)
* refactor connection delay injection from client_lb_end2end_test

* fix build

* fix build on older compilers

* clang-format

* buildifier
2022-04-06 15:23:41 -07:00
Mark D. Roth 96c19e8c98
weighted_target and xds_cluster_manager: don't update picker while update is in flight (#29313)
* weighted_target and xds_cluster_manager: don't update picker while update is in flight

* include deactivation and new child creation

* add test
2022-04-06 15:00:07 -07:00
Mark D. Roth e145c068f2
xds_cluster_resolver LB: don't reuse child numbers that might still be deactivated (#29316)
* xds_cluster_resolver LB: don't reuse child numbers that might still be deactivated

* clang-format
2022-04-06 06:43:39 -07:00
Mark D. Roth caacc7e3aa
xds_end2end_test: fix env var name in RLS tests (#29291)
* xds_end2end_test: fix env var name in RLS tests

* add ScopedExperimentalEnvVar to ensure env vars get unset after tests

* clang-format
2022-04-04 14:27:17 -07:00
Mark D. Roth a6419dde06
Revert "Adding is_optional case to RLS (#29259)" (#29299)
This reverts commit 2fd632a4c1.
2022-04-02 20:27:01 -07:00
Mark D. Roth 217ab2a793
xds_end2end_test: make each individual test start the number of backends it needs (#29271)
* move some code around

* remove num_backends parameter from XdsEnd2endTest

* remove use_xds_enabled_server param from XdsEnd2endTest

* remove xds_resource_does_not_exist_timeout_ms param from XdsEnd2endTest

* remove client_load_reporting_interval_seconds param from XdsEnd2endTest

* start moving CreateAndStartBackends() into individual tests

* finish moving CreateAndStartBackends() into individual tests

* remove unused variable

* fix test flake

* clang-tidy

* clang-format
2022-04-02 09:26:57 -07:00