* Add a provision to allow specification of separate set of channel args for the grpclb channel
* fix asan issue
* review comments
* review comments
* add missing file
* remove unused hdr
* fix sanity
* fix comments
* remove unused hdr
* ConnectionAttemptInjector: fix tsan failures
* Add hold to test
* Add hold to test
* Address comments
* Address comments
* Address comments
* Fix typo
Co-authored-by: Mark D. Roth <roth@google.com>
Previously this failed 1/1000 times with a 1s timeout, giving a
`Deadline Exceeded` error. I was able to reproduce the failure in
22/1000 times with a 500ms timeout. Changing it to a 2s timeout in this
PR, the failure did not reproduce in 5000 runs.
* FaultInjection: Fix random number generation
* Put random generation under a mutex
* Fix IWYU
* Regenerate projects
* Modify timeouts
* Dbg build knobs
* Remove unnecessary slowdown factor
* Tune error tolerance and add note on broken computation of ComputeIdealNumRpcs
* outlier_detection LB: reset state when OD is disabled
* leave interval unset when no OD config is present
* Automated change: Fix sanity tests
* fix clang-tidy
* don't check for interval of infinite duration
Co-authored-by: markdroth <markdroth@users.noreply.github.com>
* service config API: use absl::Status instead of grpc_error
* Automated change: Fix sanity tests
* add missing build deps
* attempt to work around build breakage on older compilers
* trying the work-around in more spots
* more work-arounds
* more workarounds
* Automated change: Fix sanity tests
* work around another compiler problem
* Automated change: Fix sanity tests
* Automated change: Fix sanity tests
Co-authored-by: markdroth <markdroth@users.noreply.github.com>
* client_lb_end2end_test: fix flake in RoundRobin.SingleReconnect test
* fix condition used to determine when client has seen backend 0 down
* remove duplicate counter reset
* Automated change: Fix sanity tests
Co-authored-by: markdroth <markdroth@users.noreply.github.com>
* refactor connection injectors in client_lb_end2end_test and add test for sticky TF
* round_robin: update status upon each new connection failure
* fix ring_hash too
* clang-format
* ring_hash: don't recreate ring when individual subchannel states change
* client_channel: remove synchronous subchannel connectivity state API
* change subchannel list to automatically start watching all subchannels
* use a separate loop to start watches, so list size is logged correctly
* fix RR to re-resolve on IDLE again
* fix ring_hash to delay promoting new subchannel list
* fix pick_first to wait for all subchannels to report state
* clean up SubchannelList API
* fix unused argument error
* fix another unused argument error
* clang-format
* fix RR to not re-resolve on initial IDLE state
* also don't re-resolve in initial TF state; same for ring_hash
* clang-format
* change RR and PF to initially report CONNECTING, and add second loop to priority policy
* simplify priority logic a bit
* fix grpclb to drop ref to stats object even if the subchannel call is never started
* fix memory leak in ring_hash
* fix tsan failure in grpclb code
* iwyu
* add missing BUILD deps
* update outlier_detection policy
* fix test
* fix pick_first to not report TF prematurely due to subchannel sharing
* fix test to not depend on timing
* server: add method to expose authority seen by server
* Automated change: Fix sanity tests
Co-authored-by: markdroth <markdroth@users.noreply.github.com>
* implement UniqueTypeName API
* convert security code to use UniqueTypeName
* change subchannel data producer API to use UniqueTypeName
* sanitize
* add missing build dep
* fix credentials_test
* fix certificate_provider_store_test
* fix tls_security_connector_test
* attempt to fix windows build
* avoid unnecessary allocation
* work around MSVC 2017 bug
* sanity
* change factory to not be templated
* fix sanity
* fix bug in chttp2 connector that used server creds instead of channel creds
* add missing build dep
* simplify API
* Initial skeleton for outlier detection
* fixing code review comments (modifying child policy)
* Skeleton and all tests passing except for 1
* small code review comments fix
* Adding the parsing of the policy in cds and put it in discovery
mechansim json format
* Parsing outlier detection json policy from parent
* Adding parsing of the updates
* Added Subchannel wrapper and watcher wrapper: and all states pass
through and all tests still pass
* added framework to do eject and uneject
* fixing code review comments
* restore a test
* fixing code review comments
* taking care of code review comments
* removing debug code and rebuild build files
* fixing according to code review comments
* fixing code review comments
* Adding address to subchannel map
* addressing code review comments
* adding call counter
* Refcount SubchannelState (in the map) and store them in Subcahnnel Wrapper
* fixing counterss
* Call counter and tracker skleton added
* Call counter
* addressing code review comments
* addressing code review comments
* Added CallCounter and timer
* fixing sanity; but more importantly: taking out timer temporarly as it
was causing test failures.
* sanity
* fixing according to code review comments
* addressing code review comments
* all algorithms implemented
* addressing code review comment about starting the timer
* protect private vars
* small fix
* Added one more corner case
* fixing EjectionTimer
* Fixing according to code review suggestions.
* fixing according to code reveiw comments
* taking care of code review comments
* fixing sanity issues
* Adding proto to tests
* First test
* Fixing according to code review comments
* Tests all working now
* fixing a crash
* fixing build files
* fixing sanity
* sanity
* Simplifying tests
* merge and update
* format
* sanity and format
* Fixing asan error
* fixing parsing logic and error handling
* 6 more tests done
* Added verifying unejection to tests
* Added all the tests
* fixing according to code review comments
* fixing asan and ubsan
* Fixing tests according to code review comments
* Added both algorithm tests
* added percentage enforcement tests
* fixing tsan error
* keeping debug, but fix warning
* remove debugs
* fixing IWYU and build errors after
* test comments change only but very important
* fixing code review comments
* one more refactorying of util function
* Removed debugs and added one more helper method
* one more logic fix
* Fixing last bit of code review comments and added disable tests
* fixing code review comments
* fixing IWYU
* sanity format
* protecting the feature with environment var:
registering policy and generating policy
* added a todo according to code review comments
* fixing a clang finding at import time
* build fix after synching to latest
* subchannel: report IDLE upon existing connection failure and after backoff interval
* rename AttemptToConnect() to RequestConnection()
* clang-format
* fix unused parameter warning
* fix subchannel to handle either TF or SHUTDOWN from transport
* fix handling of ConnectedSubchannel failure
* pass status up in IDLE state to communicate keepalive info
* update comment
* split pick_first and round_robin tests into their own test suites
* improve log message
* add test
* clang-format
* appease clang-tidy
* fix test to do a poor man's graceful shutdown to avoid spurious RPC failures
* simplify round_robin logic and fix test flakes
* fix grpclb bug
* WIP: add OOB backend metric API for LB policies
* fix some includes
* minor fixes
* picking this up again...
* more WIP
* health checking: cancel stream if response message fails to parse
* basic structure in place, but still have synchronization issues to address
* ORCA: implement ORCA RPC service for OOB backend metric reporting
* fix unused parameter error
* gen_upb_api
* add missing build deps
* increase test timing fudge factor
* add missing copyright header
* fix build and locking problems
* clang-format
* document API
* buildifier
* add test, but doesn't build yet
* new test working, but broke existing test, and need to fix server API
* don't register as a generic service
* update test for new orca service registration API
* fix build
* sanitize
* report interval defaults to min interval
* add channel trace event on UNIMPLEMENTED
* don't regenerate the response proto unless something changed
* add missing build dep
* fix comment
* move some code around
* remove num_backends parameter from XdsEnd2endTest
* remove use_xds_enabled_server param from XdsEnd2endTest
* remove xds_resource_does_not_exist_timeout_ms param from XdsEnd2endTest
* remove client_load_reporting_interval_seconds param from XdsEnd2endTest
* start moving CreateAndStartBackends() into individual tests
* finish moving CreateAndStartBackends() into individual tests
* remove unused variable
* remove SetEdsResourceWithDelay
* fix test flake
* clang-tidy
* clang-format
* move test framework to its own library
* fix build
* clang-format
* fix windows build
* rename TestType to XdsTestType
* move BackendServiceImpl inside of BackendServerThread
* clang-format
* move AdminServerThread to CSDS test suite
* remove unnecessary deps
* move aggregate and logical_dns cluster tests to their own file
* split aggregate and logical_dns tests into separate suites
* clang-format
* re-add flaky tag
* clang-tidy and remove unnecessary dep
* move some code around
* remove num_backends parameter from XdsEnd2endTest
* remove use_xds_enabled_server param from XdsEnd2endTest
* remove xds_resource_does_not_exist_timeout_ms param from XdsEnd2endTest
* remove client_load_reporting_interval_seconds param from XdsEnd2endTest
* start moving CreateAndStartBackends() into individual tests
* finish moving CreateAndStartBackends() into individual tests
* remove unused variable
* remove SetEdsResourceWithDelay
* fix test flake
* clang-tidy
* clang-format
* move test framework to its own library
* fix build
* clang-format
* fix windows build
* rename TestType to XdsTestType
* move BackendServiceImpl inside of BackendServerThread
* clang-format
* move AdminServerThread to CSDS test suite
* move ring_hash tests to their own file
* generate_projects
* remove unnecessary deps
* re-add flaky tag
* clang-format
* refactor connection delay injection from client_lb_end2end_test
* fix build
* fix build on older compilers
* clang-format
* buildifier
* a bit of code cleanup
* start failover time whenever the child reports CONNECTING, and don't cancel when deactivating
* clang-format
* rewrite test
* simplify logic in priority policy
* clang-format
* switch to using a bit to indicate child healthiness
* fix reversed comment
* more changes in priority and ring_hash.
priority:
- go back to starting failover timer upon CONNECTING, but only if seen
READY or IDLE more recently than TRANSIENT_FAILURE
ring_hash:
- don't flap back and forth between IDLE and CONNECTING; once we go
CONNECTING, we stay there until either TF or READY
- after the first subchannel goes TF, we proactively start another
subchannel connecting, just like we do after a second subchannel
reports TF, to ensure that we don't stay in CONNECTING indefinitely if
we aren't getting any new picks
- always return ring hash's picker, regardless of connectivity state
- update the subchannel connectivity state seen by the picker upon
subchannel list creation
- start proactive subchannel connection attempt upon subchannel list
creation if needed
* ring_hash: fix connectivity state seen by aggregation and picker
* fix obiwan error
* swap the order of ring_hash aggregation rules 3 and 4
* restore original test
* refactor connection injector QueuedAttempt code
* add test showing that ring_hash will continue connecting without picks
* clang-format
* don't actually need seen_failure_since_ready_ anymore
* fix TSAN problem
* address code review comments
* move some code around
* remove num_backends parameter from XdsEnd2endTest
* remove use_xds_enabled_server param from XdsEnd2endTest
* remove xds_resource_does_not_exist_timeout_ms param from XdsEnd2endTest
* remove client_load_reporting_interval_seconds param from XdsEnd2endTest
* start moving CreateAndStartBackends() into individual tests
* finish moving CreateAndStartBackends() into individual tests
* remove unused variable
* remove SetEdsResourceWithDelay
* fix test flake
* clang-tidy
* clang-format
* move test framework to its own library
* fix build
* clang-format
* fix windows build
* rename TestType to XdsTestType
* move BackendServiceImpl inside of BackendServerThread
* clang-format
* move AdminServerThread to CSDS test suite
* move RLS tests to their own file
* remove unnecessary deps
* generate_projects
* Fixes a flake with the LoadReporter end2end test.
I *believe* the test is wrong, based on the .proto description of the
LoadReporter.
The protocol described in src/proto/grpc/lb/v1/load_reporter.proto has
the ReportLoad rpc returns a stream of LoadReportResponse, which itself
has a repeated field of Load messages. The comment before it states:
"It is not strictly necessary to aggregate all entries into one entry
per <tag, user_id> tuple, although it is preferred to do so."
Debugging the issue shows we are in fact properly getting all 3 expected
load report types, just in two separate messages instead of a single
one.
This new test codepath will coalesce the load report responses, and also
addresses the fact the original test wasn't verifying that we were
getting the 3 expected types.
* Automated change: Fix sanity tests
* Renaming variables.
* ASSERT_ -> EXPECT_
* Automated change: Fix sanity tests
* move some code around
* remove num_backends parameter from XdsEnd2endTest
* remove use_xds_enabled_server param from XdsEnd2endTest
* remove xds_resource_does_not_exist_timeout_ms param from XdsEnd2endTest
* remove client_load_reporting_interval_seconds param from XdsEnd2endTest
* start moving CreateAndStartBackends() into individual tests
* finish moving CreateAndStartBackends() into individual tests
* remove unused variable
* remove SetEdsResourceWithDelay
* fix test flake
* clang-tidy
* clang-format
* move test framework to its own library
* fix build
* clang-format
* fix windows build
* move fault injection tests to their own file
* rename TestType to XdsTestType
* move BackendServiceImpl inside of BackendServerThread
* clang-format
* generate_projects
* appease clang-tidy
* move AdminServerThread to CSDS test suite
* remove unnecessary deps
* generate_projects
* don't mark test as flaky
* Maybe fix for PUT deprecation
* Guard PUT request accepting with a flag and add tests
* Reviewer comments
* Add fallthrough notation
* Reviewer comments
Co-authored-by: Craig Tiller <ctiller@google.com>
* Revert "Revert "ORCA: implement ORCA RPC service for OOB backend metric reporting (#29215)" (#29351)"
This reverts commit 71b355624f.
* move ORCA service to its own BUILD rule
* ORCA: implement ORCA RPC service for OOB backend metric reporting
* fix unused parameter error
* gen_upb_api
* add missing build deps
* increase test timing fudge factor
* add missing copyright header
* buildifier
* don't register as a generic service
* report interval defaults to min interval
* don't regenerate the response proto unless something changed
* use INTERNAL for proto parsing failure
* use absl::Duration in public API
* Adding is_optional case to RLS
* integrated with the updated envoy data-plane
* Fixing an old bug and adding test
* Use the same plugin map for ignore
* Remove ignore set
* Fixed another test.
* addressing code review comments.
* clean up!
* Refactor end2end tests to exercise each EventEngine
* fix incorrect bazel_only exclusions
* Automated change: Fix sanity tests
* microbenchmark fix
* sanitize, fix iOS flub
* Automated change: Fix sanity tests
* iOS fix
* reviewer feedback
* first pass at excluding EventEngine test expansion
Also caught a few cases where we should not test pollers, but should
test all engines. And two cases where we likely shouldn't be testing
either product.
* end2end fuzzers to be fuzzed differently via EventEngine.
* sanitize
* reviewer feedback
* remove misleading comment
* reviewer feedback: comments
* EE test_init needs to play with our build system
* fix golden file test runner
Co-authored-by: drfloob <drfloob@users.noreply.github.com>
* eliminate unnecessary templates
* add xDS node ID to error messages generated by XdsClient
* change test to verify that node ID is present in status message
* clang-format
* xds_cluster_resolver: fix bug that caused us to incorrectly reuse priority child names
* add debug logging
* assign child policy names within each discovery mechanism, not across all mechanisms
* clang-format
* add tests
* don't need EDS service name or LOGICAL_DNS hostname in child name
* remove unnecessary code
* client channel: record call completion even if recv_trailing_metadata was not started
* add test
* add test for opencensus filter
* remove unnecessary code
* Initial structure for RLS
* Adding and building the proto to parse the Any proto for the plugins
* re-org
* Parsing the plugin
* Parsing more into json
* Parsed proto to json
* small cleanup
* Adding prefix
* Added new rls_experimental policy
* build files
* Fixing according to code review comments
* code review comments
* Adding sym changes
* adding action name check
* fixing code review comments.
* fixing unused var error
* clean up
* fixing code review comments
* fixing code review comments
* fixing according to code review comments.
* Remove unnecessary include
* small fix
* generate more, hard-code less
* Moving to using absl::variant
* absl::string_view and absl::variant of vector of std::string are not
playing nice together.
* fixed variant
* Using absl::variant now
* Checkint used plugins
* Refactor Parsing code and separating out Parsing of the plugin
* Fixing code review comments
* code review comments
* fixing code review comments.
* Addressing code review comments
* First end-to-end test
* generated build files
* commit generated files via tools/codegen/core/gen_upb_api.sh
* Fixing rls policy parsing tests
* Restore checks for the test server
* Refactor rls_server
* added keys to rls request
* fixing small logic error
* Complete the test using all the keys
* Separating out RLS test and rls_server thread
* sanity errors
* generated build files
* Complete the rest of the tests and sanity cleanup
* fixing code review comments: using upb_JsonEncode now!
* fixing code review comments
* fixing code review comments
* Fixing code review comments
* misisng fix
* simplifying tests
* simplify tests 2
* Linking in the correct proto for rls_config
* restore metadata check
* Add disable test
* Fixing RLS test and removing environment var that is no longer necessary
* Fixing "Wrong type" type of tests after json parsing change to accept
STRING for number
* adding json_encode.h/c to src/upb/gen_build_yaml.py and generate
necessary files.
* Fixing un-used var error
* fixing sanity errors
* Fixing the upb encoding buffer
* Fixing code review comments.
* Adding nack test for unkonwn plugin proto
* Last bit of code review comments
* fixing unused variable
Based on a handful of https://abseil.io/tips, it's generally advised to
only fully-qualify namespaces when in a `using` statement, or when it's
otherwise required for compilation. In all other cases, the general
recommendation is to not fully-qualify.
This change fixes most `grpc.*` namespace uses. There are potential
challenges in trying to make blanket changes to non-gRPC namespace uses,
such as `::testing`, since there is also a `grpc::testing` namespace.
Based on a handful of https://abseil.io/tips, it's generally advised to
only fully-qualify namespaces when in a `using` statement, or when it's
otherwise required for compilation. In all other cases, the general
recommendation is to not fully-qualify.
This change fixes most `grpc.*` namespace uses. There are potential
challenges in trying to make blanket changes to non-gRPC namespace uses,
such as `::testing`, since there is also a `grpc::testing` namespace.
* [issue #28771] Fix pick_first policy to clear selected_ when promoting
a pending subchannel list to the active subchannel list when all
subchannels have been attempted and are in an error state.
* address comments
* revert idle_filter
* Passing xds server object instead of just a string name
* Adding xds server to policy
* Refactor ToJson
* Using XdsServer for load reporting
* code review comments
* fixing code review comments
* Taking care of lifetime of the XdsServer key
* code review comments
* Fixing channel_state storage and re-run tests (1 assert hit)
* Checking for server in the bootstrap file
* Adding LRS test
* adding a bootstrap file ToJson and parse test
* fixing code review comments
* fixing code review comments.
* fixing test
* break out the federation lrs test
* Fixed last bit of code review comments
* fixing error message to be more precise
* Adding TLS Key export logic to core and c++ wrappers
* Adding and end2end cpp tls key export test and updating broken test due to interface changes
* regenerate projects
* updating tls key export core logic with addition of APIs to grpc_security.h
* undoing changes to tls_security_connector_test
* regenerate projects
* changing the logging format enum name as per GRFC comments
* regenerate projects
* removing some commented code
* updating changes as per review comments
* adding GRPCAPI annotations to functions defined in grpc_security.h
* regenerate projects
* fixed some code styling issues
* removing grpc_security.h include from tls_credentials_options.h
* updating files as per review comments
* minor fixes
* moving some code around
* removing key log format from tls session key log config and converting it to a simple string
* regenerate projects
* fixing mistakes in recent merge with master
* regenerate projects
* regenerate projects
* fixing some distrib and snity errors
* fixing formatting errors
* fixing more sanity checks and raising supported openssl versions to 1.1.1
* updating min supported openssl version to 1.1.1
* updating min supported openssl version in tls_key_export_test
* updating test to fix incorrect vector initialization
* updating as per latest comments
* fixing sanity checks
* addressing review comments
* fixing sanity checks
* fixed c++ comment style
* Automated change: Fix sanity tests
* fixing review comments
Co-authored-by: Vignesh2208 <Vignesh2208@users.noreply.github.com>
Eliminate slice interning, and structures in slices to support it.
Reduces grpc_slice_refcount from 40 bytes (+ a required 8 bytes elsewhere) to 16 bytes.
Removes a pointer dereference for every slice ref/unref.
Co-authored-by: ctiller <ctiller@users.noreply.github.com>
* Empty principals checks for authenticated connection
* fix sanity check
* clang-format
* principals:[] will result in ANY
* minor correction after merging
* clang-format
* formatting
* clang-format
* remove unnecessary header
* Remove unnecessary target from BUILD
* Fix XdsClient for multiple watchers on the same resource
* xds_end2end_test: Don't use XdsCredentials for XdsRbacNackTests
* Use separate response states for EDS and RDS resources
* Reviewer comments
* Reviewer comments
* Reviewer comments
* Remove blank link
* Reviewer comments
* RBAC Authorization Engine updates based on A41 proposal
* Fix headermatcher
* Support for metadata
* Maps host to :authority, special-case te and :method
* removes method hardcoding
* add factory methods
* minor formatting
* fix test
* formatting
* remove constructors
* fix api usage
* ReqServerName test in xds_end2end_test
* Initialize bool
* xDS Federation: bootstrap and xds_resolver changes
* code review fixes
* fixing code review comments
* fixing code review comments
* fixing code review comments
* code review comments
* fixing code review comments
* First very basic test to make sure parsing and reconstruction work as
expected.
* clean up
* fixing logic error about authority
* fixing resource type parsing
* fixing code review comments
* simplify parsing!
* Parsing method signature update
* fixing code review comments
* clean up
* working progress for the test with generated bootstrap
* reorg the bootstrap file
* fixing tests
* Adding more to test authorities
* Added a test and it passes
* addressing code review comments
* code review comments to make parser cleaner and more efficient
* Merge in authority prefixes
* fixing sanity error and xds boostrap test error
* small fix
* Release all tests that pass; reduce scope for DeadUpdate
* Updated test strcuture and how to pass in the index for balancers to be
used as xds server uri and authority xds server uri
* code review comments
* code review fixes
* code review comment
* Making test structure changes
* fixing code review comments
* fixing code review comments
* Fixing test regression
* fixing bootstrap tests
* cleanup files
* enabling localhost:xxx for xds server; updated server tests and will fix
one more NameExpected test with testsetup.
* Finally removing fake reolsver for xds server
* Fixing bootstrap tests
* Rewrite builder
* Fixing code review comments
* fixing code review comments
* Fixing all tests to use Setup again
* fixing small sanity error
* Found the source of xds server nack test faiure and fix added
* small code review fixes
* Remove fake resolver! YAY!
* Fixing according to code review comments
* Setup plugin in bootfile
* Added more tests.
* Adding server test
* fixing a regression
* regression
* sanity fix
* fixing code review
* fixing code review comments
* Re-combine SecurityNaming tests.
* Add Rds new resource type and new tests
* Added PercentEncode test
* fixing code review comments
* refactor test a bit more
* fixing code review comments
* fixing according to code review comments
* fixing code review comments
* fixing code review comments
* Revert "Revert "XdsClient: remove resource-type-specific methods from XdsClient API (#28231)" (#28301)"
This reverts commit 3e779b68fe.
* remove global resource type registry
* WIP
* introduce XdsResourceType API and change Listener parsing to use it
* converted RouteConfig parsing
* convert cluster and endpoint parsing
* cleanup
* clang-format
* attempt to work around compiler problems
* move XdsResourceType to its own file, and move endpoint code out of XdsApi
* move cluster parsing to its own file
* move route config parsing to its own file
* move listener parsing to its own file
* clang-format
* minor cleanup
* plumbed XdsResourceType throughout XdsClient
* a bit of cleanup
* more cleanup
* construct full resource names before calling XdsApi::CreateAdsRequest()
* remove some unneeded code
* clean up includes and have XdsResourceType initialize the upb symtab
* more cleanup of unnecessary code
* more cleanup
* update comment
* clang-format
* add missing virtual dtor
* fix build
* remove resource-type-specific methods from XdsClient API
* have each resource type register itself upon instantiation
* remove comment
* add missing virtual dtor
* clang-format
Vignesh Babu