Commit Graph

205 Commits

Author SHA1 Message Date
Mark D. Roth 9dd6a98ed6
e2e tests: update regex used for connection failure messages (#32325)
* e2e tests: update regex used for connection failure messages

* attempt to tweak regex syntax to work on mac
2023-02-08 14:31:07 -08:00
Yousuk Seung b98f527260
Revert "Revert "Revert "Revert "server: introduce ServerMetricRecorde… (#32301)
* Revert "Revert "Revert "Revert "server: introduce ServerMetricRecorder API and move per-call reporting from a C++ interceptor to a C-core filter (#32106)" (#32272)" (#32279)" (#32293)"

This reverts commit 1f960697c5.

* Do not create CallMetricRecorder if call is null.
2023-02-06 12:00:39 -08:00
Craig Tiller 1f960697c5
Revert "Revert "Revert "server: introduce ServerMetricRecorder API and move per-call reporting from a C++ interceptor to a C-core filter (#32106)" (#32272)" (#32279)" (#32293)
This reverts commit 4475e74c6a.
2023-02-03 13:18:23 -08:00
Yousuk Seung 4475e74c6a
Revert "Revert "server: introduce ServerMetricRecorder API and move per-call reporting from a C++ interceptor to a C-core filter (#32106)" (#32272)" (#32279)
* Revert "Revert "server: introduce ServerMetricRecorder API and move per-call reporting from a C++ interceptor to a C-core filter (#32106)" (#32272)"

This reverts commit deb1e25543.

* Fix by caching call metric recording stuff in async request

PR #32106 caused msan errors in some tests while de-referencing the
server object where async calls are active after the server is
destroyed. Instead cache the ServerMetricRecorder pointer.

* copyright headers fixed

* clang fixes.
2023-02-03 11:49:32 -08:00
Eugene Ostroukhov 72872fc29d
stateful session affinity: add e2e test for draining state (#32248) 2023-02-02 14:59:10 -08:00
Xuan Wang deb1e25543
Revert "server: introduce ServerMetricRecorder API and move per-call reporting from a C++ interceptor to a C-core filter (#32106)" (#32272)
This reverts commit c7f641da0d.
2023-02-02 09:58:29 -08:00
Yousuk Seung c7f641da0d
server: introduce ServerMetricRecorder API and move per-call reporting from a C++ interceptor to a C-core filter (#32106)
* backend metric sampling

* Comments addressed.

* More comments addressed.

* Pushing changes left behind locally.

* Removed empty lines

* Update OrcaService to use ServerMetricRecorder (no named metrics yet)

* Comments addressed.

* More comments addressed

* More comments addressed.

* Comments fixed

* Comments addressed.

* Test fixed

* make seq returned always up-to-date

* skip atomic load when not cached

* Fixed ABSL_GUARDED_BY

* Comments addressed except client_lb_end2end_test

* test updated

* Comments addressed

* BUILD fix.

* BackendMetricDataState moved to a separate header

* comments addressed

* Fixed clang and buildifier errors

* More sanity check errors fixed.

* Fixed xds tests

* Ran generate_projects.sh

* Comments addressed

* comments addressed.

* generate project

* Build fixed

* generate project

* sanity check errors fixed

* test fixed

* Backup poller period override moved to main()

* Also move cfstream override

* Clang fixes, sanitize

* generate_projects.sh

* portable print format fix

* Removed outdated comment
2023-02-01 14:54:57 -08:00
Mark D. Roth 75f4ee6e5e
xDS: fix bug when receiving an update for an aggregate cluster child (#32249) 2023-01-31 14:58:03 -08:00
Eugene Ostroukhov b168e9cc9d
stateful session affinity: add end2end tests (#32009)
Co-authored-by: Mark D. Roth <roth@google.com>
2023-01-31 10:10:36 -08:00
Mark D. Roth 1c4b138322
Revert "Revert "WRR: xDS LB policy support (#32067)" (#32180)" (#32182)
This reverts commit d7193a28c4.
2023-01-23 14:14:32 -08:00
Alisha Nanda d7193a28c4
Revert "WRR: xDS LB policy support (#32067)" (#32180)
This reverts commit 8a024963f5.
2023-01-23 09:16:11 -08:00
Mark D. Roth 8a024963f5
WRR: xDS LB policy support (#32067)
* WRR: port StaticStrideScheduler to OSS

* WIP

* Automated change: Fix sanity tests

* fix build

* remove unused aliases

* fix another type mismatch

* remove unnecessary include

* move benchmarks to their own file, and don't run it on windows

* Automated change: Fix sanity tests

* add OOB reporting

* generate_projects

* clang-format

* add config parser test

* clang-tidy and minimize lock contention

* add config defaults

* add oob_reporting_period config field and add basic test

* Automated change: Fix sanity tests

* fix test

* change test to use basic RR

* WIP: started exposing peer address to LB policy API

* first WRR test passing!

* small cleanup

* port RR fix to WRR

* test helper refactoring

* more test helper refactoring

* WIP: trying to fix test to have the right weights

* more WIP -- need to make pickers DualRefCounted

* fix timer ref handling and get tests working

* clang-format

* iwyu and generate_projects

* fix build

* add test for OOB reporting

* keep only READY subchannels in the picker

* add file missed in a previous commit

* fix sanity

* iwyu

* add weight expiration period

* add tests for weight update period and OOB reporting period

* Automated change: Fix sanity tests

* lower bound for timer interval

* consistently apply grpc_test_slowdown_factor()

* cache time in test

* add blackout_period tests

* avoid some unnecessary copies

* clang-format

* add field to config test

* simplify orca watcher tracking

* attempt to fix build

* iwyu

* generate_projects

* update xds proto dependency

* add xDS LB policy entry to registry

* add "_experimental" suffix to policy name

* update LB policy name and remove debug log

* add env var protection

* generate_projects

* gen_upb_api

* WRR: update tests to cover qps plumbing

* WIP

* Automated change: Fix sanity tests

* more WIP

* basic WRR e2e test working

* add OOB test

* add xDS WRR e2e test

* clang-format

* fix sanity

* ignore duplicate addresses

* Automated change: Fix sanity tests

* add new tracer to doc/environment_variables.md

* retain scheduler state across pickers

* Automated change: Fix sanity tests

* use separate mutexes for scheduler and timer

* sort addresses to avoid index churn

* remove fetch_sub for wrap around in RR case

Co-authored-by: markdroth <markdroth@users.noreply.github.com>
2023-01-23 08:08:52 -08:00
Yash Tibrewal aa11978541
Fix include guards for src/ and test/ directories (#32167)
* Fix include guards for src/ and test/ directories

* Sanity

* Update new files
2023-01-20 16:27:27 -08:00
Mark D. Roth 7a96bed837
xDS: fix WeightedClusters total weight handling (#32134)
* xDS: fix WeightedClusters total weight handling

* iwyu
2023-01-19 08:01:26 -08:00
Craig Tiller 67f364e23e
[cleanup] Eliminate usage of GRPC_ASSERT(false...); (#31757)
* crash function

* progress

* fix

* fix

* Automated change: Fix sanity tests

* fix

* fix

* fix

* fixes

* Automated change: Fix sanity tests

* fix

* Automated change: Fix sanity tests

* fix

* fix

* use cpp attr

* Automated change: Fix sanity tests

* fix

* fix

* fix

* fix

* fix

* fix

* add exclusion

* fix

* typo

* fix

* fmt

* Update tcp_socket_utils.cc

* Automated change: Fix sanity tests

* fix

* revert php changes

* Automated change: Fix sanity tests

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2023-01-11 08:50:32 -08:00
Mark D. Roth e5a37d59fa
xDS: don't NACK RouteConfig with a VirtualHost containing no valid routes (#32069) 2023-01-11 08:03:55 -08:00
Vignesh Babu 4806da6041
Fix flakiness in xds_cluster_end2end_test: XdsTest/CdsTest.CircuitBreaking/V3 (#32027)
* Fix flakiness in xds_cluster_end2end_test: XdsTest/CdsTest.CircuitBreaking/V3

* update comment and add sleep in one more instance
2023-01-05 15:46:54 -08:00
Mark D. Roth 5192021637
Second attempt: xDS stateful session affinity: add config plumbing (#31874)
* Revert "Revert "xDS stateful session affinity: add config plumbing (#31827)" (#31873)"

This reverts commit 4f15d3dcf9.

* fix build for compilers too dumb to recognize the full set of enum values
2022-12-13 15:47:24 -08:00
Richard Belleville 4f15d3dcf9
Revert "xDS stateful session affinity: add config plumbing (#31827)" (#31873)
This reverts commit f76948b12d.
2022-12-12 14:26:27 -08:00
Mark D. Roth f76948b12d
xDS stateful session affinity: add config plumbing (#31827)
* WIP

* sanitize

* add endpoint tests

* add plumbing through LB policies

* clang-tidy
2022-12-12 08:19:31 -08:00
Cheng-Yu Chung 77b3972f95
Clean up the TODO item in build target `grpc_base` part `channel_args` (#31642)
* Clean up the TODO item in build target `grpc_base` part `channel_args`

* Update

* Update

* Update

* Automated change: Fix sanity tests

* update

* Automated change: Fix sanity tests
2022-11-23 16:01:34 -05:00
Esun Kim f3fedae7c8
Added readability-redundant-string-cstr (#31707)
* Added readability-redundant-string-cstr

* Auto fix

* No need to wrap std::string
2022-11-18 17:12:46 -08:00
Mark D. Roth 589fee0530
xDS: fix crash on wrong listener type (both client and server side) (#31684)
* xds resolver: fix crash on wrong listener type

* fix same bug on server side

* fix clang-tidy and add requested TODO
2022-11-17 07:38:05 -08:00
Mark D. Roth 64589d7ff3
xds resolver: fix edge cases in interactions between LDS and RDS (#31668)
* xds resolver: fix edge cases in interactions between LDS and RDS

* improve SwitchFromInlineRouteConfigToRds test

* clang-tidy

* Automated change: Fix sanity tests

Co-authored-by: markdroth <markdroth@users.noreply.github.com>
2022-11-16 11:04:20 -08:00
Mark D. Roth df80a31199
Second attempt: xDS RouteConfig: use ValidationErrors and add unit test (#31658)
* Revert "Revert "xDS RouteConfig: use ValidationErrors and add unit test (#31418)" (#31653)"

This reverts commit 335b74fea7.

* use auto instead of explicitly naming the protobuf map type
2022-11-15 09:31:28 -08:00
Mark D. Roth 335b74fea7
Revert "xDS RouteConfig: use ValidationErrors and add unit test (#31418)" (#31653)
This reverts commit c72a67b6da.
2022-11-14 18:34:42 -08:00
Mark D. Roth c72a67b6da
xDS RouteConfig: use ValidationErrors and add unit test (#31418)
Specific changes:
- Use absl::variant<> for HashPolicy types.
- Pull validation of resulting LB policy configs out of ClusterSpecifierPlugin registry and into RouteConfig validation.
- Don't ignore missing Any fields in HTTP filter configs if is_optional is true, since is_optional should apply only to unsupported filter types, not to malformed resource protos.
- Use ExtractXdsExtension() for HttpConnectionManager config itself.
- Minor API improvements to StatusCodeSet to make it a bit more useful in tests.
- Changed StringMatcher to return the underlying error message when a regex fails to compile.
- Fix a bug whereby a ClusterSpecifierPlugin would not be returned if it was not used in a vhost, even if it was used in another vhost.
2022-11-14 15:17:53 -08:00
Esun Kim 250f9fc026
Revived google-readability-casting (#31607)
* Added google-readability-casting

* Fix

* Use brace init for constants

* more

* Fix

* Fix 2

* More fix

* Fixf
2022-11-11 16:44:37 -08:00
Mark D. Roth ddfa85f42b
xDS: fix error message when the request does not match any route (#31593)
* improve error message when there is no matching route

* clean up ConfigSelector API

* clang-format

* fix client_lb_end2end_test

* clang-format
2022-11-09 14:29:22 -08:00
Mark D. Roth 43c8cdd2e9
xDS: fix bug when a cluster specifier plugin was not used in all vhosts (#31583) 2022-11-09 14:23:29 -08:00
Esun Kim 530b702259
Update Clang 15 (#31518)
* clang-15

* iwyu

* run clang_format_code

* Clang-Tidy config

* Fix misc-unused-using-decls

* Fixed performance-inefficient-vector-operation

* Sanity
2022-11-04 09:08:03 -07:00
Mark D. Roth a0fb351023
xds_core_e2e_test: fix federation test flake (#31487) 2022-10-31 17:15:34 -07:00
Mark D. Roth 447a058837
xDS: remove deprecated C++ APIs in the experimental namespace (#31451) 2022-10-26 14:53:57 -07:00
Mark D. Roth ba5e477446
xds_end2end_test: fix test flake (#31468) 2022-10-26 11:05:19 -07:00
Mark D. Roth b9aca8db8a
Second attempt: xDS Listener: use ValidationErrors and add unit test (#31457)
* Revert "Revert "xDS Listener: use ValidationErrors and add unit test (#31351)" (#31456)"

This reverts commit ab3d62ae8f.

* Revert "Revert "xds_http_filters_test: fix includes for import (#31454)" (#31455)"

This reverts commit 32590d110a.

* fix import

* work around internal differences in Any API (string vs. cord)

* iwyu
2022-10-25 15:03:16 -07:00
Craig Tiller ab3d62ae8f
Revert "xDS Listener: use ValidationErrors and add unit test (#31351)" (#31456)
This reverts commit 7229bea2fd.
2022-10-25 12:22:18 -07:00
Mark D. Roth 7229bea2fd
xDS Listener: use ValidationErrors and add unit test (#31351)
Additional changes:
- Use absl::variant<> to differentiate between API listeners and TCP listeners.
- Use absl::variant<> to differentiate between an RDS reference and an inline RouteConfig.
- Use common xDS extension code for transport socket extensions in both Listener and Cluster resources.
- Improve xDS HTTP filter registration mechanism.
- Use ValidationErrors for HTTP filter config validation.
2022-10-25 10:01:00 -07:00
Mark D. Roth bd4b5d70c8
xDS: remove v2 support (#31327)
* xDS: remove v2 support

* remove unnecessary templating in fake xDS server code

* remove now-unnecessary workaround for clang bug

* clang-format

* fix unit tests

* iwyu

* remove now-unused v2 proto files

* fix from merge

* clang-format

* fix typo
2022-10-18 08:45:42 -07:00
Mark D. Roth 5d0d5fe1a7
xDS: implement xds_wrr_locality LB policy and return xDS LB config from XdsClient (#31160)
* XdsBootstrap: move two more methods out of the interface

* Automated change: Fix sanity tests

* XdsClient: add unit test

* Automated change: Fix sanity tests

* fix memory leaks

* add helper method

* add unsubscription

* add test for multiple subscriptions

* clang-format

* fix build

* fix flakiness

* add checking for other node fields

* add v2 test

* add response builder

* add test for update from server

* add test for update containing only changed resources

* clang-format

* fix build

* add test for resource not existing upon subscription

* add test for stream closed by server

* add test for multiple watchers for the same resource

* add test for connection failure

* clang-format

* add test for resources wrapped in Resource wrapper message

* add test for resource validation failure

* add test for multiple invalid resources, and fix a case in XdsClient

* add test for validation failure for already-cached resource

* add test for server not resending resources after stream disconnect

* clang-format

* fix XdsClient to report channel errors to newly started watchers

* fix XdsClient to send cached errors/does-not-exists to newly started watchers

* fix watcher to ensure events arrive in the expected order

* fix tests

* clang-format

* add test for multiple resource types

* fix xds_cluster_e2e_test

* Automated change: Fix sanity tests

* cleanup

* add federation tests

* clang-format

* remove now-unnecessary XdsCertificateProviderPluginMapInterface

* code review comments

* simplify XdsResourceType::Decode() API

* XdsClient: add unit tests for XdsClusterResourceType

* add XdsClient with gRPC bootstrap config

* add LB policy tests

* started adding CertificateProvider tests

* update for recent API changes

* fix merge bugs

* xDS resource validation: identify extensions by type_url instead of name

* fix build

* migrate to ValidationErrors

* add xds_common_types_test

* finish TLS tests and add LRS tests

* move ScopedExperimentalEnvVar to its own library and remove redundant e2e tests

* add circuit breaking and outlier detection tests

* add validation to outlier detection LB policy parsing

* clang-format

* Automated change: Fix sanity tests

* fix signedness

* fix sanity

* xDS: implement xds_wrr_locality LB policy and return xDS LB config from XdsClient

* fix unused parameter

* fix sanity

* fix test

* Automated change: Fix sanity tests

* fix aggregate cluster bug

* Automated change: Fix sanity tests

* absl::make_unique -> std::make_unique

* fix sanity

* fix sanity

* iwyu

* iwyu

* update code for XdsResourceTypeImpl changes

Co-authored-by: markdroth <markdroth@users.noreply.github.com>
2022-10-17 11:22:43 -07:00
Mark D. Roth 93433d2faf
xDS RBAC: remove env var protection (#31348)
* xDS RBAC: remove env var protection

* fix federation server test

* fix sanity
2022-10-17 08:50:05 -07:00
Mark D. Roth 14c914dee0
xDS: refactor code for handling xDS extensions and add tests (#31295)
* general-purpose utility for validation xDS extensions

* plumb XdsExtension into HTTP filters and ClusterSpecifiers

* clang-format

* fix tests

* iwyu

* fix build with old compilers
2022-10-13 05:16:34 -07:00
Mark D. Roth 3475489bf3
xDS: NACK EDS resources with duplicate addresses (#31321)
* xDS: NACK EDS resources with duplicate addresses

* iwyu

* update e2e tests

* Automated change: Fix sanity tests

Co-authored-by: markdroth <markdroth@users.noreply.github.com>
2022-10-12 11:45:11 -07:00
Mark D. Roth ede4e42c7d
weighted_target LB: use uint64_t for aggregate weights to avoid overflow (#31244)
* weighted_target LB: use uint64_t for aggregate weights to avoid overflow

* iwyu

* fix undefined behavior

* iwyu

* iwyu again

* fix test weights to sum to uint32 max
2022-10-10 10:29:01 -07:00
Vignesh Babu 8f1004f435
Increase timeout in xds_cluster_end2end_test (#31260)
* Add testonly attribute to mock_event_engine

* Increase rpc timeout in xds_cluster_end2end_test.cc
2022-10-06 13:25:26 -07:00
Mark D. Roth 11fbb63279
outlier detection: remove env var protection (#31251)
* outlier detection: remove env var protection

* fix sanity
2022-10-06 11:26:29 -07:00
Mark D. Roth f2a377d35e
xDS cluster resource type: use ValidationErrors and add unit test (#31025)
* XdsBootstrap: move two more methods out of the interface

* Automated change: Fix sanity tests

* XdsClient: add unit test

* Automated change: Fix sanity tests

* fix memory leaks

* add helper method

* add unsubscription

* add test for multiple subscriptions

* clang-format

* fix build

* fix flakiness

* add checking for other node fields

* add v2 test

* add response builder

* add test for update from server

* add test for update containing only changed resources

* clang-format

* fix build

* add test for resource not existing upon subscription

* add test for stream closed by server

* add test for multiple watchers for the same resource

* add test for connection failure

* clang-format

* add test for resources wrapped in Resource wrapper message

* add test for resource validation failure

* add test for multiple invalid resources, and fix a case in XdsClient

* add test for validation failure for already-cached resource

* add test for server not resending resources after stream disconnect

* clang-format

* fix XdsClient to report channel errors to newly started watchers

* fix XdsClient to send cached errors/does-not-exists to newly started watchers

* fix watcher to ensure events arrive in the expected order

* fix tests

* clang-format

* add test for multiple resource types

* fix xds_cluster_e2e_test

* Automated change: Fix sanity tests

* cleanup

* add federation tests

* clang-format

* remove now-unnecessary XdsCertificateProviderPluginMapInterface

* code review comments

* simplify XdsResourceType::Decode() API

* XdsClient: add unit tests for XdsClusterResourceType

* add XdsClient with gRPC bootstrap config

* add LB policy tests

* started adding CertificateProvider tests

* update for recent API changes

* fix merge bugs

* xDS resource validation: identify extensions by type_url instead of name

* fix build

* migrate to ValidationErrors

* add xds_common_types_test

* finish TLS tests and add LRS tests

* move ScopedExperimentalEnvVar to its own library and remove redundant e2e tests

* add circuit breaking and outlier detection tests

* add validation to outlier detection LB policy parsing

* clang-format

* Automated change: Fix sanity tests

* fix signedness

* fix sanity

* fix sanity

* iwyu

* update code for XdsResourceTypeImpl changes

Co-authored-by: markdroth <markdroth@users.noreply.github.com>
2022-10-05 11:27:30 -07:00
Esun Kim 3c96517fc0
Simplified GRPC_ERROR_CREATE macros (#31222)
* Simplified GRPC_ERROR_CREATE macros

* Code migration
2022-10-04 15:55:47 -07:00
Craig Tiller a23eae84c9
Revert "[flakes] Update flakiness (#31121)" (#31173)
This reverts commit bdc799af73.
2022-09-29 14:06:44 -07:00
Craig Tiller bdc799af73
[flakes] Update flakiness (#31121)
* update

* Automated change: Fix sanity tests

* fix

* Automated change: Fix sanity tests

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2022-09-29 11:04:22 -07:00
Craig Tiller 9c98d67d7a
[c++14] absl::make_unique --> std::make_unique (#31159)
* [c++14] absl::make_unique --> std::make_unique

* fix

* fix
2022-09-29 06:45:17 -07:00