Commit Graph

5930 Commits

Author SHA1 Message Date
Cheng-Yu Chung cc58369279
Remove `include/grpcpp/impl/codegen/completion_queue.h` (#31059)
* Remove `include/grpcpp/impl/codegen/completion_queue.h`

* Update

* Update

* Fix build error

* Update
2022-10-10 11:35:09 -04:00
Craig Tiller 20d1efc38a
[stats] Move core stats to C++ (#30936)
* begin c++

* Automated change: Fix sanity tests

* progress

* progress

* missing-files

* Automated change: Fix sanity tests

* moved-from-stats

* remove old benchmark cruft, get tests compiling

* iwyu

* Automated change: Fix sanity tests

* fix

* fix

* fixes

* fixes

* add needed constructor

* Automated change: Fix sanity tests

* iwyu

* fix

* fix?

* fix

* fix

* Remove ResetDefaultEventEngine

Now that it is a weak_ptr, there's no need to explicitly reset it. When
the tracked shared_ptr is deleted, the weak_ptr will fail to lock, and a
new default EventEngine will be created.

* forget existing engine with FactoryReset

* add visibility

* fix

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
Co-authored-by: AJ Heller <hork@google.com>
2022-10-09 21:22:08 -07:00
Craig Tiller afa3a6c890
[event_engine] Fix for the EE/iomgr shutdown ordering problem (#31265)
* Remove ResetDefaultEventEngine

Now that it is a weak_ptr, there's no need to explicitly reset it. When
the tracked shared_ptr is deleted, the weak_ptr will fail to lock, and a
new default EventEngine will be created.

* forget existing engine with FactoryReset

* init/shutdown in event engine for now

* fix

* fix

* fix windows deadlock

* Automated change: Fix sanity tests

* fix

* better windows fix

Co-authored-by: AJ Heller <hork@google.com>
Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2022-10-07 21:35:49 -07:00
Vignesh Babu 8f1004f435
Increase timeout in xds_cluster_end2end_test (#31260)
* Add testonly attribute to mock_event_engine

* Increase rpc timeout in xds_cluster_end2end_test.cc
2022-10-06 13:25:26 -07:00
Cheng-Yu Chung d76c6f61f0
Remove `include/grpcpp/impl/codegen/method_handler.h` (#31249) 2022-10-06 15:50:54 -04:00
Mark D. Roth 11fbb63279
outlier detection: remove env var protection (#31251)
* outlier detection: remove env var protection

* fix sanity
2022-10-06 11:26:29 -07:00
Cheng-Yu Chung 488f17eb46
Remove `include/grpcpp/impl/codegen/message_allocator.h` (#31246) 2022-10-06 13:18:24 -04:00
Cheng-Yu Chung a192f155b9
Remove `include/grpcpp/impl/codegen/config.h` (#31061)
* Remove `include/grpcpp/impl/codegen/config.h`

* Update
2022-10-06 13:14:16 -04:00
Yash Tibrewal 8cffd1044e
OpenCensus: Ability to globally disable stats and tracing (#31252)
* OpenCensus: Ability to globally disable stats and tracing

* Fix docs
2022-10-05 14:38:15 -07:00
Mark D. Roth f2a377d35e
xDS cluster resource type: use ValidationErrors and add unit test (#31025)
* XdsBootstrap: move two more methods out of the interface

* Automated change: Fix sanity tests

* XdsClient: add unit test

* Automated change: Fix sanity tests

* fix memory leaks

* add helper method

* add unsubscription

* add test for multiple subscriptions

* clang-format

* fix build

* fix flakiness

* add checking for other node fields

* add v2 test

* add response builder

* add test for update from server

* add test for update containing only changed resources

* clang-format

* fix build

* add test for resource not existing upon subscription

* add test for stream closed by server

* add test for multiple watchers for the same resource

* add test for connection failure

* clang-format

* add test for resources wrapped in Resource wrapper message

* add test for resource validation failure

* add test for multiple invalid resources, and fix a case in XdsClient

* add test for validation failure for already-cached resource

* add test for server not resending resources after stream disconnect

* clang-format

* fix XdsClient to report channel errors to newly started watchers

* fix XdsClient to send cached errors/does-not-exists to newly started watchers

* fix watcher to ensure events arrive in the expected order

* fix tests

* clang-format

* add test for multiple resource types

* fix xds_cluster_e2e_test

* Automated change: Fix sanity tests

* cleanup

* add federation tests

* clang-format

* remove now-unnecessary XdsCertificateProviderPluginMapInterface

* code review comments

* simplify XdsResourceType::Decode() API

* XdsClient: add unit tests for XdsClusterResourceType

* add XdsClient with gRPC bootstrap config

* add LB policy tests

* started adding CertificateProvider tests

* update for recent API changes

* fix merge bugs

* xDS resource validation: identify extensions by type_url instead of name

* fix build

* migrate to ValidationErrors

* add xds_common_types_test

* finish TLS tests and add LRS tests

* move ScopedExperimentalEnvVar to its own library and remove redundant e2e tests

* add circuit breaking and outlier detection tests

* add validation to outlier detection LB policy parsing

* clang-format

* Automated change: Fix sanity tests

* fix signedness

* fix sanity

* fix sanity

* iwyu

* update code for XdsResourceTypeImpl changes

Co-authored-by: markdroth <markdroth@users.noreply.github.com>
2022-10-05 11:27:30 -07:00
Mark D. Roth 126393c9c1
client_lb_end2end_test: update failure regex to include "Socket closed" (#31242) 2022-10-05 10:09:17 -07:00
Craig Tiller 2d00d50c59
[event_engine] Improve scaling in threadpool (#31234)
* fixes

* Automated change: Fix sanity tests

* fix

* fix

* fix

* fix

* fixes

* fixes

* fix

* fix-win

* fix iwyu

* fix

* fix

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2022-10-05 00:17:23 -07:00
Esun Kim 3c96517fc0
Simplified GRPC_ERROR_CREATE macros (#31222)
* Simplified GRPC_ERROR_CREATE macros

* Code migration
2022-10-04 15:55:47 -07:00
Cheng-Yu Chung 4ba0f9afb5
Revert "Revert "Remove `include/grpcpp/impl/codegen/client_callback.h` (#31005)" (#31079)" (#31087)
This reverts commit da08fe1d6f.
2022-10-04 13:59:32 -04:00
Cheng-Yu Chung 0e8a6515df
Remove `include/grpcpp/impl/codegen/client_context.h` (#31006)
* Remove `include/grpcpp/impl/codegen/client_context.h`

* Update

* Update
2022-10-03 18:01:22 -04:00
Esun Kim 347bf1a1f4
Removed GRPC_ERROR_[INT|STR] macros (#31195)
* Clean-up which

* Reformat
2022-10-03 11:33:56 -07:00
Esun Kim 4bd27c524a
More changes (#31194) 2022-09-30 17:10:01 -07:00
Craig Tiller a23eae84c9
Revert "[flakes] Update flakiness (#31121)" (#31173)
This reverts commit bdc799af73.
2022-09-29 14:06:44 -07:00
Craig Tiller bdc799af73
[flakes] Update flakiness (#31121)
* update

* Automated change: Fix sanity tests

* fix

* Automated change: Fix sanity tests

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2022-09-29 11:04:22 -07:00
Craig Tiller 9c98d67d7a
[c++14] absl::make_unique --> std::make_unique (#31159)
* [c++14] absl::make_unique --> std::make_unique

* fix

* fix
2022-09-29 06:45:17 -07:00
AJ Heller 2ee2c91c92
Reland x2: Make GetDefaultEventEngine return a shared_ptr (#30619)
* Reland x2: Make GetDefaultEventEngine return a shared_ptr

* remove thread leak from NativeDNSResolver

This is not going to work for resolvers that support cancellation.

* give resolvers bounded lifetimes

Some resolver own EventEngines. EventEngines cannot run off the end of
the process since they have unjoined threads (problematic in a small set
of environments). This gives resolvers bounded lifetimes, and allows
replacement of resolvers without ASAN issues of deleting resolvers in
active use (occurs in tests).

* fix

* fix windows

* fix surface init test

* fix

* sanitize

* use after move

* the test must wait for the callback to be destroyed

* windows fix: delete the resolver on iomgr shutdown, not before

* Make TimerManager threads non-joinable

On gRPC shutdown, any unjoined TimerManager threads will cause TSAN to
detect thread leaks. This fix resolves issues I saw in end2end test
shutdown in another PR, where a single timer manager thread was always
alive after the test ended.

The long-term solution is to integrate the new ThreadPool here, but this
unblocks me for now.

* backport fix

* fix

* shared_ptr<EventEngine> in EventEngine benchmarks
2022-09-28 08:39:56 -07:00
Esun Kim d43511f4af
Removed GRPC_ERROR_NONE (#31131) 2022-09-27 23:17:39 -07:00
AJ Heller a874b8f6ca
EventEngine::Run microbenchmarks (#30769)
* [WIP] EventEngine::Run microbenchmarks

* Add fanout impl and fix tracking of time spent doing work in threads

* tune down benchmarks; fix fanout counting logic.

* tune down closure fanout tests

* format

* odr

* reviewer feedback

* unify some fanout logic; add a large-AnyInvocable test

lambdas that take an allocation are about 10x slower

* reviewer feedback

* fix invalid vector access

* rm DNS

* format

* copy params for each lambda callback

This fixes segfaults when we cannot ensure all callbacks are complete
before exiting the test.

* s/promise/Notification/g bm_exec_ctx

* ODR and leak

* fix division by zero

* fix
2022-09-27 14:57:36 -07:00
Yash Tibrewal 6d249c0af2
Observability: Experimental arg to disable client side tracing (#31093)
* Observability: Experimental arg to disable client side tracing

* Fix IWYU

* Reviewer comments

* Reviewer comments

* Reviewer comment: Move experimental/internal arg to different file

* Fix build

* Fix IWYU
2022-09-27 14:34:07 -07:00
Esun Kim ba8af0157b
Removing GRPC_ERROR_[REF|UNREF|IS_NONE] (#31089)
* Removing GRPC_ERROR_[REF|UNREF|IS_NONE]

* Clean up unnecessary error

* Fix

* Fix2

* Remove unnecessary else
2022-09-26 15:41:12 -07:00
Mark D. Roth cdb7d2c93c
XdsEndpointResourceType: use ValidationErrors and add unit tests (#31076)
* simplify XdsResourceType::Decode() API

* fix xds_client_test

* xDS endpoint: use ValidationErrors to improve error messages

* fix sanity

* add xDS endpoint resource type test

* clang-format

* remove a couple of now-unnecessary end2end tests

* generate_projects

* fix xds_csds_e2e_test

* iwyu

* attempt to avoid ubsan failure
2022-09-26 11:45:56 -07:00
Craig Tiller ebc4f236b6
[slice] Remove _internal variants of APIs (#30953)
* [slice] Remove _internal variants of APIs

* Automated change: Fix sanity tests

* fix

* reduce bloat

* fixes

* Automated change: Fix sanity tests

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2022-09-24 13:34:00 -07:00
Craig Tiller 32b43f0168
[channel_args] Make channel_stack_init take new type (#30841)
* [channel_args] Make channel_stack_init take new type

* fix

* fix

* fix

* Automated change: Fix sanity tests

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2022-09-23 11:31:03 -07:00
Yousuk Seung 747d016397
Support --max_recv_msg_size in grpc cli (#31106)
* Support --max_recv_msg_size in grpc cli

* Comment typos fixed in tests.
2022-09-23 11:13:29 -07:00
AJ Heller fb14fdf0e0
Performant thread-safe Work Queue (#30821)
* WorkQueue

* weaken the large obj stress test for Windows; documentation

* update comment

* Add WorkQueue microbenchmark. Results below ...

------------------------------------------------------------------------------------------
Benchmark                                Time             CPU   Iterations UserCounters...
------------------------------------------------------------------------------------------
BM_WorkQueueIntptrPopFront/1           297 ns          297 ns      2343500 items_per_second=3.3679M/s
BM_WorkQueueIntptrPopFront/8          7022 ns         7020 ns        99356 items_per_second=1.13956M/s
BM_WorkQueueIntptrPopFront/64        59606 ns        59590 ns        11770 items_per_second=1074k/s
BM_WorkQueueIntptrPopFront/512      477867 ns       477748 ns         1469 items_per_second=1071.7k/s
BM_WorkQueueIntptrPopFront/4096    3815786 ns      3814925 ns          184 items_per_second=1073.68k/s
I0902 19:05:22.138022069      12 test_config.cc:194]         TestEnvironment ends
================================================================================

* use int64_t for times. 0 performance change

------------------------------------------------------------------------------------------
Benchmark                                Time             CPU   Iterations UserCounters...
------------------------------------------------------------------------------------------
BM_WorkQueueIntptrPopFront/1           277 ns          277 ns      2450292 items_per_second=3.60967M/s
BM_WorkQueueIntptrPopFront/8          6718 ns         6716 ns       105497 items_per_second=1.19126M/s
BM_WorkQueueIntptrPopFront/64        56428 ns        56401 ns        12268 items_per_second=1.13474M/s
BM_WorkQueueIntptrPopFront/512      458953 ns       458817 ns         1550 items_per_second=1.11591M/s
BM_WorkQueueIntptrPopFront/4096    3686357 ns      3685120 ns          191 items_per_second=1.1115M/s
I0902 19:25:31.549382949      12 test_config.cc:194]         TestEnvironment ends
================================================================================

* add PopBack tests: same performance profile exactly

* use Mutex instead of Spinlock

It's safer, and so far equally performant in benchmarks of opt builds

* add deque test for comparison. It is faster on all tests.

* Add sparsely-populated multi-threaded benchmarks.

* fix

* fix

* refactor to help thread safety analysis

* Specialize WorkQueue for Closure*s and AnyInvocables

* remove unused callback storage

* add single-threaded benchmark for closure vs invocable

* sanitize

* missing include

* move bm_work_queue to microbenchmarks so it isn't exported

* s/workqueue/work_queue/g

* use nullptr instead of optionals for popped closures

* reviewer test suggestion

* private things are private

* add a work_queue fuzzer

Ran for 10 minutes @ 42 jobs @ 42 workers. Zero failures.

Checked in a selection of 100 good seeds after merging the thousands of
results.

* fix

* fix header guards

* nuke the corpora

* feedback

* sanitize

* Timestamp::Now

* fix

* fuzzers do not work on windows

* windows does not like multithreaded benchmark tests
2022-09-23 09:10:42 -07:00
Craig Tiller f15ba1ffc7
[tls] Remove support for pthread tls (#31040)
* [tls] Remove support for pthread tls

* fix

* fix

* fix

* lalala

* fix

* fix

* Clean up deployment target changes

* additional clean up of deployment target

* regen podspec for updated file list

* remove destination change

* deployment target override for ios cpp test

* fix?

* fix run test script & update test destination

* [tls] Remove support for pthread tls

* fix

* fix

* fix

* lalala

* fix

* fix

* Clean up deployment target changes

* additional clean up of deployment target

* regen podspec for updated file list

* remove destination change

* deployment target override for ios cpp test

* fix?

* fix run test script & update test destination

* merge

* fix

* final script fix for proper destination & target

* more deployment target fix

* objc ios test script fix

Co-authored-by: Denny C. Dai <dennycd@me.com>
Co-authored-by: dennycd <dennycd@google.com>
2022-09-22 22:39:35 -07:00
Craig Tiller 1ad4bbe78e
Revert "fix (#31018)" (#31021)
This reverts commit 2e6f268dfc.
2022-09-21 09:28:28 -07:00
Craig Tiller da08fe1d6f
Revert "Remove `include/grpcpp/impl/codegen/client_callback.h` (#31005)" (#31079)
This reverts commit 17f515744f.
2022-09-20 18:58:54 -07:00
Yash Tibrewal 9cff4d2d28
OpenCensus Plugin: Add missing measure and views for started RPCs (#31034) 2022-09-20 17:26:16 -07:00
Mark D. Roth 07df5ff9c7
json_object_loader: refactor ErrorList into its own library (#31049)
* json_object_loader: refactor ErrorList into its own library

* fix observability_config_test

* generate_projects

* iwyu
2022-09-20 11:58:27 -07:00
Cheng-Yu Chung 0cb5faf8e4
Remove `include/grpcpp/impl/codegen/client_interceptor.h` (#31054)
* Remove `include/grpcpp/impl/codegen/client_interceptor.h`

* Update comment
2022-09-20 14:03:46 -04:00
Cheng-Yu Chung 17f515744f
Remove `include/grpcpp/impl/codegen/client_callback.h` (#31005) 2022-09-20 13:51:55 -04:00
Mark D. Roth e475e165a7
LB policies: convert to new JSON API (#30468)
* Declarative JSON parser

* Automated change: Fix sanity tests

* fix

* shrinking stuff a little

* static vtables

* separate fns

* simpler?

* make maps work

* windows fixes

* Automated change: Fix sanity tests

* simplify code

* Automated change: Fix sanity tests

* vtable-test

* dont always create vec/map impls for every type

* comments

* make error consistent

* move method private

* progress

* durations!

* Automated change: Fix sanity tests

* fix

* fix

* fix

* Automated change: Fix sanity tests

* post-load

* Automated change: Fix sanity tests

* document JsonPostLoad() and add static_assert

* don't copy field names, to avoid length limitations

* use absl::Status

* accept either string or number for numeric values

* add test for direct data member of another struct type

* remove unused method

* add support for retaining part of the JSON wirthout processing

* update test for changes in Json::Parse() API

* add absl::optional support

* Automated change: Fix sanity tests

* fix tests, improve error messages, and add overload to parse to existing object

* remove overload of LoadFromJson()

* change special case for Json to instead use Json::Object

* fix build

* improve error structure, add missing types, and improve tests

* clang-format

* Automated change: Fix sanity tests

* update grpclb parsing

* fix build

* convert outlier_detection LB config

* convert priority LB policy

* convert ring_hash LB config

* add LoadJsonObjectField(), add LoadFromJson() overload that takes an ErrorList parameter, and add tests for parsing bare top-level types

* fix msan

* Automated change: Fix sanity tests

* WIP on RLS parsing conversion

* fix error message

* Automated change: Fix sanity tests

* fixed RLS parser tests

* fix error prefix in OD, priority, and ring_hash policies

* convert weighted_target policy

* convert xds_cluster_manager

* convert cds policy

* convert xds_cluster_resolver

* convert xds_cluster_impl

* Automated change: Fix sanity tests

* fix test

* fix xds_cluster_impl drop config parsing

* Automated change: Fix sanity tests

* attempt to fix mac build

* work around gcc6 problem

* Automated change: Fix sanity tests

* fix build

* add mechanism to conditionally disable individual fields

* fix build

* Automated change: Fix sanity tests

* fix move assignment operator

* fix build

* Automated change: Fix sanity tests

* fix build and simplify RLS duplicate key check

* Automated change: Fix sanity tests

* Automated change: Fix sanity tests

* fix build

* iwyu

* fix build

* fix sanity

* add LoadRefCountedFromJson() and use it to eliminate some moves

* fix build

* Automated change: Fix sanity tests

Co-authored-by: Craig Tiller <craig.tiller@gmail.com>
Co-authored-by: ctiller <ctiller@users.noreply.github.com>
Co-authored-by: Craig Tiller <ctiller@google.com>
Co-authored-by: markdroth <markdroth@users.noreply.github.com>
2022-09-20 09:14:14 -07:00
Mark D. Roth 6d792b6bce
xDS resource validation: identify extensions by type_url instead of name (#31024)
* xDS resource validation: identify extensions by type_url instead of name

* fix build
2022-09-20 09:06:01 -07:00
Yash Tibrewal 5cd8ee25f8
GcpObservabilityConfig: New changes (#31038)
* GcpObservabilityConfig: New changes

* Fix IWYU
2022-09-19 15:29:46 -07:00
Yash Tibrewal 27e5b4e187
GcpObservabilityConfig: Fall-back to GCP environment variables (#31022) 2022-09-19 11:48:35 -07:00
Yash Tibrewal 6098202c09
GcpObservabilityTest: Cleanup (#31023) 2022-09-19 11:47:53 -07:00
Craig Tiller 2e6f268dfc
fix (#31018) 2022-09-16 12:41:08 -07:00
Craig Tiller d9ac89a441
[gprpp] absl::Notification polyfill (#31008)
* [gprpp] absl::Notification polyfill

* grpc_core::

* Automated change: Fix sanity tests

* fix

* Automated change: Fix sanity tests

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2022-09-16 09:06:03 -07:00
Craig Tiller 96f5cddb5f
[time] Introduce time sources (#30815)
* [time] Introduce time sources

* make import trivial

* Automated change: Fix sanity tests

* fix

* Automated change: Fix sanity tests

* fix

* Automated change: Fix sanity tests

* Automated change: Fix sanity tests

* review feedback

* fix

* Automated change: Fix sanity tests

* ios fix

* fix

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2022-09-15 21:41:04 -07:00
Craig Tiller c2ab8c99bb
Revert "Revert "[c++] Move environment functions to C++ (#30937)" (#30986)" (#30988)
This reverts commit 96264e07b8.
2022-09-14 14:19:36 -07:00
Craig Tiller 96264e07b8
Revert "[c++] Move environment functions to C++ (#30937)" (#30986)
This reverts commit 74c0d6fe3f.
2022-09-14 11:53:55 -07:00
Mark D. Roth e5aadf985e
Revert "Revert "client_channel: allow LB policy to communicate update errors to resolver (#30809)" (#30970)" (#30981)
This reverts commit 1648bc08e0.
2022-09-14 11:46:42 -07:00
Cheng-Yu Chung b765b06155
Remove `include/grpcpp/impl/codegen/async_stream.h` (#30941)
* Remove `include/grpcpp/impl/codegen/async_stream.h`

* Fix

* Fix

* Update
2022-09-14 13:40:35 -04:00
Craig Tiller 74c0d6fe3f
[c++] Move environment functions to C++ (#30937)
* [gprpp] Move env to C++

* move headers/impl

* Automated change: Fix sanity tests

* fix

* fix

* Automated change: Fix sanity tests

* Update http_proxy.cc

* fix

* fix

* rename

* fix merge

* fix

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2022-09-14 10:17:15 -07:00