Commit Graph

113 Commits

Author SHA1 Message Date
Craig Tiller db3daf567b
[api-fuzzer] Enable fuzzing over config vars (#32736)
Add the capability for api-fuzzer to fuzz over different config
variables, to enable us to spot incompatible configurations there
sooner.

---------

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2023-03-28 16:18:12 -07:00
Craig Tiller a363b6c001
[fuzzing] Implement endpoints for FuzzingEventEngine (#32689)
Implement listeners, connection, endpoints for `FuzzingEventEngine`.
Allows the fuzzer to select write sizes and delays, connection delays,
and port assignments.

I made a few modifications to the test suite to admit this event engine
to pass the client & server tests:
1. the test factories return shared_ptr<> to admit us to return the same
event engine for both the oracle and the implementation - necessary
because FuzzingEventEngine forms a closed world of addresses & ports.
2. removed the WaitForSingleOwner calls - these seem unnecessary, and we
don't ask our users to do this - tested existing linux tests 1000x
across debug, asan, tsan with this change

Additionally, the event engine overrides the global port picker logic so
that port assignments are made by the fuzzer too.

This PR is a step along a longer journey, and has some outstanding
brethren PR's, and some follow-up work:
* #32603 will convert all the core e2e tests into a more malleable form
* we'll then use #32667 to turn all of these into fuzzers
* finally we'll integrate this into that work and turn all core e2e
tests into fuzzers over timer & callback reorderings and io
size/spacings

---------

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2023-03-27 17:43:47 -07:00
Vignesh Babu 8cf04e9a54
[EventEngine] Modify iomgr to allow creation of event engine clients and client side endpoints (#31661)
* [WIP] EventEngine iomgr endpoint shims

* [WIP] EventEngine::Endpoint iomgr shims for the PosixEventEngine

* Util functions to help with posix engine listener implementation

* sanity

* update comments in posix_engine_listener_utils.h

* review comments

* iwyu

* revert prev commit

* iwyu

* update build

* update

* regenerate projects

* regenerate projects

* minor fixes

* update BUILD

* sanity

* update build

* regenerate projects

* fix unused parameter

* sanity

* update

* sanity

* regenerate_projects

* remove unused variable

* start

* update

* regenerate_projects

* sanity

* update

* fixes

* update

* regenerate_projects

* update

* fix sanity and msan failure

* more fixes

* build failure

* update

* fix

* sanity

* fixes

* update

* regenerate projects

* fix sanity

* review comments

* An EventEngine subclass to be implemented by all posix based event engines

* sanity

* comments

* update

* review comments

* re-word

* fix

* update

* review comments

* regenerate projects

* syntax fix

* add lock free event benchmark

* releasable mutex lock

* fix build isue

* update

* start

* regenerate projects

* update

* fix

* windows build

* update

* windows portability issue

* update

* update

* update

* update

* format

* update

* update

* update

* update

* update

* fix sanity

* regenerate projects

* update

* iwyu

* Fix resolved address length related bugs in tcp_socket_utils and listener_utils

* iwyu

* cleanup src/core/lib/event_engine/tcp_socket_utils.cc

* iwyu

* fix

* regenerate projects

* fix sanity

* re-write endpoint shim

* more re-write

* cleanup

* update

* review comments

* build issue

* more build issue fixes plus adding event_engine_trace

* even more build issue fixes

* iwyu

* add static_cast

* update

* remove redundant code

* update

* deduplicate

* iwyu

* Fix review comments and regenerate_projects

* sanity

* review comments

* fix include guards

Co-authored-by: AJ Heller <hork@google.com>
2023-01-20 19:47:14 -08:00
AJ Heller 6778bb3501
[EventEngine] Relandx3: WindowsEventEngine client (#32031)
* Reland x3: "WindowsEventEngine Client implementation (#31848)"

This reverts commit 2c4d55b3a1.

* fixes

* fix

* fix
2023-01-12 10:26:17 -08:00
AJ Heller 2c4d55b3a1
Revert "Revert "Revert "WindowsEventEngine Client implementation (#31848)" (#… (#31974)" (#32018)
This reverts commit aa83319695.
2023-01-04 11:09:56 -08:00
AJ Heller aa83319695
Revert "Revert "WindowsEventEngine Client implementation (#31848)" (#… (#31974)
* Revert "Revert "WindowsEventEngine Client implementation (#31848)" (#31965)"

This reverts commit 58ad7950a0.

* iwyu & visibility
2023-01-03 16:21:29 -08:00
AJ Heller 58ad7950a0
Revert "WindowsEventEngine Client implementation (#31848)" (#31965)
This reverts commit bc7e155e55.
2022-12-22 09:59:14 -08:00
AJ Heller bc7e155e55
WindowsEventEngine Client implementation (#31848)
* rename task-handle mutex

* rename TimerClosure

* tmp commit, building, not tested

* add test for client connection to a non-listening port

* fix posix EE tests

* re-fix windows test suite after posix compatibility

* (unfinished, backup): passing the suite's NonExistingListener client test

* remove redundant windows client test

* integrate IOCP worker loop

* initial commit of echo test tool; fixes

* move echo client to test_suite/tools; I do not yet like the setup, it's about time for a macro that generates all useful test/tool targets

* cleanup

* add --target arg to echo_client. requires URI

* Automated change: Fix sanity tests

* build fixes

* fix

* fix

* reviewer feedback

* warning fix

* delete logic on cancellation

* fix

* cancel connect deadlock; improved template code

* fix build failure with linux environments building windows targets

* fix

* fix

* no ++ for atomics

* remove the test changes, to be landed separately

* Automated change: Fix sanity tests

* remnants

Co-authored-by: drfloob <drfloob@users.noreply.github.com>
2022-12-20 17:04:54 -08:00
Cheng-Yu Chung 77b3972f95
Clean up the TODO item in build target `grpc_base` part `channel_args` (#31642)
* Clean up the TODO item in build target `grpc_base` part `channel_args`

* Update

* Update

* Update

* Automated change: Fix sanity tests

* update

* Automated change: Fix sanity tests
2022-11-23 16:01:34 -05:00
Cheng-Yu Chung 7e3ada34e6
Clean up the TODO item in build target `grpc_base` -- part `closure` (#31001)
* Clean up the TODO item in build target `grpc_base` part `closure`

* Update

* Update

* Revert "Update"

This reverts commit daf3118390.

* Update

* Update

* Automated change: Fix sanity tests
2022-11-18 18:35:22 -05:00
Craig Tiller eea5d11fdd
[build] Move exec_ctx headers out of grpc_base (#31603)
* [build] Move exec_ctx headers out of grpc_base

* Automated change: Fix sanity tests

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2022-11-13 22:04:29 -08:00
Craig Tiller 5b8f006ece
[build] Stop publishing iomgr/error.h from grpc_base (#31521)
* [build] Stop publishing iomgr/error.h from grpc_base

* fixes

* fix
2022-11-08 20:04:31 -08:00
Yash Tibrewal 5f6c357e74
Move some targets from //: to //src/core: (#31402)
* temp

* temp

* temp

* top BUILD builds

* everything builds

* buildify

* temp fix_build_deps change

* fix gpr_atm

* additional changes

* fix build files

* nofixdeps

* Generate projects

* json_args

* fix context

* Generate projects

* Sprinkle some avoid_dep tags

* Run fix_build_deps

* Remove additional avoid_dep

* Revert some targets

* grpc_mock_cel
2022-10-20 20:12:53 -07:00
Esun Kim 85794d4094
Deprecated grpc_error_std_string (#31267) 2022-10-07 10:16:18 -07:00
Mark D. Roth f2a377d35e
xDS cluster resource type: use ValidationErrors and add unit test (#31025)
* XdsBootstrap: move two more methods out of the interface

* Automated change: Fix sanity tests

* XdsClient: add unit test

* Automated change: Fix sanity tests

* fix memory leaks

* add helper method

* add unsubscription

* add test for multiple subscriptions

* clang-format

* fix build

* fix flakiness

* add checking for other node fields

* add v2 test

* add response builder

* add test for update from server

* add test for update containing only changed resources

* clang-format

* fix build

* add test for resource not existing upon subscription

* add test for stream closed by server

* add test for multiple watchers for the same resource

* add test for connection failure

* clang-format

* add test for resources wrapped in Resource wrapper message

* add test for resource validation failure

* add test for multiple invalid resources, and fix a case in XdsClient

* add test for validation failure for already-cached resource

* add test for server not resending resources after stream disconnect

* clang-format

* fix XdsClient to report channel errors to newly started watchers

* fix XdsClient to send cached errors/does-not-exists to newly started watchers

* fix watcher to ensure events arrive in the expected order

* fix tests

* clang-format

* add test for multiple resource types

* fix xds_cluster_e2e_test

* Automated change: Fix sanity tests

* cleanup

* add federation tests

* clang-format

* remove now-unnecessary XdsCertificateProviderPluginMapInterface

* code review comments

* simplify XdsResourceType::Decode() API

* XdsClient: add unit tests for XdsClusterResourceType

* add XdsClient with gRPC bootstrap config

* add LB policy tests

* started adding CertificateProvider tests

* update for recent API changes

* fix merge bugs

* xDS resource validation: identify extensions by type_url instead of name

* fix build

* migrate to ValidationErrors

* add xds_common_types_test

* finish TLS tests and add LRS tests

* move ScopedExperimentalEnvVar to its own library and remove redundant e2e tests

* add circuit breaking and outlier detection tests

* add validation to outlier detection LB policy parsing

* clang-format

* Automated change: Fix sanity tests

* fix signedness

* fix sanity

* fix sanity

* iwyu

* update code for XdsResourceTypeImpl changes

Co-authored-by: markdroth <markdroth@users.noreply.github.com>
2022-10-05 11:27:30 -07:00
Craig Tiller 9c98d67d7a
[c++14] absl::make_unique --> std::make_unique (#31159)
* [c++14] absl::make_unique --> std::make_unique

* fix

* fix
2022-09-29 06:45:17 -07:00
Craig Tiller bc4f98bb36
[build] Fixes for experiments, poll strategy (#31042)
* fix

* Automated change: Fix sanity tests

* fix

* fix

* fix

* fix

* fix

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2022-09-19 22:19:36 -07:00
Craig Tiller c2ab8c99bb
Revert "Revert "[c++] Move environment functions to C++ (#30937)" (#30986)" (#30988)
This reverts commit 96264e07b8.
2022-09-14 14:19:36 -07:00
Craig Tiller 96264e07b8
Revert "[c++] Move environment functions to C++ (#30937)" (#30986)
This reverts commit 74c0d6fe3f.
2022-09-14 11:53:55 -07:00
Craig Tiller 74c0d6fe3f
[c++] Move environment functions to C++ (#30937)
* [gprpp] Move env to C++

* move headers/impl

* Automated change: Fix sanity tests

* fix

* fix

* Automated change: Fix sanity tests

* Update http_proxy.cc

* fix

* fix

* rename

* fix merge

* fix

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2022-09-14 10:17:15 -07:00
AJ Heller bf1856d2cc
A modest split of `:gpr` for mpscq support (#30798)
* A modest split of `:gpr` for mpscq support

A full split of the `gpr` target into 25 separate targets is building,
but there are some hurdles to get over with respect to ODR violations
and public API support for the gpr library.

This PR splits off a small chunk of that work.

* Automated change: Fix sanity tests

Co-authored-by: drfloob <drfloob@users.noreply.github.com>
2022-09-09 10:30:17 -07:00
Craig Tiller 381976dc99
[windows] Handling for tchar type in C++ (#30845)
* [windows] Handling for tchar type in C++

* fix

* fix

* Automated change: Fix sanity tests

* fix

* fix

* Update tchar.cc

* Update tchar.h

* Update tchar.h

* Update env_windows.cc

* Update string_util_windows.cc

* Update tmpfile_windows.cc

* Update env_windows.cc

* Update env_windows.cc

* Update subprocess_windows.cc

* Automated change: Fix sanity tests

* fix

* Update subprocess_windows.cc

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2022-09-07 14:37:37 -07:00
Craig Tiller 535168431a
[lb_policy] Move to core configuration system (#30483)
* [load_balancing] Move interfaces to src/core/lib/load_balancing

* Automated change: Fix sanity tests

* progress

* progress

* fix

* missed file

* Automated change: Fix sanity tests

* update tests

* Automated change: Fix sanity tests

* Automated change: Fix sanity tests

* fix

* fix

* Automated change: Fix sanity tests

* fix

* fixes

* fix

* fix

* fix

* fix

* fix

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2022-08-27 08:35:40 -07:00
Cheng-Yu Chung 543b290203
Build Target Reorganization Part 1 (#30518)
* Clean up gpr_base

* Remove `avoid_dep` for target `gpr`

* Add alias `gpr_base` for `gpr` for compatibility

* Fix tool error

* Update

* Remove temporary `gpr_base`

* Remove unused `alt_gpr_base_legacy`
2022-08-11 16:58:08 -04:00
Craig Tiller 2cb58b9565
[build] Auto-deps for test/core/resource_quota (#30430)
* [build] Auto-deps for test/core/resource_quota

* fix

* fix

* Automated change: Fix sanity tests

* fixes

* Automated change: Fix sanity tests

* fix

* Automated change: Fix sanity tests

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2022-08-08 05:06:56 -07:00
Craig Tiller a5cab10d6f
[load_balancing] Move interfaces to src/core/lib/load_balancing (#30465)
* [load_balancing] Move interfaces to src/core/lib/load_balancing

* Automated change: Fix sanity tests

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2022-08-03 13:45:56 -07:00
Craig Tiller 05e6bf2e4a
[build] Add test/core/util, test/core/event_engine to auto build deps (#30364)
* [build] Add test/core/util, test/core/event_engine to auto build deps

* deal with suppressions

* fix suppressions

* Automated change: Fix sanity tests

* fix

* fix

* fix

* fix

* progress

* Automated change: Fix sanity tests

* fix

* fix

* fix

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2022-07-25 21:59:25 -07:00
Esun Kim c0ec55d2f1
gtestify test/core/util (#30195)
* Gtestify cmdline_test

* gtestify histogram_test

* Fix
2022-07-11 18:18:57 -07:00
AJ Heller adfd009d3a
Make stack_tracer more broadly usable (#29660)
These changes reduce the stack_tracer's dependencies, allowing it to be
added to other targets such as `gpr_base` without dependency cycles.

Why would I want to do this? printf debugging lock problems that only
occur on the CI's Mac VMs, which we cannot access.
2022-05-12 09:17:00 -07:00
AJ Heller 18a8f6aad9
Refactor end2end tests to exercise each EventEngine (#29202)
* Refactor end2end tests to exercise each EventEngine

* fix incorrect bazel_only exclusions

* Automated change: Fix sanity tests

* microbenchmark fix

* sanitize, fix iOS flub

* Automated change: Fix sanity tests

* iOS fix

* reviewer feedback

* first pass at excluding EventEngine test expansion

Also caught a few cases where we should not test pollers, but should
test all engines. And two cases where we likely shouldn't be testing
either product.

* end2end fuzzers to be fuzzed differently via EventEngine.

* sanitize

* reviewer feedback

* remove misleading comment

* reviewer feedback: comments

* EE test_init needs to play with our build system

* fix golden file test runner

Co-authored-by: drfloob <drfloob@users.noreply.github.com>
2022-03-30 15:43:05 -07:00
Craig Tiller ff87ca02b2
Revive test/core/memory_usage (#28931)
* Revert "Deprecate gpr_get/set_allocation_functions"

This reverts commit 81df105ac8.

* memory stats

* Hack up to get per-call numbers

* expunge memory counters

* buildgen

* Automated change: Fix sanity tests

* sanity

* fix

* Automated change: Fix sanity tests

* simplify

* better

* Automated change: Fix sanity tests

* Disable windows memory_usage

* disable mac

* disable mac

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2022-02-22 15:39:40 -08:00
Jan Tattermusch 997bec7f24
Revert "Fix some clang-tidy issue (#28679)" (#28699)
This reverts commit b762dce244.
2022-01-26 08:36:02 -08:00
Esun Kim b762dce244
Fix some clang-tidy issue (#28679)
* Fix clang-tidy

* Generated projects
2022-01-25 16:19:39 -08:00
apolcyn 2a4b7f25e6
testing: add a fake TCP and UDP server to test/core/util (extracted from #27883) (#28332)
* add a fake TCP and UDP server to test/core/util
2021-12-14 08:50:17 -08:00
Craig Tiller 0794973bbb
Reland user-agent metadata work (#28109)
* Revert "Revert "user-agent metadata trait, also: grpc_core::Slice is born (#27770)" (#28108)"

This reverts commit 89d08dad9d.

* will it blend

* will it blend

* will it blend

* lcnagfmt

* sanity

* will it blend

* sleep @ end

* will it blend

* Revert "sleep @ end"

This reverts commit d6bca8ed3d.

* review feedback

* review feedback
2021-11-17 19:04:29 -08:00
Jan Tattermusch 89d08dad9d
Revert "user-agent metadata trait, also: grpc_core::Slice is born (#27770)" (#28108)
This reverts commit 7a40f50cb3.
2021-11-16 06:52:52 -08:00
Craig Tiller 7a40f50cb3
user-agent metadata trait, also: grpc_core::Slice is born (#27770)
* new slice api

* storage-classes

* Automated change: Fix sanity tests

* tweaks

* refinement

* refinement

* compiles

* Automated change: Fix sanity tests

* better impl

* convert to gtest

* clean

* fixes

* tests

* Automated change: Fix sanity tests

* more-tests

* clarity

* comments

* comments

* fixes

* comment-updates

* fixes

* spam

* Automated change: Fix sanity tests

* move transport size into transport!

* mebbefix

* fix type

* x

* fix-merge

* review feedback

* review feedback

* Automated change: Fix sanity tests

* meh

* review feedback

* fix

* resolve compile issue

* Remove slice_refcount dependency on RefCounted

* update init for channel_data in http_client_filter

* Automated change: Fix sanity tests

* remove banned function

* fixes

* will it blend

* will it blend

* fix refcount init

* fix

* fix comment

* Fix typo in bloat script

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2021-11-15 19:16:20 -08:00
Craig Tiller a629c9a03e
Reland resource quota work (#28017)
* Check if memory owner available prior to polling it

The transport may drop the memory owner during its destruction sequence

* tcp_fix

* Revert "Revert "New resource quota integration (#27643)" (#28014)"

This reverts commit 0ea2c37263.

* clang-format

* fix-path

* fix
2021-11-15 08:23:51 -08:00
Craig Tiller 0ea2c37263
Revert "New resource quota integration (#27643)" (#28014)
This reverts commit 39f0877d25.
2021-11-11 14:26:19 -08:00
Craig Tiller 39f0877d25
New resource quota integration (#27643)
* new resource quota integration

* Automated change: Fix sanity tests

* fix

* fix

* fixes

* fixes

* fixes

* Automated change: Fix sanity tests

* fixes

* fixes

* Automated change: Fix sanity tests

* fixes

* fix

* fixes

* windows-fix

* fixes

* fixes

* fix

* fix-asan

* banned

* banned

* fixes

* clang-tidy-fix

* Automated change: Fix sanity tests

* fix-cronet

* review feedback

* review feedback

* Automated change: Fix sanity tests

* fixes

* bug fix

* fixes

* compile fix

* exclude megabyte size payloads from 1byte tests

* windows fix

* start moving ios

* keep moving windows

* Get windows compilation working.

* Automated change: Fix sanity tests

* better

* fixes

* remove slice buffer from memory_allocator.h

* Revert "remove slice buffer from memory_allocator.h"

This reverts commit 234a63b6e3.

* ugh

* #fixtests

* pthread tls fixes

* Automated change: Fix sanity tests

* fixfixfix

* xxx

* add reset

* review feedback

* fix

* fix

* fixes

* fix

* mac progress

* cpp-impl-of

* rename ptr

* Automated change: Fix sanity tests

* memory-owner-is-a-memory-allocator

* fixes

* fix

* fix from prod

* fix

* Fix issue leading to bad pointers being returned on Windows.

* Automated change: Fix sanity tests

* fix multislice bug

* argh

* hyrums law fixes

* hyrums law fixes

* clang-format

* hyrums law fixes

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2021-11-10 22:53:04 -08:00
Craig Tiller 75a54f55e3
Remove trickle benchmarks (#28000)
* Remove trickle benchmarks

* Automated change: Fix sanity tests

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2021-11-10 18:21:13 -08:00
Richard Belleville 468768865b
Make Buildifier Sanity Test Strict (#27807)
* Fix all lint errors in repo.

* Use strict buildifier by default

* Whoops. That file does not exist

* Attempt fix to buildifier invocation

* Add missing copyright
2021-11-03 14:57:04 -07:00
Craig Tiller d1e01ac139
Add a test for a (now-illegal) build file construct (#27602)
* Add a test for a (now-illegal) build file construct

* add to test suite!

* fix

* fix

* fix syntax
2021-10-06 07:43:59 -07:00
Craig Tiller 9b41852a5e
channel_init --> core configuration system (#27347)
* channel_init --> core configuration system

* Automated change: Fix sanity tests

* fixes

* Automated change: Fix sanity tests

* fixes

* fixes

* eliminate grpc_base_c

* fixes

* fix merge

* fix

* update visibility

* fixes

* feedback

* fix comment

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2021-09-27 15:51:39 -07:00
AJ Heller d10617edb5
Move resource_user ownership into chttp2 transport/server/connector v2 (#27032)
Reintroducing PR #26643, which was reverted in #27029

Fixed a memory leak and added a test that would have caught it (ASAN build): ca0c8c4
2021-08-18 17:04:00 -07:00
Craig Tiller f5d3ed2db1
Revert "Move resource_user ownership into chttp2 transport/server/connector (#26643)" (#27029)
This reverts commit d1935a65a1. Will be rolled forward with a fix.
2021-08-16 19:07:39 -07:00
AJ Heller d1935a65a1
Move resource_user ownership into chttp2 transport/server/connector (#26643) 2021-08-16 13:50:56 -07:00
Craig Tiller 0bd70a7e3e
Add Match/Overload abstractions (#26640)
* match/overload abstraction

* update projects

* match should really not accept mutable args

* typo

* tests

* usage comment

* mutable version

* build stuff

* clang-format

* add an escape hatch to avoid needing port_platform.h in files that do not need port_platform.h

* unused args

* Make it possible for a test to not depend on gpr

* add tests

* compile fix

* sepelling
2021-07-09 17:30:19 -07:00
Ashitha Santhosh e438fa609b
Separate connection specific args in EvaluaetArgs. (#25969) 2021-04-16 16:57:48 -07:00
Ashitha Santhosh 8489a6050b
Revert "Rbac Matcher implementation. (#25653)" (#25957)
This reverts commit 1fb4f715db.
2021-04-12 17:22:44 -07:00