Commit Graph

107 Commits

Author SHA1 Message Date
AJ Heller fb14fdf0e0
Performant thread-safe Work Queue (#30821)
* WorkQueue

* weaken the large obj stress test for Windows; documentation

* update comment

* Add WorkQueue microbenchmark. Results below ...

------------------------------------------------------------------------------------------
Benchmark                                Time             CPU   Iterations UserCounters...
------------------------------------------------------------------------------------------
BM_WorkQueueIntptrPopFront/1           297 ns          297 ns      2343500 items_per_second=3.3679M/s
BM_WorkQueueIntptrPopFront/8          7022 ns         7020 ns        99356 items_per_second=1.13956M/s
BM_WorkQueueIntptrPopFront/64        59606 ns        59590 ns        11770 items_per_second=1074k/s
BM_WorkQueueIntptrPopFront/512      477867 ns       477748 ns         1469 items_per_second=1071.7k/s
BM_WorkQueueIntptrPopFront/4096    3815786 ns      3814925 ns          184 items_per_second=1073.68k/s
I0902 19:05:22.138022069      12 test_config.cc:194]         TestEnvironment ends
================================================================================

* use int64_t for times. 0 performance change

------------------------------------------------------------------------------------------
Benchmark                                Time             CPU   Iterations UserCounters...
------------------------------------------------------------------------------------------
BM_WorkQueueIntptrPopFront/1           277 ns          277 ns      2450292 items_per_second=3.60967M/s
BM_WorkQueueIntptrPopFront/8          6718 ns         6716 ns       105497 items_per_second=1.19126M/s
BM_WorkQueueIntptrPopFront/64        56428 ns        56401 ns        12268 items_per_second=1.13474M/s
BM_WorkQueueIntptrPopFront/512      458953 ns       458817 ns         1550 items_per_second=1.11591M/s
BM_WorkQueueIntptrPopFront/4096    3686357 ns      3685120 ns          191 items_per_second=1.1115M/s
I0902 19:25:31.549382949      12 test_config.cc:194]         TestEnvironment ends
================================================================================

* add PopBack tests: same performance profile exactly

* use Mutex instead of Spinlock

It's safer, and so far equally performant in benchmarks of opt builds

* add deque test for comparison. It is faster on all tests.

* Add sparsely-populated multi-threaded benchmarks.

* fix

* fix

* refactor to help thread safety analysis

* Specialize WorkQueue for Closure*s and AnyInvocables

* remove unused callback storage

* add single-threaded benchmark for closure vs invocable

* sanitize

* missing include

* move bm_work_queue to microbenchmarks so it isn't exported

* s/workqueue/work_queue/g

* use nullptr instead of optionals for popped closures

* reviewer test suggestion

* private things are private

* add a work_queue fuzzer

Ran for 10 minutes @ 42 jobs @ 42 workers. Zero failures.

Checked in a selection of 100 good seeds after merging the thousands of
results.

* fix

* fix header guards

* nuke the corpora

* feedback

* sanitize

* Timestamp::Now

* fix

* fuzzers do not work on windows

* windows does not like multithreaded benchmark tests
2022-09-23 09:10:42 -07:00
Craig Tiller 92f58c18a8
Revert "Revert "[chttp2] Improve huffman decode efficiency (#30479)" (#30961)" (#30962)
This reverts commit bfea47093f.
2022-09-13 13:16:55 -07:00
Craig Tiller bfea47093f
Revert "[chttp2] Improve huffman decode efficiency (#30479)" (#30961)
This reverts commit 6c7f921f5f.
2022-09-13 12:25:00 -07:00
Craig Tiller 6c7f921f5f
[chttp2] Improve huffman decode efficiency (#30479)
* first pass

* refinement #1

* fix

* opt

* 8

* 5

* 12

* 10

* 7

* 15

* 8n

* 15n

* 8a

* 16a

* 15a

* 5a

* 10a

* 7a

* 9a

* 12a

* 11a

* 11b

* 11b

* 7b

* 15b

* 14b

* 15i

* 12i

* 9i

* 10i

* finalizing

* fix

* tweak

* 10f

* 10r

* 7r

* 8r

* 9r

* more

* rething

* 8s

* 9s

* 12s

* 7s

* 10s

* 15snr

* 12snr

* 14snr

* 13snr

* 11snr

* 20snr

* 15snr

* 18snr

* 7snr

* 7snr

* 12-2stp

* 13-2stp

* 14-2stp

* 11-2stp

* 10-2stp

* 9-3stp

* 8-3stp

* 7-3stp

* 8-2stp

* auto-tune1

* max-depth-2

* max-depth-3

* max-depth-2

* abbreviate

* working version

* add benchmark

* fix

* fix

* fix

* ditch define

* better code layout

* static fns

* fix

* Automated change: Fix sanity tests

* Automated change: Fix sanity tests

* fix build

* cleanup and comment

* fmt

* fix

* test

* tag

* Automated change: Fix sanity tests

* fix

* fix

* Automated change: Fix sanity tests

* fix

* fix

* fix

* progress

* progress

* progress

* fixes

* cleanup

* Automated change: Fix sanity tests

* Automated change: Fix sanity tests

* fmt

* fix

* fix

* Automated change: Fix sanity tests

* fix

* fix

* Automated change: Fix sanity tests

* fix

* add encode/decode fuzzer

* Automated change: Fix sanity tests

* fix

* fix

* fix

* review feedback

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2022-09-13 11:24:35 -07:00
Craig Tiller 622dd886e6
Update flaky tests list (#30771)
* Update flaky tests list

* Automated change: Fix sanity tests

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2022-08-30 21:42:59 -07:00
Craig Tiller f573fd44f9
[test] Semi-automatic flaky test tracking (#30638)
* [test] Semi-automatic flaky test tracking

* fix

* Automated change: Fix sanity tests

* Update update_flakes.py

* fix formatting

* Automated change: Fix sanity tests

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2022-08-19 16:22:31 -07:00
Craig Tiller 05e6bf2e4a
[build] Add test/core/util, test/core/event_engine to auto build deps (#30364)
* [build] Add test/core/util, test/core/event_engine to auto build deps

* deal with suppressions

* fix suppressions

* Automated change: Fix sanity tests

* fix

* fix

* fix

* fix

* progress

* Automated change: Fix sanity tests

* fix

* fix

* fix

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2022-07-25 21:59:25 -07:00
Craig Tiller f5bb60d8e8
[channel_args] Improve comparison performance (#30322)
* [channel_args] Improve comparison performance

* fix

* Automated change: Fix sanity tests

* faster still

* fix

* fix

* some no cost boosts if they get hit

* fix

* final tweak for pointers

* check

* fix

* Automated change: Fix sanity tests

* fix

* ugh

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2022-07-19 15:40:09 -07:00
Craig Tiller 1b5295a4a2
[iomgr] Remove executor/...
I've tried this before in #27445 and we found some internal usage of this code. Today I can find no such usage, so let's try again.
2022-07-08 12:07:36 -07:00
AJ Heller 18a8f6aad9
Refactor end2end tests to exercise each EventEngine (#29202)
* Refactor end2end tests to exercise each EventEngine

* fix incorrect bazel_only exclusions

* Automated change: Fix sanity tests

* microbenchmark fix

* sanitize, fix iOS flub

* Automated change: Fix sanity tests

* iOS fix

* reviewer feedback

* first pass at excluding EventEngine test expansion

Also caught a few cases where we should not test pollers, but should
test all engines. And two cases where we likely shouldn't be testing
either product.

* end2end fuzzers to be fuzzed differently via EventEngine.

* sanitize

* reviewer feedback

* remove misleading comment

* reviewer feedback: comments

* EE test_init needs to play with our build system

* fix golden file test runner

Co-authored-by: drfloob <drfloob@users.noreply.github.com>
2022-03-30 15:43:05 -07:00
Craig Tiller 5fc3ff8203
grpc_millis -> Timestamp/Duration (#28119)
* wip

* Automated change: Fix sanity tests

* fixes

* progress

* progress

* grpc compiles

* Automated change: Fix sanity tests

* fixing tests

* x

* progress

* better code

* Automated change: Fix sanity tests

* progress

* progress

* windows fix

* Make Duration metadata trivial

* better message

* fix

* Automated change: Fix sanity tests

* fix

* fix

* fix

* fix

* Automated change: Fix sanity tests

* Automated change: Fix sanity tests

* fix

* progress

* fixes

* fix

* fix

* spam

* un-disable errantly disabled tests

* gain insight

* Automated change: Fix sanity tests

* fixes

* fixes

* fix

* debug

* tweak

* fix

* fix timeout

* fix comment

* fixes

* x

* better test

* tests

* Automated change: Fix sanity tests

* missed file

* fix

* x

* fix

* fix

* fix

* fix

* Automated change: Fix sanity tests

* fix

* merge

* Automated change: Fix sanity tests

* fix

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2022-03-01 11:34:05 -08:00
Craig Tiller 05056e149a
eliminate bm_error (#28944) 2022-02-22 15:39:05 -08:00
Craig Tiller 2eadef9a82
Remove bm_metadata (#28219) 2021-11-29 13:40:26 -08:00
Craig Tiller 75a54f55e3
Remove trickle benchmarks (#28000)
* Remove trickle benchmarks

* Automated change: Fix sanity tests

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2021-11-10 18:21:13 -08:00
Richard Belleville 468768865b
Make Buildifier Sanity Test Strict (#27807)
* Fix all lint errors in repo.

* Use strict buildifier by default

* Whoops. That file does not exist

* Attempt fix to buildifier invocation

* Add missing copyright
2021-11-03 14:57:04 -07:00
Craig Tiller d1e01ac139
Add a test for a (now-illegal) build file construct (#27602)
* Add a test for a (now-illegal) build file construct

* add to test suite!

* fix

* fix

* fix syntax
2021-10-06 07:43:59 -07:00
Craig Tiller 84d9e7e9fa
Speed up microbenchmark tests (#27584)
* Speed up microbenchmark tests

* Automated change: Fix sanity tests

* Update grpc_benchmark_config.bzl

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2021-10-05 07:09:02 -07:00
Craig Tiller d4407b7e59
Revert "Remove dead code (#27445)" (#27461)
This reverts commit 693f36f91c.
2021-09-23 12:56:40 -07:00
Craig Tiller 693f36f91c
Remove dead code (#27445)
* Remove dead code

* Automated change: Fix sanity tests

* build

* remove benchmark

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2021-09-22 19:31:00 -07:00
Vijay Pai af76d0321a
Restore callback microbenchmarks but make them manual for now (#25491) 2021-02-19 13:36:11 -08:00
Vijay Pai dba8bb59a0 Implement callback CQ at C++ layer using shared thread pool if non-background iomgr 2021-02-16 16:02:23 -08:00
capstan 131579d52d Roll forward of commit 7883b51: Use abseil's flags in tests.
This replaces gflags. Added TODOs where use of `absl::Duration` or `absl::FlagSaver` might be preferred in follow-up cleanup. Fixes #24493.

This reverts commit da66b7d14e.

NEW:
* Adds references to `absl/flags/declare.h`, new to LTS 2020923.2 imported in commit 5b43440.
* Works around MSVC 2017 compiler error with large help text on flags by reducing the help text.
2020-11-06 16:07:19 +01:00
Nathan Herring da66b7d14e
Revert "Abseil Flags" 2020-10-24 15:19:55 -07:00
capstan 7883b5133d Use abseil's flags in tests.
This replaces gflags. Added TODOs where use of `absl::Duration` or `absl::FlagSaver` might be preferred in follow-up cleanup. Fixes #24493.
2020-10-23 19:36:33 +02:00
Vijay Pai b92ca15356
Increase size of a test to match internal guidance 2020-07-06 09:51:05 -07:00
Esun Kim 92d31d6f99 Remove the flaky flag from bm_opencensus_plugin 2020-05-04 11:41:09 -07:00
Vijay Pai 527509abc4 Unmark flaky on streaming_pump 2020-04-08 10:31:47 -07:00
Jan Tattermusch a791e16983 mark more tests as flaky 2020-03-17 05:32:17 -04:00
Jan Tattermusch 880111fe47
Merge pull request #22355 from jtattermusch/bm_metadata_not_flaky
bm_metadata is not flaky anymore
2020-03-16 18:14:25 +01:00
Vijay Pai b4f0cad4bd
bm_arena needs more time
Internal tests show that it needs a large timeout. This has passed 100x without timeout.
2020-03-16 08:12:15 -07:00
Jan Tattermusch 29528fc026 bm_metadata is not flaky anymore 2020-03-16 03:07:25 -04:00
Jan Tattermusch cbb0ef2717 try marking some offenders as flaky 2020-02-27 22:37:32 -05:00
Jan Tattermusch 543d8796f0 dont build some microbenchmarks on windows 2020-02-26 19:56:49 -05:00
Yash Tibrewal b18ae4b784 Add notap tag to tests that cause issues timeout issues internally 2020-02-18 12:31:46 -08:00
Jan Tattermusch 5e19408ef5 disable bm_chttp2_transport on msan 2020-02-18 03:20:18 -08:00
Jan Tattermusch cb6d09ac21 add missing bm_pollset and bm_chttp2_transport targets to bazel build 2020-02-18 02:34:42 -08:00
Vijay Pai f7d857a8e4 Make threadpool and trickle tests manual only (excessive time/resources) 2020-02-13 10:12:12 -08:00
Vijay Pai 1513f3a7b4 Reduce unneeded sizes and disable tests on mac for resource limits 2020-02-13 09:34:15 -08:00
Vijay Pai 0c0f3f0556 Declare microbenchmarks as tests rather than binaries, fix issues 2020-02-13 09:34:15 -08:00
Lidi Zheng d5d6914751 Make buildifier sanity test fail-able 2019-12-18 16:16:49 -08:00
Jan Tattermusch 5d352ded8b disable a few macos bazel tests 2019-11-25 13:35:26 +01:00
Lidi Zheng c34bef4ca4 Format all Bazel files with buildifier 2019-11-05 14:03:52 -08:00
Hope Casey-Allen 32801fb5eb Remove build target for microbenchmark 2019-08-29 14:11:05 -07:00
Hope Casey-Allen 6dfe27ab08 Fix race in bm_chttp2_transport 2019-08-28 10:07:38 -07:00
Yunjia Wang c6bc2b1875 Add threadpool benchmark and build files 2019-07-19 10:23:22 -07:00
Na-Na Pang 1ea651aee3 Add assertion 2019-05-09 12:22:55 -07:00
Na-Na Pang 9b50fbcaac Solve conflict 2019-05-08 18:50:06 -07:00
Na-Na Pang 070902b871 Merge bm_callback_cq to bm_cq 2019-05-08 15:51:58 -07:00
Na-Na Pang 2d5a9750a0 Manually add echo.proto to pass Portability build test 2019-05-06 16:20:59 -07:00
Na-Na Pang 1ba5f5c701 Modify build file 2019-05-06 14:55:00 -07:00