Commit Graph

204 Commits

Author SHA1 Message Date
Craig Tiller 26df942eb4 [build] Stop compiling every test.cc file for every config (#36197)
Instead, build a library and re-use that across compilations.

This still invokes a link step per target, and we'll want to deal with that at some point too, but at least this makes some progress to not being as wasteful with our compilation resources.

Additionally: remove bm_pollset -- it was having some problems compiling, and we really don't need it anymore.

Closes #36197

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36197 from ctiller:maintain-this-shite 9955026e23
PiperOrigin-RevId: 620946543
2024-04-01 14:04:25 -07:00
Craig Tiller a0c1027bb3
[transport] Move transport interface to C++ (#34618)
Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2023-10-18 13:13:23 -07:00
Sergii Tkachenko de6ed9ba9f
[Python] Migrate from yapf to black (#33138)
- Switched  from yapf to black
- Reconfigure isort for black
- Resolve black/pylint idiosyncrasies 

Note: I used `--experimental-string-processing` because black was
producing "implicit string concatenation", similar to what described
here: https://github.com/psf/black/issues/1837. While currently this
feature is experimental, it will be enabled by default:
https://github.com/psf/black/issues/2188. After running black with the
new string processing so that the generated code merges these `"hello" "
world"` strings concatenations, then I removed
`--experimental-string-processing` for stability, and regenerated the
code again.

To the reviewer: don't even try to open "Files Changed" tab 😄 It's
better to review commit-by-commit, and ignore `run black and isort`.
2023-06-09 15:08:55 -07:00
Craig Tiller c545350633
Reland: [promises] Compression filter conversion (#31686)
* Revert "Revert "[promises] Compression filter conversion (#31204)" (#31682)"

This reverts commit fa31b36cb1.

* fix?

* fix
2022-11-17 14:08:43 -08:00
Craig Tiller fa31b36cb1
Revert "[promises] Compression filter conversion (#31204)" (#31682)
This reverts commit a8c983528a.
2022-11-16 17:01:11 -08:00
Craig Tiller a8c983528a
[promises] Compression filter conversion (#31204)
* Convert compress filter

* decompress sketch

* Automated change: Fix sanity tests

* progress

* progress

* got the new filter impls compiling

* Automated change: Fix sanity tests

* declare the bits

* allocate control types

* stub in pipe accessors

* init pipes

* send-path

* working through the state machines

* [arena] pool allocator

* add tests

* Automated change: Fix sanity tests

* better sizes

* iwyu

* Automated change: Fix sanity tests

* first pass cut fragments

* fix1

* fix

* Automated change: Fix sanity tests

* fix

* iwyu

* build-deps

* fix for windows

* fix

* fixes

* better-logs

* better-logs

* callpushpush-->tryconcurrently

* Automated change: Fix sanity tests

* Automated change: Fix sanity tests

* logging improvements

* logging improvements

* fixes

* Automated change: Fix sanity tests

* respect flag

* handle eos

* cancel_after_client_done

* handle failed sends

* more error handling

* Automated change: Fix sanity tests

* more stuff

* remove unnecessary conversion

* hackity hack hack

* fixes

* progress

* Automated change: Fix sanity tests

* capture metadata from promise to recieve up

* spacing

* debug,fix

* not-used

* fix

* fix

* tune

* unify compress/decompress

* fixes

* send_message/send_trailing_metadata ordering

* cleanup loop

* remove minimal stack bits

* better error generation

* fix-build

* Automated change: Fix sanity tests

* cancellation of recv message for forwarded receives

* fix

* compile fix

* fix clobbering

* fix test, convert to gtest

* fix

* Automated change: Fix sanity tests

* cleanup

* fixes

* fix

* fix

* Automated change: Fix sanity tests

* Automated change: Fix sanity tests

* fixes

* add test

* Automated change: Fix sanity tests

* fix

* fix

* fix

* no need to store message per filter

* Automated change: Fix sanity tests

* danke yash

* iwyu

* clang-tidy

* Automated change: Fix sanity tests

* fix

* [promise] CallPushPull -> more general TryConcurrently

* Automated change: Fix sanity tests

* [promise] Introduce map_pipe, cleanup factories

* [promise] Promise based filter changes for messages

* [log] Longer space for filenames

* [test] Add some debugability affordances to async_end2end_test

* add comment

* Automated change: Fix sanity tests

* fix-merge

* fix-merge

* fix

* [chttp2] Fix a bug whereby trailers-only is not reported

We don't report trailers-only if the trailers are parsed prior to executing recv_initial_metadata

* comments

* fix for ruby

* review feedback

* merge

* fix

* Automated change: Fix sanity tests

* fix bm?

* re-add dropped flag

* turn on some traces to try and debug ruby

* debug for ruby problem

* proposed chttp2 fix

* Revert "turn on some traces to try and debug ruby"

This reverts commit 1bf3cfd552.

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2022-11-15 20:16:18 -08:00
Craig Tiller 1f1f923a72
[benchmarks] Remove stats integration (#30900)
* remove old stats cruft

* remove

* remove

* fix

* fix
2022-09-09 04:15:12 -07:00
AJ Heller 3c558dd796
Disable performance labeler for Github PRs (#29258) 2022-03-29 19:51:33 -07:00
Craig Tiller 40adbb46f0
Exclude noisy benchmarks from labelling computations (#29039)
* Exclude noisy benchmarks from labelling computations

* Automated change: Fix sanity tests

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2022-03-07 12:14:03 -08:00
Craig Tiller 05056e149a
eliminate bm_error (#28944) 2022-02-22 15:39:05 -08:00
Jan Tattermusch eb79b6b178
More run_microbenchmark.py cleanup (#28893)
* cleanup

* update TODOs

* fixup
2022-02-17 19:14:08 +01:00
Jan Tattermusch d1db000f62
small cleanup in microbenchmark scripts (#28886) 2022-02-15 18:02:20 +01:00
Craig Tiller 2eadef9a82
Remove bm_metadata (#28219) 2021-11-29 13:40:26 -08:00
Craig Tiller d342a86f2c
Fix typo (#28019) 2021-11-11 16:22:46 -08:00
Craig Tiller 6003710534
Label microbenchmark differences similarly to bloat (#27998)
* benchmark differences as a label

* debug

* Automated change: Fix sanity tests

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2021-11-11 07:37:02 -08:00
Craig Tiller 75a54f55e3
Remove trickle benchmarks (#28000)
* Remove trickle benchmarks

* Automated change: Fix sanity tests

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2021-11-10 18:21:13 -08:00
Craig Tiller 77ef40bba5
update specs file for benchmark (#27581) 2021-10-04 17:29:41 -07:00
Richard Belleville b6cc72f129
Revert "Run 2to3 on tools directory (#26002)" (#27183)
This reverts commit a388361c3d.
2021-08-30 12:29:20 -07:00
Richard Belleville a388361c3d
Run 2to3 on tools directory (#26002)
* Run 2to3 on tools directory

* Delete github_stats_tracking

* Re-run 2to3

* Remove unused script

* Remove unused script

* Remove unused line count utility

* Yapf. Isort

* Remove accidentally included file

* Restore line_count directory
2021-08-30 09:25:21 -07:00
Lidi Zheng 2231c2ba77
Introduce Python import sorting to our sanity test suite (#26768)
* Add isort_code.sh to sanity tests

* Run tools/distrib/isort_code.sh

* Fine tune the import order for relative imports

* Make pylint and project generation happy

* Fix a few corner cases

* Use --check instead of --diff

* The import order impacts test result somehow

* Make isort print diff and check output at the same time

* Let tools/run_tests/python_utils be firstparty library

* Run isort against latest HEAD
2021-07-26 12:31:21 -07:00
AJ Heller 5139a012e7
Migrate the bm_diff benchmarks to python3 (#25619)
* Migrate the bm_diff benchmarks to python3

Includes a requirements.txt pinned at ~2017 versions, when this script
was first written.

* Replace p2 with p3 scipy/numpy dependencies.

* py2->3 for benchmark setup scripts

* upgrade pip to resolve python3 cryptography/setuptools-rust problem

* re-add jobset import (accidentally removed)

* re-add six's urllib import. This file is still used in py2 tests

* force py3 in run_if_c_cpp_modified.sh

* Fix another instance of subprocess.check_output binary mode

* Use the requirements.txt for CI perf environment setup

* Try to upgrade PyJWT. (v2.0.0 was problematic, #25053)

v2.x makes encode return strs from jwt.encode in both py2 and py3.
Previously, py3 would return bytes, and py2 a str.

* upgate cryptography lib version requirements for jwt.

* Wrap pip requirements specifier in quotes '>=x,<y'

* Decode subprocess output once instead of for every line

* Revert "Decode subprocess output once instead of for every line"

This reverts commit 28d14026431622ac7afd3535a8d7118e5be96628.

py2 doesn't support the `text` argument to subprocess.check_output.

* Address reviewer requests

* Pin a valid scipy version

* Remove scipy and tabulate dependencies from macos tests
2021-03-09 15:44:00 +01:00
Sergii Tkachenko 8db79e2e71 Python style: reformat code using updated yapf 0.30.0 2021-01-13 14:00:12 -05:00
Jan Tattermusch c81ebbd343 use bazel to build microbenchmarks in bm_build.py 2020-08-17 09:50:26 +02:00
Esun Kim 7be821ac47 Yapf all target python sources 2020-01-03 09:25:10 -08:00
apolcyn 8a2c9512a5
Merge pull request #20447 from apolcyn/fix_print
Put parenthesis around print in bm_main.py
2019-10-02 16:20:54 -07:00
Alexander Polcyn 76abd3dd03 Also fix exception syntax 2019-10-02 13:12:06 -07:00
Alexander Polcyn c6654fc02a Put parenthesis around print in bm_main.py 2019-10-02 12:06:03 -07:00
chentanjun 4ecc1fe6a4 fix the wrong word 2019-10-01 05:13:33 +08:00
Lidi Zheng 0f0822d53f WIP: Utilize the GitHub Check Feature 2018-12-13 16:40:47 -08:00
Noah Eisen b649422741
Merge pull request #15220 from ncteisen/microbenchmarks
Don't Use Benchmark Counters
2018-04-27 13:38:01 -07:00
ncteisen 99e910f58f Stop tracking call size in bm diff 2018-04-27 11:34:50 -07:00
Jan Tattermusch 746479ad80 fix performance profile jobs 2018-04-11 21:59:09 +02:00
Mehrdad Afshari 87cd994b04 Upgrade yapf to 0.20.0
Upgrade yapf version to 0.20.0 and reformat Python files.
2018-01-02 14:40:00 -08:00
ncteisen 173c477bd0 tools/profiling 2017-12-11 16:52:44 -08:00
ncteisen 4df6a64403 Change default bm_diff loops to 10 2017-11-06 15:46:37 -08:00
Ken Payson 667aa2b990 Add call size bm_diff 2017-10-27 14:33:17 -07:00
ncteisen 4c9fa854f6 Add debugging prints 2017-10-24 08:59:05 -07:00
ncteisen 6bf4bcef04 Fix bm_diff 2017-10-23 22:19:01 -07:00
ncteisen c2d5edfef5 Increase CPU cost 2017-09-28 19:20:02 -07:00
ncteisen 460af54440 Shuffle new and old jobs together 2017-09-28 16:12:15 -07:00
Craig Tiller 319652fa85 Track pings in bmdiff 2017-09-13 18:13:48 -07:00
Craig Tiller e6d2e8f16b Fix case on internal type names
Should allow microbenchmarks to get running again
2017-09-13 06:10:18 -07:00
Craig Tiller 54f3c2eefe Automatically fetch schema from BigQuery 2017-09-11 13:12:04 -07:00
ncteisen 2aab728245 regex to diff 2017-07-21 11:25:56 -07:00
ncteisen 5c92dcde1e Allow passing regex to bm_diff tools 2017-07-20 14:55:49 -07:00
ncteisen 06795cb442 Address github comments 2017-06-20 08:58:43 -07:00
ncteisen 65fe1abed2 Address github comments 2017-06-19 14:31:04 -07:00
ncteisen 3345e1ccf7 Actually enable trickle diff 2017-06-19 09:47:58 -07:00
ncteisen 45e161b1c9 Make threshold toggleable 2017-06-19 09:27:59 -07:00
ncteisen 81941b9221 Propagate counters flag change 2017-06-14 11:55:56 -07:00