Commit Graph

257 Commits

Author SHA1 Message Date
Craig Tiller 1f1f923a72
[benchmarks] Remove stats integration (#30900)
* remove old stats cruft

* remove

* remove

* fix

* fix
2022-09-09 04:15:12 -07:00
Craig Tiller 004788af3d
[cleanup] Remove profiling timers (#30779)
* [cleanup] Remove profiling timers

- nobody has used this system in years
- if we needed it, we'd probably rewrite it at this point to be something more modern
- let's remove it until that need arises

* fix

* fixes
2022-08-30 13:40:08 -07:00
nancylucy01 45460e1f87
Fix per call label (#30515) 2022-08-05 12:38:39 -07:00
nancylucy01 cc96858f0b
Add channel memory benchmark into memory_diff.py (#30507)
* Added the channel benchmark to the python script

* removing warning

* Automated change: Fix sanity tests

* added label function for channel and skip for minstack channel

* Automated change: Fix sanity tests

Co-authored-by: nancylucy01 <nancylucy01@users.noreply.github.com>
2022-08-05 09:24:24 -07:00
nancylucy01 3cd368b28d
Basic callback client and server for new perchannel benchmark (#30305)
* Added new files for channel client/server

* Committing to switch branch

* Rebasing branch

* Switching branch

* Server process getting called

* Still working

* RPC received success, with sleep

* gRPC Receive success, grpc timeout

* Earlier but Clang tidy

* Fix timeout issue, remove some logs

* Added signint handler, test passing but flaky

* added sleep to reduce flakiness, removed some dependencies, changed LOG to gpr_log

* Changed benchmark_name default back to call

* remove deleted files

* grpc shutdown timeout

* trying to add shutdown

* Some changes

* Removed shutdown

* Automated change: Fix sanity tests

* Changes for review comments

* Changed comments

* Changed benchmark driver defaults so that CI testing would happen for all benchmarks

* Automated change: Fix sanity tests

Co-authored-by: nancylucy01 <nancylucy01@users.noreply.github.com>
2022-07-25 10:04:29 -07:00
nancylucy01 24171b3b57
Changing memory benchmark driver's flags (#30276)
* Changed benchmark driver's flags and updated calling python script. Untested

* Changes made for debugging/testing

* Switching branch

* Removed debug statements and changed file permissions for memory_diff

* Minor changes for review

* Changed secure description

* Changed flag configuration for clarity

* Added dep to BUILD file

* Changed return value

* Edits for review, added comments

* Changed a print statement
2022-07-13 10:10:25 -07:00
Jan Tattermusch c33963d371
iOS binary size script cleanup (#29377)
* iOS binary size script cleanup

* add job cfg for running iOS binary size on master
2022-04-14 11:01:16 +02:00
AJ Heller 3c558dd796
Disable performance labeler for Github PRs (#29258) 2022-03-29 19:51:33 -07:00
Craig Tiller bf72db4180
Collect memory measurements of the minimal stack (#29164)
* minstack measuring

* fix

* sort

* x

* x

* x

* x

* Automated change: Fix sanity tests

* review feedback

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2022-03-25 10:09:46 -07:00
Craig Tiller 5d81683faa
Add per-call memory bloat report to CI (#29037)
* Add per-call memory bloat report to CI

* fix

* test

* Automated change: Fix sanity tests

* Update grpc_bloat_diff_in_docker.sh

* Update memory_diff.py

* review feedback

* prep for new job

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2022-03-12 11:56:48 +01:00
Craig Tiller 40adbb46f0
Exclude noisy benchmarks from labelling computations (#29039)
* Exclude noisy benchmarks from labelling computations

* Automated change: Fix sanity tests

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2022-03-07 12:14:03 -08:00
Craig Tiller 05056e149a
eliminate bm_error (#28944) 2022-02-22 15:39:05 -08:00
Jan Tattermusch eb79b6b178
More run_microbenchmark.py cleanup (#28893)
* cleanup

* update TODOs

* fixup
2022-02-17 19:14:08 +01:00
Jan Tattermusch d1db000f62
small cleanup in microbenchmark scripts (#28886) 2022-02-15 18:02:20 +01:00
Craig Tiller 2eadef9a82
Remove bm_metadata (#28219) 2021-11-29 13:40:26 -08:00
Craig Tiller f6a88e54ec
ignore dynamic linker segments in bloat severity calculations (#28149) 2021-11-18 22:52:55 -08:00
Craig Tiller 4ff6d66c06
Don't limit bloaty output lines (#28120)
* dont limit bloaty output lines

* Automated change: Fix sanity tests

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2021-11-17 16:18:42 -08:00
Craig Tiller da47e88237
Fix typo in bloat script (#28104) 2021-11-15 16:12:02 -08:00
Craig Tiller b7311aad77
Exclude debug sections from bloat severity calculations (#28089) 2021-11-15 11:42:54 -08:00
Craig Tiller f4be36f1cf
Bloat reporting improvements (#28013)
* Bloat reporting improvements

* typo

* fix

* fix

* fix
2021-11-12 20:12:19 +00:00
Craig Tiller d342a86f2c
Fix typo (#28019) 2021-11-11 16:22:46 -08:00
Craig Tiller 6003710534
Label microbenchmark differences similarly to bloat (#27998)
* benchmark differences as a label

* debug

* Automated change: Fix sanity tests

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2021-11-11 07:37:02 -08:00
Craig Tiller 75a54f55e3
Remove trickle benchmarks (#28000)
* Remove trickle benchmarks

* Automated change: Fix sanity tests

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2021-11-10 18:21:13 -08:00
Richard Belleville 59693b7e72
Migrate Infrastructure Scripts to Python 3 (#27135)
* Run 2to3 on tools directory

* Delete github_stats_tracking

* Re-run 2to3

* Remove unused script

* Remove unused script

* Remove unused line count utility

* Yapf. Isort

* Remove accidentally included file

* Migrate tools/distrib directory to python 3

* Remove unnecessary shebang

* Restore line_count directory

* Immediately convert subprocess.check_output output to string

* Take care of Python 2 shebangs

* Invoke scripts using a Python 3 interpreter

* Yapf. Isort

* Try installing Python 3 first

* See if we have any Python 3 versions installed

* Add Python 3.7 to Windows path

* Try adding a symlink

* Try to symlink differently

* Install six for Python 3

* Run run_interop_tests with python 3

* Try installing six in python3.7 explicitly

* Revert "Try installing six in python3.7 explicitly"

This reverts commit 2cf60d72f3.

* And debug some more

* Fix issue with jobset.py

* Add debug for CI failure

* Revert microbenchmark changes
2021-11-08 08:55:25 -08:00
Craig Tiller 65e300e55d
Tweak bloat thresholds (#27942) 2021-11-04 12:29:11 -07:00
Craig Tiller 577d799de0
ugh (#27937) 2021-11-03 23:58:28 -07:00
Craig Tiller aa23c675ef
Fix crash in bloat diff if diff_size != 0 (#27935)
* Update bloat_diff.py

* Automated change: Fix sanity tests (#27936)

Co-authored-by: ctiller <ctiller@users.noreply.github.com>

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2021-11-03 22:07:56 -07:00
Craig Tiller 59ed9d09a2
Reland bloat labelling code. (#27926)
* Revert "Revert "Assign a label for level of bloat diff (#27880)" (#27925)"

This reverts commit f2b96000a7.

* fix
2021-11-03 09:57:13 -07:00
Craig Tiller f2b96000a7
Revert "Assign a label for level of bloat diff (#27880)" (#27925)
This reverts commit 882cc1259e.
2021-11-03 08:14:19 -07:00
Craig Tiller 882cc1259e
Assign a label for level of bloat diff (#27880)
* upgrade bloaty

* Update check_submodules.sh

* fix

* label bloat

* set labels based on severity

* measure severity of changes

* Automated change: Fix sanity tests

* test

* test

* Update bloat_diff.py

* Update check_on_pr.py

* Automated change: Fix sanity tests

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2021-11-02 22:39:15 -07:00
Craig Tiller 77ef40bba5
update specs file for benchmark (#27581) 2021-10-04 17:29:41 -07:00
Richard Belleville b6cc72f129
Revert "Run 2to3 on tools directory (#26002)" (#27183)
This reverts commit a388361c3d.
2021-08-30 12:29:20 -07:00
Richard Belleville a388361c3d
Run 2to3 on tools directory (#26002)
* Run 2to3 on tools directory

* Delete github_stats_tracking

* Re-run 2to3

* Remove unused script

* Remove unused script

* Remove unused line count utility

* Yapf. Isort

* Remove accidentally included file

* Restore line_count directory
2021-08-30 09:25:21 -07:00
Lidi Zheng 2231c2ba77
Introduce Python import sorting to our sanity test suite (#26768)
* Add isort_code.sh to sanity tests

* Run tools/distrib/isort_code.sh

* Fine tune the import order for relative imports

* Make pylint and project generation happy

* Fix a few corner cases

* Use --check instead of --diff

* The import order impacts test result somehow

* Make isort print diff and check output at the same time

* Let tools/run_tests/python_utils be firstparty library

* Run isort against latest HEAD
2021-07-26 12:31:21 -07:00
AJ Heller 4ea47d6ae3
Fix python3 performance tests - incomplete migration from python2 (#25734)
This fixes a crash in the grpc_performance_profile_* and grpc_e2e_performance_* tests.

Example failure: https://source.cloud.google.com/results/invocations/2112f2d5-db91-4901-87cb-cc9865f351f1/targets

The daily performance test passes with this fixit branch:
http://sponge2/23d4df20-f4dd-48ee-8789-4cd96b078ed5.

Additional fixes:

reorder imports according to the google style guide
replace deprecated cgi.escape library method with html.escape
In case it comes up, with respect to upgrading from py2 to py3, the iter{items,keys,values} methods are intentionally replaced with the less-efficient-in-python2 non-iter versions. This is a recommended practice even when supporting both python versions, and this is not performance-critical code.

Additional note: there is no performance difference btw py2 and py3 (~80 minutes to finish)
2021-03-16 16:49:30 -07:00
AJ Heller 5139a012e7
Migrate the bm_diff benchmarks to python3 (#25619)
* Migrate the bm_diff benchmarks to python3

Includes a requirements.txt pinned at ~2017 versions, when this script
was first written.

* Replace p2 with p3 scipy/numpy dependencies.

* py2->3 for benchmark setup scripts

* upgrade pip to resolve python3 cryptography/setuptools-rust problem

* re-add jobset import (accidentally removed)

* re-add six's urllib import. This file is still used in py2 tests

* force py3 in run_if_c_cpp_modified.sh

* Fix another instance of subprocess.check_output binary mode

* Use the requirements.txt for CI perf environment setup

* Try to upgrade PyJWT. (v2.0.0 was problematic, #25053)

v2.x makes encode return strs from jwt.encode in both py2 and py3.
Previously, py3 would return bytes, and py2 a str.

* upgate cryptography lib version requirements for jwt.

* Wrap pip requirements specifier in quotes '>=x,<y'

* Decode subprocess output once instead of for every line

* Revert "Decode subprocess output once instead of for every line"

This reverts commit 28d14026431622ac7afd3535a8d7118e5be96628.

py2 doesn't support the `text` argument to subprocess.check_output.

* Address reviewer requests

* Pin a valid scipy version

* Remove scipy and tabulate dependencies from macos tests
2021-03-09 15:44:00 +01:00
Sergii Tkachenko 8db79e2e71 Python style: reformat code using updated yapf 0.30.0 2021-01-13 14:00:12 -05:00
Jan Tattermusch b4007c1cbc fix bloat_diff.py 2020-12-11 13:43:44 +01:00
Esun Kim bfdfc9eabc Python3 ios_bin/binary_size.py 2020-11-12 18:48:46 -08:00
Jan Tattermusch c81ebbd343 use bazel to build microbenchmarks in bm_build.py 2020-08-17 09:50:26 +02:00
Esun Kim 7be821ac47 Yapf all target python sources 2020-01-03 09:25:10 -08:00
apolcyn 8a2c9512a5
Merge pull request #20447 from apolcyn/fix_print
Put parenthesis around print in bm_main.py
2019-10-02 16:20:54 -07:00
Alexander Polcyn 76abd3dd03 Also fix exception syntax 2019-10-02 13:12:06 -07:00
Alexander Polcyn c6654fc02a Put parenthesis around print in bm_main.py 2019-10-02 12:06:03 -07:00
chentanjun 4ecc1fe6a4 fix the wrong word 2019-10-01 05:13:33 +08:00
Esun Kim f88bd06ee5 Go UPB! 2019-08-06 15:27:14 -07:00
Lidi Zheng 0f0822d53f WIP: Utilize the GitHub Check Feature 2018-12-13 16:40:47 -08:00
Muxi Yan 07f190d2d5 Fix binary_size.py 2018-08-07 17:03:25 -07:00
Muxi Yan e773a9b75f Use /tmpfs on Kokoro for ObjC tests 2018-07-27 14:09:24 -07:00
Muxi Yan 687316c025
Clarify it's Objective-C sizes in comment 2018-07-19 16:55:35 -07:00