Commit Graph

99 Commits

Author SHA1 Message Date
Leonardo Pistone dfa9d853ff Fix Python DeprecationWarning: 'pipes' (#34941)
Starting from Python 3.11, the pipes module produces this warning:

DeprecationWarning: 'pipes' is deprecated and slated for removal in Python 3.13

Turns out that in this repo the pipes module is only used for the
"quote" function which is turn directly taken from the shlex module [1].

The shlex module is not deprecated as of today and is already used in
other places in this repo. The function shlex.quote has been around
since the ancient Python 3.3.

[1] https://github.com/python/cpython/blob/3.11/Lib/pipes.py#L64-L66

<!--

If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the appropriate
lang label.

-->

Closes #34941

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/34941 from lepistone:deprecate-python-pipes 233c54c135
PiperOrigin-RevId: 588883480
2023-12-07 13:02:23 -08:00
Paulo Castello da Costa 3eca188895
[benchmark] Delete references to `node_purejs`. (#33970)
There is only one supported implementation for performance benchmarks,
so the name is unified to `node` (previously `node_purejs`).
2023-08-08 17:55:56 -07:00
Sergii Tkachenko de6ed9ba9f
[Python] Migrate from yapf to black (#33138)
- Switched  from yapf to black
- Reconfigure isort for black
- Resolve black/pylint idiosyncrasies 

Note: I used `--experimental-string-processing` because black was
producing "implicit string concatenation", similar to what described
here: https://github.com/psf/black/issues/1837. While currently this
feature is experimental, it will be enabled by default:
https://github.com/psf/black/issues/2188. After running black with the
new string processing so that the generated code merges these `"hello" "
world"` strings concatenations, then I removed
`--experimental-string-processing` for stability, and regenerated the
code again.

To the reviewer: don't even try to open "Files Changed" tab 😄 It's
better to review commit-by-commit, and ignore `run black and isort`.
2023-06-09 15:08:55 -07:00
Jan Tattermusch a3be072605
Switch .py shebang to python3 (#28963)
* switch shebang to python3

* prepare_build_macos: install latest available patch for each python
2022-03-14 10:41:30 +01:00
Jan Tattermusch d1db000f62
small cleanup in microbenchmark scripts (#28886) 2022-02-15 18:02:20 +01:00
Richard Belleville 59693b7e72
Migrate Infrastructure Scripts to Python 3 (#27135)
* Run 2to3 on tools directory

* Delete github_stats_tracking

* Re-run 2to3

* Remove unused script

* Remove unused script

* Remove unused line count utility

* Yapf. Isort

* Remove accidentally included file

* Migrate tools/distrib directory to python 3

* Remove unnecessary shebang

* Restore line_count directory

* Immediately convert subprocess.check_output output to string

* Take care of Python 2 shebangs

* Invoke scripts using a Python 3 interpreter

* Yapf. Isort

* Try installing Python 3 first

* See if we have any Python 3 versions installed

* Add Python 3.7 to Windows path

* Try adding a symlink

* Try to symlink differently

* Install six for Python 3

* Run run_interop_tests with python 3

* Try installing six in python3.7 explicitly

* Revert "Try installing six in python3.7 explicitly"

This reverts commit 2cf60d72f3.

* And debug some more

* Fix issue with jobset.py

* Add debug for CI failure

* Revert microbenchmark changes
2021-11-08 08:55:25 -08:00
Richard Belleville b6cc72f129
Revert "Run 2to3 on tools directory (#26002)" (#27183)
This reverts commit a388361c3d.
2021-08-30 12:29:20 -07:00
Richard Belleville a388361c3d
Run 2to3 on tools directory (#26002)
* Run 2to3 on tools directory

* Delete github_stats_tracking

* Re-run 2to3

* Remove unused script

* Remove unused script

* Remove unused line count utility

* Yapf. Isort

* Remove accidentally included file

* Restore line_count directory
2021-08-30 09:25:21 -07:00
Lidi Zheng 2231c2ba77
Introduce Python import sorting to our sanity test suite (#26768)
* Add isort_code.sh to sanity tests

* Run tools/distrib/isort_code.sh

* Fine tune the import order for relative imports

* Make pylint and project generation happy

* Fix a few corner cases

* Use --check instead of --diff

* The import order impacts test result somehow

* Make isort print diff and check output at the same time

* Let tools/run_tests/python_utils be firstparty library

* Run isort against latest HEAD
2021-07-26 12:31:21 -07:00
Esun Kim 7be821ac47 Yapf all target python sources 2020-01-03 09:25:10 -08:00
Esun Kim e12ff9cd58 Remove third_party/libcxx[abi] 2019-10-02 10:16:40 -07:00
Jan Tattermusch 0fbc4fc7cb yapf code 2019-09-19 03:54:00 -04:00
Jan Tattermusch 523ec7398e improve target naming 2019-09-18 13:43:38 +02:00
Jan Tattermusch e573d4180a more readable run_performance_tests.py reports 2019-09-18 10:27:29 +02:00
Jan Tattermusch 0d6eec8b8f a few fixes for run_performance_tests.py flame graph generators 2019-08-01 20:26:58 +02:00
Jan Tattermusch 84c74b67f8 build C++ in benchmarks with cmake 2018-10-22 17:56:53 +02:00
Jan Tattermusch 185d7332ad more reasonable timeouts scenario timeout 2018-10-22 15:42:06 +02:00
Jan Tattermusch 96ae5f9aaa make sure port server runs locally too 2018-10-10 18:37:50 +02:00
Michael Lumish 44f1270649
Merge pull request #15175 from murgatroid99/node_perf_tests_again
Add Node perf tests for both implementations
2018-08-16 09:39:09 -07:00
Jan Tattermusch 4bfff0a0ff yapf code 2018-08-14 17:49:25 +02:00
Jan Tattermusch 217d460ee2 fix performance benchmarks on windows 2018-08-14 10:09:02 +02:00
murgatroid99 fc566ddcc5 Increase build timeout again 2018-08-10 11:04:31 -07:00
murgatroid99 f2177c735f Increase build timeouts to account for Node build 2018-05-01 11:03:22 -07:00
murgatroid99 07c7eda3e3 Add node repo to repo archive 2018-04-30 14:20:28 -07:00
Mehrdad Afshari 87cd994b04 Upgrade yapf to 0.20.0
Upgrade yapf version to 0.20.0 and reformat Python files.
2018-01-02 14:40:00 -08:00
ncteisen 888093c6ed yapf run_tests 2017-12-11 18:08:35 -08:00
Matt Kwong b75db422bd Add multi-vm performance tests to Kokoro 2017-11-29 18:28:52 -08:00
Matt Kwong 69ce380ccc Revert "Merge pull request #12513 from grpc/revert-12289-sig_hand"
This reverts commit 81ad19d4be, reversing
changes made to ed63aac195.
2017-09-12 13:30:51 -07:00
Matt Kwong 3c8f6dbffd Remove leftover instance of clear_alarms 2017-09-12 13:30:26 -07:00
Craig Tiller 0852acd8b3 Revert "Let alarms trigger at end of jobset.py instead of clearing them" 2017-09-12 13:15:03 -07:00
Matt Kwong e72255f011 Merge branch 'master' into sig_hand 2017-09-11 12:27:44 -07:00
Matt Kwong 5920abc535 Revert "Don't clear alarm in jobset when running performance tests"
This reverts commit c15d32bbe8.
2017-08-24 12:26:05 -07:00
Michael Darakananda f570fb2743 give run_performance_tests a main method
In this way, external users can add perf tests for other languages
by adding to scenario_config.LANGUAGES then calling
run_performance_tests.main.
2017-08-17 17:09:56 +10:00
Jan Tattermusch 61c913531c fix windows performance benchmarks 2017-08-11 18:03:35 +02:00
Matt Kwong c15d32bbe8 Don't clear alarm in jobset when running performance tests 2017-07-10 17:10:27 -07:00
Jan Tattermusch 7897ae9308 auto-fix most of licenses 2017-06-08 11:22:41 +02:00
Siddharth Shukla 8e64d901fd Modify shebang on python scripts
Change shebangs to `#!/usr/bin/env python` instead of
`#!/usr/bin/env python2.7`.
2017-03-12 19:53:12 +01:00
Siddharth Shukla d194f59939 Make testing scripts python3.x compatible
Update run_tests/*.py to use six based isomorphisms and print function
from __future__ module.
2017-03-12 17:47:53 +01:00
Craig Tiller 6921820fab Ensure port server is running on remote hosts 2017-03-07 15:55:53 -08:00
Craig Tiller 0a3d5f90a9 Revert "Enable lto for performance testing" 2017-03-01 17:08:39 -08:00
Craig Tiller f9abb2921b Reserve opt for external builds, use lto for profiling 2017-03-01 13:38:37 -08:00
apolcyn cf7587e066 Merge pull request #9548 from apolcyn/improve_benchmark_worker_timeouts
shorten qps worker timeouts and run remote workers under timeout
2017-02-04 09:42:48 -08:00
Alexander Polcyn 41fe579c6d always render junit xml report after perf tests 2017-02-02 10:46:51 -08:00
Alexander Polcyn 76be306f9d shorten qps worker timeouts and run remote workers under timeout 2017-02-01 12:06:23 -08:00
apolcyn d9f5b230c1 Merge pull request #9038 from apolcyn/run_perf_on_benchmarks
add --perf_args usage to jenkins performance runner scripts
2017-01-13 17:37:09 -08:00
Yuxuan Li ac87a46225 enable uploading server cpu usage data from performance tests to big query 2017-01-11 10:01:16 +00:00
Jan Tattermusch 5c79a3199c cleanup tools/run_tests directory 2016-12-22 10:47:52 +01:00
Alexander Polcyn 66c6782751 add --perf_args usage to jenkins performance runner scripts 2016-12-20 13:29:58 -08:00
apolcyn c204647295 Merge pull request #8699 from apolcyn/run_perf_on_benchmarks
Add perf option to benchmark runner script, create text profiles and flamegraphs
2016-12-08 19:03:54 -08:00
Alexander Polcyn 9f08d110f8 add a "perf" option to benchmarks script, generate profile text reports and flamegraphs 2016-12-08 16:03:19 -08:00