Commit Graph

385 Commits

Author SHA1 Message Date
Sergii Tkachenko de6ed9ba9f
[Python] Migrate from yapf to black (#33138)
- Switched  from yapf to black
- Reconfigure isort for black
- Resolve black/pylint idiosyncrasies 

Note: I used `--experimental-string-processing` because black was
producing "implicit string concatenation", similar to what described
here: https://github.com/psf/black/issues/1837. While currently this
feature is experimental, it will be enabled by default:
https://github.com/psf/black/issues/2188. After running black with the
new string processing so that the generated code merges these `"hello" "
world"` strings concatenations, then I removed
`--experimental-string-processing` for stability, and regenerated the
code again.

To the reviewer: don't even try to open "Files Changed" tab 😄 It's
better to review commit-by-commit, and ignore `run black and isort`.
2023-06-09 15:08:55 -07:00
Wanlin Du 4b40f957a1
Add query delay flag to Prometheus query script. (#31611)
* Add query delay flag to Prometheus query script.

This PR adds the flag allows user to config the delay of querying
the Prometheus.

* Update the help message of the flag to be more clear.
2022-11-18 12:06:47 -08:00
Paulo Castello da Costa 5d2767601a
Delete obsolete import. (#31010) 2022-09-16 02:04:06 -07:00
Craig Tiller 1f1f923a72
[benchmarks] Remove stats integration (#30900)
* remove old stats cruft

* remove

* remove

* fix

* fix
2022-09-09 04:15:12 -07:00
Craig Tiller 610a248059
[stats] Optimize histogram system (#30834)
* recurse

* bsearch

* bounds

* optimize stats

* fix

* Automated change: Fix sanity tests

* Automated change: Fix sanity tests

* fix

* Automated change: Fix sanity tests

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2022-09-08 13:30:28 -07:00
Craig Tiller 0079382cfd
[cq] Track cq type to determine api usage (#30833)
* [cq] Track cq type to determine api usage

* Automated change: Fix sanity tests

* fix

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2022-09-04 00:01:47 -07:00
Craig Tiller 903e0490ea
[iomgr] Restrict TCP read allocations to 8kb or 64kb (#30626)
* [stats] Cleanup stats system

* clear out optionality

* fix

* might as well...

* Automated change: Fix sanity tests

* only allocate 8k or 64k blocks for tcp reads

* Automated change: Fix sanity tests

* fix

* Update tcp_posix.cc

* Automated change: Fix sanity tests

* respect target, min progress sizes

* cleaner loops

* clean out more unused stuff

* clean out more unused stuff

* Automated change: Fix sanity tests

* update

* tsan race

* Automated change: Fix sanity tests

* Introduce flag

* protect with flag

* [experiments] Make output more diffable/readable

* Automated change: Fix sanity tests

* buildifier sized indentations

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2022-09-01 15:46:51 -07:00
Craig Tiller 5b6dac02ac
[stats] Cleanup & re-enable stats system (#30610)
* [stats] Cleanup stats system

* clear out optionality

* fix

* might as well...

* Automated change: Fix sanity tests

* clean out more unused stuff

* clean out more unused stuff

* Automated change: Fix sanity tests

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2022-08-26 10:18:06 -07:00
Wanlin Du c8c82b1c2f
Add enablePrometheus annotation. (#30452)
* Add enablePrometheus annotation.

The PR adds the enablePrometheus annotation to load tests that are
part of PSM data plan performance tests. This annotation enables
all PSM related tests to obtain data from Prometehus, even for the
regular tests.
2022-08-02 09:23:53 -07:00
Paulo Castello da Costa cf0782bab3
Create directory tree before generating load test examples. (#30453)
PSM examples are created deeper and that may cause an error if the directories are not created before generating the examples.
2022-08-01 16:45:50 -07:00
Wanlin Du daaeec062b
Remove support for csharp(core) and ruby for PSM tests. (#29694) 2022-05-23 13:43:02 -07:00
Wanlin Du dc4ed3d60d
Fix typo in psm proxyless template for prebuilt workers. (#29473) 2022-04-21 12:58:46 -07:00
Wanlin Du 61c69db2bf
Add templates for PSM test. (#29456)
* Add templates for PSM test.

The commit adds psm related template generated from
loadtest_template.py. This commit also update the
loadtest_example.sh to generate examples based on the templates.
2022-04-20 18:01:17 -07:00
Wanlin Du 83a472a5ca
Update java psm base scenario to have 15 warm up seconds (#29465) 2022-04-20 17:42:54 -07:00
Wanlin Du fa60c082bb
Fix typo in scenario names when generated by loadtest_config.py (#29464) 2022-04-20 16:46:26 -07:00
Wanlin Du 2b42b24a94
Add scenario generation for PSM tests. (#29336)
This commit adds the PSM base scenario to each language in
scenario_config.py allowing per language generation of PSM
base scenarios. The PSM base scenarios will be further updated
in loadtest_config.py for number of client channels and the
number of async_server_thread. After update of the two
aforementioned fields, scenarios varying offered_load will
be generated from the base scenarios.

loadtest_config.py is updated to take number of client channels,
the number of async_servers and a list of targeted offered_load.
2022-04-20 14:12:20 -07:00
Jan Tattermusch 2c6ab431a1
Add support for grpc-dotnet in GKE benchmarks (#28975)
* GKE benchmarks: add support for benchmarking grpc-dotnet

* add dotnet to loadtest basic templates

* print full path for generated examples

* add grpc-dotnet scenario to loadtest_example.sh generator

* add grpc-dotnet to the experimental kokoro job

* yapf format code
2022-04-07 05:13:37 -07:00
Wanlin Du 58612ba155
Big query upload (#29197)
Update existing table schema to include prometheus data and fields (startTime and endTime) added in the scenario results.
2022-03-31 18:42:13 -07:00
Wanlin Du 3d84874dcd
Python script to query prometheus for CPU and memory data (#29057)
Python script to query prometheus for total cpu seconds for a test and average memory (in Bytes) during a test.
2022-03-31 18:34:31 -07:00
Yash Tibrewal 114e83d875
HTTP2: Initiate write for acknowledging SETTINGS frame (#29218)
* HTTP2: Initiate write for acknowledging SETTINGS frame

* Codegen

* yapf code

* Fix tests
2022-03-30 13:51:18 -07:00
Craig Tiller 766cd6b573
Remove epollex poller (#29160)
* start delete

* remove build.gn... again

* remove more

* Automated change: Fix sanity tests

* fixes

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2022-03-21 10:27:52 -07:00
Jan Tattermusch a3be072605
Switch .py shebang to python3 (#28963)
* switch shebang to python3

* prepare_build_macos: install latest available patch for each python
2022-03-14 10:41:30 +01:00
Paulo Castello da Costa ccd8d577d0
Fix python os import. (#28805) 2022-02-07 17:06:15 -08:00
Wanlin Du abc0f3e881
Update templates (#28797)
This pr updates the templates required by the change made in grpc/test-infra#267.
2022-02-07 12:08:00 -08:00
Jan Tattermusch 79b58431ed
Update csharp SDK to LTS versions (#27966)
* update C# SDK

* regenerate dockerfiles

* install .NET6 and .NET Core 3.1

* regenerate dockerfiles

* change netcoreapp2.1 targets to netcoreapp3.1

* update installed SDKs in aarch64 C# job

* update run_tests scripts to use netcoreapp3.1 for C#
2021-11-17 10:41:02 +01:00
Paulo Castello da Costa 567c9db86c
Remove `from . import` from benchmarking scripts. (#27977) 2021-11-09 02:25:55 -08:00
Richard Belleville 59693b7e72
Migrate Infrastructure Scripts to Python 3 (#27135)
* Run 2to3 on tools directory

* Delete github_stats_tracking

* Re-run 2to3

* Remove unused script

* Remove unused script

* Remove unused line count utility

* Yapf. Isort

* Remove accidentally included file

* Migrate tools/distrib directory to python 3

* Remove unnecessary shebang

* Restore line_count directory

* Immediately convert subprocess.check_output output to string

* Take care of Python 2 shebangs

* Invoke scripts using a Python 3 interpreter

* Yapf. Isort

* Try installing Python 3 first

* See if we have any Python 3 versions installed

* Add Python 3.7 to Windows path

* Try adding a symlink

* Try to symlink differently

* Install six for Python 3

* Run run_interop_tests with python 3

* Try installing six in python3.7 explicitly

* Revert "Try installing six in python3.7 explicitly"

This reverts commit 2cf60d72f3.

* And debug some more

* Fix issue with jobset.py

* Add debug for CI failure

* Revert microbenchmark changes
2021-11-08 08:55:25 -08:00
brandonpaiz 27731a9081
Revert low Java throughput hotfix; implement permanent fix (#27919) 2021-11-03 15:47:24 -07:00
Jan Tattermusch a923eff71f
Rename experimental benchmark job from "..._v2" to "..._gke_experimental" (#27869)
* make it obvious which gke benchmark script is experimental

* add cfg file for new kokoro jobs with a more fitting name

* update performance readme after renaming _v2 benchmark job

* fix typo
2021-11-02 05:52:02 -07:00
Jan Tattermusch 6001bc636b
Stop running some more esoteric C++ benchmark scenarios continuously (#27872)
* stop running some more esoteric C++ benchamk scenarios continuously

* regenerate qps scenarios for bazel unit tests
2021-11-02 11:27:00 +01:00
Jan Tattermusch cf99a32109
set threads_per_cq=2 for C++ unconstrained scenarios for best throughput and lower variance (#27870) 2021-11-02 06:57:53 +01:00
Jan Tattermusch 0a81089efc
hotfix for low java throughput (#27868) 2021-10-29 14:09:14 -07:00
Paulo Castello da Costa da752c2c0b
Update basic template to match examples. (#27856)
Examples were updated by https://github.com/grpc/test-infra/pull/228.
2021-10-29 09:37:49 +02:00
Jan Tattermusch b3a1f68606
Reorganize docs for end2end benchmarks (#27484)
* Reorganize docs for end2end benchmarks

* Update README.md

* Restore heading "gRPC OSS benchmarks" (#27646)

This heading is used in cross-references from other repositories and sites. Also fix formatting.

Co-authored-by: Paulo Castello da Costa <6579971+paulosjca@users.noreply.github.com>
2021-10-08 19:37:04 +02:00
Paulo Castello da Costa aea060b530
Rename `prefix` to `index` in index generator. (#27576) 2021-10-06 13:42:29 +02:00
Paulo Castello da Costa 931be3f7ef
Fix load test name format for more than 10 runs per test. (#27558) 2021-10-04 15:30:43 +02:00
Jan Tattermusch 7c10e1b9fe
stop running protobuf_async_streaming_from_server_qps_unconstrained_secure scenario on CI (#27346) 2021-09-15 10:00:51 +02:00
Donghao Qiu dc19d6c132
add file existance check (#27133) 2021-09-02 03:32:56 -07:00
Richard Belleville b6cc72f129
Revert "Run 2to3 on tools directory (#26002)" (#27183)
This reverts commit a388361c3d.
2021-08-30 12:29:20 -07:00
Richard Belleville a388361c3d
Run 2to3 on tools directory (#26002)
* Run 2to3 on tools directory

* Delete github_stats_tracking

* Re-run 2to3

* Remove unused script

* Remove unused script

* Remove unused line count utility

* Yapf. Isort

* Remove accidentally included file

* Restore line_count directory
2021-08-30 09:25:21 -07:00
Donghao Qiu 1e84143126
Save node info to BigQuery (#27113)
* save nodeinfo to bigquery

* correct format
2021-08-24 21:02:40 -07:00
Wanlin Du 971e95b906
Update related scripts to populate BigQuery fields (#26778)
* Update related scripts to populate BigQuery fields
2021-08-24 07:42:35 -07:00
Lidi Zheng 2231c2ba77
Introduce Python import sorting to our sanity test suite (#26768)
* Add isort_code.sh to sanity tests

* Run tools/distrib/isort_code.sh

* Fine tune the import order for relative imports

* Make pylint and project generation happy

* Fix a few corner cases

* Use --check instead of --diff

* The import order impacts test result somehow

* Make isort print diff and check output at the same time

* Let tools/run_tests/python_utils be firstparty library

* Run isort against latest HEAD
2021-07-26 12:31:21 -07:00
Paulo Castello da Costa 28ec34ac68
Update README. (#26572)
Updates the README file for performance tests.
2021-06-30 16:39:18 -07:00
Wanlin Du 82871fe497
Update templates for php7 and php7_protobuf_c examples (#26578)
* Update templates for php7 and php7_protobuf_c examples
* Update templates and loadtest_examples.sh
2021-06-29 20:51:07 -07:00
Wanlin Du d18b6f7df9
Fix executable path for Java workers (#26545)
This commit fix executable path in the prebuilt template for Java
tests.
2021-06-24 05:56:06 -07:00
Wanlin Du f133d8334d
Feature/updating templates to support worker timeouts. (#26516)
This PR update templates for both basic and prebuilt load tests to support worker timeouts.

See also #26505.
2021-06-23 15:02:11 -07:00
Paulo Castello da Costa 41d2db8cb0
Name driver when generating load test configs. (#26501)
This change ensures that a valid driver configuration is always included in generated load test configurations, and that the driver pod is named with an index (`0`, since there is only one driver), in the same way as client and server pods.

Generated examples can be found in https://github.com/grpc/test-infra/pull/189.

With this change, it is no longer necessary to specify a driver image in order to specify a driver name and pool, so that is removed from the kokoro jobs.
2021-06-22 09:15:00 -07:00
Paulo Castello da Costa 8c9f3c2b30
Update tools in preparation for load test timeouts. (#26505)
- Support multiline strings in template generation (as already supported in config generation, supporting roundtrip).
- Ignore substitution of variables that are set by the controller at runtime (`DRIVER_PORT`, `KILL_AFTER`, `POD_TIMEOUT`).
2021-06-21 12:06:30 -07:00
Paulo Castello da Costa 22cbc3a388
Ensure valid formatting of LoadTest names. (#26477)
Names must consist of lowercase alphanumeric characters and dashes, start with an alphabetical character, and not end with a dash.
2021-06-14 16:39:11 -07:00