Commit Graph

543 Commits

Author SHA1 Message Date
Jan Tattermusch ce5b4e949f
Faster and streamlined PHP distribtests (for linux and macos) (#27101)
* faster and streamlined PHP distribtests

* add kokoro job for PHP distribtests

* php distribtests job for macos

* add trailing newline
2021-08-31 09:23:13 +02:00
Lidi Zheng 3dab256776
Make Workload Identity optional (#27189)
* Make Workload Identity optional

* Update tools/run_tests/xds_k8s_test_driver/framework/test_app/server_app.py

Co-authored-by: Sergii Tkachenko <hi@sergii.org>

* Update tools/run_tests/xds_k8s_test_driver/framework/xds_k8s_flags.py

Co-authored-by: Sergii Tkachenko <hi@sergii.org>

* Flip the bool flag naming

* Correct the flag help description

Co-authored-by: Sergii Tkachenko <hi@sergii.org>
2021-08-30 16:51:37 -07:00
Ming-Chuan 73003f8527
Add BinderTransport example server app (#27067)
Some code are commented out because the server interface has not been
merged yet.

After this, user should be able to install both apps on Android device
and test if basic unary call works or not
2021-08-30 18:26:49 +08:00
Jan Tattermusch 7e532fd198
add streamlined and faster ruby distribtest (#27124) 2021-08-27 09:23:50 +02:00
Wanlin Du 2d1ca227a6
Populate the correct value for annotations (#27136)
The commits populates the correct values for gitCommit
and gitActualCommit.
2021-08-25 19:09:08 -07:00
Wanlin Du 2d2576513d
Add annotations to gRPC benchmark CI master workflow. (#27117) 2021-08-24 21:23:07 -07:00
Wanlin Du 793cb25d4a
Fix buildUrl annotation in kokoro job (#27105) 2021-08-24 14:13:21 -07:00
Wanlin Du 971e95b906
Update related scripts to populate BigQuery fields (#26778)
* Update related scripts to populate BigQuery fields
2021-08-24 07:42:35 -07:00
Eric Gribkoff 594e936c41
use fixed c++ server image for python tests (#27096)
* use fixed c++ server image for python tests

* use variable name
2021-08-24 06:34:53 -07:00
Paulo Castello da Costa afb660b595
Use get to update go version in kokoro runner. (#27088)
Installed version is old and does not support install for this purpose.
2021-08-23 06:58:11 -07:00
Paulo Castello da Costa 04f774de3c
Delete logic to list leftover load tests. (#27087)
Tests have strict timeouts, so this logic is no longer needed.
2021-08-23 06:14:29 -07:00
Paulo Castello da Costa 4faaecdfc9
Update xml file location and update go version to 1.17. (#27076)
* Add subdirectories to support change in https://github.com/grpc/test-infra/pull/218.
* Update go version in kokoro runners to go1.17.
2021-08-23 03:50:31 -07:00
Eric Gribkoff 51aea5df15
fix bash endofline (#27075)
* fix bash endofline

* set --server_image

* populate server image global
2021-08-22 23:33:24 -07:00
Eric Gribkoff df3a13a2a3
remove extra url map flagfile (#27071) 2021-08-21 11:18:54 -07:00
Ming-Chuan 9da755a61e
Import binder transport channel create interface (#27007)
Also try use the API in example apk to make sure the binder transport
code and the rest of gRPC compiles with the Android toolchain

We will properly expose our interface later.
2021-08-20 13:03:21 +08:00
Lidi Zheng 9b3ee20242
[xDS UrlMap] Move client_runner's ownership to test classes (#27031)
* [xDS UrlMap] Move client_runner's ownership to test classes

* Extend the Kokoro url map timeout for C++ and Python
2021-08-17 12:48:33 -07:00
Ming-Chuan f0489c8f5f
Add Java helper code for establishing binder transport connection (#26862)
These Java code will be invoked by binder transport C++ implementation
through JNI to establish the connection between client and server.

The code is locally tested with other pending changes to make sure it
works correctly.

For now we only make sure it builds in CI. We will port proper tests
from internal repository later.

A new local repository is created for the Android-only Java code because
1. The analysis of its BUILD will fail without Android SDK configured
2. We want to prevent clang-tidy (and maybe other scripts)'s automatic
expansion of '...' to include it as they typically don't have Android
SDK installed
2021-08-13 09:12:31 +08:00
Menghan Li 3e34e3aac7
xds/testing: print timestamp of commands in xds tests cripts to help debug timeout failures (#27000)
* xds/testing: print timestamp of commands in xds tests cripts to help debug timeout failures

There were recently many kokoro timeout, after job running for 6 hours.
In those, the run_xds_tests.py only ran less then 2 hours.

There was no enough timestamp to find what was taking so long (either
run_xds_tests.py is stuck, or something before it took too long). This
should add more timestamps to help debug.

* print script start time

* format
2021-08-12 17:12:08 -07:00
Richard Belleville 56c18f99b3
And use Python 3 explicitly in C++ interop as well (#26987) 2021-08-11 17:11:04 -07:00
Eric Gribkoff 5fc98218f4
Add Kokoro scripts for k8s insecure tests (#26883)
* Add Kokoro scripts for k8s insecure tests

* change cluster and job names

* update name in cfg

* do not exit immediately if test case fails

* better error message, loop

* fix arr
2021-08-10 22:26:44 -07:00
Lidi Zheng 42e30a26d1
Update xDS client/server image per-branch tag after build (#26955)
* Update xDS client/server image per-branch tag after build

* Address reviewers' comments

* Replace the entire string instead of only prefix

* Update regex

* Update tools/internal_ci/linux/grpc_xds_k8s_install_test_driver.sh

Co-authored-by: Sergii Tkachenko <hi@sergii.org>

Co-authored-by: Sergii Tkachenko <hi@sergii.org>
2021-08-10 13:50:16 -07:00
Richard Belleville 8e0f8beb1f
Ensure we use the Python 3 environment for creating the virtual environment (#26953) 2021-08-09 15:36:23 -07:00
Donghao Qiu 98239e5272
Change master continuous integration test to run on benchmarks-prod2 cluster (#26891) 2021-08-05 14:31:34 -07:00
Donghao Qiu 9cda17468b
Change experimental performance test to run on original cluster (#26877) 2021-08-05 10:40:02 -07:00
Paulo Castello da Costa ce483be968
Specify go version for infra build. (#26881)
This change depends on https://github.com/grpc/test-infra/pull/211.
2021-08-04 21:10:34 -07:00
Donghao Qiu 65ded7c77f
Change experimental performance test run on a new cluster (#26821) 2021-07-28 19:52:29 -07:00
Ming-Chuan db9fa2eaa0
Add Kokoro job for binder transport (#26783)
1. A new Dockerfile is added
2. Required configurations and scripts are added
2021-07-29 06:09:13 +08:00
Ming-Chuan 746eaf6c48
Add BinderTransport example app (#26747)
This commit adds an app that will be served as BinderTransport example
later.

Currently the app simply calls C++ function when button is pressed. In
the future the C++ function will run BinderTransport gRPC example
instead.

Tests not included in this commit, later we will add a build test to
CI to make sure this apk is always build-able. (This app will also be
used to make sure our BinderTransport implementation is compile-able
with Android toolchain, on GitHub) For now we exclude this target in
bazel build test.

Changes are made to WORKSPACE file in order to support android builds.

Build instructions are documented at
examples/android/binder/java/io/grpc/binder/cpp/example/README.md
2021-07-27 11:18:59 +08:00
Lidi Zheng 48ce79f7e5
Allow url-map tests to run concurrently (#26764)
* Allow url-map tests to run concurrently

* Clean-up client namespace if everything goes smoothly

* Make isort happy

* Update tools/internal_ci/linux/grpc_xds_url_map_python.sh

Co-authored-by: Sergii Tkachenko <hi@sergii.org>

* Add logging && change suffix generation

* Move the suffix generation to helpers

* Remove unused import

Co-authored-by: Sergii Tkachenko <hi@sergii.org>
2021-07-23 15:26:13 -07:00
Donghao Qiu e98fa13a57
change to new cluster prod2 (#26749) 2021-07-22 12:25:48 -07:00
Donghao Qiu 69228e202c
add php7 to kokoro run (#26682)
* add php7 to kokoro run

* add php7 to master flow
2021-07-14 09:27:58 -07:00
Lidi Zheng a990f6f987
Revert gRPC Python Kokoro cfg (#26678) 2021-07-13 15:48:55 -07:00
Lidi Zheng 048d6be581
Halt and save GCP resources in our interop tests (#26669)
* Halt and save GCP resources in our interop tests

* Fix typo in argparser

* Add logging when halt after fail is triggered
2021-07-13 08:38:38 -07:00
Sergii Tkachenko b6d9aedba0
xds-k8s: Adapt URL Map tests to use resource prefix/suffix (#26671) 2021-07-12 18:36:12 -07:00
Lidi Zheng 69a130c6ec
Revert the accidentally touched grpc_xds_k8s.sh (#26560) 2021-06-25 14:21:52 -07:00
Lidi Zheng ee4f6854bd
Migrating urlMap related test cases to K8s framework (#26367)
* Build url-map test case class & migrate two test cases

- Migrated header matching
- Migrated path matching

* Polish some details

* Give sanity test the trailing line return it wants

* Address comments & add timeout cases

* Add fault injection and CSDS test cases

* Squashed commit of the following:

commit 0bf96a0e3eae59d76a456b7deed2c6c1314f4228
Author: Lidi Zheng <lidiz@google.com>
Date:   Tue Jun 8 12:35:19 2021 -0700

    Bazel

commit 81b4bf926b656295f1db78ab4c2ff934ab5602af
Author: Lidi Zheng <lidiz@google.com>
Date:   Mon Jun 7 00:04:28 2021 -0700

    Enable Bazel

commit 245026ed696357308141944aa8c4c9f5b2013a4b
Author: Lidi Zheng <lidiz@google.com>
Date:   Fri Jun 4 10:37:48 2021 -0700

    Reuse

commit 564d39d28b2201803bfdb379f78e8298623c4029
Author: Lidi Zheng <lidiz@google.com>
Date:   Fri Jun 4 01:44:56 2021 -0700

    Keep`

commit 83c437d72a76b71948314f7d5389aadca39b3d07
Author: Lidi Zheng <lidiz@google.com>
Date:   Thu Jun 3 18:50:47 2021 -0700

    1800s timeout

commit 1e790a53cd01a1e54bf7df6793381812a9c58c01
Author: Lidi Zheng <lidiz@google.com>
Date:   Thu Jun 3 18:45:41 2021 -0700

    Go

commit 68a99640a100ecf2989f0808bd69a8e17f7156bd
Author: Lidi Zheng <lidiz@google.com>
Date:   Thu Jun 3 17:26:29 2021 -0700

    No

commit d4f6fa5d4a1208c9dfac76e58ca15daa64231c2e
Author: Lidi Zheng <lidiz@google.com>
Date:   Thu Jun 3 17:20:46 2021 -0700

    L

commit 5ccd048f185c1f5a8d5fdd4359f3157d8d64ea2a
Author: Lidi Zheng <lidiz@google.com>
Date:   Thu Jun 3 16:30:55 2021 -0700

    Go

commit 411887c7181a593ba3c98fe09bf9f74d52a6ad9c
Author: Lidi Zheng <lidiz@google.com>
Date:   Thu Jun 3 15:51:34 2021 -0700

    Test

commit 8d005e5203e608c18ffe89fb2f60f81b1ba4ffa8
Author: Lidi Zheng <lidiz@google.com>
Date:   Thu Jun 3 15:11:44 2021 -0700

    Go

commit 7c189cb59077ecfb981ca99a8458cf44f3c10526
Author: Lidi Zheng <lidiz@google.com>
Date:   Thu Jun 3 15:11:03 2021 -0700

    Go

commit 8fa65a68ca9d5b71c47c8bdf5367ed3189e49cba
Author: Lidi Zheng <lidiz@google.com>
Date:   Thu Jun 3 13:08:11 2021 -0700

    Go

* Port changes from dev branch and polish

* Remove unused code

* Fix Bazel build

* Finally find out the secret about rpc-behavior

* Prepare for merging

* Fix bash script and use a more serious name

* Enable port forwarding

* Resolve comments

* Reuse existing code && stylish changes

* Update for interface changes

* Refactor to remove load_tests and explicit global variables

* Improve readability

* Disable AlwaysDelay for now

* Fix issues with resource creation

* Fix unexpected behavior of TestLoader.discover

* Add load_tests to timeout_test

* Improve documentation

* I really shouldn't run tests in parallel at this point

* Resolve reviewers' comments && revert DO-NOT-MERGE changes

* Two small cleanups

* Fix GCP resource creation path

* Polish some test logic

* Correct the non-failfast logic

* Put the accidentally deleted copyright line back

* Separate the Kokoro job from security tests

* Deflake configure and get_stats && remove DO-NOT-MERGE changes
2021-06-25 13:45:33 -07:00
Paulo Castello da Costa 41d2db8cb0
Name driver when generating load test configs. (#26501)
This change ensures that a valid driver configuration is always included in generated load test configurations, and that the driver pod is named with an index (`0`, since there is only one driver), in the same way as client and server pods.

Generated examples can be found in https://github.com/grpc/test-infra/pull/189.

With this change, it is no longer necessary to specify a driver image in order to specify a driver name and pool, so that is removed from the kokoro jobs.
2021-06-22 09:15:00 -07:00
Jan Tattermusch 7351fbb7ac
Make ARM64 C/C++ bazel tests greener (#26518)
* add no_arm64 tag to resolver_component_tests_runner_invoker tests

* skip no_arm64 tests when running on arm64

* increase kokoro jobs timeout for ARM64 C/C++ bazel tests

* use 8 core instance for arm64 bazel C/C++ tests
2021-06-21 18:31:54 +02:00
Sergii Tkachenko 8211702eef
buildscripts: add option to use xds-k8s test driver from a fork (#26490) 2021-06-17 12:29:13 -07:00
Jan Tattermusch 288fe131b4
Reintroduce #22964 with extra fixes (#26465)
* Revert "Revert "Upgrade dotnet SDK 3 for C# build (#22964)" (#26462)"

This reverts commit 736d914cfd.

* Revert "Exclude csharp from performance test continuous runs. (#26458)"

This reverts commit 6744e8f84a.

* switch remaining tests to C# buster image

* remove C#'s global.json (which requires dotnet SDK 3+)

* dotnet SDK3 currently breaks C# package build

* regenerate dockerfiles
2021-06-15 18:50:13 +02:00
Paulo Castello da Costa cbc6f0b0f8
Generate reports for performance tests. (#26475)
This PR integrates report generation into performance test continuous runs.
2021-06-14 09:41:51 -07:00
Paulo Castello da Costa 6744e8f84a
Exclude csharp from performance test continuous runs. (#26458)
This is a workaround for a breakage introduced by https://github.com/grpc/grpc/pull/22964.
2021-06-10 11:19:01 -07:00
Paulo Castello da Costa e76afe5457
Update load test template and config generation. (#26398)
* Update load test template and config generation.

This change includes the following features and fixes:

* Add a script to generate load test examples.
* Update template generation logic to support round trip from configs to templates (handling of repeated clients and servers for the same language and of named clients and servers in source configs).
* Integrate safe language names from scenario config.
* Update template and config formatting (now that we generate in round trip).
* Fix shellcheck lint warnings.
* Update README.md.
2021-06-02 12:57:32 -07:00
Jan Tattermusch ea0e700ea3
Fix typo in python aarch64 testing script. (#26397) 2021-06-01 20:51:11 +02:00
Jan Tattermusch 7f9ff71180
AWS integration: allow copying test reports back to the kokoro instance. (#26380)
* support copying back test reports

* improve exicode logic
2021-05-31 10:44:39 +02:00
Paulo Castello da Costa a63bfcc5b8
Build test-infra tools using the makefile. (#26387) 2021-05-27 20:29:01 -07:00
Paulo Castello da Costa 8bdbb74425
Fix driver configuration in loadtest templates. (#26372)
This change fixes the format and location of the images in the driver configuration, so tests run with the proper driver configuration instead of a default.
2021-05-26 05:02:20 -07:00
Jan Tattermusch e065f22f27
Add kokoro AWS integration and corresponding test jobs. (#26349)
* milestone 1: static build instance, cmake

* on-demand vm per build, and use bazel

* PR cleanup

* pr cleanup: use builtin bazel wrapper

* pr: misc cleanup

* less verbose unzip

* small cleanup of shell scripts and config file

* using rsync for copying workspace is much faster

* simpler way to increase worker disk size

* simplify bazel build

* increase job timeout

* make max instance lifespan setting more obvious

* refactor the exitcode logic

* shutdown AWS instance once possible

* sudo shudown is required

* add useful AWS instance tags

* move aws integration scripts under AWS folder

* adjust scripts

* make sure ssh session closes even if stdout/stderr remains open

* add test scripts for multiple languages

* improvements to the run_remote_test script

* add cfg files for aws kokoro jobs

Co-authored-by: Alexander Midlash <amidlash@google.com>
2021-05-25 20:54:24 +02:00
Paulo Castello da Costa d8c852f101
List leftover tests that have running pods and are in Errored state. (#26296)
* List annotations of tests that have running pods and are in Errored state.

* Fix format.

* Use pod owner reference instead of "loadtest" label.

* Delete loadtests that have running pods and are in Errored state.

* Improve jsonpath expressions.

* Add comment.

* List tests but do not delete them.
2021-05-21 11:35:30 -07:00
Paulo Castello da Costa 1e187b58db
Run all kokoro performance tests on dedicated node pools. (#26286)
* Run all kokoro performance tests on dedicated node pools.

Both official and experimental tests run with kokoro are set to run on separate node pools with the suffix "-ci" (drivers-ci, workers-8core-ci, workers-32core-ci), separate from the default pools used for manual runs.
2021-05-18 22:49:23 -07:00