Commit Graph

1161 Commits

Author SHA1 Message Date
Jan Tattermusch c1216357a3
Upgrade ccache on windows (#32668)
Try to avoid the `ccache: error: Failed to read from file stream` build
failures that started happening once we upgraded to the grpc-win2019
image.

e.g.
https://source.cloud.google.com/results/invocations/e94c2f35-6d1f-4d89-853a-be7e1e1747de/targets/github%2Fgrpc%2Ftoplevel_run_tests_invocations%2Frun_tests_c_windows_opt_native/tests

```
C:\ccache\ccache.exe C:\PROGRA~2\MICROS~1\2017\COMMUN~1\VC\Tools\MSVC\1416~1.270\bin\Hostx64\x86\cl.exe  /nologo /TP -DCARES_STATICLIB -D_CRT_SECURE_NO_WARNINGS -D_SCL_SECURE_NO_WARNINGS -D_WIN32_WINNT=0x600 -D_WINSOCK_DEPRECATED_NO_WARNINGS -IT:\altsrc\github\grpc\workspace_c_windows_opt_native\third_party\re2 -IT:\altsrc\github\grpc\workspace_c_windows_opt_native\third_party\zlib -IT:\altsrc\github\grpc\workspace_c_windows_opt_native -IT:\altsrc\github\grpc\workspace_c_windows_opt_native\include -IT:\altsrc\github\grpc\workspace_c_windows_opt_native\third_party\address_sorting\include -IT:\altsrc\github\grpc\workspace_c_windows_opt_native\cmake\build\third_party\re2 -IT:\altsrc\github\grpc\workspace_c_windows_opt_native\third_party\boringssl-with-bazel\src\include -IT:\altsrc\github\grpc\workspace_c_windows_opt_native\src\core\ext\upb-generated -IT:\altsrc\github\grpc\workspace_c_windows_opt_native\src\core\ext\upbdefs-generated -IT:\altsrc\github\grpc\workspace_c_windows_opt_native\third_party\upb -IT:\altsrc\github\grpc\workspace_c_windows_opt_native\third_party\xxhash -IT:\altsrc\github\grpc\workspace_c_windows_opt_native\cmake\build\third_party\zlib -IT:\altsrc\github\grpc\workspace_c_windows_opt_native\third_party\abseil-cpp -IT:\altsrc\github\grpc\workspace_c_windows_opt_native\cmake\build\third_party\cares\cares -IT:\altsrc\github\grpc\workspace_c_windows_opt_native\third_party\cares\cares -IT:\altsrc\github\grpc\workspace_c_windows_opt_native\third_party\cares\cares\include /DWIN32 /D_WINDOWS /W3 /GR /EHsc  /MP8 /wd4065 /wd4506 /wd4200 /wd4291 /wd4244 /wd4267 /wd4987 /wd4774 /wd4819 /wd4996 /wd4619 /wd4503 /utf-8 /MD /O2 /Ob2 /DNDEBUG /showIncludes /FoCMakeFiles\invalid_call_argument_test.dir\test\core\end2end\cq_verifier.cc.obj /FdCMakeFiles\invalid_call_argument_test.dir\ /FS -c T:\altsrc\github\grpc\workspace_c_windows_opt_native\test\core\end2end\cq_verifier.cc
ccache: error: Failed to read from file stream
```
2023-03-22 08:51:55 +01:00
Jan Tattermusch 5873ba96dc
Ensure compatibility with the new custom kokoro win2019 image (#32649)
Fix incompatibilities identified when running adhoc runs on the new
custom win2019 image.
After merging this, it should be possible to switch to the new image
without breaking any tests.

- for most fixes I added a comment that explains why they're necessary.
- the new image won't have VS2015 installed, so I'm switching the protoc
artifact build to VS2017

This PR will need to be backported to older release branches to ensure
the windows tests continue working on those branches as well (IMHO I
haven't made any changes that would be difficult to backport and I tried
to keeps the diff as small as possible to avoid issues when
backporting).

After we switch to the new image (and all the windows tests are green),
we can incrementally move the builds that are still using VS2017 to
VS2019.
2023-03-20 16:03:50 +01:00
Eugene Ostroukhov 4c5771d27c
PSM Interop: revert adding XDS protos to protoc args (#32640)
This dependency is no longer needed after #32631
2023-03-17 18:50:55 -07:00
Eugene Ostroukhov 555f3e26a4
PSM Interop: add orca proto to the legacy test driver dependencies (#32620)
ref b/273575071
2023-03-14 19:45:02 -07:00
Sergii Tkachenko c16338581d
PSM Interop: add orca proto to the new test driver dependencies (#32619)
Now `messages.proto` requires `xds/v3/orca_load_report.proto`.
The dependency introduced in https://github.com/grpc/grpc/pull/32524.

ref b/273575071
2023-03-14 18:43:34 -07:00
Richard Belleville 86bd0721ea
[fork] Generate GDB backtraces in fork tests on Kokoro (#32535)
<!--

If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the
appropriate
lang label.

-->
2023-03-08 10:30:46 -08:00
Jan Tattermusch 740932c899
Make remaining objc test jobs to pass on MacOS monterey (#32493)
Make remaining objC jobs compatible with kokoro monterey workers and
prepare for boringssl upgrade.

The changes here are taken from https://github.com/grpc/grpc/pull/32357,
but they should be merged in a separate PR
(we need the changes to be able to upgrade to monterey anyway and
there's no reason to make the boringssl upgrade PR more complicated by
bundling more fixes into it).

I've checked that the grpc_basictests_objc_examples and
grpc_ios_binary_size are green if switched to monterey.
Unfortunately it's hard to make grpc_basictests_objc_examples pass on
both monterey and mojave, so I suggest merging this PR at the same time
as CL to upgrade the kokoro jobs to monterey.
- that way both PR and continuous runs will remain green
- older branches would need a backport anyway

---------

Co-authored-by: Hannah Shi <hannahshisfb@gmail.com>
2023-03-01 09:55:33 +01:00
Paulo Castello da Costa d5d509d3f9
Stop php7 image build in OSS benchmarks CI. (#32437)
Follow-up to https://github.com/grpc/grpc/pull/32420.
2023-02-21 10:02:23 -08:00
Paulo Castello da Costa 758f1c2b11
Exclude php7 from OSS benchmarks CI. (#32420)
PHP7 build is failing, removing from CI while investigating the failure.




<!--

If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the
appropriate
lang label.

-->
2023-02-17 12:48:27 -08:00
sanjaypujare 7139bb1d2e
xds-interop-testing: move TESTING_VERSION check to after calling kokoro_setup_test_driver func and fix script_dir (#32405) 2023-02-15 16:59:18 -08:00
sanjaypujare 9dd4b7757d
xds-interop-tests: add cross branch testing to the script (#31569)
Co-authored-by: Sergii Tkachenko <hi@sergii.org>
2023-02-14 12:28:43 -08:00
Hannah Shi 2ac1b1708d
[ObjC] run cpp ios cronet test with bazel (#31808)
Cleanup and remove ios cpp test cronet

To test manually:
./tools/bazel test //src/objective-c/tests:CppCronetTests

@sampajano 
<!--

If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the
appropriate
lang label.

-->
2023-02-13 17:51:03 -08:00
Richard Belleville c608379c70
Freeze xDS interop test runner dependencies (#31859)
* Freeze xDS interop test runner dependencies

* Address review comments

* Reviewer comments on scripts

* Create venv in tmpdir instead

* Moved freeze.sh to bin directory

* Fix shellcheck

* Update tools/run_tests/xds_k8s_test_driver/bin/freeze.sh

Co-authored-by: Sergii Tkachenko <hi@sergii.org>

* Revert

---------

Co-authored-by: Sergii Tkachenko <hi@sergii.org>
2023-02-07 18:32:23 -08:00
Esun Kim ac66562349
Bazel update (#32251) 2023-02-06 13:52:54 -08:00
Esun Kim 6943c1841f
Excluding more tests from build_submodule_at_head (#32290) 2023-02-03 10:25:29 -08:00
Esun Kim 0014c91858
Increased timeout for sanity test variants (#32288) 2023-02-03 09:49:23 -08:00
Sergii Tkachenko 947b30b231
PSM Interop: Switch to the new GKE clusters (except the URL map) (#32280)
Our test clusters are in the broken/dirty state (except the URL map's
"basic" cluster, which isn't affected).

This PR switches to the newly created clusters to:

1. Get a data point on whether newly created clusters are affected
   by the same issues.
2. Allow for descriptive work on the old clusters.
3. Hopefully, bring our tests back to green.

Bonus: more sensible cluster names.
2023-02-02 19:57:49 -08:00
Paulo Castello da Costa 1a5b4e31e4
Filter out tags when getting remote head commit. (#32276) 2023-02-02 17:06:04 -08:00
Richard Belleville 190d095a62
Fix Python epoll1 Fork Support (#32196)
* WIP. A seemingly properly failing test

* WIP. Pre-fork handlers now work

* Roughly working.

* Clean up

* Clean up more

* Add to CI

* Format

* Ugh. Remove swap file

* And another

* clean up

* Add copyright

* Format

* Remove another debug line

* Add stub forkable methods

* Remove use of 3.9+ function

* Remove unintentional double copyright

* drfloob review comments

* Only hold lock during Close once

* Create separate job for fork test

* Bump up gdb timeout

* Format
2023-02-02 14:46:16 -08:00
Esun Kim 9403786b11
Changing parallelism of linux/grpc_portability_build_only (#32246)
* Change args

* Fix args
2023-01-31 14:53:40 -08:00
Yash Tibrewal 82c697e2e7
CI: Move kokoro bazel output root to /tmpfs/bazel (#32156) 2023-01-19 15:32:04 -08:00
Sergii Tkachenko 80e5f98139
xds interop: Fix buildscripts not continuing on a failed test suite (#32093)
Apparently there's a difference between bash 3 and bash 4. OSX comes with bash 3 out-of-box, so for whoever wrote this logic it "worked on my machine".
2023-01-13 18:13:09 -08:00
Sergii Tkachenko 70d0b21e2a
xDS interop: resume circuit_breaking test (#32038)
This is a test running on legacy (GCE) framework.
It was disabled in https://github.com/grpc/grpc/pull/27355,
however it has never been ported to the new framework.
Unless there were reasons I'm not aware of, disabling it is a mistake.
Other languages still run it:

- Java: cc5378453f/buildscripts/kokoro/xds_v3.sh (L21)
- Go: f2fbb0e07e/test/kokoro/xds.sh (L30)
- Node: 81083bd229/packages/grpc-js-xds/scripts/xds.sh (L57)
2023-01-06 14:49:01 -08:00
apolcyn e31cfaaa11
Update default ruby version to 3.2 on linux/macos ruby builds (#32010)
* Use ruby 3.2 in macos and linux ruby builds
2023-01-03 13:38:15 -08:00
apolcyn b27d726342
testing: bump timeout of grpc/abseil-head build job from 4 to 5 hours (#31936) 2023-01-03 12:35:44 +01:00
Hannah Shi 0ca4bbfb9f
run cf stream test with bazel (#31555) 2022-12-06 15:52:16 -08:00
Jan Tattermusch 7ab2a95979
More cleanup after moving testing docker images to artifact registry. (#31689)
* remove DOCKERHUB_ORGANIZATION from prepare_build_interop_rc

* switch rubys rake-compiler-dock to .current_version files

* remove DOCKERHUB_ORGANIZATION from prepare_build_linux_rc

* fix .current_version files for third_party/rake-compiler-dock

* fix: avoid .current_version files named */.current_version

* fix: avoid pushing in local only mode
2022-11-24 14:09:16 +01:00
Sergii Tkachenko 7e850a101f
[tests] Restore grpc_interop_matrix_adhoc.cfg (#31709)
Previously deleted in https://github.com/grpc/grpc/pull/31423.
We still need this build config to execute adhoc run of interop matrix
tests per https://github.com/grpc/grpc/blob/master/tools/interop_matrix/README.md
instructions.
2022-11-18 16:46:58 -08:00
Sergii Tkachenko 517e96ee11
PSM Interop: add missing GCP service account to the cleanup job (#31577)
Recently we had an issue with Service Account IAM policy getting too large.
RC is the cleanup script missing GCP SA account.
This PR adds adds missing `gcp_service_account` to the cleanup script.
2022-11-08 11:11:41 -08:00
Jan Tattermusch 1b00515f90
php7.4 (#31568) 2022-11-07 21:49:50 +01:00
Jan Tattermusch 78fd1f637a
Run grpc_android job under docker for better maintainability. (#31504)
* run grpc_android in a docker container

* add TODO

* fixup android_ndk dockerfile
2022-11-02 11:09:19 +01:00
Jan Tattermusch 7b0fc97bfc
run peformance_profile jobs in docker (#31505) 2022-11-02 08:53:46 +01:00
sanjaypujare 06cd65e813
upgrade to testing 1.50.x (#31512) 2022-10-31 15:27:55 -07:00
Jan Tattermusch 5bfc367cc5
run microbenchmarks_diff under docker (#31503) 2022-10-31 20:51:29 +01:00
Jan Tattermusch 3cf3ed7d59
Revert "Revert "Add back C# distribtests for Grpc.Tools"" (#31436)
* Revert "Revert "Add back C# distribtests for Grpc.Tools (#31399)" (#31412)"

This reverts commit dbc45208e2.

* avoid clash with internal protos
2022-10-27 09:59:06 +02:00
Hannah Shi c7a2fc8e75
[ObjC] Run ios perf test from bazel (#31019)
* run ios CronetTest from bazel

* run ios perf test from bazel

* ci fix

Co-authored-by: Denny C. Dai <dennycd@me.com>
2022-10-24 23:58:32 -07:00
Craig Tiller 29175b97ee
[tests] Remove interop tests from PR flows (#31423) 2022-10-21 11:15:15 -07:00
Alisha Nanda dbc45208e2
Revert "Add back C# distribtests for Grpc.Tools (#31399)" (#31412)
This reverts commit 50bad293c9.
2022-10-20 12:08:29 -07:00
Jan Tattermusch 50bad293c9
Add back C# distribtests for Grpc.Tools (#31399)
* remove remains of grpc_csharp_ext artifact build completely

* add back part of C# distribtests

* redirect C# linux distribtests

* comment out some distribtests

* switch updateversion to Grpc.tools

* fix .proto

* fix distribtest complile

* temporarily comment out service in .proto

* cleanup

* improvements

* reenable mac distribtests

* fixes
2022-10-20 16:42:29 +02:00
Xuan Wang 216fb9b77c
Add back Six for internal_ci scripts (#31390) 2022-10-18 13:33:30 -07:00
Jan Tattermusch 2ebd03d9e2
cleanup grpc_sample_fuzzers job (#31358) 2022-10-17 21:04:07 +02:00
Jan Tattermusch db693b37c2
Move objc TvTests to bazel (and make it "build-only") (#31046)
* try TvTests with bazel

* remove tvos-test-basictests from run_tests.py

* add a build-only test for TvTests

* style fixes
2022-10-17 21:03:50 +02:00
Wanlin Du faf7999a42
Remove installaton of go 1.17 in the CI jobs. (#31379) 2022-10-17 10:38:05 -07:00
Xuan Wang 7aca6d4007
[Remove Six] Remove dependency on six (#31340)
* [Python][Remove Six] Remove Six from requirements
2022-10-17 09:34:45 -07:00
Sergii Tkachenko 6016651794
xDS Interop: update default prefix in the cleanup script (#31325)
We don't have a separate security prefix anymore. The default script was updated in https://github.com/grpc/grpc/pull/30754.
2022-10-12 12:00:40 -07:00
Sergii Tkachenko 279b97ecaa
xDS Interop: require Python 3.7 (#31323) 2022-10-12 11:56:25 -07:00
Jan Tattermusch 1267b0c0f4
fix broken objc test jobs (#31304) 2022-10-10 21:16:32 +02:00
Jan Tattermusch 108444e9f7
Fix python tests on kokoro macos monterey (#31126)
* fix python build on kokoro macos monterey

* avoid using realpath

* fix grpc_distribtests_python on macos monterey

* try upgrade setuptools
2022-10-10 17:45:31 +02:00
Wanlin Du cf917bc365
Move PSM benchmark to a separated cluster. (#31223)
This PR moves the PSM benchmark away from the benchmark-prod2
cluster to psm-benchmarks-performance cluster. This is to avoid
any unexpected change on the OSS benchmarks due to the tuning
we may perform for PSM benchmarks.

The psm-benchmarks-performance cluster has 2 system nodes,
1 driver-ci and 2 worker-ci nodes. The node type is the same as
the benchmark-prod2 cluster.
2022-10-04 15:43:07 -07:00
Sergii Tkachenko 42e45f93a8
[CPP] xDS interop GCE framework: pin grpcio-tools to use protobuf 3.x (#31214) 2022-10-03 11:45:40 -07:00