Commit Graph

223 Commits

Author SHA1 Message Date
Richard Belleville 12897ab0fb [Interop] Make C++ xDS Interop Client and Server Responsive to SIGTERM (#34518)
This PR adds `tini` to the C++ interop client and server to make them responsive to SIGTERM.

Closes #34518

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/34518 from gnossen:ssa_interop_server a762b27178
PiperOrigin-RevId: 610766377
2024-02-27 08:52:21 -08:00
apolcyn 014087ad4e [ruby] fix interop test build (#35948)
Followup to https://github.com/grpc/grpc/pull/35934

Trying to fix:

```
Required ruby-3.0.0 is not installed.
To install do: 'rvm install "ruby-3.0.0"'
```

Closes #35948

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35948 from apolcyn:fix_ri cc92b928f3
PiperOrigin-RevId: 608688718
2024-02-20 12:11:25 -08:00
apolcyn 7a43130d10 [ruby] fix ruby interop build (#35934)
Followup fix for #35911

Closes #35934

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35934 from apolcyn:ruby_stuff fdafeffc05
PiperOrigin-RevId: 607836125
2024-02-16 16:46:00 -08:00
Esun Kim 2ca568663f [CI] Upgraded the docker images for Ruby Distrib Tests (#35911)
- Dropped Ruby 2.7 test (Ruby 2.7 hit EOL 1 year ago)
- Upgraded the base image of Ruby docker images from Debian:10 to Debian:11 (Debian:10 hit EOL)

Closes #35911

PiperOrigin-RevId: 607088383
2024-02-14 13:34:40 -08:00
Yash Tibrewal 264f4f4f54 [interop] Fix ruby docker build (#35438)
Closes #35438

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35438 from yashykt:FixRubyInteropBuild 0442988e8d
PiperOrigin-RevId: 595529644
2024-01-03 16:16:02 -08:00
Eugene Ostroukhov 4295f0250c
[Interop Test] Update Docker images (#34210)
Co-authored-by: Sergii Tkachenko <hi@sergii.org>
2023-08-31 10:25:21 -07:00
Jan Tattermusch a526a091ae
[docker images] Consolidate git directory ownership workaround and upgrade ccache (#34035)
Based on https://github.com/grpc/grpc/pull/34033

Bunch of cleanup and rebuilding many docker images from scratch
- consolidate the workaround for "dubious ownership" issue reported by
git. Other team members have run into this recently and used similar but
not identical workarounds so some cleanup is due.
- rebuilding many images increases the chance that we fix the "dubious
ownership" git issue early on rather than later on in the one-at-a-time
fashion in the future (and the former will prevent many teammembers from
wasting time on this weird issue).
- Newer version of ccache is needed for some portability tests to be
able to benefit from caching (e.g. the GCC 12 portability test to get
benefits of local disk caching) - this is a prerequisite for reenabling
the bazelified gcc12 portability test.
- upgrade node interop images to debian:11 (since debian jessie is long
past EOL).
2023-08-17 00:28:21 +02:00
Jan Tattermusch 4c3ff07dac
[bazel] Reintroduce bazelified ruby and python tests (#34033)
Reintroduce https://github.com/grpc/grpc/pull/33959.

I added a fix for the python arm64 build (which is the reason why the
change has been reverted earlier).
2023-08-16 19:00:19 +02:00
Xuan Wang 08659ad1c3
Revert "[bazel] Add bazelified run_tests.py tests for ruby and python" (#33999)
Reverts grpc/grpc#33959
2023-08-07 12:43:55 -07:00
Jan Tattermusch cb235a3e22
[bazel] Add bazelified run_tests.py tests for ruby and python (#33959)
Context: https://github.com/grpc/grpc/pull/33707

Add python and ruby tests. Both required a few fixes so I wanted to add
them separately from https://github.com/grpc/grpc/pull/33707

Manual run of all the tests:
https://source.cloud.google.com/results/invocations/24c7b6ed-a8b3-4367-a4bf-8828919a9340
2023-08-07 16:14:17 +02:00
Eugene Ostroukhov 1c39493d96
[Interop test] Install curl (#33982)
Curl is required to install Bazel during the build. This also unifies
all build images to use Python base image.
2023-08-03 14:57:41 -07:00
Eugene Ostroukhov 30c68edfcb
[Interop Test] Rollback image changes (#33979)
Fixes build issues
2023-08-03 20:05:48 +00:00
Eugene Ostroukhov 78eb191da5
[Interop Test] Use distroless for C++ xDS tests (#33925)
This should drastically reduce number of vulnerabilities.

Tested locally by running the test
2023-07-31 11:30:45 -07:00
Eugene Ostroukhov e399f1d447
[Build] Update Phusion baseimage (#33767)
Fixes b/282107030

I tested this locally by building the images and running some PSM
interop tests.
2023-07-20 11:26:15 -07:00
Moritz 2c336ac7fd
[tools] Switch to Dart stable (#33498)
Fixing b/288120247 together with
https://github.com/grpc/grpc-dart/pull/644.
2023-06-21 08:44:33 -07:00
Yash Tibrewal f2d5c47ff3
xDS Interop: Update tracers (#32352)
<!--

If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the
appropriate
lang label.

-->
2023-02-10 18:22:07 -08:00
Doug Fawley a65382d9f9
Use go1.19 for go interop tests and for future interop_matrix tests (#31732) 2022-11-22 13:07:36 -08:00
Jan Tattermusch ed39634fe8
Move docker images from dockerhub to artifact registry. (#31591)
* push_testing_images.sh overhaul

* simplify dockerize scripts

* update run_in_docker.sh script

* add generated .current_version files for docker images

* improvements to push_testing_images.sh

* add local-only check for push_testing_images.sh to sanity tests

* fix sanity check

* add expiration for TRANSFER_FROM_DOCKERHUB

* add a readme file
2022-11-17 10:12:39 +01:00
Jan Tattermusch 401c751cb4
Update dart SDK version in dart interop image (#31542)
* update dart SDK version in dart interop image

* fix dart interop docker build
2022-11-04 09:27:27 -07:00
Eric Anderson 9098d8b8db
Upgrade Java interop to Bullseye and Java 11 (#30599) 2022-10-20 13:44:18 -07:00
Jan Tattermusch 3f1c3f23bd
Python virtualenv dockerfile cleanup (#31139)
* python tests install pinned virtualenv themselves

* more alpine image simplifications

* regenerate alpine image

* introduce dockerfile template for xds interop tests harness deps

* cleanup interop_http2 docker image, remove python_deps.include
2022-10-04 19:20:02 +02:00
apolcyn 8b941bb648
Drop support for ruby 2.5 (#30699)
* Drop ruby 2.5 support
2022-08-25 21:39:31 -07:00
Eric Anderson f4c162f30d
Allow Gradle to use more memory when building Java interop (#30249)
Should fix "Expiring Daemon because JVM heap space is exhausted".

https://github.com/grpc/grpc-java/pull/9269 probably pushed the build
over the edge, but there's been evidence via flakes for a good while
that we've been reaching the limit.

b/238438006
2022-07-08 13:00:12 -07:00
Michael Lumish 7b10852a98
xDS k8s: Enable outlier detection tracing in client images (#30227) 2022-07-06 15:17:44 -07:00
Lidi Zheng 347d9ed3ea
Add ring_hash_lb to trace logs for Python/C++ xds interop clients (#29956) 2022-06-08 15:41:29 -07:00
Yash Tibrewal 1e9fe0b8b3
Istio Echo Server Implementation (#29940)
* Istio Echo Server Implementation

* New line fix

* Fix race
2022-06-07 19:13:01 -07:00
Menghan Li 4a855552a9
xds/interop: add a testing server for istio integration test (#29590) 2022-05-20 13:50:16 -07:00
Jan Tattermusch e8404698d5
Remove C# implementation (individual packages will continue to be maintained through v2.46.x patches or moved to grpc-dotnet) (#29225)
* remove examples/csharp

* remove reference to examples/csharp

* remove C# references from doc/

* remove src/csharp

* update test_creds locations

* remove C# distribtests

* remove buildgen templates for files under src/csharp

* remove grpc_csharp_ext C build target

* regenerate projects

* update tools/run_tests/README.md

* change many C# test/build scripts to noop

* unbreak run_tests.py -l csharp

* unbreak C# build_package targets

* unbreak C# distribtests

* remove C# pre-build steps from prepare_build_windows.bat

* remove src/csharp from clang_format

* remove csharp and csharpcoreclr from run_interop_tests.py

* remove C# interop test docker images

* Revert "remove src/csharp"

This reverts commit 97f2b6d707.

* remove src/csharp, but keep Grpc.Tools

* add back templates/src/csharp/build/dependencies.props.template

* remove everything but Grpc.Tools and Grpc.Tools.Tests from sln

* Revert "unbreak run_tests.py -l csharp"

This reverts commit 529b5e2ec1.

* revive Grpc.Tools.Tests through run_tests.py

* Revert "unbreak C# build_package targets"

This reverts commit 79d8d83c8b.

* experiment with build_nuget.sh fixes

* simplify create_fake_native_artifacts.sh

* building of Grpc.Tools in package_targets.py

* move shared files from Grpc.Core to Grpc.Tools

* yapf format: distribtest_targets.py

* Revert "remove C# pre-build steps from prepare_build_windows.bat"

This reverts commit bd38956a2b.

* update C# README.md

* disable C# in gke OSS benchmarks
2022-04-28 12:00:34 +02:00
Michael Lumish 28f1453ad5
Use a non-root user in Node interop docker images (#29062)
* Use a non-root user in Node interop docker images

* Make node interop scripts without being root

* Update cwd in Node interop tests

* Update cwd in other Node interop tests

* Update cwd in Node interop tests relative to hardcoded path

* Fix updated Node interop test cwd

* Make service account file location more generic

* Use service account creds directly from mounted directory

* Fixing source path is recursive local submodule clone.

* Move creds file to /root, make /root accessible

Co-authored-by: Nicolas "Pixel" Noble <pixel@nobis-crew.org>
2022-03-15 14:41:21 -07:00
Jan Tattermusch 6d3c45ff7b
Add dockerized ARM64 tests (bazel C/C++, C#, PHP, ruby, python) (#28966)
* C# arm64 docker image

* add ruby arm64 docker image

* add bazel_arm64 docker image

* add python arm64 image

* add php arm64 docker image

* regenerate

* add support for running arm64 tests with run_tests.py

* restrict arm64 testing to python3.9

* first version of bazel arm64 C++ tests

* add arm64 job configs

* add initial checks to push_testing_images.sh
2022-03-14 13:30:52 +01:00
Jan Tattermusch d15711d2bb
Docker image cleanup (prepare for ARM64 tests) (#29051)
* remove gcc-multilib from base image

* remove useless "apt-get --allow-releaseinfo-change update" from docker images

* get rid of no-longer-useful python_debian11_base.include

* regenerate dockerfiles
2022-03-08 20:31:32 +01:00
Jan Tattermusch 1289cdae2e
Upgrade PHP docker images to debian11 (#28701)
* upgrade PHP docker images to debian11

* regenerate dockerfiles

* adjust testrunners to php debian11 images
2022-01-27 06:45:52 +01:00
Jan Tattermusch b0cdd3cba2
Cleanup (followup for ccache addition) (#28702)
* nit in python interop dockerfile

* address TODO in prepare_ccache_symlinks_rc

* enable ccache for C# and ruby distribtest jobs

* use debian11 instead of bullseye consistently
2022-01-26 18:56:46 +01:00
Jan Tattermusch 1cf6b085f2
Enable ccache in more builds (#28665)
* upgrade ruby docker images to debian 11

* upgrade C# docker images to debian11

* update sanity dockerimage to debian11

* upgrade cxx interop to debian11

* add ccache to python interop images

* enable use of ccache for interop tests

* adjust run_tests.py to new docker images

* add ccache to rake-compiler-dock docker images

* improve prepare_ccache_symlinks_rc

* enable use of ccache in rake-compiler-dock docker containers

* add ccache support for python_manylinux2014_aarch64

* add ccache support for python_linux_armv7

* deduplicate python3.9 install

* ccache for crosscompiled darwin gems is broken

* fix bash -l resetting of PATH in grpc_artifact_python_linux_armv7
2022-01-25 20:37:04 +01:00
apolcyn 3c368a5b2c
Don't gem install rake-compiler on macos setup scripts (#28415)
* Don't gem install rake-compiler on macos setup scripts
2021-12-28 09:57:33 -08:00
Jan Tattermusch 205d7569ea
Big dockerimage cleanup (#28010)
* use python3 for running run_tests.py

* update dockerimage templates

* update dockerfiles

* fix bazel build in the bazel docker image

* adjust run_tests.py to renamed php docker image

* add previously forgotten new templates

* add missing images

* php dockerfile remove whitespaces

* restore accidentally deleted binder_transport_apk image

* add back cxx_buster_x64 image

* add back php zts stretch image

* add back binder_transport_apk image

* fix bazel builds by temporarily adding back python2-dev

* check_redundant_namespace_qualifiers.py to python3

* fix clang-tidy

* make run_tests_matrix.py use python3

* fixup cxx_debian9 templates

* regenerate

* stop running dummy grpc-node tests for protobuf-at-head and boringssl-at-head jobs

* remove cxx_buster image (replaced by cxx_debian9)

* add back python_alpine_x64 image

* run_tests_in_workspace using python3

* unbreak python macos basictests by partially reverting run_tests_matrix to python2

* fix cxx_gcc_X docker images
2021-12-01 12:56:26 +01:00
Jan Tattermusch 79b58431ed
Update csharp SDK to LTS versions (#27966)
* update C# SDK

* regenerate dockerfiles

* install .NET6 and .NET Core 3.1

* regenerate dockerfiles

* change netcoreapp2.1 targets to netcoreapp3.1

* update installed SDKs in aarch64 C# job

* update run_tests scripts to use netcoreapp3.1 for C#
2021-11-17 10:41:02 +01:00
Richard Belleville 179a7c7459
Fix Python Interop (#27620) (#27703) (#27749)
* WIP. Attempt to fix interop

* Yapf
2021-10-15 18:22:29 -07:00
Esun Kim 11d2f86e5e
Upgrade PHP7 docker image to use Debian 9 (#27537)
* Upgrade PHP7 docker image to use Debian 9

* update
2021-10-01 08:33:19 -07:00
Esun Kim 420a98f96b
Update docker images (#27109) 2021-08-24 18:03:45 -07:00
Jan Tattermusch 288fe131b4
Reintroduce #22964 with extra fixes (#26465)
* Revert "Revert "Upgrade dotnet SDK 3 for C# build (#22964)" (#26462)"

This reverts commit 736d914cfd.

* Revert "Exclude csharp from performance test continuous runs. (#26458)"

This reverts commit 6744e8f84a.

* switch remaining tests to C# buster image

* remove C#'s global.json (which requires dotnet SDK 3+)

* dotnet SDK3 currently breaks C# package build

* regenerate dockerfiles
2021-06-15 18:50:13 +02:00
Esun Kim 736d914cfd
Revert "Upgrade dotnet SDK 3 for C# build (#22964)" (#26462)
This reverts commit a44bafec08.
2021-06-10 16:46:44 -07:00
Jan Tattermusch a44bafec08
Upgrade dotnet SDK 3 for C# build (#22964)
* upgrade dotnet SDK 3 for C# build

* add C# global.json

* install dotnet SDK 3 on macos

* upgrade C# docker images to debian buster

* regenerate dockerfiles
2021-06-09 09:25:48 +02:00
Yash Tibrewal 39ab85a6d9
Add logging to xDS client server images (#26327) 2021-05-21 12:53:21 -07:00
Richard Belleville cc22375572
Fix Python xDS k8s Interop (#26158)
* Increase core logging verbosity

* Make repo URL injectable

* Apparently that's not allowed

* Use ipv4 in interop server

* Add CSDS to Python interop client

* Revert CI script change

* Scale back logging

* Move env variable to dockerfile

* And Python too
2021-05-20 17:58:29 -07:00
Jan Tattermusch 802375393a
Use go1.16 for go interop tests and for newly added interop_matrix tests. (#26124)
* use go1.16 for go interop tests

* add go1.16 runtime support for interop_matrix tests

* update copyright headers
2021-04-30 13:29:21 +02:00
Yash Tibrewal 57cb063fb7
Add kokoro job for xds psm security tests (#26033)
* add kokoro job for xds psm security

* Fix scripts

* Cleanup

* Add copyright
2021-04-20 19:23:28 -07:00
Ivan Inozemtsev 5c95dd5c2c
Update dart to 2.12 release (#25657) 2021-03-09 15:39:42 +01:00
Richard Belleville 1437dc9816
Fix pypa link (#25647)
* Fix pypa link

* And 3.5 as well
2021-03-08 17:26:52 -08:00
Lidi Zheng a6c5d6c43e
[xDS Proto] Pin PyPI dependencies in Dockerfiles (#25510) 2021-02-22 10:10:56 -08:00