Compare commits

...

12 Commits

Author SHA1 Message Date
Sergii Tkachenko 61eee8e32c
[PSM Interop] Simplify PSM Interop Kokoro buildscripts (v1.57.x backport) (#36540)
Backport of #36450 to v1.57.x.
---
Integrates the new features of the the Kokoro PSM Interop install
library introduced in grpc/psm-interop#73.

Nearly all common functionality was moved from per-language/per-branch
PSM Interop build scripts to
[psm_interop_kokoro_lib.sh](https://github.com/grpc/psm-interop/blob/main/.kokoro/psm_interop_kokoro_lib.sh):
1. The list of tests in the each test suite 
2. Per-test-suite flag customization
3. `run_test` methods
4. `build_docker_images_if_needed` methods
5. Generic `build_test_app_docker_images` methods (simple docker build +
docker push + docker tag). grpc-java is one exception, as it doesn't run
docker directly, but a cloudbuild flow.

Now all PSM Interop jobs share the same buildscripts by all test suites:
1. buildscript that invokes the test: `psm-interop-test-{language}.sh`
(configured as `build_file` in the build cfg)
2. buildscript that builds the xDS test client/server and publishes them
as a Docker image: `psm-interop-build-{language}.sh` (conventional name
called from `psm_interop_kokoro_lib.sh`)

`psm-interop-test-{language}.sh`:
1. Sets `GRPC_LANGUAGE`, `BUILD_SCRIPT_DIR` environment variables.
2. Downloads the shared `psm_interop_kokoro_lib.sh` from the main branch
of the psm-interop repo.
3. Sources `psm-interop-build-{language}.sh`
4. Calls `psm::run "${PSM_TEST_SUITE}"` (`PSM_TEST_SUITE` configured in
the cfg file).

`psm-interop-build-{language}.sh`:
1. Defines `psm::lang::build_docker_images` which is called from
`psm_interop_kokoro_lib.sh`.
2. Invokes any repo-specific logic.
3. May use `psm::build::docker_images_generic` for generic Docker build,
tag, push, or provide implement its own build/publish method.

References:
- b/288578634
- See the full list of the new features at grpc/psm-interop#73.
- Additional fixes to the shared lib: grpc/psm-interop#78,
grpc/psm-interop#79
2024-05-06 15:15:03 -07:00
Sergii Tkachenko 8478a2978c
[PSM Interop] Migrate to Artifact Registry (v1.57.x backport) (#36316)
Backport of #36273 to v1.57.x.
---
Migrate PSM Interop images from Container Registry (gcr.io) to Artifact
Registry (pkg.dev).
2024-04-09 14:03:10 -07:00
Xuan Wang 78e18edec3
[PSM Interop] Use the Kokoro shared install lib from the new repo (v1.57.x backport) (#35338)
Backport of #35327 to v1.57.x.
---
Source:
4f7ead2c7b/tools/internal_ci/linux/grpc_xds_k8s_install_test_driver.sh
New repo: https://github.com/grpc/psm-interop
New path: `.kokoro/psm_interop_kokoro_lib.sh`

Ref https://github.com/grpc/psm-interop/pull/16

Co-authored-by: Sergii Tkachenko <sergiitk@google.com>
2023-12-20 10:14:15 -08:00
Esun Kim d9c68cba70
[Release] Bump version to 1.57.1 (on v1.57.x branch) (#34623)
Change was created by the release automation script. See go/grpc-release
2023-10-09 10:08:41 -07:00
Esun Kim eccd75cc1a
[Cord] Backport-1.57: Fix MakeCordFromSlice memory bug (#34557)
Backport of https://github.com/grpc/grpc/pull/34549
2023-10-06 16:00:56 -07:00
Eugene Ostroukhov 6c7eb423cc
[Interop Tests][v1.57] Update Docker images (#34358) 2023-09-15 09:34:56 -07:00
AJ Heller a61640173d
[Release] Bump version to 1.57.0 (on v1.57.x branch) (#34008)
Change was created by the release automation script. See go/grpc-release
2023-08-08 10:56:15 -07:00
Eugene Ostroukhov 53053479f9
[v1.57.x][Interop] Backport Python image update (#33868)
Update Phusion base image to address security issues. Original PR:
#33847
2023-07-25 13:28:10 -07:00
apolcyn dfae18fd2f
[ruby] backport "[ruby] remove unnecessary background thread startup wait logic that interferes with forking #33805" to v1.57.x (#33846)
Backports #33805
2023-07-25 08:46:09 -07:00
AJ Heller c1285fe482
[backport][test] Run PHP distribtests with make -j8 (#33831) (#33834)
Related to https://github.com/grpc/grpc/pull/33795.

The tests are still OOMing, and I found that this distribtest script is
still running `make -j ...` on a 16-core machine. Reducing it to `8` to
see if that helps. Example OOM:

https://fusion2.corp.google.com/invocations/c306e6b9-24e3-498d-8971-3e914feea29b/log
2023-07-24 11:51:53 -07:00
AJ Heller 74d8f5c332
[Release] Bump version to 1.57.0-pre1 (on v1.57.x branch) (#33826)
Change was created by the release automation script. See go/grpc-release
2023-07-24 10:24:33 -07:00
Sergii Tkachenko 1e904e82bd
[PSM Interop] Legacy tests: fix xDS test client build (v1.57.x backport) (#33806)
Backport of #33796 to v1.57.x.
---
https://github.com/grpc/grpc/pull/33699 incorrectly changed the legacy
builds to not just use the test driver from the master, but also to
build from it. This PR fixes the issue, and also updates the python job
to work use the driver from master.
2023-07-21 13:38:20 -07:00
72 changed files with 337 additions and 1203 deletions

2
BUILD
View File

@ -215,7 +215,7 @@ g_stands_for = "grounded" # @unused
core_version = "34.0.0" # @unused
version = "1.57.0-dev" # @unused
version = "1.57.1" # @unused
GPR_PUBLIC_HDRS = [
"include/grpc/support/alloc.h",

8
CMakeLists.txt generated
View File

@ -25,12 +25,12 @@
cmake_minimum_required(VERSION 3.8)
set(PACKAGE_NAME "grpc")
set(PACKAGE_VERSION "1.57.0-dev")
set(PACKAGE_VERSION "1.57.1")
set(gRPC_CORE_VERSION "34.0.0")
set(gRPC_CORE_SOVERSION "34")
set(gRPC_CPP_VERSION "1.57.0-dev")
set(gRPC_CPP_VERSION "1.57.1")
set(gRPC_CPP_SOVERSION "1.57")
set(gRPC_CSHARP_VERSION "2.57.0-dev")
set(gRPC_CSHARP_VERSION "2.57.1")
set(gRPC_CSHARP_SOVERSION "2.57")
set(PACKAGE_STRING "${PACKAGE_NAME} ${PACKAGE_VERSION}")
set(PACKAGE_TARNAME "${PACKAGE_NAME}-${PACKAGE_VERSION}")
@ -299,7 +299,7 @@ endif()
if (gRPC_XDS_USER_AGENT_IS_CSHARP)
# The value of the defines needs to contain quotes.
# See https://github.com/grpc/grpc/blob/fbf32836a418cc84f58786700273b65cb9174e1d/src/core/ext/xds/xds_api.cc#L854
add_definitions("-DGRPC_XDS_USER_AGENT_NAME_SUFFIX=\"csharp\"" "-DGRPC_XDS_USER_AGENT_VERSION_SUFFIX=\"2.57.0-dev\"")
add_definitions("-DGRPC_XDS_USER_AGENT_NAME_SUFFIX=\"csharp\"" "-DGRPC_XDS_USER_AGENT_VERSION_SUFFIX=\"2.57.1\"")
endif()
if(UNIX AND NOT HAIKU)

4
Makefile generated
View File

@ -411,8 +411,8 @@ Q = @
endif
CORE_VERSION = 34.0.0
CPP_VERSION = 1.57.0-dev
CSHARP_VERSION = 2.57.0-dev
CPP_VERSION = 1.57.1
CSHARP_VERSION = 2.57.1
CPPFLAGS_NO_ARCH += $(addprefix -I, $(INCLUDES)) $(addprefix -D, $(DEFINES))
CPPFLAGS += $(CPPFLAGS_NO_ARCH) $(ARCH_FLAGS)

View File

@ -14,4 +14,4 @@
# AUTO-GENERATED FROM `$REPO_ROOT/templates/_metadata.py.template`!!!
__version__ = """1.57.0.dev0"""
__version__ = """1.57.1"""

View File

@ -16,7 +16,7 @@ settings:
csharp_major_version: 2
g_stands_for: grounded
protobuf_version: 3.23.4
version: 1.57.0-dev
version: 1.57.1
configs:
asan:
CC: clang

2
config.m4 generated
View File

@ -1323,7 +1323,7 @@ if test "$PHP_GRPC" != "no"; then
-D_HAS_EXCEPTIONS=0 -DNOMINMAX -DGRPC_ARES=0 \
-DGRPC_POSIX_FORK_ALLOW_PTHREAD_ATFORK=1 \
-DGRPC_XDS_USER_AGENT_NAME_SUFFIX='"\"PHP\""' \
-DGRPC_XDS_USER_AGENT_VERSION_SUFFIX='"\"1.57.0dev\""')
-DGRPC_XDS_USER_AGENT_VERSION_SUFFIX='"\"1.57.1\""')
PHP_ADD_BUILD_DIR($ext_builddir/src/core/ext/filters/backend_metrics)
PHP_ADD_BUILD_DIR($ext_builddir/src/core/ext/filters/census)

View File

@ -56,4 +56,4 @@
- 1.54 'g' stands for ['gracious'](https://github.com/grpc/grpc/tree/v1.54.x)
- 1.55 'g' stands for ['grandslam'](https://github.com/grpc/grpc/tree/v1.55.x)
- 1.56 'g' stands for ['galvanized'](https://github.com/grpc/grpc/tree/v1.56.x)
- 1.57 'g' stands for ['grounded'](https://github.com/grpc/grpc/tree/master)
- 1.57 'g' stands for ['grounded'](https://github.com/grpc/grpc/tree/v1.57.x)

2
gRPC-C++.podspec generated
View File

@ -22,7 +22,7 @@
Pod::Spec.new do |s|
s.name = 'gRPC-C++'
# TODO (mxyan): use version that match gRPC version when pod is stabilized
version = '1.57.0-dev'
version = '1.57.1'
s.version = version
s.summary = 'gRPC C++ library'
s.homepage = 'https://grpc.io'

2
gRPC-Core.podspec generated
View File

@ -21,7 +21,7 @@
Pod::Spec.new do |s|
s.name = 'gRPC-Core'
version = '1.57.0-dev'
version = '1.57.1'
s.version = version
s.summary = 'Core cross-platform gRPC library, written in C'
s.homepage = 'https://grpc.io'

2
gRPC-ProtoRPC.podspec generated
View File

@ -21,7 +21,7 @@
Pod::Spec.new do |s|
s.name = 'gRPC-ProtoRPC'
version = '1.57.0-dev'
version = '1.57.1'
s.version = version
s.summary = 'RPC library for Protocol Buffers, based on gRPC'
s.homepage = 'https://grpc.io'

View File

@ -21,7 +21,7 @@
Pod::Spec.new do |s|
s.name = 'gRPC-RxLibrary'
version = '1.57.0-dev'
version = '1.57.1'
s.version = version
s.summary = 'Reactive Extensions library for iOS/OSX.'
s.homepage = 'https://grpc.io'

2
gRPC.podspec generated
View File

@ -20,7 +20,7 @@
Pod::Spec.new do |s|
s.name = 'gRPC'
version = '1.57.0-dev'
version = '1.57.1'
s.version = version
s.summary = 'gRPC client library for iOS/OSX'
s.homepage = 'https://grpc.io'

View File

@ -185,10 +185,17 @@ class ProtoBufferReader : public grpc::protobuf::io::ZeroCopyInputStream {
// This function takes ownership of slice and return a newly created Cord off
// of it.
static absl::Cord MakeCordFromSlice(grpc_slice slice) {
// slice_for_cord is created to keep inlined data of the given slice
grpc_slice* slice_for_cord = new grpc_slice;
*slice_for_cord = slice;
return absl::MakeCordFromExternal(
absl::string_view(reinterpret_cast<char*>(GRPC_SLICE_START_PTR(slice)),
GRPC_SLICE_LENGTH(slice)),
[slice](absl::string_view /* view */) { grpc_slice_unref(slice); });
absl::string_view(
reinterpret_cast<char*>(GRPC_SLICE_START_PTR(*slice_for_cord)),
GRPC_SLICE_LENGTH(*slice_for_cord)),
[slice_for_cord](absl::string_view /* view */) {
grpc_slice_unref(*slice_for_cord);
delete slice_for_cord;
});
}
#endif // GRPC_PROTOBUF_CORD_SUPPORT_ENABLED

View File

@ -20,8 +20,8 @@
#define GRPC_CPP_VERSION_MAJOR 1
#define GRPC_CPP_VERSION_MINOR 57
#define GRPC_CPP_VERSION_PATCH 0
#define GRPC_CPP_VERSION_TAG "dev"
#define GRPC_CPP_VERSION_STRING "1.57.0-dev"
#define GRPC_CPP_VERSION_PATCH 1
#define GRPC_CPP_VERSION_TAG ""
#define GRPC_CPP_VERSION_STRING "1.57.1"
#endif // GRPCPP_VERSION_INFO_H

10
package.xml generated
View File

@ -13,16 +13,16 @@
<date>2019-09-24</date>
<time>16:06:07</time>
<version>
<release>1.57.0dev</release>
<api>1.57.0dev</api>
<release>1.57.1</release>
<api>1.57.1</api>
</version>
<stability>
<release>beta</release>
<api>beta</api>
<release>stable</release>
<api>stable</api>
</stability>
<license>Apache 2.0</license>
<notes>
- gRPC Core 1.57.0 update
- gRPC Core 1.57.1 update
</notes>
<contents>
<dir baseinstalldir="/" name="/">

View File

@ -1,7 +1,7 @@
<!-- This file is generated -->
<Project>
<PropertyGroup>
<GrpcCsharpVersion>2.57.0-dev</GrpcCsharpVersion>
<GrpcCsharpVersion>2.57.1</GrpcCsharpVersion>
<GoogleProtobufVersion>3.23.4</GoogleProtobufVersion>
</PropertyGroup>
</Project>

View File

@ -42,7 +42,7 @@ Pod::Spec.new do |s|
# exclamation mark ensures that other "regular" pods will be able to find it as it'll be installed
# before them.
s.name = '!ProtoCompiler-gRPCCppPlugin'
v = '1.57.0-dev'
v = '1.57.1'
s.version = v
s.summary = 'The gRPC ProtoC plugin generates C++ files from .proto services.'
s.description = <<-DESC

View File

@ -42,7 +42,7 @@ Pod::Spec.new do |s|
# exclamation mark ensures that other "regular" pods will be able to find it as it'll be installed
# before them.
s.name = '!ProtoCompiler-gRPCPlugin'
v = '1.57.0-dev'
v = '1.57.1'
s.version = v
s.summary = 'The gRPC ProtoC plugin generates Objective-C files from .proto services.'
s.description = <<-DESC

View File

@ -22,4 +22,4 @@
// instead. This file can be regenerated from the template by running
// `tools/buildgen/generate_projects.sh`.
#define GRPC_OBJC_VERSION_STRING @"1.57.0-dev"
#define GRPC_OBJC_VERSION_STRING @"1.57.1"

View File

@ -22,5 +22,5 @@
// instead. This file can be regenerated from the template by running
// `tools/buildgen/generate_projects.sh`.
#define GRPC_OBJC_VERSION_STRING @"1.57.0-dev"
#define GRPC_OBJC_VERSION_STRING @"1.57.1"
#define GRPC_C_VERSION_STRING @"34.0.0"

View File

@ -2,7 +2,7 @@
"name": "grpc/grpc-dev",
"description": "gRPC library for PHP - for Development use only",
"license": "Apache-2.0",
"version": "1.57.0",
"version": "1.57.1",
"require": {
"php": ">=7.0.0",
"google/protobuf": "^v3.3.0"

View File

@ -20,6 +20,6 @@
#ifndef VERSION_H
#define VERSION_H
#define PHP_GRPC_VERSION "1.57.0dev"
#define PHP_GRPC_VERSION "1.57.1"
#endif /* VERSION_H */

View File

@ -14,4 +14,4 @@
# AUTO-GENERATED FROM `$REPO_ROOT/templates/src/python/grpcio/grpc/_grpcio_metadata.py.template`!!!
__version__ = """1.57.0.dev0"""
__version__ = """1.57.1"""

View File

@ -14,4 +14,4 @@
# AUTO-GENERATED FROM `$REPO_ROOT/templates/src/python/grpcio/grpc_version.py.template`!!!
VERSION = '1.57.0.dev0'
VERSION = '1.57.1'

View File

@ -14,4 +14,4 @@
# AUTO-GENERATED FROM `$REPO_ROOT/templates/src/python/grpcio_admin/grpc_version.py.template`!!!
VERSION = '1.57.0.dev0'
VERSION = '1.57.1'

View File

@ -14,4 +14,4 @@
# AUTO-GENERATED FROM `$REPO_ROOT/templates/src/python/grpcio_channelz/grpc_version.py.template`!!!
VERSION = '1.57.0.dev0'
VERSION = '1.57.1'

View File

@ -14,4 +14,4 @@
# AUTO-GENERATED FROM `$REPO_ROOT/templates/src/python/grpcio_csds/grpc_version.py.template`!!!
VERSION = '1.57.0.dev0'
VERSION = '1.57.1'

View File

@ -14,4 +14,4 @@
# AUTO-GENERATED FROM `$REPO_ROOT/templates/src/python/grpcio_health_checking/grpc_version.py.template`!!!
VERSION = '1.57.0.dev0'
VERSION = '1.57.1'

View File

@ -14,4 +14,4 @@
# AUTO-GENERATED FROM `$REPO_ROOT/templates/src/python/grpcio_reflection/grpc_version.py.template`!!!
VERSION = '1.57.0.dev0'
VERSION = '1.57.1'

View File

@ -14,4 +14,4 @@
# AUTO-GENERATED FROM `$REPO_ROOT/templates/src/python/grpcio_status/grpc_version.py.template`!!!
VERSION = '1.57.0.dev0'
VERSION = '1.57.1'

View File

@ -14,4 +14,4 @@
# AUTO-GENERATED FROM `$REPO_ROOT/templates/src/python/grpcio_testing/grpc_version.py.template`!!!
VERSION = '1.57.0.dev0'
VERSION = '1.57.1'

View File

@ -14,4 +14,4 @@
# AUTO-GENERATED FROM `$REPO_ROOT/templates/src/python/grpcio_tests/grpc_version.py.template`!!!
VERSION = '1.57.0.dev0'
VERSION = '1.57.1'

View File

@ -1,11 +1,6 @@
FROM phusion/baseimage:master@sha256:65ea10d5f757e5e86272625f8675d437dd83d8db64bdb429e2354d58f5462750
FROM python:3.9-slim-bookworm
RUN apt-get update -y && \
apt-get install -y \
build-essential \
clang \
python3 \
python3-dev
RUN apt-get update -y && apt-get upgrade -y && apt-get install -y build-essential clang curl
WORKDIR /workdir
@ -16,13 +11,18 @@ COPY . .
RUN tools/bazel build -c dbg //src/python/grpcio_tests/tests_py3_only/interop:xds_interop_client
RUN cp -rL /workdir/bazel-bin/src/python/grpcio_tests/tests_py3_only/interop/xds_interop_client* /artifacts/
FROM phusion/baseimage:master@sha256:65ea10d5f757e5e86272625f8675d437dd83d8db64bdb429e2354d58f5462750
COPY --from=0 /artifacts ./
FROM python:3.9-slim-bookworm
ENV GRPC_VERBOSITY="DEBUG"
ENV GRPC_TRACE="xds_client,xds_resolver,xds_cluster_manager_lb,cds_lb,xds_cluster_resolver_lb,priority_lb,xds_cluster_impl_lb,weighted_target_lb,ring_hash_lb,outlier_detection_lb"
RUN apt-get update -y && apt-get install -y python3
RUN apt-get update -y \
&& apt-get install -y python3 \
&& apt-get -y autoremove \
&& rm -rf /var/lib/apt/lists/*
RUN ln -s /usr/bin/python3 /usr/bin/python
COPY --from=0 /artifacts ./
ENTRYPOINT ["/xds_interop_client"]

View File

@ -1,11 +1,6 @@
FROM phusion/baseimage:master@sha256:65ea10d5f757e5e86272625f8675d437dd83d8db64bdb429e2354d58f5462750
FROM python:3.9-slim-bookworm
RUN apt-get update -y && \
apt-get install -y \
build-essential \
clang \
python3 \
python3-dev
RUN apt-get update -y && apt-get upgrade -y && apt-get install -y build-essential clang curl
WORKDIR /workdir
@ -16,13 +11,18 @@ COPY . .
RUN tools/bazel build -c dbg //src/python/grpcio_tests/tests_py3_only/interop:xds_interop_server
RUN cp -rL /workdir/bazel-bin/src/python/grpcio_tests/tests_py3_only/interop/xds_interop_server* /artifacts/
FROM phusion/baseimage:master@sha256:65ea10d5f757e5e86272625f8675d437dd83d8db64bdb429e2354d58f5462750
COPY --from=0 /artifacts ./
FROM python:3.9-slim-bookworm
ENV GRPC_VERBOSITY="DEBUG"
ENV GRPC_TRACE="xds_client,xds_resolver,xds_cluster_manager_lb,cds_lb,xds_cluster_resolver_lb,priority_lb,xds_cluster_impl_lb,weighted_target_lb"
RUN apt-get update -y && apt-get install -y python3
RUN apt-get update -y \
&& apt-get install -y python3 \
&& apt-get -y autoremove \
&& rm -rf /var/lib/apt/lists/*
RUN ln -s /usr/bin/python3 /usr/bin/python
COPY --from=0 /artifacts ./
ENTRYPOINT ["/xds_interop_server"]

View File

@ -0,0 +1,44 @@
#!/usr/bin/env ruby
#
# Copyright 2016 gRPC authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
ENV['GRPC_ENABLE_FORK_SUPPORT'] = "1"
fail "forking only supported on linux" unless RUBY_PLATFORM =~ /linux/
this_dir = File.expand_path(File.dirname(__FILE__))
protos_lib_dir = File.join(this_dir, 'lib')
grpc_lib_dir = File.join(File.dirname(this_dir), 'lib')
$LOAD_PATH.unshift(grpc_lib_dir) unless $LOAD_PATH.include?(grpc_lib_dir)
$LOAD_PATH.unshift(protos_lib_dir) unless $LOAD_PATH.include?(protos_lib_dir)
$LOAD_PATH.unshift(this_dir) unless $LOAD_PATH.include?(this_dir)
require 'grpc'
require 'end2end_common'
def main
10_000.times do
# The prefork and postfork APIs are meant to be used before and after a
# fork. So this is not technically correct usage of the API. However, the
# current implementation doesn't actually care about a "fork" call happening
# in between prefork and postfork_parent, and this is unlikely to change anytime
# soon. Also note the goal of this test is mainly to test the background thread startup
# and shutdown that happens in prefork and postfork_parent. If we were to actually
# fork in this test, it would take much longer to run.
GRPC.prefork
GRPC.postfork_parent
end
end
main

View File

@ -97,8 +97,6 @@ static bg_watched_channel* bg_watched_channel_list_head = NULL;
static void grpc_rb_channel_try_register_connection_polling(
bg_watched_channel* bg);
static void* wait_until_channel_polling_thread_started_no_gil(void*);
static void wait_until_channel_polling_thread_started_unblocking_func(void*);
static void* channel_init_try_register_connection_polling_without_gil(
void* arg);
@ -111,7 +109,6 @@ static grpc_completion_queue* g_channel_polling_cq;
static gpr_mu global_connection_polling_mu;
static gpr_cv global_connection_polling_cv;
static int g_abort_channel_polling = 0;
static int g_channel_polling_thread_started = 0;
static gpr_once g_once_init = GPR_ONCE_INIT;
static VALUE g_channel_polling_thread = Qnil;
@ -224,12 +221,6 @@ static VALUE grpc_rb_channel_init(int argc, VALUE* argv, VALUE self) {
channel_init_try_register_stack stack;
grpc_ruby_fork_guard();
int stop_waiting_for_thread_start = 0;
rb_thread_call_without_gvl(
wait_until_channel_polling_thread_started_no_gil,
&stop_waiting_for_thread_start,
wait_until_channel_polling_thread_started_unblocking_func,
&stop_waiting_for_thread_start);
/* "3" == 3 mandatory args */
rb_scan_args(argc, argv, "3", &target, &channel_args, &credentials);
@ -286,7 +277,6 @@ static void* get_state_without_gil(void* arg) {
get_state_stack* stack = (get_state_stack*)arg;
gpr_mu_lock(&global_connection_polling_mu);
GPR_ASSERT(g_abort_channel_polling || g_channel_polling_thread_started);
if (stack->bg->channel_destroyed) {
stack->out = GRPC_CHANNEL_SHUTDOWN;
} else {
@ -423,15 +413,8 @@ static void grpc_rb_channel_maybe_recreate_channel_after_fork(
bg_watched_channel* bg = wrapper->bg_wrapped;
if (bg->channel_destroyed) {
// There must be one ref at this point, held by the ruby-level channel
// object.
// object, drop this one here.
GPR_ASSERT(bg->refcount == 1);
// Wait for channel polling thread to re-initialize
int stop_waiting_for_thread_start = 0;
rb_thread_call_without_gvl(
wait_until_channel_polling_thread_started_no_gil,
&stop_waiting_for_thread_start,
wait_until_channel_polling_thread_started_unblocking_func,
&stop_waiting_for_thread_start);
rb_thread_call_without_gvl(channel_safe_destroy_without_gil, bg, NULL,
NULL);
// re-create C-core channel
@ -635,9 +618,6 @@ static void grpc_rb_channel_try_register_connection_polling(
bg_watched_channel* bg) {
grpc_connectivity_state conn_state;
watch_state_op* op = NULL;
GPR_ASSERT(g_channel_polling_thread_started || g_abort_channel_polling);
if (bg->refcount == 0) {
GPR_ASSERT(bg->channel_destroyed);
bg_watched_channel_list_free_and_remove(bg);
@ -647,7 +627,6 @@ static void grpc_rb_channel_try_register_connection_polling(
if (bg->channel_destroyed || g_abort_channel_polling) {
return;
}
conn_state = grpc_channel_check_connectivity_state(bg->channel, 0);
if (conn_state == GRPC_CHANNEL_SHUTDOWN) {
return;
@ -655,7 +634,6 @@ static void grpc_rb_channel_try_register_connection_polling(
GPR_ASSERT(bg_watched_channel_list_lookup(bg));
// prevent bg from being free'd by GC while background thread is watching it
bg->refcount++;
op = gpr_zalloc(sizeof(watch_state_op));
op->op_type = CONTINUOUS_WATCH;
op->op.continuous_watch_callback_args.bg = bg;
@ -678,9 +656,6 @@ static void* run_poll_channels_loop_no_gil(void* arg) {
gpr_log(GPR_DEBUG, "GRPC_RUBY: run_poll_channels_loop_no_gil - begin");
gpr_mu_lock(&global_connection_polling_mu);
GPR_ASSERT(!g_abort_channel_polling);
GPR_ASSERT(!g_channel_polling_thread_started);
g_channel_polling_thread_started = 1;
gpr_cv_broadcast(&global_connection_polling_cv);
gpr_mu_unlock(&global_connection_polling_mu);
@ -761,31 +736,6 @@ static VALUE run_poll_channels_loop(VALUE arg) {
return Qnil;
}
static void* wait_until_channel_polling_thread_started_no_gil(void* arg) {
int* stop_waiting = (int*)arg;
gpr_log(GPR_DEBUG, "GRPC_RUBY: wait for channel polling thread to start");
gpr_mu_lock(&global_connection_polling_mu);
while (!g_channel_polling_thread_started && !g_abort_channel_polling &&
!*stop_waiting) {
gpr_cv_wait(&global_connection_polling_cv, &global_connection_polling_mu,
gpr_inf_future(GPR_CLOCK_REALTIME));
}
gpr_mu_unlock(&global_connection_polling_mu);
return NULL;
}
static void wait_until_channel_polling_thread_started_unblocking_func(
void* arg) {
int* stop_waiting = (int*)arg;
gpr_mu_lock(&global_connection_polling_mu);
gpr_log(GPR_DEBUG,
"GRPC_RUBY: interrupt wait for channel polling thread to start");
*stop_waiting = 1;
gpr_cv_broadcast(&global_connection_polling_cv);
gpr_mu_unlock(&global_connection_polling_mu);
}
static void* set_abort_channel_polling_without_gil(void* arg) {
(void)arg;
gpr_mu_lock(&global_connection_polling_mu);
@ -814,7 +764,6 @@ void grpc_rb_channel_polling_thread_start() {
gpr_once_init(&g_once_init, do_basic_init);
GPR_ASSERT(!RTEST(g_channel_polling_thread));
GPR_ASSERT(!g_abort_channel_polling);
GPR_ASSERT(!g_channel_polling_thread_started);
GPR_ASSERT(g_channel_polling_cq == NULL);
g_channel_polling_cq = grpc_completion_queue_create_for_next(NULL);
@ -841,7 +790,6 @@ void grpc_rb_channel_polling_thread_stop() {
// we can start again later
g_channel_polling_thread = Qnil;
g_abort_channel_polling = false;
g_channel_polling_thread_started = false;
g_channel_polling_cq = NULL;
}

View File

@ -14,5 +14,5 @@
# GRPC contains the General RPC module.
module GRPC
VERSION = '1.57.0.dev'
VERSION = '1.57.1'
end

View File

@ -14,6 +14,6 @@
module GRPC
module Tools
VERSION = '1.57.0.dev'
VERSION = '1.57.1'
end
end

View File

@ -23,6 +23,6 @@ cp -r "$EXTERNAL_GIT_ROOT"/input_artifacts/grpc-*.tgz .
# the exact version string in advance)
GRPC_PEAR_PACKAGE_NAME=$(find . -regex '.*/grpc-[0-9].*.tgz' | sed 's|./||')
MAKEFLAGS=-j pecl install "${GRPC_PEAR_PACKAGE_NAME}"
MAKEFLAGS=-j8 pecl install "${GRPC_PEAR_PACKAGE_NAME}"
php -d extension=grpc.so -d max_execution_time=300 distribtest.php

View File

@ -14,5 +14,5 @@
# AUTO-GENERATED FROM `$REPO_ROOT/templates/tools/distrib/python/grpcio_tools/grpc_version.py.template`!!!
VERSION = '1.57.0.dev0'
VERSION = '1.57.1'
PROTOBUF_VERSION = '3.23.4'

View File

@ -14,5 +14,5 @@
# AUTO-GENERATED FROM `$REPO_ROOT/templates/tools/distrib/python/grpcio_tools/grpc_version.py.template`!!!
VERSION = '1.57.0.dev0'
VERSION = '1.57.1'
PROTOBUF_VERSION = '3.23.4'

View File

@ -14,14 +14,9 @@
# Dockerfile for building //test/cpp/interop:xds_interop_client
FROM phusion/baseimage:master@sha256:e757fe8c7adcb9f798c0eb9dfff31bbf7d91538a1002031d7cdf3e5bf9cf71fc
FROM python:3.9-slim-bookworm
RUN apt-get update -y && \
apt-get install -y \
build-essential \
clang \
python3 \
python3-dev
RUN apt-get update -y && apt-get upgrade -y && apt-get install -y build-essential clang curl
WORKDIR /workdir
@ -32,10 +27,16 @@ COPY . .
RUN tools/bazel build //test/cpp/interop:xds_interop_client
RUN cp -rL /workdir/bazel-bin/test/cpp/interop/xds_interop_client /artifacts/
FROM phusion/baseimage:master@sha256:e757fe8c7adcb9f798c0eb9dfff31bbf7d91538a1002031d7cdf3e5bf9cf71fc
COPY --from=0 /artifacts ./
FROM python:3.9-slim-bookworm
ENV GRPC_VERBOSITY="DEBUG"
ENV GRPC_TRACE="xds_client,xds_resolver,xds_cluster_manager_lb,cds_lb,xds_cluster_resolver_lb,priority_lb,xds_cluster_impl_lb,weighted_target_lb,xds_server_config_fetcher,ring_hash_lb,outlier_detection_lb,xds_wrr_locality_lb,xds_override_host_lb"
RUN apt-get update \
&& apt-get -y upgrade \
&& apt-get -y autoremove \
&& rm -rf /var/lib/apt/lists/*
COPY --from=0 /artifacts ./
ENTRYPOINT ["/xds_interop_client"]

View File

@ -14,14 +14,9 @@
# Dockerfile for building //test/cpp/interop:xds_interop_client
FROM phusion/baseimage:master@sha256:e757fe8c7adcb9f798c0eb9dfff31bbf7d91538a1002031d7cdf3e5bf9cf71fc
FROM python:3.9-slim-bookworm
RUN apt-get update -y && \
apt-get install -y \
build-essential \
clang \
python3 \
python3-dev
RUN apt-get update -y && apt-get upgrade -y && apt-get install -y build-essential clang curl
WORKDIR /workdir
@ -32,10 +27,16 @@ COPY . .
RUN tools/bazel build //test/cpp/interop:xds_interop_server
RUN cp -rL /workdir/bazel-bin/test/cpp/interop/xds_interop_server /artifacts/
FROM phusion/baseimage:master@sha256:e757fe8c7adcb9f798c0eb9dfff31bbf7d91538a1002031d7cdf3e5bf9cf71fc
COPY --from=0 /artifacts ./
FROM python:3.9-slim-bookworm
ENV GRPC_VERBOSITY="DEBUG"
ENV GRPC_TRACE="xds_client,xds_resolver,xds_cluster_manager_lb,cds_lb,xds_cluster_resolver_lb,priority_lb,xds_cluster_impl_lb,weighted_target_lb,xds_server_config_fetcher,ring_hash_lb,outlier_detection_lb,xds_wrr_locality_lb,xds_override_host_lb"
RUN apt-get update \
&& apt-get -y upgrade \
&& apt-get -y autoremove \
&& rm -rf /var/lib/apt/lists/*
COPY --from=0 /artifacts ./
ENTRYPOINT ["/xds_interop_server"]

View File

@ -40,7 +40,7 @@ PROJECT_NAME = "GRPC C++"
# could be handy for archiving the generated documentation or if some version
# control system is used.
PROJECT_NUMBER = 1.57.0-dev
PROJECT_NUMBER = 1.57.1
# Using the PROJECT_BRIEF tag one can provide an optional one line description
# for a project that appears at the top of each page and should give viewer a

View File

@ -40,7 +40,7 @@ PROJECT_NAME = "GRPC C++"
# could be handy for archiving the generated documentation or if some version
# control system is used.
PROJECT_NUMBER = 1.57.0-dev
PROJECT_NUMBER = 1.57.1
# Using the PROJECT_BRIEF tag one can provide an optional one line description
# for a project that appears at the top of each page and should give viewer a

View File

@ -40,7 +40,7 @@ PROJECT_NAME = "GRPC Objective-C"
# could be handy for archiving the generated documentation or if some version
# control system is used.
PROJECT_NUMBER = 1.57.0-dev
PROJECT_NUMBER = 1.57.1
# Using the PROJECT_BRIEF tag one can provide an optional one line description
# for a project that appears at the top of each page and should give viewer a

View File

@ -40,7 +40,7 @@ PROJECT_NAME = "GRPC Objective-C"
# could be handy for archiving the generated documentation or if some version
# control system is used.
PROJECT_NUMBER = 1.57.0-dev
PROJECT_NUMBER = 1.57.1
# Using the PROJECT_BRIEF tag one can provide an optional one line description
# for a project that appears at the top of each page and should give viewer a

View File

@ -40,7 +40,7 @@ PROJECT_NAME = "GRPC PHP"
# could be handy for archiving the generated documentation or if some version
# control system is used.
PROJECT_NUMBER = 1.57.0-dev
PROJECT_NUMBER = 1.57.1
# Using the PROJECT_BRIEF tag one can provide an optional one line description
# for a project that appears at the top of each page and should give viewer a

View File

@ -16,11 +16,9 @@
trap 'date' DEBUG
set -ex -o igncr || set -ex
mkdir -p /var/local/git
git clone /var/local/jenkins/grpc /var/local/git/grpc
(cd /var/local/jenkins/grpc/ && git submodule foreach 'cd /var/local/git/grpc \
&& git submodule update --init --reference /var/local/jenkins/grpc/${name} \
${name}')
git clone -b master --single-branch --depth=1 https://github.com/grpc/grpc.git /var/local/git/grpc
cd /var/local/git/grpc
python3 -m pip install virtualenv
@ -62,13 +60,14 @@ touch "$TOOLS_DIR"/src/proto/grpc/health/v1/__init__.py
--grpc_python_out=${TOOLS_DIR} \
${HEALTH_PROTO_SOURCE_DIR}/health.proto
cd /var/local/jenkins/grpc/
bazel build //src/python/grpcio_tests/tests_py3_only/interop:xds_interop_client
# Test cases "path_matching" and "header_matching" are not included in "all",
# because not all interop clients in all languages support these new tests.
export PYTHONUNBUFFERED=true
GRPC_VERBOSITY=debug GRPC_TRACE=xds_client,xds_resolver,xds_cluster_manager_lb,cds_lb,xds_cluster_resolver_lb,priority_lb,xds_cluster_impl_lb,weighted_target_lb "$PYTHON" \
tools/run_tests/run_xds_tests.py \
/var/local/git/grpc/tools/run_tests/run_xds_tests.py \
--halt_after_fail \
--test_case="ping_pong,circuit_breaking" \
--project_id=grpc-testing \

View File

@ -59,6 +59,7 @@ touch "$TOOLS_DIR"/src/proto/grpc/health/v1/__init__.py
--grpc_python_out=${TOOLS_DIR} \
${HEALTH_PROTO_SOURCE_DIR}/health.proto
cd /var/local/jenkins/grpc/
bazel build test/cpp/interop:xds_interop_client
# Test cases "path_matching" and "header_matching" are not included in "all",
@ -67,7 +68,7 @@ bazel build test/cpp/interop:xds_interop_client
# TODO: remove "path_matching" and "header_matching" from --test_case after
# they are added into "all".
GRPC_VERBOSITY=debug GRPC_TRACE=xds_client,xds_resolver,xds_cluster_manager_lb,cds_lb,xds_cluster_resolver_lb,priority_lb,xds_cluster_impl_lb,weighted_target_lb "$PYTHON" \
tools/run_tests/run_xds_tests.py \
/var/local/git/grpc/tools/run_tests/run_xds_tests.py \
--halt_after_fail \
--test_case="ping_pong,circuit_breaking" \
--project_id=grpc-testing \

View File

@ -15,8 +15,8 @@
# Config file for the internal CI (in protobuf text format)
# Location of the continuous shell script in repository.
build_file: "grpc/tools/internal_ci/linux/grpc_xds_k8s_lb.sh"
timeout_mins: 180
build_file: "grpc/tools/internal_ci/linux/psm-interop-test-cpp.sh"
timeout_mins: 240
action {
define_artifacts {
regex: "artifacts/**/*sponge_log.xml"
@ -24,3 +24,7 @@ action {
strip_prefix: "artifacts"
}
}
env_vars {
key: "PSM_TEST_SUITE"
value: "lb"
}

View File

@ -1,189 +0,0 @@
#!/usr/bin/env bash
# Copyright 2021 gRPC authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
set -eo pipefail
# Constants
readonly GITHUB_REPOSITORY_NAME="grpc"
readonly TEST_DRIVER_INSTALL_SCRIPT_URL="https://raw.githubusercontent.com/${TEST_DRIVER_REPO_OWNER:-grpc}/grpc/${TEST_DRIVER_BRANCH:-master}/tools/internal_ci/linux/grpc_xds_k8s_install_test_driver.sh"
## xDS test client Docker images
readonly SERVER_IMAGE_NAME="gcr.io/grpc-testing/xds-interop/cpp-server"
readonly CLIENT_IMAGE_NAME="gcr.io/grpc-testing/xds-interop/cpp-client"
readonly FORCE_IMAGE_BUILD="${FORCE_IMAGE_BUILD:-0}"
readonly BUILD_APP_PATH="interop-testing/build/install/grpc-interop-testing"
#######################################
# Builds test app Docker images and pushes them to GCR
# Globals:
# BUILD_APP_PATH
# SERVER_IMAGE_NAME: Test server Docker image name
# CLIENT_IMAGE_NAME: Test client Docker image name
# GIT_COMMIT: SHA-1 of git commit being built
# Arguments:
# None
# Outputs:
# Writes the output of `gcloud builds submit` to stdout, stderr
#######################################
build_test_app_docker_images() {
echo "Building C++ xDS interop test app Docker images"
docker build -f "${SRC_DIR}/tools/dockerfile/interoptest/grpc_interop_cxx_xds/Dockerfile.xds_client" -t "${CLIENT_IMAGE_NAME}:${GIT_COMMIT}" "${SRC_DIR}"
docker build -f "${SRC_DIR}/tools/dockerfile/interoptest/grpc_interop_cxx_xds/Dockerfile.xds_server" -t "${SERVER_IMAGE_NAME}:${GIT_COMMIT}" "${SRC_DIR}"
gcloud -q auth configure-docker
docker push "${CLIENT_IMAGE_NAME}:${GIT_COMMIT}"
docker push "${SERVER_IMAGE_NAME}:${GIT_COMMIT}"
if is_version_branch "${TESTING_VERSION}"; then
tag_and_push_docker_image "${CLIENT_IMAGE_NAME}" "${GIT_COMMIT}" "${TESTING_VERSION}"
tag_and_push_docker_image "${SERVER_IMAGE_NAME}" "${GIT_COMMIT}" "${TESTING_VERSION}"
fi
}
#######################################
# Builds test app and its docker images unless they already exist
# Globals:
# SERVER_IMAGE_NAME: Test server Docker image name
# CLIENT_IMAGE_NAME: Test client Docker image name
# GIT_COMMIT: SHA-1 of git commit being built
# FORCE_IMAGE_BUILD
# Arguments:
# None
# Outputs:
# Writes the output to stdout, stderr
#######################################
build_docker_images_if_needed() {
# Check if images already exist
server_tags="$(gcloud_gcr_list_image_tags "${SERVER_IMAGE_NAME}" "${GIT_COMMIT}")"
printf "Server image: %s:%s\n" "${SERVER_IMAGE_NAME}" "${GIT_COMMIT}"
echo "${server_tags:-Server image not found}"
client_tags="$(gcloud_gcr_list_image_tags "${CLIENT_IMAGE_NAME}" "${GIT_COMMIT}")"
printf "Client image: %s:%s\n" "${CLIENT_IMAGE_NAME}" "${GIT_COMMIT}"
echo "${client_tags:-Client image not found}"
# Build if any of the images are missing, or FORCE_IMAGE_BUILD=1
if [[ "${FORCE_IMAGE_BUILD}" == "1" || -z "${server_tags}" || -z "${client_tags}" ]]; then
build_test_app_docker_images
else
echo "Skipping C++ test app build"
fi
}
#######################################
# Executes the test case
# Globals:
# TEST_DRIVER_FLAGFILE: Relative path to test driver flagfile
# KUBE_CONTEXT: The name of kubectl context with GKE cluster access
# SECONDARY_KUBE_CONTEXT: The name of kubectl context with secondary GKE cluster access, if any
# TEST_XML_OUTPUT_DIR: Output directory for the test xUnit XML report
# SERVER_IMAGE_NAME: Test server Docker image name
# CLIENT_IMAGE_NAME: Test client Docker image name
# GIT_COMMIT: SHA-1 of git commit being built
# TESTING_VERSION: version branch under test: used by the framework to determine the supported PSM
# features.
# Arguments:
# Test case name
# Outputs:
# Writes the output of test execution to stdout, stderr
# Test xUnit report to ${TEST_XML_OUTPUT_DIR}/${test_name}/sponge_log.xml
#######################################
run_test() {
# Test driver usage:
# https://github.com/grpc/grpc/tree/master/tools/run_tests/xds_k8s_test_driver#basic-usage
local test_name="${1:?Usage: run_test test_name}"
local out_dir="${TEST_XML_OUTPUT_DIR}/${test_name}"
mkdir -pv "${out_dir}"
set -x
python3 -m "tests.${test_name}" \
--flagfile="${TEST_DRIVER_FLAGFILE}" \
--kube_context="${KUBE_CONTEXT}" \
--secondary_kube_context="${SECONDARY_KUBE_CONTEXT}" \
--server_image="${SERVER_IMAGE_NAME}:${GIT_COMMIT}" \
--client_image="${CLIENT_IMAGE_NAME}:${GIT_COMMIT}" \
--testing_version="${TESTING_VERSION}" \
--force_cleanup \
--collect_app_logs \
--log_dir="${out_dir}" \
--xml_output_file="${out_dir}/sponge_log.xml" \
${@:2} \
|& tee "${out_dir}/sponge_log.log"
}
run_alpha_test() {
local test_name=$1
run_test ${test_name} \
--compute_api_version="v1alpha"
}
#######################################
# Main function: provision software necessary to execute tests, and run them
# Globals:
# KOKORO_ARTIFACTS_DIR
# GITHUB_REPOSITORY_NAME
# SRC_DIR: Populated with absolute path to the source repo
# TEST_DRIVER_REPO_DIR: Populated with the path to the repo containing
# the test driver
# TEST_DRIVER_FULL_DIR: Populated with the path to the test driver source code
# TEST_DRIVER_FLAGFILE: Populated with relative path to test driver flagfile
# TEST_XML_OUTPUT_DIR: Populated with the path to test xUnit XML report
# GIT_ORIGIN_URL: Populated with the origin URL of git repo used for the build
# GIT_COMMIT: Populated with the SHA-1 of git commit being built
# GIT_COMMIT_SHORT: Populated with the short SHA-1 of git commit being built
# KUBE_CONTEXT: Populated with name of kubectl context with GKE cluster access
# SECONDARY_KUBE_CONTEXT: Populated with name of kubectl context with secondary GKE cluster access, if any
# Arguments:
# None
# Outputs:
# Writes the output of test execution to stdout, stderr
#######################################
main() {
local script_dir
script_dir="$(dirname "$0")"
# Source the test captured from the master branch.
echo "Sourcing test driver install captured from: ${TEST_DRIVER_INSTALL_SCRIPT_URL}"
source /dev/stdin <<< "$(curl -s "${TEST_DRIVER_INSTALL_SCRIPT_URL}")"
activate_gke_cluster GKE_CLUSTER_PSM_LB
activate_secondary_gke_cluster GKE_CLUSTER_PSM_LB
set -x
if [[ -n "${KOKORO_ARTIFACTS_DIR}" ]]; then
kokoro_setup_test_driver "${GITHUB_REPOSITORY_NAME}"
else
local_setup_test_driver "${script_dir}"
fi
build_docker_images_if_needed
# Run tests
cd "${TEST_DRIVER_FULL_DIR}"
local failed_tests=0
run_alpha_test subsetting_test || (( ++failed_tests ))
test_suites=(
"app_net_test"
"affinity_test"
"api_listener_test"
"change_backend_service_test"
"custom_lb_test"
"failover_test"
"outlier_detection_test"
"remove_neg_test"
"round_robin_test"
)
for test in "${test_suites[@]}"; do
run_test $test || (( ++failed_tests ))
done
echo "Failed test suites: ${failed_tests}"
}
main "$@"

View File

@ -15,7 +15,7 @@
# Config file for the internal CI (in protobuf text format)
# Location of the continuous shell script in repository.
build_file: "grpc/tools/internal_ci/linux/grpc_xds_k8s_lb_python.sh"
build_file: "grpc/tools/internal_ci/linux/psm-interop-test-python.sh"
timeout_mins: 180
action {
define_artifacts {
@ -24,3 +24,7 @@ action {
strip_prefix: "artifacts"
}
}
env_vars {
key: "PSM_TEST_SUITE"
value: "lb"
}

View File

@ -1,192 +0,0 @@
#!/usr/bin/env bash
# Copyright 2021 gRPC authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
set -eo pipefail
# Constants
readonly GITHUB_REPOSITORY_NAME="grpc"
readonly TEST_DRIVER_INSTALL_SCRIPT_URL="https://raw.githubusercontent.com/${TEST_DRIVER_REPO_OWNER:-grpc}/grpc/${TEST_DRIVER_BRANCH:-master}/tools/internal_ci/linux/grpc_xds_k8s_install_test_driver.sh"
## xDS test client Docker images
readonly SERVER_IMAGE_NAME="gcr.io/grpc-testing/xds-interop/python-server"
readonly CLIENT_IMAGE_NAME="gcr.io/grpc-testing/xds-interop/python-client"
readonly FORCE_IMAGE_BUILD="${FORCE_IMAGE_BUILD:-0}"
readonly BUILD_APP_PATH="interop-testing/build/install/grpc-interop-testing"
readonly LANGUAGE_NAME="Python"
#######################################
# Builds test app Docker images and pushes them to GCR
# Globals:
# BUILD_APP_PATH
# CLIENT_IMAGE_NAME: Test client Docker image name
# GIT_COMMIT: SHA-1 of git commit being built
# Arguments:
# None
# Outputs:
# Writes the output of `gcloud builds submit` to stdout, stderr
#######################################
build_test_app_docker_images() {
echo "Building ${LANGUAGE_NAME} xDS interop test app Docker images"
pushd "${SRC_DIR}"
docker build \
-f src/python/grpcio_tests/tests_py3_only/interop/Dockerfile.client \
-t "${CLIENT_IMAGE_NAME}:${GIT_COMMIT}" \
.
docker build \
-f src/python/grpcio_tests/tests_py3_only/interop/Dockerfile.server \
-t "${SERVER_IMAGE_NAME}:${GIT_COMMIT}" \
.
popd
gcloud -q auth configure-docker
docker push "${CLIENT_IMAGE_NAME}:${GIT_COMMIT}"
docker push "${SERVER_IMAGE_NAME}:${GIT_COMMIT}"
if is_version_branch "${TESTING_VERSION}"; then
tag_and_push_docker_image "${CLIENT_IMAGE_NAME}" "${GIT_COMMIT}" "${TESTING_VERSION}"
tag_and_push_docker_image "${SERVER_IMAGE_NAME}" "${GIT_COMMIT}" "${TESTING_VERSION}"
fi
}
#######################################
# Builds test app and its docker images unless they already exist
# Globals:
# SERVER_IMAGE_NAME: Test server Docker image name
# CLIENT_IMAGE_NAME: Test client Docker image name
# GIT_COMMIT: SHA-1 of git commit being built
# FORCE_IMAGE_BUILD
# Arguments:
# None
# Outputs:
# Writes the output to stdout, stderr
#######################################
build_docker_images_if_needed() {
# Check if images already exist
server_tags="$(gcloud_gcr_list_image_tags "${SERVER_IMAGE_NAME}" "${GIT_COMMIT}")"
printf "Server image: %s:%s\n" "${SERVER_IMAGE_NAME}" "${GIT_COMMIT}"
echo "${server_tags:-Server image not found}"
client_tags="$(gcloud_gcr_list_image_tags "${CLIENT_IMAGE_NAME}" "${GIT_COMMIT}")"
printf "Client image: %s:%s\n" "${CLIENT_IMAGE_NAME}" "${GIT_COMMIT}"
echo "${client_tags:-Client image not found}"
# Build if any of the images are missing, or FORCE_IMAGE_BUILD=1
if [[ "${FORCE_IMAGE_BUILD}" == "1" || -z "${server_tags}" || -z "${client_tags}" ]]; then
build_test_app_docker_images
else
echo "Skipping ${LANGUAGE_NAME} test app build"
fi
}
#######################################
# Executes the test case
# Globals:
# TEST_DRIVER_FLAGFILE: Relative path to test driver flagfile
# KUBE_CONTEXT: The name of kubectl context with GKE cluster access
# SECONDARY_KUBE_CONTEXT: The name of kubectl context with secondary GKE cluster access, if any
# TEST_XML_OUTPUT_DIR: Output directory for the test xUnit XML report
# CLIENT_IMAGE_NAME: Test client Docker image name
# GIT_COMMIT: SHA-1 of git commit being built
# TESTING_VERSION: version branch under test: used by the framework to determine the supported PSM
# features.
# Arguments:
# Test case name
# Outputs:
# Writes the output of test execution to stdout, stderr
# Test xUnit report to ${TEST_XML_OUTPUT_DIR}/${test_name}/sponge_log.xml
#######################################
run_test() {
# Test driver usage:
# https://github.com/grpc/grpc/tree/master/tools/run_tests/xds_k8s_test_driver#basic-usage
local test_name="${1:?Usage: run_test test_name}"
local out_dir="${TEST_XML_OUTPUT_DIR}/${test_name}"
mkdir -pv "${out_dir}"
set -x
python3 -m "tests.${test_name}" \
--flagfile="${TEST_DRIVER_FLAGFILE}" \
--kube_context="${KUBE_CONTEXT}" \
--secondary_kube_context="${SECONDARY_KUBE_CONTEXT}" \
--client_image="${CLIENT_IMAGE_NAME}:${GIT_COMMIT}" \
--server_image="${SERVER_IMAGE_NAME}:${GIT_COMMIT}" \
--testing_version="${TESTING_VERSION}" \
--force_cleanup \
--collect_app_logs \
--log_dir="${out_dir}" \
--xml_output_file="${out_dir}/sponge_log.xml" \
|& tee "${out_dir}/sponge_log.log"
}
#######################################
# Main function: provision software necessary to execute tests, and run them
# Globals:
# KOKORO_ARTIFACTS_DIR
# GITHUB_REPOSITORY_NAME
# SRC_DIR: Populated with absolute path to the source repo
# TEST_DRIVER_REPO_DIR: Populated with the path to the repo containing
# the test driver
# TEST_DRIVER_FULL_DIR: Populated with the path to the test driver source code
# TEST_DRIVER_FLAGFILE: Populated with relative path to test driver flagfile
# TEST_XML_OUTPUT_DIR: Populated with the path to test xUnit XML report
# GIT_ORIGIN_URL: Populated with the origin URL of git repo used for the build
# GIT_COMMIT: Populated with the SHA-1 of git commit being built
# GIT_COMMIT_SHORT: Populated with the short SHA-1 of git commit being built
# KUBE_CONTEXT: Populated with name of kubectl context with GKE cluster access
# SECONDARY_KUBE_CONTEXT: Populated with name of kubectl context with secondary GKE cluster access, if any
# Arguments:
# None
# Outputs:
# Writes the output of test execution to stdout, stderr
#######################################
main() {
local script_dir
script_dir="$(dirname "$0")"
# Source the test driver from the master branch.
echo "Sourcing test driver install script from: ${TEST_DRIVER_INSTALL_SCRIPT_URL}"
source /dev/stdin <<< "$(curl -s "${TEST_DRIVER_INSTALL_SCRIPT_URL}")"
activate_gke_cluster GKE_CLUSTER_PSM_LB
activate_secondary_gke_cluster GKE_CLUSTER_PSM_LB
set -x
if [[ -n "${KOKORO_ARTIFACTS_DIR}" ]]; then
kokoro_setup_test_driver "${GITHUB_REPOSITORY_NAME}"
else
local_setup_test_driver "${script_dir}"
fi
build_docker_images_if_needed
# Run tests
cd "${TEST_DRIVER_FULL_DIR}"
local failed_tests=0
test_suites=(
"app_net_test"
"api_listener_test"
"change_backend_service_test"
"failover_test"
"outlier_detection_test"
"remove_neg_test"
"round_robin_test"
)
for test in "${test_suites[@]}"; do
run_test $test || (( ++failed_tests ))
done
echo "Failed test suites: ${failed_tests}"
}
main "$@"

View File

@ -14,7 +14,7 @@
# limitations under the License.
## xDS test server/client Docker images
readonly IMAGE_REPO="gcr.io/grpc-testing/xds-interop"
readonly IMAGE_REPO="us-docker.pkg.dev/grpc-testing/psm-interop"
find_latest() {
gcloud container images list-tags --filter='tags~v1\.\d+\.x' "${IMAGE_REPO}/${1}-${2}" --flatten='tags[]' --format='value(tags)' | sort --version-sort | tail -n 1

View File

@ -17,7 +17,7 @@ set -eo pipefail
# Constants
readonly GITHUB_REPOSITORY_NAME="grpc"
readonly TEST_DRIVER_INSTALL_SCRIPT_URL="https://raw.githubusercontent.com/${TEST_DRIVER_REPO_OWNER:-grpc}/grpc/${TEST_DRIVER_BRANCH:-master}/tools/internal_ci/linux/grpc_xds_k8s_install_test_driver.sh"
readonly TEST_DRIVER_INSTALL_SCRIPT_URL="https://raw.githubusercontent.com/${TEST_DRIVER_REPO_OWNER:-grpc}/psm-interop/${TEST_DRIVER_BRANCH:-main}/.kokoro/psm_interop_kokoro_lib.sh"
readonly LANGS="cpp go java"
readonly MAIN_BRANCH="${MAIN_BRANCH:-master}"

View File

@ -17,7 +17,7 @@ set -eo pipefail
# Constants
readonly GITHUB_REPOSITORY_NAME="grpc"
readonly TEST_DRIVER_INSTALL_SCRIPT_URL="https://raw.githubusercontent.com/${TEST_DRIVER_REPO_OWNER:-grpc}/grpc/${TEST_DRIVER_BRANCH:-master}/tools/internal_ci/linux/grpc_xds_k8s_install_test_driver.sh"
readonly TEST_DRIVER_INSTALL_SCRIPT_URL="https://raw.githubusercontent.com/${TEST_DRIVER_REPO_OWNER:-grpc}/psm-interop/${TEST_DRIVER_BRANCH:-main}/.kokoro/psm_interop_kokoro_lib.sh"
## xDS test server/client Docker images
readonly SERVER_LANGS="cpp go java"
readonly CLIENT_LANGS="cpp go java"

View File

@ -17,7 +17,7 @@ set -ex
# consts
readonly GITHUB_REPOSITORY_NAME="grpc"
readonly TEST_DRIVER_INSTALL_SCRIPT_URL="https://raw.githubusercontent.com/${TEST_DRIVER_REPO_OWNER:-grpc}/grpc/${TEST_DRIVER_BRANCH:-master}/tools/internal_ci/linux/grpc_xds_k8s_install_test_driver.sh"
readonly TEST_DRIVER_INSTALL_SCRIPT_URL="https://raw.githubusercontent.com/${TEST_DRIVER_REPO_OWNER:-grpc}/psm-interop/${TEST_DRIVER_BRANCH:-main}/.kokoro/psm_interop_kokoro_lib.sh"
cd "$(dirname "$0")/../../.."

View File

@ -15,7 +15,7 @@
# Config file for the internal CI (in protobuf text format)
# Location of the continuous shell script in repository.
build_file: "grpc/tools/internal_ci/linux/grpc_xds_url_map.sh"
build_file: "grpc/tools/internal_ci/linux/psm-interop-test-cpp.sh"
timeout_mins: 90
action {
define_artifacts {
@ -24,3 +24,7 @@ action {
strip_prefix: "artifacts"
}
}
env_vars {
key: "PSM_TEST_SUITE"
value: "url_map"
}

View File

@ -1,149 +0,0 @@
#!/usr/bin/env bash
# Copyright 2021 gRPC authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
set -eo pipefail
# Constants
readonly GITHUB_REPOSITORY_NAME="grpc"
readonly TEST_DRIVER_INSTALL_SCRIPT_URL="https://raw.githubusercontent.com/${TEST_DRIVER_REPO_OWNER:-grpc}/grpc/${TEST_DRIVER_BRANCH:-master}/tools/internal_ci/linux/grpc_xds_k8s_install_test_driver.sh"
## xDS test client Docker images
readonly CLIENT_IMAGE_NAME="gcr.io/grpc-testing/xds-interop/cpp-client"
readonly FORCE_IMAGE_BUILD="${FORCE_IMAGE_BUILD:-0}"
readonly BUILD_APP_PATH="interop-testing/build/install/grpc-interop-testing"
#######################################
# Builds test app Docker images and pushes them to GCR
# Globals:
# BUILD_APP_PATH
# CLIENT_IMAGE_NAME: Test client Docker image name
# GIT_COMMIT: SHA-1 of git commit being built
# Arguments:
# None
# Outputs:
# Writes the output of `gcloud builds submit` to stdout, stderr
#######################################
build_test_app_docker_images() {
echo "Building C++ xDS interop test app Docker images"
docker build -f "${SRC_DIR}/tools/dockerfile/interoptest/grpc_interop_cxx_xds/Dockerfile.xds_client" -t "${CLIENT_IMAGE_NAME}:${GIT_COMMIT}" "${SRC_DIR}"
gcloud -q auth configure-docker
docker push "${CLIENT_IMAGE_NAME}:${GIT_COMMIT}"
if is_version_branch "${TESTING_VERSION}"; then
tag_and_push_docker_image "${CLIENT_IMAGE_NAME}" "${GIT_COMMIT}" "${TESTING_VERSION}"
fi
}
#######################################
# Builds test app and its docker images unless they already exist
# Globals:
# CLIENT_IMAGE_NAME: Test client Docker image name
# GIT_COMMIT: SHA-1 of git commit being built
# FORCE_IMAGE_BUILD
# Arguments:
# None
# Outputs:
# Writes the output to stdout, stderr
#######################################
build_docker_images_if_needed() {
# Check if images already exist
client_tags="$(gcloud_gcr_list_image_tags "${CLIENT_IMAGE_NAME}" "${GIT_COMMIT}")"
printf "Client image: %s:%s\n" "${CLIENT_IMAGE_NAME}" "${GIT_COMMIT}"
echo "${client_tags:-Client image not found}"
# Build if any of the images are missing, or FORCE_IMAGE_BUILD=1
if [[ "${FORCE_IMAGE_BUILD}" == "1" || -z "${client_tags}" ]]; then
build_test_app_docker_images
else
echo "Skipping C++ test app build"
fi
}
#######################################
# Executes the test case
# Globals:
# TEST_DRIVER_FLAGFILE: Relative path to test driver flagfile
# KUBE_CONTEXT: The name of kubectl context with GKE cluster access
# TEST_XML_OUTPUT_DIR: Output directory for the test xUnit XML report
# CLIENT_IMAGE_NAME: Test client Docker image name
# GIT_COMMIT: SHA-1 of git commit being built
# TESTING_VERSION: version branch under test: used by the framework to determine the supported PSM
# features.
# Arguments:
# Test case name
# Outputs:
# Writes the output of test execution to stdout, stderr
# Test xUnit report to ${TEST_XML_OUTPUT_DIR}/${test_name}/sponge_log.xml
#######################################
run_test() {
# Test driver usage:
# https://github.com/grpc/grpc/tree/master/tools/run_tests/xds_k8s_test_driver#basic-usage
local test_name="${1:?Usage: run_test test_name}"
local out_dir="${TEST_XML_OUTPUT_DIR}/${test_name}"
mkdir -pv "${out_dir}"
set -x
python3 -m "tests.${test_name}" \
--flagfile="${TEST_DRIVER_FLAGFILE}" \
--flagfile="config/url-map.cfg" \
--kube_context="${KUBE_CONTEXT}" \
--client_image="${CLIENT_IMAGE_NAME}:${GIT_COMMIT}" \
--testing_version="${TESTING_VERSION}" \
--collect_app_logs \
--log_dir="${out_dir}" \
--xml_output_file="${out_dir}/sponge_log.xml" \
|& tee "${out_dir}/sponge_log.log"
}
#######################################
# Main function: provision software necessary to execute tests, and run them
# Globals:
# KOKORO_ARTIFACTS_DIR
# GITHUB_REPOSITORY_NAME
# SRC_DIR: Populated with absolute path to the source repo
# TEST_DRIVER_REPO_DIR: Populated with the path to the repo containing
# the test driver
# TEST_DRIVER_FULL_DIR: Populated with the path to the test driver source code
# TEST_DRIVER_FLAGFILE: Populated with relative path to test driver flagfile
# TEST_XML_OUTPUT_DIR: Populated with the path to test xUnit XML report
# GIT_ORIGIN_URL: Populated with the origin URL of git repo used for the build
# GIT_COMMIT: Populated with the SHA-1 of git commit being built
# GIT_COMMIT_SHORT: Populated with the short SHA-1 of git commit being built
# KUBE_CONTEXT: Populated with name of kubectl context with GKE cluster access
# Arguments:
# None
# Outputs:
# Writes the output of test execution to stdout, stderr
#######################################
main() {
local script_dir
script_dir="$(dirname "$0")"
# Source the test driver from the master branch.
echo "Sourcing test driver install script from: ${TEST_DRIVER_INSTALL_SCRIPT_URL}"
source /dev/stdin <<< "$(curl -s "${TEST_DRIVER_INSTALL_SCRIPT_URL}")"
activate_gke_cluster GKE_CLUSTER_PSM_BASIC
set -x
if [[ -n "${KOKORO_ARTIFACTS_DIR}" ]]; then
kokoro_setup_test_driver "${GITHUB_REPOSITORY_NAME}"
else
local_setup_test_driver "${script_dir}"
fi
build_docker_images_if_needed
# Run tests
cd "${TEST_DRIVER_FULL_DIR}"
run_test url_map || echo "Failed url_map test"
}
main "$@"

View File

@ -15,7 +15,7 @@
# Config file for the internal CI (in protobuf text format)
# Location of the continuous shell script in repository.
build_file: "grpc/tools/internal_ci/linux/grpc_xds_url_map_python.sh"
build_file: "grpc/tools/internal_ci/linux/psm-interop-test-python.sh"
timeout_mins: 90
action {
define_artifacts {
@ -24,3 +24,7 @@ action {
strip_prefix: "artifacts"
}
}
env_vars {
key: "PSM_TEST_SUITE"
value: "url_map"
}

View File

@ -1,159 +0,0 @@
#!/usr/bin/env bash
# Copyright 2021 gRPC authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
set -eo pipefail
# Constants
readonly GITHUB_REPOSITORY_NAME="grpc"
readonly TEST_DRIVER_INSTALL_SCRIPT_URL="https://raw.githubusercontent.com/${TEST_DRIVER_REPO_OWNER:-grpc}/grpc/${TEST_DRIVER_BRANCH:-master}/tools/internal_ci/linux/grpc_xds_k8s_install_test_driver.sh"
## xDS test client Docker images
readonly CLIENT_IMAGE_NAME="gcr.io/grpc-testing/xds-interop/python-client"
readonly FORCE_IMAGE_BUILD="${FORCE_IMAGE_BUILD:-0}"
readonly BUILD_APP_PATH="interop-testing/build/install/grpc-interop-testing"
readonly LANGUAGE_NAME="Python"
#######################################
# Builds test app Docker images and pushes them to GCR
# Globals:
# BUILD_APP_PATH
# CLIENT_IMAGE_NAME: Test client Docker image name
# GIT_COMMIT: SHA-1 of git commit being built
# Arguments:
# None
# Outputs:
# Writes the output of `gcloud builds submit` to stdout, stderr
#######################################
build_test_app_docker_images() {
echo "Building ${LANGUAGE_NAME} xDS interop test app Docker images"
pushd "${SRC_DIR}"
docker build \
-f src/python/grpcio_tests/tests_py3_only/interop/Dockerfile.client \
-t "${CLIENT_IMAGE_NAME}:${GIT_COMMIT}" \
.
popd
gcloud -q auth configure-docker
docker push "${CLIENT_IMAGE_NAME}:${GIT_COMMIT}"
if is_version_branch "${TESTING_VERSION}"; then
tag_and_push_docker_image "${CLIENT_IMAGE_NAME}" "${GIT_COMMIT}" "${TESTING_VERSION}"
fi
}
#######################################
# Builds test app and its docker images unless they already exist
# Globals:
# CLIENT_IMAGE_NAME: Test client Docker image name
# GIT_COMMIT: SHA-1 of git commit being built
# FORCE_IMAGE_BUILD
# Arguments:
# None
# Outputs:
# Writes the output to stdout, stderr
#######################################
build_docker_images_if_needed() {
# Check if images already exist
client_tags="$(gcloud_gcr_list_image_tags "${CLIENT_IMAGE_NAME}" "${GIT_COMMIT}")"
printf "Client image: %s:%s\n" "${CLIENT_IMAGE_NAME}" "${GIT_COMMIT}"
echo "${client_tags:-Client image not found}"
# Build if any of the images are missing, or FORCE_IMAGE_BUILD=1
if [[ "${FORCE_IMAGE_BUILD}" == "1" || -z "${client_tags}" ]]; then
build_test_app_docker_images
else
echo "Skipping ${LANGUAGE_NAME} test app build"
fi
}
#######################################
# Executes the test case
# Globals:
# TEST_DRIVER_FLAGFILE: Relative path to test driver flagfile
# KUBE_CONTEXT: The name of kubectl context with GKE cluster access
# TEST_XML_OUTPUT_DIR: Output directory for the test xUnit XML report
# CLIENT_IMAGE_NAME: Test client Docker image name
# GIT_COMMIT: SHA-1 of git commit being built
# TESTING_VERSION: version branch under test: used by the framework to determine the supported PSM
# features.
# Arguments:
# Test case name
# Outputs:
# Writes the output of test execution to stdout, stderr
# Test xUnit report to ${TEST_XML_OUTPUT_DIR}/${test_name}/sponge_log.xml
#######################################
run_test() {
# Test driver usage:
# https://github.com/grpc/grpc/tree/master/tools/run_tests/xds_k8s_test_driver#basic-usage
local test_name="${1:?Usage: run_test test_name}"
local out_dir="${TEST_XML_OUTPUT_DIR}/${test_name}"
mkdir -pv "${out_dir}"
set -x
python3 -m "tests.${test_name}" \
--flagfile="${TEST_DRIVER_FLAGFILE}" \
--flagfile="config/url-map.cfg" \
--kube_context="${KUBE_CONTEXT}" \
--client_image="${CLIENT_IMAGE_NAME}:${GIT_COMMIT}" \
--testing_version="${TESTING_VERSION}" \
--collect_app_logs \
--log_dir="${out_dir}" \
--xml_output_file="${out_dir}/sponge_log.xml" \
|& tee "${out_dir}/sponge_log.log"
}
#######################################
# Main function: provision software necessary to execute tests, and run them
# Globals:
# KOKORO_ARTIFACTS_DIR
# GITHUB_REPOSITORY_NAME
# SRC_DIR: Populated with absolute path to the source repo
# TEST_DRIVER_REPO_DIR: Populated with the path to the repo containing
# the test driver
# TEST_DRIVER_FULL_DIR: Populated with the path to the test driver source code
# TEST_DRIVER_FLAGFILE: Populated with relative path to test driver flagfile
# TEST_XML_OUTPUT_DIR: Populated with the path to test xUnit XML report
# GIT_ORIGIN_URL: Populated with the origin URL of git repo used for the build
# GIT_COMMIT: Populated with the SHA-1 of git commit being built
# GIT_COMMIT_SHORT: Populated with the short SHA-1 of git commit being built
# KUBE_CONTEXT: Populated with name of kubectl context with GKE cluster access
# Arguments:
# None
# Outputs:
# Writes the output of test execution to stdout, stderr
#######################################
main() {
local script_dir
script_dir="$(dirname "$0")"
# Source the test driver from the master branch.
echo "Sourcing test driver install script from: ${TEST_DRIVER_INSTALL_SCRIPT_URL}"
source /dev/stdin <<< "$(curl -s "${TEST_DRIVER_INSTALL_SCRIPT_URL}")"
activate_gke_cluster GKE_CLUSTER_PSM_BASIC
set -x
if [[ -n "${KOKORO_ARTIFACTS_DIR}" ]]; then
kokoro_setup_test_driver "${GITHUB_REPOSITORY_NAME}"
else
local_setup_test_driver "${script_dir}"
fi
build_docker_images_if_needed
# Run tests
cd "${TEST_DRIVER_FULL_DIR}"
run_test url_map || echo "Failed url_map test"
}
main "$@"

View File

@ -0,0 +1,34 @@
#!/usr/bin/env bash
# Copyright 2024 gRPC authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
set -eo pipefail
#######################################
# Builds test app Docker images and pushes them to GCR.
# Called from psm_interop_kokoro_lib.sh.
#
# Globals:
# SRC_DIR: Absolute path to the source repo on Kokoro VM
# SERVER_IMAGE_NAME: Test server Docker image name
# CLIENT_IMAGE_NAME: Test client Docker image name
# GIT_COMMIT: SHA-1 of git commit being built
# DOCKER_REGISTRY: Docker registry to push to
# Outputs:
# Writes the output of docker image build stdout, stderr
#######################################
psm::lang::build_docker_images() {
local client_dockerfile="tools/dockerfile/interoptest/grpc_interop_cxx_xds/Dockerfile.xds_client"
local server_dockerfile="tools/dockerfile/interoptest/grpc_interop_cxx_xds/Dockerfile.xds_server"
psm::build::docker_images_generic "${client_dockerfile}" "${server_dockerfile}"
}

View File

@ -0,0 +1,34 @@
#!/usr/bin/env bash
# Copyright 2024 gRPC authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
set -eo pipefail
#######################################
# Builds test app Docker images and pushes them to GCR.
# Called from psm_interop_kokoro_lib.sh.
#
# Globals:
# SRC_DIR: Absolute path to the source repo on Kokoro VM
# SERVER_IMAGE_NAME: Test server Docker image name
# CLIENT_IMAGE_NAME: Test client Docker image name
# GIT_COMMIT: SHA-1 of git commit being built
# DOCKER_REGISTRY: Docker registry to push to
# Outputs:
# Writes the output of docker image build stdout, stderr
#######################################
psm::lang::build_docker_images() {
local client_dockerfile="src/python/grpcio_tests/tests_py3_only/interop/Dockerfile.client"
local server_dockerfile="src/python/grpcio_tests/tests_py3_only/interop/Dockerfile.server"
psm::build::docker_images_generic "${client_dockerfile}" "${server_dockerfile}"
}

View File

@ -0,0 +1,32 @@
#!/usr/bin/env bash
# Copyright 2024 gRPC authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
set -eo pipefail
# Used locally.
readonly TEST_DRIVER_INSTALL_SCRIPT_URL="https://raw.githubusercontent.com/${TEST_DRIVER_REPO_OWNER:-grpc}/psm-interop/${TEST_DRIVER_BRANCH:-main}/.kokoro/psm_interop_kokoro_lib.sh"
psm::lang::source_install_lib() {
echo "Sourcing test driver install script from: ${TEST_DRIVER_INSTALL_SCRIPT_URL}"
local install_lib
# Download to a tmp file.
install_lib="$(mktemp -d)/psm_interop_kokoro_lib.sh"
curl -s --retry-connrefused --retry 5 -o "${install_lib}" "${TEST_DRIVER_INSTALL_SCRIPT_URL}"
# Checksum.
if command -v sha256sum &> /dev/null; then
echo "Install script checksum:"
sha256sum "${install_lib}"
fi
source "${install_lib}"
}

View File

@ -0,0 +1,24 @@
#!/usr/bin/env bash
# Copyright 2024 gRPC authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
set -eo pipefail
# Input parameters to psm:: methods of the install script.
readonly GRPC_LANGUAGE="cpp"
readonly BUILD_SCRIPT_DIR="$(dirname "$0")"
source "${BUILD_SCRIPT_DIR}/psm-interop-install-lib.sh"
psm::lang::source_install_lib
source "${BUILD_SCRIPT_DIR}/psm-interop-build-${GRPC_LANGUAGE}.sh"
psm::run "${PSM_TEST_SUITE}"

View File

@ -0,0 +1,24 @@
#!/usr/bin/env bash
# Copyright 2024 gRPC authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
set -eo pipefail
# Input parameters to psm:: methods of the install script.
readonly GRPC_LANGUAGE="python"
readonly BUILD_SCRIPT_DIR="$(dirname "$0")"
source "${BUILD_SCRIPT_DIR}/psm-interop-install-lib.sh"
psm::lang::source_install_lib
source "${BUILD_SCRIPT_DIR}/psm-interop-build-${GRPC_LANGUAGE}.sh"
psm::run "${PSM_TEST_SUITE}"

View File

@ -15,7 +15,7 @@
# Config file for the internal CI (in protobuf text format)
# Location of the continuous shell script in repository.
build_file: "grpc/tools/internal_ci/linux/psm-security-python.sh"
build_file: "grpc/tools/internal_ci/linux/psm-interop-test-python.sh"
timeout_mins: 240
action {
define_artifacts {
@ -24,3 +24,7 @@ action {
strip_prefix: "artifacts"
}
}
env_vars {
key: "PSM_TEST_SUITE"
value: "security"
}

View File

@ -1,184 +0,0 @@
#!/usr/bin/env bash
# Copyright 2021 gRPC authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
set -eo pipefail
# Constants
readonly GITHUB_REPOSITORY_NAME="grpc"
readonly TEST_DRIVER_INSTALL_SCRIPT_URL="https://raw.githubusercontent.com/${TEST_DRIVER_REPO_OWNER:-grpc}/grpc/${TEST_DRIVER_BRANCH:-master}/tools/internal_ci/linux/grpc_xds_k8s_install_test_driver.sh"
## xDS test server/client Docker images
readonly SERVER_IMAGE_NAME="gcr.io/grpc-testing/xds-interop/python-server"
readonly CLIENT_IMAGE_NAME="gcr.io/grpc-testing/xds-interop/python-client"
readonly FORCE_IMAGE_BUILD="${FORCE_IMAGE_BUILD:-0}"
readonly BUILD_APP_PATH="interop-testing/build/install/grpc-interop-testing"
readonly LANGUAGE_NAME="Python"
#######################################
# Builds test app Docker images and pushes them to GCR
# Globals:
# BUILD_APP_PATH
# SERVER_IMAGE_NAME: Test server Docker image name
# CLIENT_IMAGE_NAME: Test client Docker image name
# GIT_COMMIT: SHA-1 of git commit being built
# TESTING_VERSION: version branch under test, f.e. v1.42.x, master
# Arguments:
# None
# Outputs:
# Writes the output of `gcloud builds submit` to stdout, stderr
#######################################
build_test_app_docker_images() {
echo "Building ${LANGUAGE_NAME} xDS interop test app Docker images"
pushd "${SRC_DIR}"
docker build \
-f src/python/grpcio_tests/tests_py3_only/interop/Dockerfile.client \
-t "${CLIENT_IMAGE_NAME}:${GIT_COMMIT}" \
.
docker build \
-f src/python/grpcio_tests/tests_py3_only/interop/Dockerfile.server \
-t "${SERVER_IMAGE_NAME}:${GIT_COMMIT}" \
.
popd
gcloud -q auth configure-docker
docker push "${CLIENT_IMAGE_NAME}:${GIT_COMMIT}"
docker push "${SERVER_IMAGE_NAME}:${GIT_COMMIT}"
if is_version_branch "${TESTING_VERSION}"; then
tag_and_push_docker_image "${CLIENT_IMAGE_NAME}" "${GIT_COMMIT}" "${TESTING_VERSION}"
tag_and_push_docker_image "${SERVER_IMAGE_NAME}" "${GIT_COMMIT}" "${TESTING_VERSION}"
fi
}
#######################################
# Builds test app and its docker images unless they already exist
# Globals:
# SERVER_IMAGE_NAME: Test server Docker image name
# CLIENT_IMAGE_NAME: Test client Docker image name
# GIT_COMMIT: SHA-1 of git commit being built
# FORCE_IMAGE_BUILD
# Arguments:
# None
# Outputs:
# Writes the output to stdout, stderr
#######################################
build_docker_images_if_needed() {
# Check if images already exist
server_tags="$(gcloud_gcr_list_image_tags "${SERVER_IMAGE_NAME}" "${GIT_COMMIT}")"
printf "Server image: %s:%s\n" "${SERVER_IMAGE_NAME}" "${GIT_COMMIT}"
echo "${server_tags:-Server image not found}"
client_tags="$(gcloud_gcr_list_image_tags "${CLIENT_IMAGE_NAME}" "${GIT_COMMIT}")"
printf "Client image: %s:%s\n" "${CLIENT_IMAGE_NAME}" "${GIT_COMMIT}"
echo "${client_tags:-Client image not found}"
# Build if any of the images are missing, or FORCE_IMAGE_BUILD=1
if [[ "${FORCE_IMAGE_BUILD}" == "1" || -z "${server_tags}" || -z "${client_tags}" ]]; then
build_test_app_docker_images
else
echo "Skipping ${LANGUAGE_NAME} test app build"
fi
}
#######################################
# Executes the test case
# Globals:
# TEST_DRIVER_FLAGFILE: Relative path to test driver flagfile
# KUBE_CONTEXT: The name of kubectl context with GKE cluster access
# TEST_XML_OUTPUT_DIR: Output directory for the test xUnit XML report
# SERVER_IMAGE_NAME: Test server Docker image name
# CLIENT_IMAGE_NAME: Test client Docker image name
# GIT_COMMIT: SHA-1 of git commit being built
# TESTING_VERSION: version branch under test: used by the framework to determine the supported PSM
# features.
# Arguments:
# Test case name
# Outputs:
# Writes the output of test execution to stdout, stderr
# Test xUnit report to ${TEST_XML_OUTPUT_DIR}/${test_name}/sponge_log.xml
#######################################
run_test() {
# Test driver usage:
# https://github.com/grpc/grpc/tree/master/tools/run_tests/xds_k8s_test_driver#basic-usage
local test_name="${1:?Usage: run_test test_name}"
local out_dir="${TEST_XML_OUTPUT_DIR}/${test_name}"
mkdir -pv "${out_dir}"
set -x
python3 -m "tests.${test_name}" \
--flagfile="${TEST_DRIVER_FLAGFILE}" \
--kube_context="${KUBE_CONTEXT}" \
--server_image="${SERVER_IMAGE_NAME}:${GIT_COMMIT}" \
--client_image="${CLIENT_IMAGE_NAME}:${GIT_COMMIT}" \
--testing_version="${TESTING_VERSION}" \
--nocheck_local_certs \
--force_cleanup \
--collect_app_logs \
--log_dir="${out_dir}" \
--xml_output_file="${out_dir}/sponge_log.xml" \
|& tee "${out_dir}/sponge_log.log"
set +x
}
#######################################
# Main function: provision software necessary to execute tests, and run them
# Globals:
# KOKORO_ARTIFACTS_DIR
# GITHUB_REPOSITORY_NAME
# SRC_DIR: Populated with absolute path to the source repo
# TEST_DRIVER_REPO_DIR: Populated with the path to the repo containing
# the test driver
# TEST_DRIVER_FULL_DIR: Populated with the path to the test driver source code
# TEST_DRIVER_FLAGFILE: Populated with relative path to test driver flagfile
# TEST_XML_OUTPUT_DIR: Populated with the path to test xUnit XML report
# GIT_ORIGIN_URL: Populated with the origin URL of git repo used for the build
# GIT_COMMIT: Populated with the SHA-1 of git commit being built
# GIT_COMMIT_SHORT: Populated with the short SHA-1 of git commit being built
# KUBE_CONTEXT: Populated with name of kubectl context with GKE cluster access
# Arguments:
# None
# Outputs:
# Writes the output of test execution to stdout, stderr
#######################################
main() {
local script_dir
script_dir="$(dirname "$0")"
# Source the test driver from the master branch.
echo "Sourcing test driver install script from: ${TEST_DRIVER_INSTALL_SCRIPT_URL}"
source /dev/stdin <<< "$(curl -s "${TEST_DRIVER_INSTALL_SCRIPT_URL}")"
activate_gke_cluster GKE_CLUSTER_PSM_SECURITY
set -x
if [[ -n "${KOKORO_ARTIFACTS_DIR}" ]]; then
kokoro_setup_test_driver "${GITHUB_REPOSITORY_NAME}"
else
local_setup_test_driver "${script_dir}"
fi
build_docker_images_if_needed
# Run tests
cd "${TEST_DRIVER_FULL_DIR}"
local failed_tests=0
test_suites=("baseline_test" "security_test" "authz_test")
for test in "${test_suites[@]}"; do
run_test $test || (( ++failed_tests ))
done
echo "Failed test suites: ${failed_tests}"
}
main "$@"

View File

@ -15,7 +15,7 @@
# Config file for the internal CI (in protobuf text format)
# Location of the continuous shell script in repository.
build_file: "grpc/tools/internal_ci/linux/psm-security.sh"
build_file: "grpc/tools/internal_ci/linux/psm-interop-test-cpp.sh"
timeout_mins: 240
action {
define_artifacts {
@ -24,3 +24,7 @@ action {
strip_prefix: "artifacts"
}
}
env_vars {
key: "PSM_TEST_SUITE"
value: "security"
}

View File

@ -1,168 +0,0 @@
#!/usr/bin/env bash
# Copyright 2021 gRPC authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
set -eo pipefail
# Constants
readonly GITHUB_REPOSITORY_NAME="grpc"
readonly TEST_DRIVER_INSTALL_SCRIPT_URL="https://raw.githubusercontent.com/${TEST_DRIVER_REPO_OWNER:-grpc}/grpc/${TEST_DRIVER_BRANCH:-master}/tools/internal_ci/linux/grpc_xds_k8s_install_test_driver.sh"
## xDS test server/client Docker images
readonly SERVER_IMAGE_NAME="gcr.io/grpc-testing/xds-interop/cpp-server"
readonly CLIENT_IMAGE_NAME="gcr.io/grpc-testing/xds-interop/cpp-client"
readonly FORCE_IMAGE_BUILD="${FORCE_IMAGE_BUILD:-0}"
readonly BUILD_APP_PATH="interop-testing/build/install/grpc-interop-testing"
#######################################
# Builds test app Docker images and pushes them to GCR
# Globals:
# BUILD_APP_PATH
# SERVER_IMAGE_NAME: Test server Docker image name
# CLIENT_IMAGE_NAME: Test client Docker image name
# GIT_COMMIT: SHA-1 of git commit being built
# TESTING_VERSION: version branch under test, f.e. v1.42.x, master
# Arguments:
# None
# Outputs:
# Writes the output of `gcloud builds submit` to stdout, stderr
#######################################
build_test_app_docker_images() {
echo "Building C++ xDS interop test app Docker images"
docker build -f "${SRC_DIR}/tools/dockerfile/interoptest/grpc_interop_cxx_xds/Dockerfile.xds_client" -t "${CLIENT_IMAGE_NAME}:${GIT_COMMIT}" "${SRC_DIR}"
docker build -f "${SRC_DIR}/tools/dockerfile/interoptest/grpc_interop_cxx_xds/Dockerfile.xds_server" -t "${SERVER_IMAGE_NAME}:${GIT_COMMIT}" "${SRC_DIR}"
gcloud -q auth configure-docker
docker push "${CLIENT_IMAGE_NAME}:${GIT_COMMIT}"
docker push "${SERVER_IMAGE_NAME}:${GIT_COMMIT}"
if is_version_branch "${TESTING_VERSION}"; then
tag_and_push_docker_image "${CLIENT_IMAGE_NAME}" "${GIT_COMMIT}" "${TESTING_VERSION}"
tag_and_push_docker_image "${SERVER_IMAGE_NAME}" "${GIT_COMMIT}" "${TESTING_VERSION}"
fi
}
#######################################
# Builds test app and its docker images unless they already exist
# Globals:
# SERVER_IMAGE_NAME: Test server Docker image name
# CLIENT_IMAGE_NAME: Test client Docker image name
# GIT_COMMIT: SHA-1 of git commit being built
# FORCE_IMAGE_BUILD
# Arguments:
# None
# Outputs:
# Writes the output to stdout, stderr
#######################################
build_docker_images_if_needed() {
# Check if images already exist
server_tags="$(gcloud_gcr_list_image_tags "${SERVER_IMAGE_NAME}" "${GIT_COMMIT}")"
printf "Server image: %s:%s\n" "${SERVER_IMAGE_NAME}" "${GIT_COMMIT}"
echo "${server_tags:-Server image not found}"
client_tags="$(gcloud_gcr_list_image_tags "${CLIENT_IMAGE_NAME}" "${GIT_COMMIT}")"
printf "Client image: %s:%s\n" "${CLIENT_IMAGE_NAME}" "${GIT_COMMIT}"
echo "${client_tags:-Client image not found}"
# Build if any of the images are missing, or FORCE_IMAGE_BUILD=1
if [[ "${FORCE_IMAGE_BUILD}" == "1" || -z "${server_tags}" || -z "${client_tags}" ]]; then
build_test_app_docker_images
else
echo "Skipping C++ test app build"
fi
}
#######################################
# Executes the test case
# Globals:
# TEST_DRIVER_FLAGFILE: Relative path to test driver flagfile
# KUBE_CONTEXT: The name of kubectl context with GKE cluster access
# TEST_XML_OUTPUT_DIR: Output directory for the test xUnit XML report
# SERVER_IMAGE_NAME: Test server Docker image name
# CLIENT_IMAGE_NAME: Test client Docker image name
# GIT_COMMIT: SHA-1 of git commit being built
# TESTING_VERSION: version branch under test: used by the framework to determine the supported PSM
# features.
# Arguments:
# Test case name
# Outputs:
# Writes the output of test execution to stdout, stderr
# Test xUnit report to ${TEST_XML_OUTPUT_DIR}/${test_name}/sponge_log.xml
#######################################
run_test() {
# Test driver usage:
# https://github.com/grpc/grpc/tree/master/tools/run_tests/xds_k8s_test_driver#basic-usage
local test_name="${1:?Usage: run_test test_name}"
local out_dir="${TEST_XML_OUTPUT_DIR}/${test_name}"
mkdir -pv "${out_dir}"
set -x
python3 -m "tests.${test_name}" \
--flagfile="${TEST_DRIVER_FLAGFILE}" \
--kube_context="${KUBE_CONTEXT}" \
--server_image="${SERVER_IMAGE_NAME}:${GIT_COMMIT}" \
--client_image="${CLIENT_IMAGE_NAME}:${GIT_COMMIT}" \
--testing_version="${TESTING_VERSION}" \
--nocheck_local_certs \
--force_cleanup \
--collect_app_logs \
--log_dir="${out_dir}" \
--xml_output_file="${out_dir}/sponge_log.xml" \
|& tee "${out_dir}/sponge_log.log"
}
#######################################
# Main function: provision software necessary to execute tests, and run them
# Globals:
# KOKORO_ARTIFACTS_DIR
# GITHUB_REPOSITORY_NAME
# SRC_DIR: Populated with absolute path to the source repo
# TEST_DRIVER_REPO_DIR: Populated with the path to the repo containing
# the test driver
# TEST_DRIVER_FULL_DIR: Populated with the path to the test driver source code
# TEST_DRIVER_FLAGFILE: Populated with relative path to test driver flagfile
# TEST_XML_OUTPUT_DIR: Populated with the path to test xUnit XML report
# GIT_ORIGIN_URL: Populated with the origin URL of git repo used for the build
# GIT_COMMIT: Populated with the SHA-1 of git commit being built
# GIT_COMMIT_SHORT: Populated with the short SHA-1 of git commit being built
# KUBE_CONTEXT: Populated with name of kubectl context with GKE cluster access
# Arguments:
# None
# Outputs:
# Writes the output of test execution to stdout, stderr
#######################################
main() {
local script_dir
script_dir="$(dirname "$0")"
# Source the test driver from the master branch.
echo "Sourcing test driver install script from: ${TEST_DRIVER_INSTALL_SCRIPT_URL}"
source /dev/stdin <<< "$(curl -s "${TEST_DRIVER_INSTALL_SCRIPT_URL}")"
activate_gke_cluster GKE_CLUSTER_PSM_SECURITY
set -x
if [[ -n "${KOKORO_ARTIFACTS_DIR}" ]]; then
kokoro_setup_test_driver "${GITHUB_REPOSITORY_NAME}"
else
local_setup_test_driver "${script_dir}"
fi
build_docker_images_if_needed
# Run tests
cd "${TEST_DRIVER_FULL_DIR}"
local failed_tests=0
test_suites=("baseline_test" "security_test" "authz_test")
for test in "${test_suites[@]}"; do
run_test $test || (( ++failed_tests ))
done
echo "Failed test suites: ${failed_tests}"
}
main "$@"

View File

@ -945,6 +945,7 @@ class RubyLanguage(object):
"src/ruby/end2end/fork_test.rb",
"src/ruby/end2end/simple_fork_test.rb",
"src/ruby/end2end/prefork_without_using_grpc_test.rb",
"src/ruby/end2end/prefork_postfork_loop_test.rb",
"src/ruby/end2end/secure_fork_test.rb",
"src/ruby/end2end/bad_usage_fork_test.rb",
"src/ruby/end2end/sig_handling_test.rb",
@ -969,6 +970,7 @@ class RubyLanguage(object):
"src/ruby/end2end/secure_fork_test.rb",
"src/ruby/end2end/bad_usage_fork_test.rb",
"src/ruby/end2end/prefork_without_using_grpc_test.rb",
"src/ruby/end2end/prefork_postfork_loop_test.rb",
]:
if platform_string() == "mac":
# Skip fork tests on mac, it's only supported on linux.