Commit Graph

12468 Commits

Author SHA1 Message Date
Jan Tattermusch f23f1bb51f
Use ccache for selected C++ and python builds (with redis server as cache) (#28661)
* add cmake support for ccache

* cleanup: use --env-file for docker run invocations

* make python build compatible with using ccache

* enable building using ccache in selected kokoro jobs

* print ccache stats and the end of run_tests.py
2022-01-25 19:18:58 +01:00
Jan Tattermusch 27b6b45c32
Upgrade C++ and Python docker images to debian stable (debian 11) (#28664)
* upgrade C++ and python docker images

* regenerate dockerfiles

* upgrade grpc_artifact_python docker images

* adjust run_tests scripts to dockerimages upgrades
2022-01-25 10:23:01 +01:00
Menghan Li e523825c75
xds/cleanup: update build in cleanup kokoro script (#28654) 2022-01-24 13:27:13 -08:00
Easwar Swaminathan 9bc0732b44
Add v1.43.0 release of grpc-go to interop matrix (#28359) 2022-01-24 08:04:57 -08:00
yifeizhuang b4b6862352
Add api listener test for k8s (#27534)
This ports the api listener test case to k8s.
2022-01-21 15:07:19 -08:00
Craig Tiller d2042c1c05
Convert client_authority_filter to a promise. (#28565)
* Initial ideation

* progress

* progress

* x

* Automated change: Fix sanity tests

* erorrs

* better ordering

* working on it

* xx

* comment

* comment

* Get metadata pointer semantics temporarily right

* fix

* fix

* fix

* fix

* fix

* retry: send at most one cancel_stream op on each call attempt

* fixes

* revert earlier change

* split filter defn from impl

* add tests

* fixes

* Automated change: Fix sanity tests

* review feedback

* review feedback

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
Co-authored-by: Mark D. Roth <roth@google.com>
2022-01-21 14:50:07 -08:00
Esun Kim 55d7631a99
Bump clang 12 to 13 (#28639) 2022-01-21 14:47:04 -08:00
Esun Kim 01011ab259
Revert "Buildify Envoy upb (#28558)" (#28648)
This reverts commit 97584d8346.
2022-01-21 08:35:13 -08:00
Esun Kim 97584d8346
Buildify Envoy upb (#28558)
* Buildigy xds

* Generate project

* Buildify envoy

* Added new upb files

* Removed unused upb targets

* Fix grpc_cel_engine

* Update envoy-api to the latest

* Regen upb

* Regen projects

* Fix bazel build on MacOS

* More fix on bazel mac

* Disable grpc_tool_test on Mac

* Regen projects
2022-01-20 22:29:58 -08:00
Craig Tiller 74a2cb6e2b
Revert "Revert "Reland slice changes (#28601)" (#28615)" (#28624)
* Revert "Revert "Reland slice changes (#28601)" (#28615)"

This reverts commit 939bbfc336.

* Change random seed back to nanoseconds
2022-01-20 12:47:33 -08:00
Jan Tattermusch 402981be8e
Bump version to 1.45.0-dev (#28553)
* bump version to 1.45.0-dev

* regenerate projects
2022-01-20 09:23:06 +01:00
Craig Tiller 939bbfc336
Revert "Reland slice changes (#28601)" (#28615)
This reverts commit b33e0d40af.
2022-01-19 09:40:19 -08:00
Jan Tattermusch fda6158600
Build ruby artifacts in parallel (#28243)
* build ruby artifacts in parallel

* fine tune grpc_distribtests_ruby.sh parallelism

* fine tune linux/grpc_build_artifacts.sh parallelism

* cleanup in bundle_install_wrapper

* address review comments
2022-01-19 18:28:27 +01:00
Vignesh Babu aeea02fab8
TLS Session Keys export for GRPC C++ (#26812)
* Adding TLS Key export logic to core and c++ wrappers

* Adding and end2end cpp tls key export test and updating broken test due to interface changes

* regenerate projects

* updating tls key export core logic with addition of APIs to grpc_security.h

* undoing changes to tls_security_connector_test

* regenerate projects

* changing the logging format enum name as per GRFC comments

* regenerate projects

* removing some commented code

* updating changes as per review comments

* adding GRPCAPI annotations to functions defined in grpc_security.h

* regenerate projects

* fixed some code styling issues

* removing grpc_security.h include from tls_credentials_options.h

* updating files as per review comments

* minor fixes

* moving some code around

* removing key log format from tls session key log config and converting it to a simple string

* regenerate projects

* fixing mistakes in recent merge with master

* regenerate projects

* regenerate projects

* fixing some distrib and snity errors

* fixing formatting errors

* fixing more sanity checks and raising supported openssl versions to 1.1.1

* updating min supported openssl version to 1.1.1

* updating min supported openssl version in tls_key_export_test

* updating test to fix incorrect vector initialization

* updating as per latest comments

* fixing sanity checks

* addressing review comments

* fixing sanity checks

* fixed c++ comment style

* Automated change: Fix sanity tests

* fixing review comments

Co-authored-by: Vignesh2208 <Vignesh2208@users.noreply.github.com>
2022-01-18 14:44:55 -08:00
Jan Tattermusch a215992f0e
speedup C# basictests: only build C# once for coreclr and mono tests (#28587)
* only build C# once for coreclr and mono tests

* one more attempt

* only build C# once for coreclr and mono
2022-01-18 11:09:49 -08:00
Jan Tattermusch fd1f89a28b
speedup grpc_csharp_distribtests (#28560) 2022-01-18 19:14:42 +01:00
Easwar Swaminathan 8694170579
tools: set psm-security-python timeout to 3h (#28574) 2022-01-18 18:11:37 +00:00
Jan Tattermusch 7924a5fde7
add task_runner.py --inner_jobs support for C++ distribtests (#28594)
* add support for task_runner.py --inner_jobs for C++ distribtests

* fine tune linux c++ distribtests parallelism

* yapf code
2022-01-18 18:45:13 +01:00
Craig Tiller b33e0d40af
Reland slice changes (#28601)
* Revert "Revert "Eliminate slice interning (#28363)" (#28598)"

This reverts commit 03bf699600.

* fix?
2022-01-18 09:29:49 -08:00
Jan Tattermusch 03bf699600
Revert "Eliminate slice interning (#28363)" (#28598)
This reverts commit 6703186b7a.
2022-01-18 15:44:18 +01:00
Craig Tiller 6703186b7a
Eliminate slice interning (#28363)
Eliminate slice interning, and structures in slices to support it.
Reduces grpc_slice_refcount from 40 bytes (+ a required 8 bytes elsewhere) to 16 bytes.
Removes a pointer dereference for every slice ref/unref.

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2022-01-17 20:25:45 -08:00
Lidi Zheng 8015ae1648
Remove the explicit IO platform setting (#28569) 2022-01-14 14:12:06 -08:00
Lidi Zheng 7069770ee0
Merge the 3 repeating Python binary compilations (Attempt 2) (#28543)
* Revert "Revert "Merge the 3 repeating Python binary compilations (#28500)" (#28539)"

This reverts commit 0554cbee9c.

* Update hardcoded Python path

* Repect existing images and only use new path for newer releases

* fix interop_matrix testcases

Co-authored-by: Jan Tattermusch <jtattermusch@google.com>
2022-01-14 09:25:33 -08:00
Yousuk Seung c02fe64bea
Support custom xDS channel creds (#28486)
This patch introduces a factory to allow supporting custom xDS channel
creds. Three types currently supported (fake, insecure, google_default)
are registered by default for backward-compatibility.
2022-01-13 23:00:10 -08:00
Esun Kim e0a3a513a9
Buildify upb targets for validate & udpa (#28531)
* Buildify upb targets for validate & udpa

* Fix sanity test
2022-01-13 16:11:56 -08:00
Sergii Tkachenko fd3dbcb371
xds-k8s: Fix ModuleNotFoundError: No module named 'packaging' (#28556)
`packaging` was explicitly used in xds_url_map_testcase.py,
but wasn't added to the requirements.txt.
It (unintentionally) worked before because `packaging` is
a transitive dependency of `google-api-python-client@1.12.8`
via `google-api-core@1.31.5`.

With `google-api-python-client` upgraded to `1.12.10`, it's not
the case anymore.
2022-01-13 16:00:33 -08:00
Esun Kim b38b706200
Removed meshca.proto (#28535) 2022-01-13 11:10:43 -08:00
Jan Tattermusch 32b087e674
Ruby artifact speedup (#28542)
* fix ruby native extension build parallelism defaults

* allow overriding ruby artifact build parallelism with GRPC_RUBY_BUILD_PROCS

* honor --inner_jobs for ruby artifact build

* always start building ruby artifacts first

* fine tune grpc_distribtests_ruby.sh

* address review feedback

* strip newline from nproc output
2022-01-13 14:58:48 +01:00
Jan Tattermusch d61b564ac0
Update third_party/boringssl-with-bazel (#28510)
* Update third_party/boringssl-with-bazel

* regenerate projects

* regenerate boringssl prefix headers

* Increase boringssl podspec version

* regenerate projects (2nd time)
2022-01-12 18:39:29 +01:00
Jan Tattermusch 42c08b9b56
Update third_party/protobuf to v3.19.2 (#28511)
* Update third_party/protobuf

* run tools/distrib/python/make_grpcio_tools.py

* regenerate protos for csharp, ruby, php

* update build_handwritten.yaml

* regenerate projects
2022-01-12 18:35:07 +01:00
Jan Tattermusch 54036ef62d
bump C-core version for upcoming release (#28527)
* bump C-core version for upcoming release

* regenerate projects
2022-01-12 18:31:26 +01:00
Jan Tattermusch 0554cbee9c
Revert "Merge the 3 repeating Python binary compilations (#28500)" (#28539)
This reverts commit 2d4f3c5600.
2022-01-12 18:31:02 +01:00
Craig Tiller e29bdfe4e8
Add a test for includes without paths (#28532)
* Add a test for includes without paths

* fix path

* fix

* Automated change: Fix sanity tests

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2022-01-12 07:00:02 -08:00
Jan Tattermusch 98999225be
Speedup linux portability build_only tests (#28461)
* try speeding up linux portability test

* remove gcc8.3 portability test

* c-ares is the default resolver on linux

* dont set parallel run_tests jobs too high

* original parallelism might lead to better results
2022-01-12 14:56:28 +01:00
Lidi Zheng 2d4f3c5600
Merge the 3 repeating Python binary compilations (#28500)
* Merge the 3 repeating Python binary compilations

* Restore grpcio_metadata.py

* run_tests.py is running on <3.6

* Restore the Windows gevent version pin
2022-01-11 17:14:22 -08:00
Craig Tiller d2d2d0650a
Allow MetadataMap to be a status in TrySeq (#28490)
* Enable metadata to be a 'status' for TrySeq

Begins to allow us to return trailing metadata and get the right
behaviors.

* comment
2022-01-11 13:51:43 -08:00
apolcyn 95b313de82
Remove ruby 2.4 support (#28522)
* changes to remove ruby 2.4 support

* some fixes

* remove opensuse since eol

* fixes

* remove unused x86 dockerfile

* stop building for ruby 2.4

* update build scripts

* fix scripts

* fix sanity

Co-authored-by: Jan Tattermusch <jtattermusch@google.com>
2022-01-11 13:30:58 +01:00
Jan Tattermusch f23e1d9e85
task_runner.py improvements (foundations for future speedup) (#28480)
* add --dry_run support for task_runner.py

* dont sort task_runner targets before running

* fixup dry run

* support task_runner.py --inner_jobs

* pass inner_jobs to build_jobspec for task_runner.py

* support inner_jobs for C# artifacts

* support inner_jobs for protoc artifacts

* inner jobs support for src/csharp/experimental/build_native_ext_for_android.sh

* address review feedback
2022-01-11 10:26:32 +01:00
Menghan Li dde00b6b91
xds/interop: add gke TD resource cleanup to daily cleanup script (#28370) 2022-01-07 11:39:47 -08:00
Craig Tiller 4b881c5947
Remove grpc_mdelem (#28267)
* Automated change: Fix sanity tests

* content-type

* Automated change: Fix sanity tests

* clang-format

* fix

* Move colon prefixed metadata

* Automated change: Fix sanity tests

* Automated change: Fix sanity tests

* try to fix windows failure

* try and scale sizes better

* ambiguity fix?

* wip metadatavalueasslice

* Fix status code for resource exhaustion

* Revert "Revert "Move a bunch of slice typed metadata to new system (#28107)" (#28208)"

This reverts commit 7717587063.

* fix test

* Automated change: Fix sanity tests

* Automated change: Fix sanity tests

* slice helper

* x

* noinline

* try and scale sizes better

* Automated change: Fix sanity tests

* fixes

* fix

* fix

* fixes

* fix build

* fix overflow

* progress

* Automated change: Fix sanity tests

* fix

* initial work

* progress

* fix

* fix

* Automated change: Fix sanity tests

* progress

* fix

* fix

* fix

* Automated change: Fix sanity tests

* fix

* compressor for path/authority

* Automated change: Fix sanity tests

* Automated change: Fix sanity tests

* legalize

* legalize

* status-enc

* fmt

* fix

* fix

* fix

* fix

* fix/opt

* fix

* fix

* fix

* fix

* Automated change: Fix sanity tests

* fix

* comment

* fmt

* remove arg

* Automated change: Fix sanity tests

* remove name

* Automated change: Fix sanity tests

* sketch

* Automated change: Fix sanity tests

* progress

* add specialized encoders for compression metadata

* progress

* review feedback

* fix

* missoing files

* remove crud

* xxxxx

* more

* Small improvement in memory usage and performance

* Automated change: Fix sanity tests

* Automated change: Fix sanity tests

* Fix caching

* Automated change: Fix sanity tests

* omg

* fix crash in alts

* default everything

* Automated change: Fix sanity tests

* review feedback

* fixes

* fixes

* fixes

* Automated change: Fix sanity tests

* speedup

* fix

* fix

* clang-format

* fixes

* progress

* fixes

* fixes

* Automated change: Fix sanity tests

* fix

* fix

* progress

* update timeout encoding algorithm

* Automated change: Fix sanity tests

* fix

* fix

* review feedback

* progress

* progress

* progress

* progress

* builds

* Automated change: Fix sanity tests

* remove debug code

* fix ub

* unname things

* coax compilers

* reduce failures

* fix

* fix

* fix

* fix

* cleanup

* compile fix

* preserve concatenation rule

* spew

* fix

* fix

* postfix operator++ for msvc

* fix

* fix

* remove unused code

* lower cost of hpack table construction

* fix refcounting

* review feedback

* fixes

* fixes

* support multivalued traits

* Automated change: Fix sanity tests

* fix

* clearer clear

* appeasing 4.9

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2022-01-07 09:42:27 -08:00
Craig Tiller 55b365f287
Move resolver code around a little (#27846)
* move service config

* service config should not depend on grpc_base

* move resolver, deal with fallout

* Automated change: Fix sanity tests

* fix

* fix

* Automated change: Fix sanity tests

* review feedback

* review feedback

* fix

* Automated change: Fix sanity tests

* fix

* Automated change: Fix sanity tests

* fix vis

* fix

* fixes

* Automated change: Fix sanity tests

* visibility

* fix

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2022-01-06 10:53:40 -08:00
krestofur 7759632a2d
Add CRL checking to gRPC Core (#28418) 2022-01-05 14:28:19 -08:00
Jan Tattermusch 6b869a12fe
remove python2 from bazel docker images (#28458) 2022-01-05 20:10:19 +01:00
Jan Tattermusch c0564aa21f
Speedup grpc_portability_build_only windows kokoro job (#28460)
* only run windows C++ portability build only tests on win

* remove .cfg files for no longer used kokoro windows jobs
2022-01-04 20:29:46 +01:00
Jan Tattermusch 7dfdf3d679
Delete no-longer-used kokoro jobs .cfg files. (#28465)
* remove .cfg files for non-existent kokoro jobs

* more cleanup
2022-01-04 20:26:25 +01:00
Jan Tattermusch c796d5786b
Reintroduce #28396 (Cleanup of windows CI .bat scripts) (#28457)
* Revert "Revert "Cleanup of windows CI .bat scripts (#28396)""

This reverts commit 530187f865.

* fixup
2022-01-04 18:30:05 +01:00
Craig Tiller a7dcceb365
Revert "Cleanup of windows CI .bat scripts (#28396)" (#28446)
This reverts commit 44e7be448d.
2021-12-29 12:03:22 -08:00
Yash Tibrewal ab6d7b4f02
Reland rbac filter (#28442)
* Revert "Revert "xDS: Add support for RBAC HTTP filter (#28309)" (#28441)"

This reverts commit 7aae5c66df.

* Fix internal clang errors

* Ipv6 addresses are 128 bit
2021-12-29 11:42:58 -08:00
Jan Tattermusch 44e7be448d
Cleanup of windows CI .bat scripts (#28396)
* cleanup of windows CI .bat scripts

* fixup

* fixup

* fixup
2021-12-29 12:15:48 +01:00
Craig Tiller 27662173e7
Clip long summaries (#28401) 2021-12-28 21:03:15 +00:00