Commit Graph

248 Commits

Author SHA1 Message Date
Craig Tiller ef6a5b92c9
[build] Automatic BUILD dependency generation (#29597)
* [build] Automatic dependency analysis

* add to sanity

* Automated change: Fix sanity tests

* review feedback

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2022-05-19 13:42:52 -07:00
Jan Tattermusch e8404698d5
Remove C# implementation (individual packages will continue to be maintained through v2.46.x patches or moved to grpc-dotnet) (#29225)
* remove examples/csharp

* remove reference to examples/csharp

* remove C# references from doc/

* remove src/csharp

* update test_creds locations

* remove C# distribtests

* remove buildgen templates for files under src/csharp

* remove grpc_csharp_ext C build target

* regenerate projects

* update tools/run_tests/README.md

* change many C# test/build scripts to noop

* unbreak run_tests.py -l csharp

* unbreak C# build_package targets

* unbreak C# distribtests

* remove C# pre-build steps from prepare_build_windows.bat

* remove src/csharp from clang_format

* remove csharp and csharpcoreclr from run_interop_tests.py

* remove C# interop test docker images

* Revert "remove src/csharp"

This reverts commit 97f2b6d707.

* remove src/csharp, but keep Grpc.Tools

* add back templates/src/csharp/build/dependencies.props.template

* remove everything but Grpc.Tools and Grpc.Tools.Tests from sln

* Revert "unbreak run_tests.py -l csharp"

This reverts commit 529b5e2ec1.

* revive Grpc.Tools.Tests through run_tests.py

* Revert "unbreak C# build_package targets"

This reverts commit 79d8d83c8b.

* experiment with build_nuget.sh fixes

* simplify create_fake_native_artifacts.sh

* building of Grpc.Tools in package_targets.py

* move shared files from Grpc.Core to Grpc.Tools

* yapf format: distribtest_targets.py

* Revert "remove C# pre-build steps from prepare_build_windows.bat"

This reverts commit bd38956a2b.

* update C# README.md

* disable C# in gke OSS benchmarks
2022-04-28 12:00:34 +02:00
Lidi Zheng ec5b3b22c3
Remove var type annotations in extract_metadata_from_bazel_xml (#29463)
* Remove var type annotations in extract_metadata_from_bazel_xml

* Use plain-old Python class to store data

* No more f-string

* Remove version check in generate_projects.py
2022-04-21 12:53:20 -07:00
Lidi Zheng b3ded99cd4
Add a raise for generate_projects if Python is too old (#29432)
* Add a raise for generate_projects if Python is too old

* Update the comment

* Resolve the YAPF difference between local 0.30 and kokoro's 0.30

* YAPF
2022-04-18 15:24:48 -07:00
Lidi Zheng 99752b173c
[xDS Proto] Enhence gRPC buildgen for 3rd party proto compilation (#29254)
* [xDS Proto] Enhence gRPC buildgen for 3rd party proto compilation

* Rebase from master to update envoy-api version

* Address reviewer's comments

* Address reviewer's comment

* Regenerate project

* Rename external_library

* Address reviewer's comments

* Add comments for the internals of generate C++ proto code

* Add proto file as a dependency to the custom command
2022-04-13 14:26:08 -07:00
Richard Belleville e79b2a9b32
Add Documentation for Supported Bazel Versions (#29191)
* Create Bazel support range docs

* Review comments

* Move supported versions file
2022-03-25 10:45:06 -07:00
AJ Heller 762cde4d24
Delete end2end nosec tests (#28946)
* rm nosec tests (TODO: identify coverage differences)

* grpc::load_reporter

* delete unused nosec test files & config

* trigger build

* re-delete nosec test file (snuck in with a merge)

* revert TODO deletion
2022-03-07 08:52:30 -08:00
Craig Tiller 7c74d58cbe
Disable memory_usage test on cmake builds (#28948) 2022-02-23 17:21:55 +01:00
Esun Kim f1e79853ed
Revert "Revert "Buildify Envoy upb (#28558)" (#28648)" (#28649)
This reverts commit 01011ab259.
2022-01-25 11:19:41 -08:00
Esun Kim 01011ab259
Revert "Buildify Envoy upb (#28558)" (#28648)
This reverts commit 97584d8346.
2022-01-21 08:35:13 -08:00
Esun Kim 97584d8346
Buildify Envoy upb (#28558)
* Buildigy xds

* Generate project

* Buildify envoy

* Added new upb files

* Removed unused upb targets

* Fix grpc_cel_engine

* Update envoy-api to the latest

* Regen upb

* Regen projects

* Fix bazel build on MacOS

* More fix on bazel mac

* Disable grpc_tool_test on Mac

* Regen projects
2022-01-20 22:29:58 -08:00
Esun Kim e0a3a513a9
Buildify upb targets for validate & udpa (#28531)
* Buildify upb targets for validate & udpa

* Fix sanity test
2022-01-13 16:11:56 -08:00
Esun Kim 779701ab76
google/api/... (#27928) 2021-11-30 21:36:34 -08:00
Esun Kim 9be7ca5a55
Roll-forward #27780 (#27951)
* Revert "Revert #27780 (#27944)"

This reverts commit 26e7560309.

* Fix google_api_upbdefs
2021-11-18 18:18:02 -08:00
Richard Belleville 59693b7e72
Migrate Infrastructure Scripts to Python 3 (#27135)
* Run 2to3 on tools directory

* Delete github_stats_tracking

* Re-run 2to3

* Remove unused script

* Remove unused script

* Remove unused line count utility

* Yapf. Isort

* Remove accidentally included file

* Migrate tools/distrib directory to python 3

* Remove unnecessary shebang

* Restore line_count directory

* Immediately convert subprocess.check_output output to string

* Take care of Python 2 shebangs

* Invoke scripts using a Python 3 interpreter

* Yapf. Isort

* Try installing Python 3 first

* See if we have any Python 3 versions installed

* Add Python 3.7 to Windows path

* Try adding a symlink

* Try to symlink differently

* Install six for Python 3

* Run run_interop_tests with python 3

* Try installing six in python3.7 explicitly

* Revert "Try installing six in python3.7 explicitly"

This reverts commit 2cf60d72f3.

* And debug some more

* Fix issue with jobset.py

* Add debug for CI failure

* Revert microbenchmark changes
2021-11-08 08:55:25 -08:00
Esun Kim 26e7560309
Revert #27780 (#27944) 2021-11-04 15:00:37 -07:00
Esun Kim d5971f49c4
Added a new bazel-integrated upb code-gen (#27780)
* Added bazel-integrated upb code-gen

* Update by review

* Update by review
2021-11-01 12:51:48 -07:00
Yash Tibrewal 6d2a641dd3
XdsChannelStackModifier class (#27744)
* ChannelStackModifier class

* Regenerate projects

* clang-tidy

* Use CoreConfiguration for inserting xDS HTTP filters

* New lines

* Move test to test/core/xds

* Add comment for placement of filter stack

* Fix sanity

* Fix memory leak - destroy builder

* Don't build xds_channel_stack_modifier_test on non-bazel build systems due to census dependency
2021-10-22 13:48:30 -07:00
Craig Tiller a67a9a5cd5
Exclude microbenchmarks from cmake (#27588)
* Exclude microbenchmarks from cmake

* Trigger automation

* fix
2021-10-05 10:13:57 -07:00
Craig Tiller 87a973f01d
Add structurally aware fuzzing support (#27405)
* Add a dependency on libprotobuf-mutator

* Changes for structured fuzzing

* fuzznificent

* add a config that will build fuzzers

* bazel run --config=fuzzer_asan //path/to:fuzzer fuzzes!

* fixes

* Automated change: Fix sanity tests

* Automated change: Fix sanity tests

* fixes

* fix

* sanitizer fix

* fix directory

* regen stuff

* compile error

* remove fuzzers from cmake build

* Automated change: Fix sanity tests

* review feedback

* remove unnecessary load

* review feedback

* review feedback

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2021-09-22 14:40:43 -07:00
AJ Heller 2d16865693
Delete libuv-iomgr implementation and GRPC_UV build option (#27188)
This has been unmaintained for years, last supported in gRPC-core v1.24.
2021-08-31 13:26:43 -07:00
Richard Belleville b6cc72f129
Revert "Run 2to3 on tools directory (#26002)" (#27183)
This reverts commit a388361c3d.
2021-08-30 12:29:20 -07:00
Richard Belleville a388361c3d
Run 2to3 on tools directory (#26002)
* Run 2to3 on tools directory

* Delete github_stats_tracking

* Re-run 2to3

* Remove unused script

* Remove unused script

* Remove unused line count utility

* Yapf. Isort

* Remove accidentally included file

* Restore line_count directory
2021-08-30 09:25:21 -07:00
Lidi Zheng 2231c2ba77
Introduce Python import sorting to our sanity test suite (#26768)
* Add isort_code.sh to sanity tests

* Run tools/distrib/isort_code.sh

* Fine tune the import order for relative imports

* Make pylint and project generation happy

* Fix a few corner cases

* Use --check instead of --diff

* The import order impacts test result somehow

* Make isort print diff and check output at the same time

* Let tools/run_tests/python_utils be firstparty library

* Run isort against latest HEAD
2021-07-26 12:31:21 -07:00
AJ Heller 4d4ee609c1
gRPC EventEngine Interface (#25795) 2021-04-05 09:26:16 -07:00
Jan Tattermusch 8cae49174c
extract_metadata_from_bazel_xml.py changes from #25272 (#25758)
* tools/buildgen/extract_metadata_from_bazel_xml.py changes from #25272

* regenerate projects

* get rid of useless _deduplicate_append

* extract_deps worked for both collapsed_sources and collapsed_deps only due to mismatch of bazel label vs target name mismatch

* special handling for absl has no effect

* improve comments

* remove changes that are not needed yet
2021-03-18 12:07:53 -07:00
Lidi Zheng e7d9cccf9a
Fix build abseil/boringssl/protobuf at head (#25638)
* Update PyYAML install steps

* F-string is 3.6+

* Regenerate project
2021-03-05 14:00:32 -08:00
Jan Tattermusch a2697e4592
simplify build.yaml: get rid of the "secure" field (#25547)
* get rid of useless "secure" field from build.yaml

* remove nonsensical "liblibssl.a" dependency from makefile

* regenerate projects
2021-03-04 15:07:24 +01:00
Lidi Zheng 39cc797025
[xDS Proto] Modernize buildgen scripts (#25512)
* [xDS Proto] Modernize buildgen scripts

* This PR only covers some buildgen scripts that will be used by xDS
proto
* This PR also improves the debuggability of some scripts
* Merge with master

* Adopt reviewer's advices
2021-03-02 11:25:01 -08:00
Lidi Zheng d515270e86
Fix the generated_projects.sh again (#25572)
* Double quote for shell string comparison
2021-02-26 13:21:31 -08:00
Lidi Zheng 4a90992d03
Update PyYAML in generate_project.sh if needed (#25563) 2021-02-25 13:35:14 -08:00
Jan Tattermusch 53d32b748f
Cleanup: Remove no-longer-needed expand_filegroups.py (#25532)
* get rid of expand_filegroups.py

* regenerate projects
2021-02-24 19:46:58 +01:00
Jan Tattermusch 3b6ea094dd
get rid of dll_only in build.yaml (#25524) 2021-02-23 07:23:42 +01:00
Jan Tattermusch 0c66144692
Simplify build.yaml's "deps_linkage" logic (#25521)
* regenerate to fix sanity

* makefile template always assumes deps_linkage=static

* stop setting no-longer-used deps_linkage field in build_autogenerated.yaml

* regenerate projects
2021-02-22 19:58:09 +01:00
Lidi Zheng a6c5d6c43e
[xDS Proto] Pin PyPI dependencies in Dockerfiles (#25510) 2021-02-22 10:10:56 -08:00
Jan Tattermusch 6dbc6ed4e9 remove useless build plugin "generate_vsprojects.py" 2021-02-11 11:28:29 +01:00
Yash Tibrewal 513edce7cb Replace disrespectful term 2021-02-08 13:19:27 -08:00
Sergii Tkachenko 8db79e2e71 Python style: reformat code using updated yapf 0.30.0 2021-01-13 14:00:12 -05:00
Jan Tattermusch 600dd1b86c
Merge pull request #21439 from apolcyn/revert_grpclb_in_dns_interop
Delete unused grpclb-in-dns interop test suite
2020-12-09 13:20:37 +01:00
Alexander Polcyn 70f3a25125 Delete unused grpclb-in-dns interop test suite 2020-12-08 17:02:29 +01:00
Richard Belleville 9c0497a0b0 Yapf. Index from beginning 2020-12-07 13:41:18 -08:00
Richard Belleville 38124ab0c6 Review comments 2020-12-07 11:52:44 -08:00
Richard Belleville 010d62175e Pin protobuf version dependency for grpcio-tools 2020-12-04 17:40:31 -08:00
Lidi Zheng da399bc970 Install yaml in users's directory 2020-11-17 16:34:02 -08:00
Lidi Zheng 5ea36eb47c
Revert "Revert "Resolve warnings and deprecations in several scripts in tools"" 2020-11-17 16:30:26 -08:00
Esun Kim d910557f22
Merge pull request #24690 from Capstan/remove-gflags
Remove gflags.
2020-11-16 10:30:59 -08:00
Jan Tattermusch dcb3f8c73e
Revert "Resolve warnings and deprecations in several scripts in tools" 2020-11-16 16:21:28 +01:00
capstan 1f670397ce Remove gflags.
With the move to abseil's flags, this removed the now-unnecessary submodule and installation commands.
2020-11-14 03:03:55 +01:00
Lidi Zheng 35f48d584d Explicitly import "util" 2020-11-13 13:56:31 -08:00
Lidi Zheng 6339ae9a4a Use Python3 for build_cleaner 2020-11-13 13:07:55 -08:00