- Add a new docker image "rbe_ubuntu2004" that is built in a way that's
analogous to how our other testing docker images are built (this gives
us control over what exactly is contained in the docker image and
ability to fine-tune our RBE configuration)
- Switch RBE on linux to the new image (which gives us ubuntu20.04-based
builds)
For some reason, RBE seems to have trouble pulling the docker image from
Google Artifact Registry (GAR), which is where our public testing images
normally live, so for now, I used a workaround and I upload a copy of
the rbe_ubuntu2004 docker image to GCR as well, and that makes RBE works
just fine (see comment in the `renerate_linux_rbe_configs.sh` script).
More followup items (config cleanup, getting local sanitizer builds
working etc.) are in go/grpc-rbe-tech-debt-2023
First step in the modernization of our RBE stack (see
go/rbe-tech-debt-notes).
- Get rid of the deprecated rbe_autoconfig and start using
[rbe_configs_gen](https://github.com/bazelbuild/bazel-toolchains#rbe_configs_gen---cli-tool-to-generate-configs)
+ check in the generated toolchain configs.
- Switch from marketplace.gcr.io/google/rbe-ubuntu16-04 to
marketplace.gcr.io/google/rbe-ubuntu18-04 (this image is still not owned
by us, but at least it's newer and demonstrates how a switch to a newer
docker image is done).
- provide script for generating the linux RBE toolchain configs.
- cleanup RBE configuration in the bazelrc files used for remote build
* update win RBE docker image and toolchain
and switch win RBE builds to using it.
* delete legacy RBE windows toolchain config
* switching to VS2019 makes #28544 obsolete
* Remove the toolchains //third_party/toolchains:local and //third_party/toolchains:local_large.
* Remove the platforms :rbe_ubuntu1604, :rbe_ubuntu1604_large, :local and :local_large.
* No longer inherit from @rbe_default//config:platform but instead use it directly. It is now the only non-windows platform.
* When creating @rbe_default//config:platform directly with rbe_autoconfig, set dockerAddCapabilities and dockerPrivileged directly in the exec_properties field. No need to set dockerNetwork to "off" and dockerSiblingContainers to false since these are the defaults.
* Also set gceMachineType = "n1-highmem-2" on the default platform. This value can be overridden by specific targets that want to use LARGE_MACHINE.
* Use create_exec_properties_dict where appropriate.
* Use custom_exec_properties to define LARGE_MACHINE.
I wasn't able to test thoroughly that this PR does not break any existing targets. I was not able to run anything on windows/mac and I also don't have access to gRPC's RBE setup.
This should fix Bazel Basic Tests for Python (Local) and other
local CI runs that are now failing.
I also have a plan to fix Bazel RBE Opt C/C++.
Bazel RBE UBSAN C/C++ might need to wait for gRPC to upgrade to
using 0.23.0
Jan Tattermusch