Commit Graph

17 Commits

Author SHA1 Message Date
Xuan Wang 7d04e020df [Python Dist] Fix grpc_distribtests_python (#36352)
`setuptools` made a decision to change the artifact name it builds (from `grpcio-health-checking` to `grpcio_health_checking`) in their latest release (https://github.com/pypa/setuptools/issues/4300).
As a result, we need broaden our regex so that our tests can pickup the correct files.

### Note
* Using `[_-]*` instead of `[_-]?` to match one character since `bash` uses a different flavor of regular expressions called basic regular expressions (BREs) which do not support the optional quantifier `?`.

<!--

If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the appropriate
lang label.

-->

Closes #36352

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36352 from XuanWang-Amos:fix_distribution_test 8dfcc4e9a3
PiperOrigin-RevId: 625083784
2024-04-15 14:29:41 -07:00
Xuan Wang 9b424b5dd4 [Python o11y] Fix Python O11Y artifacts name (#35965)
Update observability to PYPI failed because the artifact name is not correct.

This PR:
* Fix the artifacts name.
* Add step to test observability artifacts in `test_packages.sh`.
* Added `-fno-ipa-cp` compile flag.
  * We're seeing `inlining failed in call to always_inline 'vsnprintf': function body can be overwritten at link time` errors when building from source using musl libc.
  * Based on [investigation](https://gitlab.alpinelinux.org/alpine/aports/-/issues/8626), it's because we're using `-flto` flag.
  * One solution is to [disable fortify by adding this flag](https://gitlab.alpinelinux.org/alpine/aports/-/issues/8626).

After this PR, the observability artifacts have the correct name:
* [Distribution Tests Python Linux](https://pantheon.corp.google.com/storage/browser/grpc-testing-kokoro-prod/test_result_public/prod/grpc/core/pull_request/linux/grpc_distribtests_python/28687/20240223-105306/github/grpc/artifacts;tab=objects?pageState=(%22StorageObjectListTable%22:(%22f%22:%22%255B%255D%22))&e=13802955&mods=-logs_tg_prod&prefix=&forceOnObjectsSortingFiltering=false)

Also tested that the artifacts build in this PR can be successfully uploaded to testpypi:
* https://test.pypi.org/project/grpcio-observability/1.63.0.dev0

<!--

If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the appropriate
lang label.

-->

Closes #35965

PiperOrigin-RevId: 611268154
2024-02-28 16:33:28 -08:00
Xuan Wang c7a1336566
[Python 3.12] Deprecate distutil (#34186)
### Background

* `distutils` is deprecated with removal planned for Python 3.12
([pep-0632](https://peps.python.org/pep-0632/)), thus we're trying to
replace all distutils usage with setuptools.
* Please note that user still have access to `distutils` if setuptools
is installed and `SETUPTOOLS_USE_DISTUTILS` is set to `local` (The
default in setuptools, more details can be found [in this
discussion](https://github.com/pypa/setuptools/issues/2806#issuecomment-1193336591)).

### How we decide the replacement

* We're following setuptools [Porting from Distutils
guide](https://setuptools.pypa.io/en/latest/deprecated/distutils-legacy.html#porting-from-distutils)
when deciding the replacement.

#### Replacement not mentioned in the guide

* Replaced `distutils.utils.get_platform()` with
`sysconfig.get_platform()`.
* Based on the [answer
here](https://stackoverflow.com/questions/71664875/what-is-the-replacement-for-distutils-util-get-platform),
and also checked the document that `sysconfig.get_platform()` is good
enough for our use cases.
* Replaced `DistutilsOptionError` with `OptionError`.
* `setuptools.error` is exporting it as `OptionError` [in the
code](https://github.com/pypa/setuptools/blob/v59.6.0/setuptools/errors.py).
* Upgrade `setuptools` in `test_packages.sh` and changed the version
ping to `59.6.0` in `build_artifact_python.bat`.
* `distutils.errors.*` is not fully re-exported until `59.0.0` (See
[this issue](https://github.com/pypa/setuptools/issues/2698) for more
details).

### Changes not included in this PR

* We're patching some compiler related functions provided by distutils
in our code
([example](ee4efc31c1/src/python/grpcio/_spawn_patch.py (L30))),
but since `setuptools` doesn't have similar interface (See [this issue
for more details](https://github.com/pypa/setuptools/issues/2806)), we
don't have a clear path to replace them yet.


<!--

If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the
appropriate
lang label.

-->
2023-09-06 14:32:30 -07:00
Lidi Zheng cd4261b946
Support musllinux binary wheels on x64 and x86 (#28092)
* Support musllinux binary wheels

* Skip aarach64 for now
* Consolidate the difference of mktemp
* Extend linux artifact building time && install bash for distribtest
* Stop using grpc.tools, use grpc_tools
* Update the README to use grpc_tools
* Force static link libc++ for alpine binaries
* Rebase recent build script changes

* Install ccache for musllinux distribtest images

* Revert timeout change to grpc_build_artifacts
2022-02-10 14:14:56 -08:00
Lidi Zheng 5820e152cd
Remove Python 2.7 binary wheel generations (#26691)
* Remove Python 2.7 binary wheels

* Make sure pip won't pick-up newer versions

* Attempt to fix the distribtest

* Fallback to virtualenv==16.7.9 for 34 and 35

* Remove 34 from binary wheel distribtests
2021-07-23 08:20:39 -07:00
Lidi Zheng f42ad52c44 Update `--dest-dir` to `-d` 2019-03-13 12:40:10 -07:00
Lidi Zheng 64b29fba5d An attempt to fix distrib test 2019-03-13 09:58:54 -07:00
Lidi Zheng fa7f010795 Fix the hash checking mechanism 2019-03-08 10:47:04 -08:00
Lidi Zheng 127a6c1d9e Revert "Revert "Roll foward "Strip Python wheel binary"""
This reverts commit 98fc902200.
2019-03-08 10:29:00 -08:00
Lidi Zheng 98fc902200
Revert "Roll foward "Strip Python wheel binary"" 2019-03-07 17:34:45 -08:00
Lidi Zheng 50a1ddab5c Revert "Revert "Strip Python wheel binary""
This reverts commit 04609b1ea5.
2019-03-06 07:57:21 -08:00
Jan Tattermusch 04609b1ea5
Revert "Strip Python wheel binary" 2019-03-06 11:00:13 +01:00
Lidi Zheng 168df1cb5f Check hashes for pip wheels before installing 2019-03-04 16:01:45 -08:00
Mehrdad Afshari 2cb912c716 Add grpcio-testing to Python distribtests 2018-06-20 10:06:32 -07:00
Mehrdad Afshari 1a92cd0b52 More consistent source/binary Python distribtests 2018-06-20 10:06:32 -07:00
Mehrdad Afshari f4bea02a72 Fix the Python source distribtest script 2018-04-27 06:59:51 -07:00
Mehrdad Afshari b189923376 Split binary and source distribtests for Python 2018-04-11 14:48:48 -07:00