Commit Graph

38 Commits

Author SHA1 Message Date
Xuan Wang 9b424b5dd4 [Python o11y] Fix Python O11Y artifacts name (#35965)
Update observability to PYPI failed because the artifact name is not correct.

This PR:
* Fix the artifacts name.
* Add step to test observability artifacts in `test_packages.sh`.
* Added `-fno-ipa-cp` compile flag.
  * We're seeing `inlining failed in call to always_inline 'vsnprintf': function body can be overwritten at link time` errors when building from source using musl libc.
  * Based on [investigation](https://gitlab.alpinelinux.org/alpine/aports/-/issues/8626), it's because we're using `-flto` flag.
  * One solution is to [disable fortify by adding this flag](https://gitlab.alpinelinux.org/alpine/aports/-/issues/8626).

After this PR, the observability artifacts have the correct name:
* [Distribution Tests Python Linux](https://pantheon.corp.google.com/storage/browser/grpc-testing-kokoro-prod/test_result_public/prod/grpc/core/pull_request/linux/grpc_distribtests_python/28687/20240223-105306/github/grpc/artifacts;tab=objects?pageState=(%22StorageObjectListTable%22:(%22f%22:%22%255B%255D%22))&e=13802955&mods=-logs_tg_prod&prefix=&forceOnObjectsSortingFiltering=false)

Also tested that the artifacts build in this PR can be successfully uploaded to testpypi:
* https://test.pypi.org/project/grpcio-observability/1.63.0.dev0

<!--

If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the appropriate
lang label.

-->

Closes #35965

PiperOrigin-RevId: 611268154
2024-02-28 16:33:28 -08:00
Xuan Wang c7a1336566
[Python 3.12] Deprecate distutil (#34186)
### Background

* `distutils` is deprecated with removal planned for Python 3.12
([pep-0632](https://peps.python.org/pep-0632/)), thus we're trying to
replace all distutils usage with setuptools.
* Please note that user still have access to `distutils` if setuptools
is installed and `SETUPTOOLS_USE_DISTUTILS` is set to `local` (The
default in setuptools, more details can be found [in this
discussion](https://github.com/pypa/setuptools/issues/2806#issuecomment-1193336591)).

### How we decide the replacement

* We're following setuptools [Porting from Distutils
guide](https://setuptools.pypa.io/en/latest/deprecated/distutils-legacy.html#porting-from-distutils)
when deciding the replacement.

#### Replacement not mentioned in the guide

* Replaced `distutils.utils.get_platform()` with
`sysconfig.get_platform()`.
* Based on the [answer
here](https://stackoverflow.com/questions/71664875/what-is-the-replacement-for-distutils-util-get-platform),
and also checked the document that `sysconfig.get_platform()` is good
enough for our use cases.
* Replaced `DistutilsOptionError` with `OptionError`.
* `setuptools.error` is exporting it as `OptionError` [in the
code](https://github.com/pypa/setuptools/blob/v59.6.0/setuptools/errors.py).
* Upgrade `setuptools` in `test_packages.sh` and changed the version
ping to `59.6.0` in `build_artifact_python.bat`.
* `distutils.errors.*` is not fully re-exported until `59.0.0` (See
[this issue](https://github.com/pypa/setuptools/issues/2698) for more
details).

### Changes not included in this PR

* We're patching some compiler related functions provided by distutils
in our code
([example](ee4efc31c1/src/python/grpcio/_spawn_patch.py (L30))),
but since `setuptools` doesn't have similar interface (See [this issue
for more details](https://github.com/pypa/setuptools/issues/2806)), we
don't have a clear path to replace them yet.


<!--

If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the
appropriate
lang label.

-->
2023-09-06 14:32:30 -07:00
Sergii Tkachenko de6ed9ba9f
[Python] Migrate from yapf to black (#33138)
- Switched  from yapf to black
- Reconfigure isort for black
- Resolve black/pylint idiosyncrasies 

Note: I used `--experimental-string-processing` because black was
producing "implicit string concatenation", similar to what described
here: https://github.com/psf/black/issues/1837. While currently this
feature is experimental, it will be enabled by default:
https://github.com/psf/black/issues/2188. After running black with the
new string processing so that the generated code merges these `"hello" "
world"` strings concatenations, then I removed
`--experimental-string-processing` for stability, and regenerated the
code again.

To the reviewer: don't even try to open "Files Changed" tab 😄 It's
better to review commit-by-commit, and ignore `run black and isort`.
2023-06-09 15:08:55 -07:00
Lidi Zheng cd4261b946
Support musllinux binary wheels on x64 and x86 (#28092)
* Support musllinux binary wheels

* Skip aarach64 for now
* Consolidate the difference of mktemp
* Extend linux artifact building time && install bash for distribtest
* Stop using grpc.tools, use grpc_tools
* Update the README to use grpc_tools
* Force static link libc++ for alpine binaries
* Rebase recent build script changes

* Install ccache for musllinux distribtest images

* Revert timeout change to grpc_build_artifacts
2022-02-10 14:14:56 -08:00
Lidi Zheng 5820e152cd
Remove Python 2.7 binary wheel generations (#26691)
* Remove Python 2.7 binary wheels

* Make sure pip won't pick-up newer versions

* Attempt to fix the distribtest

* Fallback to virtualenv==16.7.9 for 34 and 35

* Remove 34 from binary wheel distribtests
2021-07-23 08:20:39 -07:00
Jan Tattermusch d61db49b48
Add python3.8 aarch64 wheel distribtest (#26136)
* add python3.8 aarch64 distribtest

* register qemu emulator for linux distribtests
2021-05-03 08:16:11 +02:00
Lidi Zheng f42ad52c44 Update `--dest-dir` to `-d` 2019-03-13 12:40:10 -07:00
Lidi Zheng 64b29fba5d An attempt to fix distrib test 2019-03-13 09:58:54 -07:00
Lidi Zheng fa7f010795 Fix the hash checking mechanism 2019-03-08 10:47:04 -08:00
Lidi Zheng 127a6c1d9e Revert "Revert "Roll foward "Strip Python wheel binary"""
This reverts commit 98fc902200.
2019-03-08 10:29:00 -08:00
Lidi Zheng 98fc902200
Revert "Roll foward "Strip Python wheel binary"" 2019-03-07 17:34:45 -08:00
Lidi Zheng 50a1ddab5c Revert "Revert "Strip Python wheel binary""
This reverts commit 04609b1ea5.
2019-03-06 07:57:21 -08:00
Jan Tattermusch 04609b1ea5
Revert "Strip Python wheel binary" 2019-03-06 11:00:13 +01:00
Lidi Zheng 168df1cb5f Check hashes for pip wheels before installing 2019-03-04 16:01:45 -08:00
Mehrdad Afshari 2cb912c716 Add grpcio-testing to Python distribtests 2018-06-20 10:06:32 -07:00
Mehrdad Afshari 1a92cd0b52 More consistent source/binary Python distribtests 2018-06-20 10:06:32 -07:00
Mehrdad Afshari f4bea02a72 Fix the Python source distribtest script 2018-04-27 06:59:51 -07:00
Mehrdad Afshari b189923376 Split binary and source distribtests for Python 2018-04-11 14:48:48 -07:00
Mehrdad Afshari 0a400c6c0a Fix python/run_distrib_test.sh to pass shellcheck 2018-02-07 13:48:14 -08:00
Jan Tattermusch 7897ae9308 auto-fix most of licenses 2017-06-08 11:22:41 +02:00
Nathaniel Manista 5450f05e0c Migrate distrib, interop, and stress to GA API 2016-08-11 14:07:37 +00:00
Masood Malekghassemi 302bfd1a10 Test Python source distribution installation 2016-07-15 23:48:25 -07:00
Jan Tattermusch 4e2f7727a9 rename grpc.protoc.compiler to grpc.tools.protoc 2016-05-06 12:28:58 -07:00
Jan Tattermusch 6f0fe0f327 fix python distribtests with docker on overlay 2016-05-06 12:16:28 -07:00
Jan Tattermusch 4ead664c33 small addendum for python distribtests 2016-05-05 08:01:40 -07:00
Masood Malekghassemi 916c960e59 Fix Python distrib test 2016-05-04 15:30:30 -07:00
Masood Malekghassemi e9407c2e74 Test binary Python distributions 2016-05-04 15:30:30 -07:00
Craig Tiller 6169d5f7b0 Update copyrights 2016-03-31 07:46:18 -07:00
Craig Tiller c0473cc823 Merge github.com:grpc/grpc into compress 2016-02-11 08:51:16 -08:00
Jan Tattermusch 0474a1c393 python fixes 2016-02-09 08:34:02 -08:00
Jan Tattermusch 6677a86283 do not hardcode the python package version 2016-02-09 08:00:41 -08:00
Craig Tiller 55e813f946 Merge pull request #5088 from murgatroid99/core_long_int64_t
Replace 'long' with 'int64_t' in public core headers
2016-02-08 10:22:42 -08:00
Craig Tiller 6b5555ead1 Fix docker files, distrib test script 2016-02-05 16:07:18 -08:00
Craig Tiller 5c04760d46 Change default on Python install option 2016-02-05 14:25:56 -08:00
murgatroid99 7a55684a13 Clang format and fix copyrights 2016-02-05 11:33:30 -08:00
Craig Tiller f3f8fdbc54 Fix copyright 2016-02-05 10:46:43 -08:00
Jan Tattermusch eae8fc4a8b python distrib smoketest 2016-02-04 14:29:50 -08:00
Craig Tiller b2afc1f680 Starting Python 2016-02-02 12:30:08 -08:00