Repository navigation
Conversation
Before this change the coverage wheels bundled with the hermetic toolchain were stored as a nested dict keyed by the python ABI and the toolchain platform, and the platform filtering was done in the Python update script. Now the update script stores the wheels as a flat list using a format similar to `uv.lock` - `struct(url = ..., hash = "sha256:...")` - and the Starlark side uses `select_whl` to pick the wheel that is compatible with the python version and the toolchain platform. The update script no longer filters by platform, but it accepts multiple coverage versions, each with a range of python versions to take the wheels for, because newer coverage releases drop support for older python versions, e.g.: ``` bazel run //dev/update_deps:update_coverage_deps -- 7.10.7:cp39-cp314 7.17.0:cp315 ``` The wheel that is selected for each toolchain is unchanged and the unsupported toolchain platforms still fall back to the `py3-none-any` wheel. This prepares for selecting the platform specific wheels for more of the toolchain platforms. Work towards bazel-contrib#3902
…ions Instead of a python tag range for each coverage version, use PEP 508 markers on the wheels to express which coverage version is for which python versions.
…atforms Before this change the bundled coverage wheels were only selected for the glibc `x86_64` and `aarch64` Linux and the macOS toolchains. All of the other toolchain platforms fell back to the pure Python `py3-none-any` wheel, which lacks the C tracer and is slower. This adds `python_toolchain_platform`, which creates the `select_whl` target platform (the PEP 508 env and the whl ABI and platform tags) from the python version and the python toolchain platform, e.g. `x86_64-unknown-linux-musl` or `aarch64-apple-darwin-freethreaded`. The coverage wheel selection now uses it, so the musl `x86_64`, `i386`, `ppc64le` (Python 3.15) and `riscv64` Linux toolchains get the platform specific wheels. The selected wheels for the other toolchains are unchanged. * Also updates the coverage docs, which said that the platforms without a bundled wheel get no coverage tool. Stacked on bazel-contrib#4414. Work towards bazel-contrib#3902
…ge tool Before this change the coverage collection was only exercised by running `bazel coverage` on a few targets in CI and the collected data was not checked. This adds a test that runs as part of `bazel test`. A `py_reconfig_test` is built with `--collect_code_coverage` for a Python 3.12 toolchain that has the bundled coverage tool configured, and `coverage_collection_test` runs it with the same environment variables that `bazel coverage` sets and checks the line coverage in the produced lcov file. * Adds the `collect_code_coverage` attribute to `py_reconfig_*` rules. * Configures the bundled coverage tool for the Python 3.12 dev toolchain. Stacked on bazel-contrib#4415. Work towards bazel-contrib#3902
Before this change the coverage updater queried the PyPI JSON API for each coverage version that was passed on the command line together with a marker. Now the bundled coverage versions are defined in `//dev/update_deps/coverage:pyproject.toml` with `python_version` markers and locked with `uv lock` via the `lock` rule. The updater parses the resulting `uv.lock` and stores the same values (the wheel `url`, `hash` and the `resolution-markers` of the package as the `marker`) in `python/private/coverage_deps.bzl`. To bump the coverage versions, edit the `pyproject.toml` and run: ``` bazel run //dev/update_deps/coverage:uv_lock.update bazel run //dev/update_deps:update_coverage_deps ``` The bundled wheels are unchanged. The markers are now written in the form that `uv` normalizes them to, e.g. `python_full_version < '3.15'`, which only differs from the previous `python_version < "3.15"` for the 3.15 pre-release interpreters. Stacked on bazel-contrib#4417. Work towards bazel-contrib#3902
Registering the toolchain with the `3.12` minor version made it take precedence over the `3.12.16` toolchain that //tests/toolchains expects.
… versions Use the same form of the markers that uv writes in the uv.lock files.
…overage-platform-whls # Conflicts: # python/private/coverage_deps.bzl
…ge tool Before this change the coverage collection was only exercised by running `bazel coverage` on a few targets in CI and the collected data was not checked. This adds a test that runs as part of `bazel test`. A `py_reconfig_test` is built with `--collect_code_coverage` for a Python 3.12 toolchain that has the bundled coverage tool configured, and `coverage_collection_test` runs it with the same environment variables that `bazel coverage` sets and checks the line coverage in the produced lcov file. * Adds the `collect_code_coverage` attribute to `py_reconfig_*` rules. * Configures the bundled coverage tool for the Python 3.12 dev toolchain. Stacked on bazel-contrib#4415. Work towards bazel-contrib#3902
Registering the toolchain with the `3.12` minor version made it take precedence over the `3.12.16` toolchain that //tests/toolchains expects.
…ests Run the instrumented program once and check the line and branch coverage of particular lines in separate tests.
…overage-uv-lock # Conflicts: # dev/update_deps/update_coverage_deps.py # python/private/coverage_deps.bzl
…rage-uv-lock # Conflicts: # tests/coverage/coverage_collection_test.py # tests/coverage/instrumented.py # tests/coverage/lib.py
Allow py_runtime.coverage_tool to be a py_library of coverage.py, such as @pypi//coverage, by using its coverage/__main__.py file as the entry point. This allows users to use the coverage version that is locked in their requirements.
Add python.single_version_override.coverage_tool and the coverage_tool argument to python_register_toolchains so that the coverage tool can be set once for all platforms of a Python version instead of once per platform via single_version_platform_override.
…s no urls When single_version_override was used without urls, e.g. to only set libpython or distutils, the strip_prefix of all platforms was reset to the tag default. The archives of the full flavor, which are used for the freethreaded builds of some versions, need a different strip_prefix, so keep the existing strip_prefix when the archives are not overridden.
…rary' into aignas.test.coverage-from-pypi
…de-keep-config' into aignas.test.coverage-from-pypi
…ool' into aignas.test.coverage-from-pypi # Conflicts: # tests/python/python_tests.bzl
A py_library depends on the Python toolchain, so using @pypi//coverage as the coverage_tool of the toolchain creates a dependency cycle. Instead find the coverage/__main__.py entry point in the files of the target, so that @pypi//coverage:extracted_whl_files can be used.
…coverage-from-pypi
…p hub Configure a 3.13 toolchain with the coverage tool from the extracted coverage wheel of a pip.parse hub and run the same coverage collection checks as for the bundled coverage tool.
Collaborator
Author
|
This PR was drafted by an AI coding agent (Claude Code) working for @aignas as part of the plan in #3902. It stays a draft until @aignas has reviewed it. Review comments will be handled by @aignas, possibly with the help of the agent; any agent-written replies will be marked as such. Agent-generated comment, posted automatically on behalf of the user. |
1 of 11 tasks
Collaborator
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Check the coverage collection with the
coveragepackage from apip.parsehub.
A dev
coverage_piphub is created from theuv.lockfile of the bundledcoverage versions and a separate 3.13 toolchain uses its
@coverage_pip//coverage:extracted_whl_filesas the coverage tool viapython.single_version_override. The same checks as for the bundled coveragetool are run against a program built with that toolchain.
This is a stacked PR, it contains #4418 (and the PRs it builds on), #4433,
#4434 and #4435. Only the last commit is new.
Work towards #3902