Repository navigation
Conversation
Summary: - Thread deps from uv.lock file into the hub_repository - Use them to define the dep graph - Alter the pkg_aliases and render_pkg_aliases to be able to use the dep graph information if it is provided. TODO: - [ ] Split the whl_library_targets into separate files, I think they have stabilized enough. - [ ] Have a separate PR just for threading the dependencies to render_pkg_aliases. We probably need to handle various edge-cases where we have different package versions in the uv.lock file. - [ ] Have a separate PR to enable usage of the dependency graph. This is now probably doing a few shortcuts. - [ ] If we have uv.lock, define fewer wheel repos - only define the sources without passing the dep_template in there. - [ ] Test coverage for the above.
This is the common denominator and for now no warnings are printed, but there are opportunities to do this. This approach is way more surgical than the previous one. Fixes bazel-contrib#2948
rickeylev
pushed a commit
to rickeylev/rules_python
that referenced
this pull request
Aug 25, 2026
…ontrib#4105) Summary: - If `requires_dist` are passed we create a `py_library` using the aliased `srcs` target. - Currently this is only exercised by tests. Split out of bazel-contrib#4059
…gle_dep_whl_library
The hub repository references the srcs and whl_file targets of each spoke repository, so whl_deps_repo needs to alias them to the repository holding the extracted wheel.
The experimental_index_url is resolved using envsubst to decide if the bazel downloader should be used. Pass the unresolved template to the simpleapi_download and whl_library instead of the resolved value, so that credentials that may be stored in the environment variables do not end up in the whl_library arguments or the lock file.
Extract each wheel downloaded by the bazel downloader once for every unique URL, filename and integrity combination and replace the whl_library repositories with whl_deps_repo repositories that wire the dependencies by reading the METADATA from the extracted repository. The download arguments, such as the index_url and envsubst, are taken from the first registration of the wheel. The URLs are compared without resolving envsubst, so that credentials do not end up in the arguments. Wheels with patches, annotations or extra hub aliases are still extracted in the whl_library repository.
The wheels downloaded by the bazel downloader are now extracted in a repository named after the wheel filename, so check for the python tag of the wheel in addition to the python version in the spoke repo name.
Labels are rendered with a single @ in WORKSPACE mode.
Move the lookup of the repos that have extra hub aliases into shared_whl_srcs so that it is covered by the unit tests.
Wheels that are extracted differently, e.g. with a different pip_data_exclude, now get their own extracted repo instead of falling back to a regular whl_library. Also name the envsubst news entry after the PR.
The previous approach on this branch is dropped: parts of it landed separately (bazel-contrib#4087, bazel-contrib#4101, bazel-contrib#4105, bazel-contrib#4106) and the rest is obsolete. The tree is reset to the state of bazel-contrib#3856 (shared extracted wheel repos) so that the uv.lock dependency graph plumbing can be reimplemented on top of it.
When `pkg_aliases` creates the `pkg` and `whl` targets in the hub for a package that is part of a dependency group, the public labels are aliases to the group. Create the `_pkg` and `_whl` targets instead, which the group implementation in `//_groups` depends on, so that the names do not clash.
Render the dependency edges of each uv.lock entry as PEP 508 requirements, keeping the extras and the markers of the edges. The optional dependencies are included only for the extras that the dependents request. If the entries of a package that are used on the target platforms have different dependencies, the dependencies are left unknown, so that the wheel METADATA is used instead.
The downloaded wheels are now extracted in a repository named after the wheel filename, which also owns the lazily fetched archive repository.
The hub has a single dependency graph for all of its python versions, so the uv.lock dependencies of a package are used only if they are the same for all python versions. The dependencies on packages that are not in the hub, e.g. ones only used on other platforms, are dropped, like the `whl_library` does with the `packages` from `config.bzl`.
When `pkg_aliases` is given `requires_dist`, `whl_library_deps_targets` creates the `py_library` and `env_marker_setting` targets, which were missing from the default `rules` struct, because only the unit tests with mocks used this code path.
Pass the uv.lock dependencies of each package to the `hub_repository` and render them as the `requires_dist` of `pkg_aliases`, so that the hub creates the `pkg` and `whl` targets wired with the dependencies from the uv.lock file instead of the wheel METADATA. The members of a dependency group are passed as `group_deps`.
…ibrary' into aignas.chore.alias-src-targets
uv.lock always writes the extras of a dependency as a list, so drop the string handling and fix the test fixture that used a string.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When a hub is created from a
uv.lockfile, wire the package dependenciesin the hub from the
uv.lockdependency graph instead of the wheelMETADATA.parse_requirementsrenders the dependency edges of eachuv.lockentryas PEP 508 requirements, keeping the extras and the markers of the edges.
The optional dependencies of a package are included only for the extras
that its dependents request. The
hub_buildercollects them per hub anddrops the edges to packages that are not in the hub, e.g. ones only used
on other platforms. The
hub_repositorypasses them topkg_aliasesasrequires_dist, which then creates thepkgandwhltargets in thehub, and the members of a dependency group as
group_deps.The hub keeps using the wheel
METADATAfor a package when:uv.lockentries of the package that are used on the targetplatforms have different dependencies, e.g. forks per Python version;
pip.parsecalls for differentPython versions of the same hub;
This threads the dependencies through the existing
pkg_aliasessupport (#4105) instead of passing them to the
whl_deps_repoas inlinemetadata, because the hub then owns the dependency graph and theper-package repos are only needed for the sources.
pkg_aliasesisfixed to create the
_pkgand_whlgroup implementation targets andto pass the rules it needs, as this code path was only used with mocks.
Follow-ups:
whl_mapdirectly at the shared extracted wheel reposfor the packages with known dependencies and skip creating their
whl_deps_repo. This needspkg_aliasesto also create the targetsfor packages without dependencies.
uv.lockentries using theirresolution-markersinstead of falling back toMETADATA.pip_parsewithuv_lockstill uses the wheelMETADATA.Stacked on #3856.