diff --git a/crates/socket-patch-cli/CLI_CONTRACT.md b/crates/socket-patch-cli/CLI_CONTRACT.md index a3ed909d6..bb63f9523 100644 --- a/crates/socket-patch-cli/CLI_CONTRACT.md +++ b/crates/socket-patch-cli/CLI_CONTRACT.md @@ -1267,7 +1267,7 @@ v5.0 replaces v4's per-purl reverts and whole-ledger reverse replay (`revert_rem * **vlt** — `vlt-lock.json`: slot [2] from the registry's `dist.integrity`, slot [3] per the lock's own convention (see the vlt hosted-mode contract); every hosted instance of the pin together. * **cargo** — `Cargo.lock` back on crates.io (source + the sparse index's checksum, `SOCKET_CRATES_INDEX`). When the lock already holds a crates.io block for the same `name@version` (a later dependent locked its own copy, #679), the Socket block merges into it instead: it is dropped (with a v1 lock's `[metadata]` line), and dependents' references take the spelling cargo writes, so no duplicate block is left that cargo cannot parse (#863), and no index lookup is needed; every `Cargo.toml` declaration loses its `registry = "socket-patch-"` pin (the shorthand the rewriter produced collapses back); every `[registries.socket-patch-]` block no manifest or lock still references leaves the project cargo config — including a superseded patch generation's block an earlier re-pin left behind (#864). A declaration it cannot unpin refuses. * **golang** — the hosted `replace` (in the root `go.mod`, an in-root workspace member's `go.mod`, or `go.work`) and the socket module's sum lines (its `go.sum`, `go.work.sum`, and for a `go.work` pin each in-root member's `go.sum`) go; the upstream module's two go.sum lines come back: for a `go.mod` pin into the `go.sum` beside it; for a `go.work` pin, which leaves the members' pair in place, only when no sum file holds them, into each sum file that held the socket pair, else `go.work.sum`. A `go.work.sum` left empty is removed; hashed from the module proxy (`SOCKET_GOPROXY`, else `GOPROXY` / `GONOPROXY` / `GOPRIVATE` as go reads them) and cross-checked against the checksum database (`SOCKET_GOSUMDB_URL`, else `sum.golang.org` unless `GOSUMDB=off` / `GONOSUMDB` / `GOPRIVATE` say go would not ask it). A `replace` the user had before the hosted run is not recorded anywhere, so the restore lands on the plain upstream module. - * **pypi** — `Pipfile.lock`, `requirements.txt` (+ in-root `-r` includes), Hatch PEP 508 direct references (`pyproject.toml` / `hatch.toml`), `poetry.lock`, `pdm.lock`, `uv.lock`, PEP 723 script locks and PEP 751 `pylock*.toml` (+ the paired `pyproject.toml` / script metadata): hashes re-derived from PyPI's JSON API (`SOCKET_PYPI_JSON_API`). A restored `requirements.txt` line gets `--hash` options only when the file is in pip's hash-checking mode. The mode is read off the file's other requirement lines (an `-e` / `--editable` line means unhashed). When every requirement is a hosted pin, it is read off the hosted line itself (`--hash` vs a `#sha256=` url fragment) (#410). Refused: a `static_urls` `pdm.lock` whose project installs from an index other than PyPI (a `[[tool.pdm.source]]` in the sibling `pyproject.toml`, a project `pypi.url` (`.pdm.toml`, then `pdm.toml`), or, with neither, another package's file URL in the lock off PyPI's file host), since restoring PyPI's file URLs would make `pdm sync` bypass that index (#413); a `pdm.lock` without `cross_platform`, or a uv / script / pylock lock, whose release has a wheel that is not pure Python 3 (which files the lock keeps is not re-derivable); a uv lock whose options filter files (`exclude-newer`, `no-binary`, `no-build`, or an `exclude-newer-package` cutoff for the restored package; one of `false` exempts it from the global cutoff here and in pylocks, as in uv) (#1604); a pylock from `uv pip compile` whose header command line (`--no-build`, `--only-binary`, `--no-binary`, `--exclude-newer`, `--exclude-newer-package`) or uv settings (the nearest `uv.toml`, else a pyproject's `[tool.uv]` / `[tool.uv.pip]`, then the user's and the system's `uv.toml` unless `UV_NO_CONFIG` is set, merged as uv does with `[pip]` over the top level; none under `--no-config`, and any `--config-file` refuses; `UV_CONFIG_FILE` or `UV_EXCLUDE_NEWER` set in the restoring environment also refuses, since the header does not record uv's environment) filter the restored package's files, or whose header records no `uv pip compile` command line (`--no-header`, `UV_CUSTOM_COMPILE_COMMAND`), since uv 0.9.16 records only the files those filters let through (#1613); a uv lock whose other registry packages name no registry, several, or one other than PyPI's simple index; a pylock whose other registry packages show neither an `index` nor (as `uv pip compile` writes them) only PyPI files with none, which restores the entry without an `index` too; uv 0.2 `[[distribution]]` locks. Restored artifact fields keep the spelling the lock's other entries show, including the `upload_time` that uv 0.6.15–0.6.17 write. A restored pylock entry's `upload-time`s are whole seconds, as uv writes them, unless the lock's other entries show fractions. Its artifacts come back in the TOML spelling the other entries use: uv's inline `wheels = [{ … }]`, or the standard tables `pip lock` writes (`[[packages.wheels]]` with a `[packages.wheels.hashes]` sub-table, `[packages.sdist]`). A `pip lock` file (`created-by = "pip"`) records only the artifact pip selected, so the entry is restored with only the release's wheel (its sdist when it has none), and a release with several wheels is refused. A transitive `override-dependencies` entry hosted mode added is removed (`upstream_uv_override_removed`). Hosted mode keeps no ledger, so it marks the entry it adds with a `# socket-patch hosted: …` comment line above it and removes only a marked entry; a user's own `==` override (which the hosted rewrite reuses rather than adding a second one) is kept, and the lock's `[manifest] overrides` record of it gets its specifier back. + * **pypi** — `Pipfile.lock`, `requirements.txt` (+ in-root `-r` includes), Hatch PEP 508 direct references (`pyproject.toml` / `hatch.toml`), `poetry.lock`, `pdm.lock`, `uv.lock`, PEP 723 script locks and PEP 751 `pylock*.toml` (+ the paired `pyproject.toml` / script metadata): hashes re-derived from PyPI's JSON API (`SOCKET_PYPI_JSON_API`). A restored `requirements.txt` line gets `--hash` options only when the file is in pip's hash-checking mode. The mode is read off the file's other requirement lines (an `-e` / `--editable` line means unhashed). When every requirement is a hosted pin, it is read off the hosted line itself (`--hash` vs a `#sha256=` url fragment) (#410). Refused: a `static_urls` `pdm.lock` whose project installs from an index other than PyPI (a `[[tool.pdm.source]]` in the sibling `pyproject.toml`, a project `pypi.url` (`.pdm.toml`, then `pdm.toml`), or, with neither, another package's file URL in the lock off PyPI's file host), since restoring PyPI's file URLs would make `pdm sync` bypass that index (#413); a `pdm.lock` without `cross_platform`, or a uv / script / pylock lock, whose release has a wheel that is not pure Python 3 (which files the lock keeps is not re-derivable); a uv lock whose options filter files (`exclude-newer`, `no-binary`, `no-build`, or an `exclude-newer-package` cutoff for the restored package; one of `false` exempts it from the global cutoff here and in pylocks, as in uv) (#1604); a pylock from `uv pip compile` whose header command line (`--no-build`, `--only-binary`, `--no-binary`, `--exclude-newer`, `--exclude-newer-package`) or uv settings (the nearest `uv.toml`, else a pyproject's `[tool.uv]` / `[tool.uv.pip]`, then the user's and the system's `uv.toml` unless `UV_NO_CONFIG` is set, merged as uv does with `[pip]` over the top level; none under `--no-config`, and any `--config-file` refuses; `UV_CONFIG_FILE` or `UV_EXCLUDE_NEWER` set in the restoring environment also refuses, since the header does not record uv's environment) filter the restored package's files, or whose header records no `uv pip compile` command line (`--no-header`, `UV_CUSTOM_COMPILE_COMMAND`), since uv 0.9.16 records only the files those filters let through (#1613); a uv lock whose other registry packages name no registry, several, or one other than PyPI's simple index; a pylock whose other registry packages show neither an `index` nor (as `uv pip compile` writes them) only PyPI files with none, which restores the entry without an `index` too. Restored artifact fields keep the spelling the lock's other entries show, including the `upload_time` that uv 0.6.15–0.6.17 write. A uv 0.2 `[[distribution]]` lock (uv 0.1.45–0.2.34) is restored like a `[[package]]` one, in the grammar the hosted rewrite wrote the entry in, which is the entry's own: its source keeps its spelling (a string `"direct+…"` back to `"registry+…"`, uv ≤ 0.2.17, or an inline table), its artifacts their grammar (`[distribution.sdist]` / `[[distribution.wheel]]` sub-tables, uv ≤ 0.2.5, or inline), and dependents' references, inline or `[[distribution.dependencies]]` tables, are pointed back. Refused there: an entry with an sdist and inline `dependencies` when no other registry entry of the lock carries both (uv 0.2.18–0.2.22 and 0.2.23–0.2.34 order them differently; hosted mode refuses to wire such an entry, `redirect_uv_lock_unsupported`); and, for the inline-artifact grammar, an `exclude-newer` cutoff the lock does not record: the project's settings (`uv.toml`, else the pyproject's `[tool.uv]`), the user's or system `uv.toml`, or `UV_EXCLUDE_NEWER` (uv 0.2.12–0.2.27 apply these without recording them; hosted mode refuses to wire a lock whose project sets one). A cutoff uv 0.2.28–0.2.33 recorded at the lock's top level refuses like an `[options]` one. A cutoff passed to uv 0.2.6–0.2.27 only on the command line leaves no trace and cannot be detected. A restored pylock entry's `upload-time`s are whole seconds, as uv writes them, unless the lock's other entries show fractions. Its artifacts come back in the TOML spelling the other entries use: uv's inline `wheels = [{ … }]`, or the standard tables `pip lock` writes (`[[packages.wheels]]` with a `[packages.wheels.hashes]` sub-table, `[packages.sdist]`). A `pip lock` file (`created-by = "pip"`) records only the artifact pip selected, so the entry is restored with only the release's wheel (its sdist when it has none), and a release with several wheels is refused. A transitive `override-dependencies` entry hosted mode added is removed (`upstream_uv_override_removed`). Hosted mode keeps no ledger, so it marks the entry it adds with a `# socket-patch hosted: …` comment line above it and removes only a marked entry; a user's own `==` override (which the hosted rewrite reuses rather than adding a second one) is kept, and the lock's `[manifest] overrides` record of it gets its specifier back. * **gem** — `Gemfile.lock` / `gems.locked` + `Gemfile` / `gems.rb`: the spec moves back into the upstream `GEM` section (or the Socket remote leaves a merged section), the `source "" do … end` block is undone (also when a superseding hosted re-scan on a lock with no `CHECKSUMS` section has moved that block to the newer patch of the same name and version while the lock still pins the older one; several such blocks are refused), the `CHECKSUMS` entry is re-pinned from the rubygems.org compact index (`SOCKET_RUBYGEMS_URL`) and the `DEPENDENCIES` pin loses its `!`, unless the declaration comes back inside the user's own `source "…" do` block (Bundler writes the `!` for any dependency declared in one, rubygems.org included), where the `!` stays. The declaration's original constraint is not recorded, so it comes back as the exact pin `gem "", ""`. A transitive gem (one the manifest never declared) gets an appended block with a blank line before it; the restore removes that block, its blank line and the `DEPENDENCIES` entry, so the pair comes back byte for byte. An appended block with no blank line before it (written by a release before this one) can't be told apart from an in-place rewrite, so it still comes back as the exact pin. Refused: an ambiguous upstream section, an upstream remote other than rubygems.org. The manifest pair can't make a committed bundler cache upstream: a `-.gem` left in Bundler's cache dir that isn't the upstream archive is named by `upstream_gem_stale_cache`, never deleted. * **composer** — `composer.lock`: `dist` and the deleted `source` block from packagist's composer v2 metadata (`SOCKET_PACKAGIST_URL`). Refused unless the entry is packagist-sourced and packagist still serves the lock's `dist.reference` for the version. * **maven** — `pom.xml` (the `-socket.` version suffix, the added `` / `` entry) and the `.mvn/maven.config` / `.mvn/checksums/checksums.sha256` lines hosted mode writes: **no network**, so it restores under `--offline` too. `.mvn` files holding anything else keep the resolver lines (`maven_trusted_checksums_left`). diff --git a/crates/socket-patch-core/src/hosted/engine.rs b/crates/socket-patch-core/src/hosted/engine.rs index 09901663b..e4f186b28 100644 --- a/crates/socket-patch-core/src/hosted/engine.rs +++ b/crates/socket-patch-core/src/hosted/engine.rs @@ -1388,7 +1388,7 @@ pub fn wheel_targets<'a>( files: &BTreeMap, ) -> Vec<(&'a DepOverride, &'a str)> { use crate::utils::python_lock::{ArtifactSource, PythonLockSession}; - let mut probes: Option> = None; + let mut probes: Option> = None; let mut out = Vec::new(); for dep in candidates .iter() @@ -1413,14 +1413,13 @@ pub fn wheel_targets<'a>( .filter(|(path, _)| { *path == "uv.lock" || crate::utils::python_lock::is_script_lock_name(path) }) - .map(|(_, text)| (text.as_str(), PythonLockSession::new(text))) + .map(|(_, text)| PythonLockSession::new(text)) .collect() }) .iter() - .any(|(text, session)| { + .any(|session| { matches!( session.plan( - text, &dep.name, &dep.version, ArtifactSource::Url(&dep.artifact_url), diff --git a/crates/socket-patch-core/src/patch/redirect/mod.rs b/crates/socket-patch-core/src/patch/redirect/mod.rs index 0c778ac74..a2f323059 100644 --- a/crates/socket-patch-core/src/patch/redirect/mod.rs +++ b/crates/socket-patch-core/src/patch/redirect/mod.rs @@ -6758,9 +6758,16 @@ fn rewrite_uv_lock( .iter() .any(|(path, _)| is_pep751_lock_name(path)) .then(|| pylock_uv_settings(files)); + // The project settings' `exclude-newer`, which a `[[distribution]]` + // lock does not record (read once, as the unwind reads them). + let project_cutoff = distribution_project_cutoff(&pylock_uv_settings(files)); for (path, original) in locks { let mut content = original.clone(); let mut session = PythonLockSession::new(original); + // The cutoff a uv lock records, which the unwind refuses to undo. + let lock_doc = (!is_pep751_lock_name(path)) + .then(|| original.parse::().ok()) + .flatten(); let pipenv_reads = pipenv_reads_pylock(path, |rel| files.contains_key(rel)); // Pipenv installs the pylocks it reads, and reads no uv setting. let uv_pip_settings = pip_settings @@ -6768,7 +6775,7 @@ fn rewrite_uv_lock( .filter(|_| is_pep751_lock_name(path) && !pipenv_reads); for &(dep, sha256) in &usable { let artifact = ArtifactSource::Url(&dep.artifact_url); - let planned = session.plan(&content, &dep.name, &dep.version, artifact); + let planned = session.plan(&dep.name, &dep.version, artifact); // Every patched artifact is a wheel, which uv refuses to // install under no-binary (#1603). let no_binary = match (&planned, uv_pip_settings) { @@ -6787,6 +6794,12 @@ fn rewrite_uv_lock( }); continue; } + let lock_cutoff = lock_doc.as_ref().and_then(|doc| { + upstream::lock_cutoff( + doc, + &crate::crawlers::python_crawler::canonicalize_pypi_name(&dep.name), + ) + }); let plan = match planned { Ok(Some(_)) if pipenv_reads => { result @@ -6795,6 +6808,35 @@ fn rewrite_uv_lock( result.warnings.push(pipenv_pylock_refusal(path, dep)); continue; } + Ok(Some(_)) if lock_cutoff.is_some() => { + result + .refused_python_lock_uuids + .insert(dep.patch_uuid.clone()); + result.warnings.push(RewriteWarning { + code: "redirect_uv_lock_unsupported".into(), + detail: format!( + "{path}: the lock's `{}` option filters {}'s release files by \ + upload time, so rollback could not tell which files to restore; \ + re-lock without it to redirect this package", + lock_cutoff.unwrap_or_default(), + dep.name + ), + }); + continue; + } + Ok(Some(plan)) if plan.distribution_inline_artifacts() => match &project_cutoff { + Some(why) => { + result + .refused_python_lock_uuids + .insert(dep.patch_uuid.clone()); + result.warnings.push(RewriteWarning { + code: "redirect_uv_lock_unsupported".into(), + detail: format!("{path}: {why}"), + }); + continue; + } + None => plan, + }, Ok(Some(plan)) => plan, Ok(None) => { result.warnings.push(RewriteWarning { @@ -6882,6 +6924,24 @@ fn rewrite_uv_lock( } } +/// Why a uv 0.2 `[[distribution]]` lock with inline artifacts may not be +/// restorable once rewritten: the project's uv settings (`settings`, as +/// [`pylock_uv_settings`] finds them: `uv.toml`, else the pyproject's +/// `[tool.uv]`, else the settings file above the root, as uv and the +/// unwind read them) set an `exclude-newer` cutoff, which uv 0.2.12–0.2.27 +/// apply without recording it in the lock, so the unwind could not tell +/// which release files it kept. +fn distribution_project_cutoff(settings: &[(String, toml_edit::Table)]) -> Option { + settings.iter().find_map(|(file, table)| { + let key = crate::utils::python_lock::sets_exclude_newer(table)?; + Some(format!( + "{file} sets uv `{key}`, which uv 0.2.12–0.2.27 apply to a `[[distribution]]` lock \ + without recording it, so rollback could not tell which release files the lock \ + kept; re-lock with uv >=0.2.35" + )) + }) +} + // ── composer.lock ──────────────────────────────────────────────────────────── /// Whether `text` points at `artifact_url` in any spelling a rewritten file may /// carry: the raw url every rewriter emits — composer.lock included, since @@ -10076,6 +10136,61 @@ mod tests { } } + /// A uv lock that records an `exclude-newer` cutoff for the package + /// (globally under `[options]` or at the top level, or per package) is + /// never wired: the unwind refuses to undo such a pin, so hosted must + /// not write one. A package the per-package table exempts is wired. + #[test] + fn uv_lock_recording_a_cutoff_is_not_wired() { + const SIX: &str = "[[package]]\nname = \"six\"\nversion = \"1.16.0\"\nsource = { registry = \"https://pypi.org/simple\" }\nwheels = [{ url = \"https://files.pythonhosted.org/packages/six-1.16.0-py2.py3-none-any.whl\", hash = \"sha256:8abb2f1d86890a2dfb989f9a77cfcfd3e47c2a354b01111771326f8aa26e0254\" }]\n"; + let dep = pypi_override( + "six", + "1.16.0", + "https://patch.test/six-1.16.0-py2.py3-none-any.whl", + &"c".repeat(64), + ); + let lock = |head: &str| { + let mut files = BTreeMap::new(); + files.insert( + "uv.lock".to_string(), + format!("version = 1\nrequires-python = \">=3.9\"\n{head}\n{SIX}"), + ); + rewrite_registry_redirect(&files, std::slice::from_ref(&dep)) + }; + for head in [ + "\n[options]\nexclude-newer = \"2024-06-01T00:00:00Z\"\n", + "exclude-newer = \"2024-06-01 00:00:00 UTC\"\n", + "\n[options]\nexclude-newer-package = { six = \"2024-06-01T00:00:00Z\" }\n", + ] { + let r = lock(head); + assert!(r.files.is_empty(), "{head}: {:?}", r.files); + assert!( + r.refused_python_lock_uuids.contains(&dep.patch_uuid), + "{head}" + ); + assert!( + r.warnings + .iter() + .any(|w| w.code == "redirect_uv_lock_unsupported" + && w.detail.contains("exclude-newer")), + "{head}: {:?}", + r.warnings + ); + } + for head in [ + "", + "\n[options]\nexclude-newer = \"2024-06-01T00:00:00Z\"\nexclude-newer-package = { six = false }\n", + "\n[options]\nexclude-newer-package = { other = \"2024-06-01T00:00:00Z\" }\n", + ] { + let r = lock(head); + assert!( + r.files.get("uv.lock").is_some_and(|l| l.contains("patch.test")), + "{head}: {:?}", + r.warnings + ); + } + } + /// Re-running a rewriter over its own output must be a no-op: zero new /// edits, byte-identical files. Recorded edits whose `original` is the /// already-redirected value would grow the committed ledger on every diff --git a/crates/socket-patch-core/src/patch/redirect/upstream/mod.rs b/crates/socket-patch-core/src/patch/redirect/upstream/mod.rs index 1b1e54f55..c35e85a32 100644 --- a/crates/socket-patch-core/src/patch/redirect/upstream/mod.rs +++ b/crates/socket-patch-core/src/patch/redirect/upstream/mod.rs @@ -45,8 +45,8 @@ mod vlt; use client::rubygems_base; pub(crate) use client::UpstreamClient; pub(crate) use uv::{ - declared_lock_specifier, pylock_no_binary, respell_lock_specifier, root_uv_settings, - LockRequirementArray, + declared_lock_specifier, lock_cutoff, pylock_no_binary, respell_lock_specifier, + root_uv_settings, LockRequirementArray, }; use super::staged::{flush_staged, read_rel, Staged, StagedBytes}; diff --git a/crates/socket-patch-core/src/patch/redirect/upstream/uv.rs b/crates/socket-patch-core/src/patch/redirect/upstream/uv.rs index dd9a3e818..131ca7f54 100644 --- a/crates/socket-patch-core/src/patch/redirect/upstream/uv.rs +++ b/crates/socket-patch-core/src/patch/redirect/upstream/uv.rs @@ -52,8 +52,26 @@ //! release has none and is kept (#411); //! * the metadata's `[tool.uv.sources]. = { url }` is removed. //! -//! uv 0.2 `[[distribution]]` locks are refused (their artifact grammar -//! changed across 0.2.x releases). +//! uv 0.2 `[[distribution]]` locks (uv 0.1.45–0.2.34) come back the same +//! way, in the grammar the hosted rewrite wrote the entry in — the one the +//! entry itself carried (`utils::python_lock`'s shared detectors): its +//! source keeps its spelling (`"direct+"` back to `"registry+"`, +//! uv ≤ 0.2.17, or `{ url }` back to `{ registry }`), and its artifacts +//! their grammar (`[distribution.sdist]` / `[[distribution.wheel]]` +//! sub-tables ahead of the entry's other sub-tables, uv ≤ 0.2.5, or inline +//! `sdist` / `wheels`). Every one of those uv releases records each file of +//! a release its `requires-python` admits, whatever the platform, so a +//! pure-Python 3 release re-derives faithfully. Two things are not in the +//! lock and are refused when unknown: where uv 0.2.18–0.2.22 (before) and +//! 0.2.23–0.2.34 (after) put an `sdist` beside inline `dependencies`, read +//! off another registry entry carrying both; and an `exclude-newer` cutoff, +//! which uv 0.2.12–0.2.27 apply from the project's or user's settings or +//! `UV_EXCLUDE_NEWER` without recording it (0.2.28–0.2.33 record it at the +//! lock's top level). One passed only on the command line leaves no trace. +//! Dependents' references, inline or `[[distribution.dependencies]]` +//! tables, get the registry source back in the spelling they carry. Those +//! locks have no `[manifest]` or `[package.metadata]` for the rewrite to +//! touch. use std::collections::{BTreeMap, BTreeSet}; @@ -65,7 +83,10 @@ use super::pypi::{ }; use super::{by_uuid, read_or_refuse, refuse_all_in, Ctx, FormatResult, HostedPin, View}; use crate::crawlers::python_crawler::canonicalize_pypi_name; -use crate::utils::python_lock::{paired_metadata_path, preserve_line_endings, UvSource}; +use crate::utils::python_lock::{ + distribution_artifact_tables, distribution_sdist_before_dependencies, paired_metadata_path, + preserve_line_endings, table_likes, UvSource, +}; use crate::vendor::common::pep508_name; /// One hosted entry of a lock. @@ -78,6 +99,7 @@ struct Hit { } /// How the lock's other registry packages record their artifacts. +#[derive(Clone)] struct Shape { /// The one registry index they name. registry: String, @@ -93,8 +115,10 @@ struct Shape { datetime: bool, /// `wheels` puts one entry per line. multiline: bool, - /// pylock: artifacts are standard tables (`[[packages.wheels]]`, - /// `[packages.sdist]`), as `pip lock` writes them, not inline ones. + /// Artifacts are standard tables, not inline ones: pylock + /// `[[packages.wheels]]` / `[packages.sdist]`, as `pip lock` writes + /// them, or uv ≤ 0.2.5's `[[distribution.wheel]]` / + /// `[distribution.sdist]`. tables: bool, /// With `tables`: `hashes` is a sub-table (`[packages.wheels.hashes]`). hash_tables: bool, @@ -323,13 +347,6 @@ async fn restore_lock( result.refuse(&hit.uuid, format!("{rel}: {why}")); } }; - if collection == "distribution" { - return refuse_hits( - result, - "uv 0.2 `[[distribution]]` locks changed their artifact grammar across releases; \ - which one this lock follows is not derivable", - ); - } let version_ok = if pep751 { doc.get("lock-version").and_then(Item::as_str) == Some("1.0") } else { @@ -338,23 +355,8 @@ async fn restore_lock( if !version_ok { return refuse_hits(result, "unsupported lock version"); } - let options = doc.get("options").and_then(Item::as_table_like); - let global_cutoff = options.and_then(|o| { - o.iter() - .map(|(k, _)| k) - .find(|k| k.starts_with("exclude-newer") && *k != "exclude-newer-package") - }); - // A per-package cutoff (#1604) filters only the packages it names, and - // one of `false` exempts its package from the global cutoff. - let cutoffs = options.and_then(|o| o.get("exclude-newer-package")); for hit in &hits { - let key = if cutoffs.is_some_and(|c| names_package(c, &hit.name)) { - "exclude-newer-package" - } else if let Some(key) = - global_cutoff.filter(|_| !cutoffs.is_some_and(|c| exempts(c, &hit.name))) - { - key - } else { + let Some(key) = lock_cutoff(&doc, &hit.name) else { continue; }; result.refuse( @@ -369,6 +371,40 @@ async fn restore_lock( Ok(shape) => shape, Err(why) => return refuse_hits(result, &why), }; + let packages = doc + .get(collection) + .and_then(Item::as_array_of_tables) + .expect("lock_shape found the package array"); + // uv 0.2 `[[distribution]]` locks: each hit's artifact grammar (the + // one the rewrite wrote it in), and where this uv writes an `sdist` + // beside inline `dependencies`. + let legacy: BTreeMap = if collection == "distribution" { + let skip: BTreeSet = hits.iter().map(|h| h.index).collect(); + let sdist_before_dependencies = + distribution_sdist_before_dependencies(packages, |i| skip.contains(&i)); + hits.iter() + .map(|h| { + let legacy = Legacy { + tables: distribution_artifact_tables(packages, h.index), + sdist_before_dependencies, + }; + (h.uuid.clone(), legacy) + }) + .collect() + } else { + BTreeMap::new() + }; + let distribution_inline = legacy.values().any(|l| !l.tables); + if distribution_inline { + // uv 0.2.12–0.2.27 apply an `exclude-newer` cutoff from the + // project's or the user's settings, or the environment, without + // recording it in the lock (uv ≤ 0.2.5, the sub-table grammar, + // apply none). Those are the best evidence of the cutoff the lock + // was made under; one passed only on the command line leaves none. + if let Err(why) = distribution_cutoff(view, rel).await { + return refuse_hits(result, &why); + } + } if pep751 && pip_compiled(&doc, &text, &shape) { // Without uv's command line (`--no-header`, or a header replaced by // `UV_CUSTOM_COMPILE_COMMAND`) its filter flags are unknowable. @@ -489,6 +525,14 @@ async fn restore_lock( let Some(release) = released.get(&(hit.name.clone(), hit.version.clone())) else { continue; }; + let legacy = legacy.get(&hit.uuid).copied(); + let shape = match legacy { + Some(legacy) => Shape { + tables: legacy.tables, + ..shape.clone() + }, + None => shape.clone(), + }; let artifacts = match render_artifacts(release, &shape) { Ok(artifacts) => artifacts, Err(why) => { @@ -505,7 +549,7 @@ async fn restore_lock( let restore = if pep751 { restore_pylock_entry(&mut doc, hit, &shape, artifacts) } else { - restore_uv_entry(&mut doc, hit, &shape, artifacts, ctx) + restore_uv_entry(&mut doc, collection, legacy, hit, &shape, artifacts, ctx) .and_then(|()| restore_requirements(&mut doc, hit, metadata.as_ref(), &styles, ctx)) }; if let Err(why) = restore { @@ -615,23 +659,12 @@ impl SiblingRegistry<'_> { } } -/// A package's `wheels` and `sdist` artifact tables, in either PEP 751 -/// spelling: inline (`wheels = [{ … }]`, as uv writes them) or standard -/// tables (`[[packages.wheels]]` / `[packages.sdist]`, as `pip lock` does). +/// A package's `wheels` and `sdist` artifact tables, in either spelling: +/// inline (`wheels = [{ … }]`, as uv writes them) or standard tables +/// (`[[packages.wheels]]` / `[packages.sdist]`, as `pip lock` does, and uv +/// ≤ 0.2.5's `[[distribution.wheel]]` / `[distribution.sdist]`). fn artifact_tables(package: &toml_edit::Table) -> Vec<&dyn TableLike> { - let mut tables: Vec<&dyn TableLike> = Vec::new(); - match package.get("wheels") { - Some(Item::Value(Value::Array(wheels))) => tables.extend( - wheels - .iter() - .filter_map(Value::as_inline_table) - .map(|t| t as &dyn TableLike), - ), - Some(Item::ArrayOfTables(wheels)) => { - tables.extend(wheels.iter().map(|t| t as &dyn TableLike)) - } - _ => {} - } + let mut tables = table_likes(package.get("wheels").or_else(|| package.get("wheel"))); tables.extend(package.get("sdist").and_then(Item::as_table_like)); tables } @@ -685,6 +718,29 @@ fn env_set(name: &str) -> bool { std::env::var_os(name).is_some_and(|v| !v.is_empty()) } +/// `Err(why)` when a uv setting sets the `exclude-newer` cutoff that uv +/// 0.2.12–0.2.27 apply to the `[[distribution]]` lock `rel` without +/// recording it: the project's (`uv.toml`, else a pyproject's `[tool.uv]`, +/// in the lock's directory or above), the user's or the system's +/// `uv.toml`, or `UV_EXCLUDE_NEWER`. +async fn distribution_cutoff(view: &mut View<'_>, rel: &str) -> Result<(), String> { + let consequence = "which uv 0.2.12–0.2.27 apply to a `[[distribution]]` lock without \ + recording it, so which release files the lock kept is not derivable"; + if std::env::var_os("UV_EXCLUDE_NEWER").is_some_and(|v| !v.is_empty() && v != "false") { + return Err(format!( + "`UV_EXCLUDE_NEWER` is set in this environment, {consequence}" + )); + } + let project = pip_settings(view, rel).await?; + let global = global_uv_settings(&|name| std::env::var_os(name), cfg!(windows)).await?; + for (path, settings) in project.iter().chain(&global) { + if let Some(key) = crate::utils::python_lock::sets_exclude_newer(settings) { + return Err(format!("{path} sets uv `{key}`, {consequence}")); + } + } + Ok(()) +} + /// Whether a header command line is `uv pip compile`'s. uv records global /// options where they were given, before `pip` or between it and /// `compile` (`uv --no-config pip compile …`). @@ -881,6 +937,26 @@ fn names_package(item: &Item, name: &str) -> bool { } } +/// The `exclude-newer` option uv recorded in lock `doc` that filters +/// canonical package `name`'s release files by upload time, if any: a +/// per-package `exclude-newer-package` naming it (#1604), else a global +/// cutoff it is not exempted from. uv 0.2.28–0.2.33 record the global +/// cutoff at the lock's top level, later releases under `[options]`. The +/// unwind cannot tell which files such a cutoff kept, so it refuses the +/// pin, and the hosted rewrite refuses to write one it could not unwind. +pub(crate) fn lock_cutoff<'a>(doc: &'a DocumentMut, name: &str) -> Option<&'a str> { + let options = doc.get("options").and_then(Item::as_table_like); + let global = options + .into_iter() + .chain(Some(doc.as_table() as &dyn TableLike)) + .find_map(crate::utils::python_lock::sets_exclude_newer); + let cutoffs = options.and_then(|o| o.get("exclude-newer-package")); + if cutoffs.is_some_and(|c| names_package(c, name)) { + return Some("exclude-newer-package"); + } + global.filter(|_| !cutoffs.is_some_and(|c| exempts(c, name))) +} + /// Whether a per-package cutoff table exempts `name` from the global /// cutoff (an entry of `false`). fn exempts(cutoffs: &Item, name: &str) -> bool { @@ -1194,7 +1270,7 @@ fn lock_shape( .get("wheels") .and_then(Item::as_array) .is_none_or(|a| a.to_string().contains('\n')); - let tables = match package.get("wheels") { + let tables = match package.get("wheels").or_else(|| package.get("wheel")) { Some(Item::ArrayOfTables(wheels)) => !wheels.is_empty(), Some(Item::Value(Value::Array(wheels))) if !wheels.is_empty() => false, _ => package.get("sdist").is_some_and(Item::is_table), @@ -1423,63 +1499,175 @@ fn inline_source(registry: &str) -> Value { .expect("a registry source renders as TOML") } +/// Point `source` back at `registry`, in its own spelling, when it routes +/// `hit` to its hosted artifact: `{ url }` (or a `[….source]` table) +/// becomes `{ registry }`, and uv 0.2's string `"direct+"` becomes +/// `"registry+"`. `false` when it names something else. +fn restore_source(source: &mut Item, hit: &Hit, registry: &str, ctx: &Ctx<'_>) -> bool { + if let Item::Value(value) = source { + return restore_source_value(value, hit, registry, ctx); + } + let names_hit = UvSource::of_item(source) + .and_then(UvSource::url) + .is_some_and(|url| names_hosted(url, hit, ctx)); + let Some(table) = source.as_table_mut().filter(|_| names_hit) else { + return false; + }; + table.clear(); + table.insert("registry", toml_edit::value(registry)); + true +} + +/// [`restore_source`] of a `source` value (a string or an inline table). +fn restore_source_value(source: &mut Value, hit: &Hit, registry: &str, ctx: &Ctx<'_>) -> bool { + let Some(parsed) = UvSource::of_value(source) else { + return false; + }; + if !parsed.url().is_some_and(|url| names_hosted(url, hit, ctx)) { + return false; + } + let restored = match parsed { + UvSource::Str(_) => Value::from(format!("registry+{registry}")), + UvSource::Table(_) => inline_source(registry), + }; + replace_keeping_decor(source, restored); + true +} + +/// Whether `url` is `hit`'s hosted artifact. +fn names_hosted(url: &str, hit: &Hit, ctx: &Ctx<'_>) -> bool { + ctx.hosted_uuid(url).is_some_and(|u| u == hit.uuid) +} + /// Replace `item`'s value, keeping its decor. fn replace_keeping_decor(slot: &mut Value, mut value: Value) { *value.decor_mut() = slot.decor().clone(); *slot = value; } +/// How a uv 0.2 `[[distribution]]` hit is laid out (see +/// [`distribution_artifact_tables`] and +/// [`distribution_sdist_before_dependencies`]). +#[derive(Clone, Copy)] +struct Legacy { + tables: bool, + sdist_before_dependencies: Option, +} + /// The package entry itself, and every dependent reference to it. fn restore_uv_entry( doc: &mut DocumentMut, + collection: &str, + legacy: Option, hit: &Hit, shape: &Shape, (sdist, wheels): (Option, Option), ctx: &Ctx<'_>, ) -> Result<(), String> { let package = doc - .get_mut("package") + .get_mut(collection) .and_then(Item::as_array_of_tables_mut) .and_then(|p| p.get_mut(hit.index)) .ok_or("the package entry vanished")?; - match package.get_mut("source").and_then(Item::as_value_mut) { - Some(slot) => replace_keeping_decor(slot, inline_source(&shape.registry)), - None => return Err("the package source is not an inline table".to_string()), - } + let source = package + .get_mut("source") + .ok_or("the package entry has no `source`")?; + if !restore_source(source, hit, &shape.registry, ctx) { + return Err("the package `source` no longer names the hosted artifact".to_string()); + } + // uv 0.2.18–0.2.22 write `sdist` before inline `dependencies`, + // 0.2.23–0.2.34 after them. + let inline_dependencies = package + .get("dependencies") + .is_some_and(|d| d.as_array().is_some()); + let sdist_before_dependencies = match legacy { + Some(legacy) if sdist.is_some() && inline_dependencies => { + legacy.sdist_before_dependencies.ok_or( + "no other registry package of the lock carries both an `sdist` and inline \ + `dependencies`, which uv 0.2.18–0.2.22 and 0.2.23–0.2.34 order differently, so \ + where the entry's `sdist` goes is not derivable", + )? + } + _ => false, + }; for key in ["sdist", "wheels", "wheel", "archive"] { package.remove(key); } if let Some(sdist) = sdist { package.insert("sdist", sdist); } + let tables = legacy.is_some_and(|l| l.tables); if let Some(wheels) = wheels { - package.insert("wheels", wheels); + // uv ≤ 0.2.5 names its wheel sub-tables `[[distribution.wheel]]`. + package.insert(if tables { "wheel" } else { "wheels" }, wheels); + } + if tables { + // uv ≤ 0.2.5 write an entry's `[distribution.sdist]` and + // `[[distribution.wheel]]` before its `[[distribution.dependencies]]` + // (and every other sub-table), and a new sub-table renders where + // its key sits. + let rank = |key: &str| match key { + "sdist" => 0, + "wheel" => 1, + _ => 2, + }; + package.sort_values_by(|a, _, b, _| rank(a.get()).cmp(&rank(b.get()))); + } else if sdist_before_dependencies { + move_before(package, "sdist", "dependencies"); } package.remove("metadata"); restore_refs(doc.as_item_mut(), hit, &shape.registry, ctx); Ok(()) } -/// Dependents' `{ name, source = { url } }` references back to the registry. +/// Move `key` right before `anchor` in `table`'s key order. +fn move_before(table: &mut toml_edit::Table, key: &str, anchor: &str) { + let order: Vec = table + .iter() + .map(|(k, _)| k.to_string()) + .filter(|k| k != key) + .flat_map(|k| { + if k == anchor { + vec![key.to_string(), k] + } else { + vec![k] + } + }) + .collect(); + let rank = |k: &str| order.iter().position(|o| o == k); + table.sort_values_by(|a, _, b, _| rank(a.get()).cmp(&rank(b.get()))); +} + +/// Dependents' references back to the registry: inline `{ name, source }` +/// tables, and uv 0.2's `[[distribution.dependencies]]` tables. fn restore_refs(item: &mut Item, hit: &Hit, registry: &str, ctx: &Ctx<'_>) { match item { Item::Value(value) => restore_ref_value(value, hit, registry, ctx), - Item::Table(table) => { - for (_, item) in table.iter_mut() { - restore_refs(item, hit, registry, ctx); - } - } + Item::Table(table) => restore_ref_table(table, hit, registry, ctx), Item::ArrayOfTables(tables) => { for table in tables.iter_mut() { - for (_, item) in table.iter_mut() { - restore_refs(item, hit, registry, ctx); - } + restore_ref_table(table, hit, registry, ctx); } } Item::None => {} } } +fn restore_ref_table(table: &mut toml_edit::Table, hit: &Hit, registry: &str, ctx: &Ctx<'_>) { + let names_hit = table + .get("name") + .and_then(Item::as_str) + .is_some_and(|n| canonicalize_pypi_name(n) == hit.name); + if names_hit { + if let Some(source) = table.get_mut("source") { + restore_source(source, hit, registry, ctx); + } + } + for (_, item) in table.iter_mut() { + restore_refs(item, hit, registry, ctx); + } +} + fn restore_ref_value(value: &mut Value, hit: &Hit, registry: &str, ctx: &Ctx<'_>) { match value { Value::Array(array) => { @@ -1492,16 +1680,9 @@ fn restore_ref_value(value: &mut Value, hit: &Hit, registry: &str, ctx: &Ctx<'_> .get("name") .and_then(Value::as_str) .is_some_and(|n| canonicalize_pypi_name(n) == hit.name); - let hosted = table - .get("source") - .and_then(Value::as_inline_table) - .and_then(|s| s.get("url")) - .and_then(Value::as_str) - .and_then(|u| ctx.hosted_uuid(u)) - .is_some_and(|u| u == hit.uuid); - if names_hit && hosted { - if let Some(slot) = table.get_mut("source") { - replace_keeping_decor(slot, inline_source(registry)); + if names_hit { + if let Some(source) = table.get_mut("source") { + restore_source_value(source, hit, registry, ctx); } } for (_, value) in table.iter_mut() { diff --git a/crates/socket-patch-core/src/utils/python_lock.rs b/crates/socket-patch-core/src/utils/python_lock.rs index 626245e7c..35d8cc86c 100644 --- a/crates/socket-patch-core/src/utils/python_lock.rs +++ b/crates/socket-patch-core/src/utils/python_lock.rs @@ -123,13 +123,26 @@ impl<'t> UvSource<'t> { /// `package`'s `source`; `None` when absent or neither a string nor a /// table. pub(crate) fn of(package: &'t dyn TableLike) -> Option { - let source = package.get("source")?; + Self::of_item(package.get("source")?) + } + + /// A `source` item: a string, or an inline or standard table. + pub(crate) fn of_item(source: &'t Item) -> Option { match source.as_str() { Some(s) => Some(Self::Str(s)), None => source.as_table_like().map(Self::Table), } } + /// A `source` value: a string or an inline table. + pub(crate) fn of_value(source: &'t Value) -> Option { + match source { + Value::String(s) => Some(Self::Str(s.value())), + Value::InlineTable(t) => Some(Self::Table(t)), + _ => None, + } + } + /// The `+` value of the string form, or the table's string `key`. fn field(self, prefix: &str, key: &str) -> Option<&'t str> { match self { @@ -164,6 +177,73 @@ impl<'t> UvSource<'t> { } } +/// Whether the uv 0.2 `[[distribution]]` entry `index` of `packages` keeps +/// its artifacts in uv ≤ 0.2.5's `[distribution.sdist]` / +/// `[[distribution.wheel]]` sub-tables rather than inline `sdist` / +/// `wheels` (uv 0.2.6 and later). The entry's own artifacts decide — on a +/// hosted pin, the ones the rewrite wrote in the grammar it chose — and an +/// entry with none takes any other entry's. The hosted rewrite and the +/// hosted unwind both read the grammar here. +/// +/// uv 0.2.17 parses an unexpected `[[distribution.wheel]]` but ignores it, +/// then treats the direct wheel URL as a source archive ("Unsupported +/// archive type: …whl"), so the shape must be the lock's own. +pub(crate) fn distribution_artifact_tables(packages: &ArrayOfTables, index: usize) -> bool { + let tables = |p: &Table| { + p.get("wheel").is_some_and(Item::is_array_of_tables) + || p.get("sdist").is_some_and(Item::is_table) + }; + let has_artifacts = |p: &Table| { + ["wheel", "wheels", "sdist"] + .iter() + .any(|k| p.contains_key(k)) + }; + match packages.get(index) { + Some(own) if has_artifacts(own) => tables(own), + _ => packages.iter().any(tables), + } +} + +/// Where the uv 0.2 `[[distribution]]` grammar writes an entry's inline +/// `sdist` relative to an inline `dependencies` array, as the lock's +/// registry entries other than `skip` show it: `Some(true)` before it +/// (uv 0.2.18–0.2.22), `Some(false)` after it (uv 0.2.23–0.2.34), `None` +/// when no such entry carries both. Nothing else in the lock tells those +/// releases apart, and a hosted entry has lost its `sdist`, so the hosted +/// unwind can restore one with both only from this evidence (and the +/// hosted rewrite refuses an entry whose unwind would lack it). +pub(crate) fn distribution_sdist_before_dependencies( + packages: &ArrayOfTables, + skip: impl Fn(usize) -> bool, +) -> Option { + packages.iter().enumerate().find_map(|(i, p)| { + if skip(i) || UvSource::of(p).and_then(UvSource::registry).is_none() { + return None; + } + let at = |key: &str| p.iter().position(|(k, _)| k == key); + let inline_dependencies = p + .get("dependencies") + .is_some_and(|d| d.as_array().is_some()); + match (at("sdist"), at("dependencies")) { + (Some(sdist), Some(dependencies)) if inline_dependencies => Some(sdist < dependencies), + _ => None, + } + }) +} + +/// The `exclude-newer` cutoff a uv project setting sets (`[tool.uv]` of a +/// `pyproject.toml`, or the top level of a `uv.toml`): uv 0.2.12–0.2.27 +/// apply it when they lock without recording it in a `[[distribution]]` +/// lock, so which of a release's files that lock kept is not derivable. +/// `exclude-newer-package` is not such a cutoff: those uv releases predate +/// it and ignore the key, and a later uv records it in the lock. +pub(crate) fn sets_exclude_newer(settings: &dyn TableLike) -> Option<&str> { + settings + .iter() + .map(|(k, _)| k) + .find(|k| k.starts_with("exclude-newer") && *k != "exclude-newer-package") +} + /// A uv package's install location: `source = { url | path }`, or the uv /// 0.2.x string grammar `direct+` / `path+`. Registry, git, /// editable, virtual and directory sources yield `None`. @@ -794,7 +874,6 @@ struct PythonLockPlan { fn plan_python_lock_rewrite( document: &DocumentMut, - text: &str, name: &str, version: &str, artifact: ArtifactSource<'_>, @@ -851,18 +930,7 @@ fn plan_python_lock_rewrite( // still write string sources but already use inline `sdist = { … }` / // `wheels = [ … ]` instead of the `[distribution.sdist]` / // `[[distribution.wheel]]` tables of uv ≤0.2.5. - // uv 0.2.17 parses an unexpected `[[distribution.wheel]]` but ignores it, - // then treats the direct wheel URL as a source archive ("Unsupported - // archive type: …whl"). Decide from the entry's own artifact keys, then - // from any sibling entry in the document. - let legacy_artifact_tables = legacy - && (package.get("wheel").is_some_and(Item::is_array_of_tables) - || package.get("sdist").is_some_and(Item::is_table) - || (package.get("wheel").is_none() - && package.get("wheels").is_none() - && package.get("sdist").is_none() - && (text.contains("[[distribution.wheel]]") - || text.contains("[distribution.sdist]")))); + let legacy_artifact_tables = legacy && distribution_artifact_tables(packages, index); if !pep751 && !original_source.as_ref().is_some_and(|source| { source_identity(source).is_some_and(|(kind, value)| { @@ -878,6 +946,24 @@ fn plan_python_lock_rewrite( if pep751 && (package.contains_key("vcs") || package.contains_key("directory")) { return Ok(None); } + // uv 0.2.18–0.2.22 write an entry's `sdist` before its inline + // `dependencies`, 0.2.23–0.2.34 after them. The rewrite drops the sdist, + // so the unwind can put it back only where another registry entry shows + // it; refuse an entry it could not restore. + if legacy + && package.contains_key("sdist") + && package + .get("dependencies") + .is_some_and(|d| d.as_array().is_some()) + && distribution_sdist_before_dependencies(packages, |i| i == index).is_none() + { + return Err(format!( + "no other registry package of this uv 0.2 `[[distribution]]` lock carries both an \ + `sdist` and inline `dependencies`, which uv 0.2.18–0.2.22 and 0.2.23–0.2.34 order \ + differently, so rollback could not restore {name}=={version}'s entry; re-lock \ + with uv >=0.2.35" + )); + } // An entry resolved from a direct URL (uv.lock `source = { url }`, a // pylock `archive = { url }`) is the user's own source choice unless it // is socket-patch's earlier hosted artifact for this release. @@ -952,6 +1038,15 @@ pub(crate) struct PythonLockSession { /// A dep [`PythonLockSession::plan`] settled as rewritable. pub(crate) struct PlannedPythonLockRewrite(PythonLockPlan); +impl PlannedPythonLockRewrite { + /// The entry is in a uv 0.2 `[[distribution]]` lock with inline + /// artifacts (uv 0.2.6–0.2.34), the releases that can apply an + /// `exclude-newer` cutoff (see [`sets_exclude_newer`]). + pub(crate) fn distribution_inline_artifacts(&self) -> bool { + self.0.legacy && !self.0.legacy_artifact_tables + } +} + impl PythonLockSession { pub(crate) fn new(text: &str) -> Self { Self { @@ -961,20 +1056,17 @@ impl PythonLockSession { } } - /// [`rewrite_python_lock`]'s refusal / not-applicable verdict, where - /// `text` is the session's current rendering. + /// [`rewrite_python_lock`]'s refusal / not-applicable verdict on the + /// session's current document. pub(crate) fn plan( &self, - text: &str, name: &str, version: &str, artifact: ArtifactSource<'_>, ) -> Result, String> { let document = self.document.as_ref().map_err(String::clone)?; - Ok( - plan_python_lock_rewrite(document, text, name, version, artifact)? - .map(PlannedPythonLockRewrite), - ) + Ok(plan_python_lock_rewrite(document, name, version, artifact)? + .map(PlannedPythonLockRewrite)) } /// [`check_python_lock_source_scope`] on the current rendering. @@ -1032,7 +1124,7 @@ pub fn rewrite_python_lock( let mut document: DocumentMut = text .parse() .map_err(|error| format!("invalid Python lock: {error}"))?; - let Some(plan) = plan_python_lock_rewrite(&document, text, name, version, artifact)? else { + let Some(plan) = plan_python_lock_rewrite(&document, name, version, artifact)? else { return Ok(None); }; apply_python_lock_plan(&mut document, plan, version, artifact, sha256); @@ -1174,6 +1266,37 @@ pub fn paired_metadata_path(lock: &str) -> Option { mod tests { use super::*; + /// An entry the rewrite does not apply to (a non-registry, non-direct + /// string source) is not applicable, not refused: the sdist-order + /// refusal only judges entries the rewrite would change. + #[test] + fn a_not_applicable_distribution_entry_is_not_refused_for_its_sdist() { + let lock = "version = 1\nrequires-python = \">=3.9\"\n\n[[distribution]]\nname = \"urllib3\"\nversion = \"1.26.18\"\nsource = \"git+https://github.com/urllib3/urllib3?rev=1.26.18#abc\"\nsdist = { url = \"https://example.test/urllib3-1.26.18.tar.gz\", hash = \"sha256:aa\" }\ndependencies = [{ name = \"idna\" }]\n"; + let session = PythonLockSession::new(lock); + let planned = session.plan( + "urllib3", + "1.26.18", + ArtifactSource::Url("https://patch.test/urllib3-1.26.18-py3-none-any.whl"), + ); + assert!(matches!(planned, Ok(None)), "{:?}", planned.err()); + } + + /// `exclude-newer-package` names packages (or exempts them with `false`); + /// uv 0.2 ignores it, so it is not the global cutoff that blocks a + /// `[[distribution]]` rewrite. + #[test] + fn exclude_newer_package_is_not_a_global_cutoff() { + let table = |text: &str| text.parse::().unwrap(); + let per_package = table("exclude-newer-package = { six = \"2024-01-01\" }\n"); + assert_eq!(sets_exclude_newer(per_package.as_table()), None); + let exempt = table("exclude-newer-package = { six = false }\n"); + assert_eq!(sets_exclude_newer(exempt.as_table()), None); + let global = table( + "exclude-newer-package = { six = false }\nexclude-newer = \"2024-01-01T00:00:00Z\"\n", + ); + assert_eq!(sets_exclude_newer(global.as_table()), Some("exclude-newer")); + } + #[test] fn script_locks_pair_with_their_script_and_uv_lock_with_pyproject() { assert!(is_script_lock_name("tool.py.lock")); @@ -1343,6 +1466,55 @@ wheels = [{ url = "https://pypi.org/urllib3-2.0.0-py3-none-any.whl", hash = "sha assert_eq!(package["sdist"]["url"].as_str(), Some(url)); } + /// The grammar detectors both hosted directions share: an entry's own + /// artifacts decide its grammar before any other entry's, and the + /// `sdist` / inline `dependencies` order comes only from registry + /// entries other than the skipped ones. + #[test] + fn distribution_grammar_reads_the_entry_before_its_siblings() { + let doc: DocumentMut = r#"version = 1 +[[distribution]] +name = "inline" +version = "1" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/inline-1.tar.gz" } +dependencies = [{ name = "bare" }] +[[distribution]] +name = "tables" +version = "1" +source = "registry+https://pypi.org/simple" +[[distribution.wheel]] +url = "https://files.pythonhosted.org/tables-1-py3-none-any.whl" +[[distribution]] +name = "bare" +version = "1" +source = "registry+https://pypi.org/simple" +[[distribution]] +name = "hosted" +version = "1" +source = { url = "https://patch.socket.dev/hosted-1-py3-none-any.whl" } +dependencies = [{ name = "bare" }] +sdist = { url = "https://patch.socket.dev/hosted-1.tar.gz" } +"# + .parse() + .unwrap(); + let packages = doc["distribution"].as_array_of_tables().unwrap(); + assert!(!distribution_artifact_tables(packages, 0)); + assert!(distribution_artifact_tables(packages, 1)); + // No artifacts of its own: any entry's sub-tables decide. + assert!(distribution_artifact_tables(packages, 2)); + // `inline` shows `sdist` before `dependencies`; `hosted` (not a + // registry entry) shows the reverse and is never evidence. + assert_eq!( + distribution_sdist_before_dependencies(packages, |_| false), + Some(true) + ); + assert_eq!( + distribution_sdist_before_dependencies(packages, |i| i == 0), + None + ); + } + #[test] fn legacy_distribution_updates_source_qualified_edges() { let text = r#"version = 1 diff --git a/crates/socket-patch-core/tests/fixtures/upstream/uv-0.2/0.1.45/pyproject.toml b/crates/socket-patch-core/tests/fixtures/upstream/uv-0.2/0.1.45/pyproject.toml new file mode 100644 index 000000000..189fa288e --- /dev/null +++ b/crates/socket-patch-core/tests/fixtures/upstream/uv-0.2/0.1.45/pyproject.toml @@ -0,0 +1,5 @@ +[project] +name = "proj" +version = "0.1.0" +requires-python = ">=3.9" +dependencies = ["urllib3==1.26.18", "python-dateutil==2.9.0.post0", "isodate==0.6.1"] diff --git a/crates/socket-patch-core/tests/fixtures/upstream/uv-0.2/0.1.45/uv.lock b/crates/socket-patch-core/tests/fixtures/upstream/uv-0.2/0.1.45/uv.lock new file mode 100644 index 000000000..5c329a6e5 --- /dev/null +++ b/crates/socket-patch-core/tests/fixtures/upstream/uv-0.2/0.1.45/uv.lock @@ -0,0 +1,94 @@ +version = 1 + +[[distribution]] +name = "isodate" +version = "0.6.1" +source = "registry+https://pypi.org/simple" + +[distribution.sdist] +url = "https://files.pythonhosted.org/packages/db/7a/c0a56c7d56c7fa723988f122fa1f1ccf8c5c4ccc48efad0d214b49e5b1af/isodate-0.6.1.tar.gz" +hash = "sha256:48c5881de7e8b0a0d648cb024c8062dc84e7b840ed81e864c7614fd3c127bde9" +size = 28443 + +[[distribution.wheel]] +url = "https://files.pythonhosted.org/packages/b6/85/7882d311924cbcfc70b1890780763e36ff0b140c7e51c110fc59a532f087/isodate-0.6.1-py2.py3-none-any.whl" +hash = "sha256:0751eece944162659049d35f4f549ed815792b38793f07cf73381c1c87cbed96" +size = 41722 + +[[distribution.dependencies]] +name = "six" +version = "1.17.0" +source = "registry+https://pypi.org/simple" + +[[distribution]] +name = "proj" +version = "0.1.0" +source = "directory+file:///project" + +[distribution.sdist] +url = "file:///project" + +[[distribution.dependencies]] +name = "isodate" +version = "0.6.1" +source = "registry+https://pypi.org/simple" + +[[distribution.dependencies]] +name = "python-dateutil" +version = "2.9.0.post0" +source = "registry+https://pypi.org/simple" + +[[distribution.dependencies]] +name = "urllib3" +version = "1.26.18" +source = "registry+https://pypi.org/simple" + +[[distribution]] +name = "python-dateutil" +version = "2.9.0.post0" +source = "registry+https://pypi.org/simple" + +[distribution.sdist] +url = "https://files.pythonhosted.org/packages/66/c0/0c8b6ad9f17a802ee498c46e004a0eb49bc148f2fd230864601a86dcf6db/python-dateutil-2.9.0.post0.tar.gz" +hash = "sha256:37dd54208da7e1cd875388217d5e00ebd4179249f90fb72437e91a35459a0ad3" +size = 342432 + +[[distribution.wheel]] +url = "https://files.pythonhosted.org/packages/ec/57/56b9bcc3c9c6a792fcbaf139543cee77261f3651ca9da0c93f5c1221264b/python_dateutil-2.9.0.post0-py2.py3-none-any.whl" +hash = "sha256:a8b2bc7bffae282281c8140a97d3aa9c14da0b136dfe83f850eea9a5f7470427" +size = 229892 + +[[distribution.dependencies]] +name = "six" +version = "1.17.0" +source = "registry+https://pypi.org/simple" + +[[distribution]] +name = "six" +version = "1.17.0" +source = "registry+https://pypi.org/simple" + +[distribution.sdist] +url = "https://files.pythonhosted.org/packages/94/e7/b2c673351809dca68a0e064b6af791aa332cf192da575fd474ed7d6f16a2/six-1.17.0.tar.gz" +hash = "sha256:ff70335d468e7eb6ec65b95b99d3a2836546063f63acc5171de367e834932a81" +size = 34031 + +[[distribution.wheel]] +url = "https://files.pythonhosted.org/packages/b7/ce/149a00dd41f10bc29e5921b496af8b574d8413afcd5e30dfa0ed46c2cc5e/six-1.17.0-py2.py3-none-any.whl" +hash = "sha256:4721f391ed90541fddacab5acf947aa0d3dc7d27b2e1e8eda2be8970586c3274" +size = 11050 + +[[distribution]] +name = "urllib3" +version = "1.26.18" +source = "registry+https://pypi.org/simple" + +[distribution.sdist] +url = "https://files.pythonhosted.org/packages/0c/39/64487bf07df2ed854cc06078c27c0d0abc59bd27b32232876e403c333a08/urllib3-1.26.18.tar.gz" +hash = "sha256:f8ecc1bba5667413457c529ab955bf8c67b45db799d159066261719e328580a0" +size = 305687 + +[[distribution.wheel]] +url = "https://files.pythonhosted.org/packages/b0/53/aa91e163dcfd1e5b82d8a890ecf13314e3e149c05270cc644581f77f17fd/urllib3-1.26.18-py2.py3-none-any.whl" +hash = "sha256:34b97092d7e0a3a8cf7cd10e386f401b3737364026c45e622aa02903dffe0f07" +size = 143835 diff --git a/crates/socket-patch-core/tests/fixtures/upstream/uv-0.2/0.2.15/pyproject.toml b/crates/socket-patch-core/tests/fixtures/upstream/uv-0.2/0.2.15/pyproject.toml new file mode 100644 index 000000000..189fa288e --- /dev/null +++ b/crates/socket-patch-core/tests/fixtures/upstream/uv-0.2/0.2.15/pyproject.toml @@ -0,0 +1,5 @@ +[project] +name = "proj" +version = "0.1.0" +requires-python = ">=3.9" +dependencies = ["urllib3==1.26.18", "python-dateutil==2.9.0.post0", "isodate==0.6.1"] diff --git a/crates/socket-patch-core/tests/fixtures/upstream/uv-0.2/0.2.15/uv.lock b/crates/socket-patch-core/tests/fixtures/upstream/uv-0.2/0.2.15/uv.lock new file mode 100644 index 000000000..2330dfb5d --- /dev/null +++ b/crates/socket-patch-core/tests/fixtures/upstream/uv-0.2/0.2.15/uv.lock @@ -0,0 +1,61 @@ +version = 1 +requires-python = ">=3.9" + +[[distribution]] +name = "isodate" +version = "0.6.1" +source = "registry+https://pypi.org/simple" +sdist = { url = "https://files.pythonhosted.org/packages/db/7a/c0a56c7d56c7fa723988f122fa1f1ccf8c5c4ccc48efad0d214b49e5b1af/isodate-0.6.1.tar.gz", hash = "sha256:48c5881de7e8b0a0d648cb024c8062dc84e7b840ed81e864c7614fd3c127bde9", size = 28443 } +wheels = [{ url = "https://files.pythonhosted.org/packages/b6/85/7882d311924cbcfc70b1890780763e36ff0b140c7e51c110fc59a532f087/isodate-0.6.1-py2.py3-none-any.whl", hash = "sha256:0751eece944162659049d35f4f549ed815792b38793f07cf73381c1c87cbed96", size = 41722 }] + +[[distribution.dependencies]] +name = "six" +version = "1.17.0" +source = "registry+https://pypi.org/simple" + +[[distribution]] +name = "proj" +version = "0.1.0" +source = "editable+." +sdist = { path = "." } + +[[distribution.dependencies]] +name = "isodate" +version = "0.6.1" +source = "registry+https://pypi.org/simple" + +[[distribution.dependencies]] +name = "python-dateutil" +version = "2.9.0.post0" +source = "registry+https://pypi.org/simple" + +[[distribution.dependencies]] +name = "urllib3" +version = "1.26.18" +source = "registry+https://pypi.org/simple" + +[[distribution]] +name = "python-dateutil" +version = "2.9.0.post0" +source = "registry+https://pypi.org/simple" +sdist = { url = "https://files.pythonhosted.org/packages/66/c0/0c8b6ad9f17a802ee498c46e004a0eb49bc148f2fd230864601a86dcf6db/python-dateutil-2.9.0.post0.tar.gz", hash = "sha256:37dd54208da7e1cd875388217d5e00ebd4179249f90fb72437e91a35459a0ad3", size = 342432 } +wheels = [{ url = "https://files.pythonhosted.org/packages/ec/57/56b9bcc3c9c6a792fcbaf139543cee77261f3651ca9da0c93f5c1221264b/python_dateutil-2.9.0.post0-py2.py3-none-any.whl", hash = "sha256:a8b2bc7bffae282281c8140a97d3aa9c14da0b136dfe83f850eea9a5f7470427", size = 229892 }] + +[[distribution.dependencies]] +name = "six" +version = "1.17.0" +source = "registry+https://pypi.org/simple" + +[[distribution]] +name = "six" +version = "1.17.0" +source = "registry+https://pypi.org/simple" +sdist = { url = "https://files.pythonhosted.org/packages/94/e7/b2c673351809dca68a0e064b6af791aa332cf192da575fd474ed7d6f16a2/six-1.17.0.tar.gz", hash = "sha256:ff70335d468e7eb6ec65b95b99d3a2836546063f63acc5171de367e834932a81", size = 34031 } +wheels = [{ url = "https://files.pythonhosted.org/packages/b7/ce/149a00dd41f10bc29e5921b496af8b574d8413afcd5e30dfa0ed46c2cc5e/six-1.17.0-py2.py3-none-any.whl", hash = "sha256:4721f391ed90541fddacab5acf947aa0d3dc7d27b2e1e8eda2be8970586c3274", size = 11050 }] + +[[distribution]] +name = "urllib3" +version = "1.26.18" +source = "registry+https://pypi.org/simple" +sdist = { url = "https://files.pythonhosted.org/packages/0c/39/64487bf07df2ed854cc06078c27c0d0abc59bd27b32232876e403c333a08/urllib3-1.26.18.tar.gz", hash = "sha256:f8ecc1bba5667413457c529ab955bf8c67b45db799d159066261719e328580a0", size = 305687 } +wheels = [{ url = "https://files.pythonhosted.org/packages/b0/53/aa91e163dcfd1e5b82d8a890ecf13314e3e149c05270cc644581f77f17fd/urllib3-1.26.18-py2.py3-none-any.whl", hash = "sha256:34b97092d7e0a3a8cf7cd10e386f401b3737364026c45e622aa02903dffe0f07", size = 143835 }] diff --git a/crates/socket-patch-core/tests/fixtures/upstream/uv-0.2/0.2.16/pyproject.toml b/crates/socket-patch-core/tests/fixtures/upstream/uv-0.2/0.2.16/pyproject.toml new file mode 100644 index 000000000..189fa288e --- /dev/null +++ b/crates/socket-patch-core/tests/fixtures/upstream/uv-0.2/0.2.16/pyproject.toml @@ -0,0 +1,5 @@ +[project] +name = "proj" +version = "0.1.0" +requires-python = ">=3.9" +dependencies = ["urllib3==1.26.18", "python-dateutil==2.9.0.post0", "isodate==0.6.1"] diff --git a/crates/socket-patch-core/tests/fixtures/upstream/uv-0.2/0.2.16/uv.lock b/crates/socket-patch-core/tests/fixtures/upstream/uv-0.2/0.2.16/uv.lock new file mode 100644 index 000000000..f09b924e4 --- /dev/null +++ b/crates/socket-patch-core/tests/fixtures/upstream/uv-0.2/0.2.16/uv.lock @@ -0,0 +1,50 @@ +version = 1 +requires-python = ">=3.9" + +[[distribution]] +name = "isodate" +version = "0.6.1" +source = "registry+https://pypi.org/simple" +sdist = { url = "https://files.pythonhosted.org/packages/db/7a/c0a56c7d56c7fa723988f122fa1f1ccf8c5c4ccc48efad0d214b49e5b1af/isodate-0.6.1.tar.gz", hash = "sha256:48c5881de7e8b0a0d648cb024c8062dc84e7b840ed81e864c7614fd3c127bde9", size = 28443 } +wheels = [{ url = "https://files.pythonhosted.org/packages/b6/85/7882d311924cbcfc70b1890780763e36ff0b140c7e51c110fc59a532f087/isodate-0.6.1-py2.py3-none-any.whl", hash = "sha256:0751eece944162659049d35f4f549ed815792b38793f07cf73381c1c87cbed96", size = 41722 }] + +[[distribution.dependencies]] +name = "six" + +[[distribution]] +name = "proj" +version = "0.1.0" +source = "editable+." + +[[distribution.dependencies]] +name = "isodate" + +[[distribution.dependencies]] +name = "python-dateutil" + +[[distribution.dependencies]] +name = "urllib3" + +[[distribution]] +name = "python-dateutil" +version = "2.9.0.post0" +source = "registry+https://pypi.org/simple" +sdist = { url = "https://files.pythonhosted.org/packages/66/c0/0c8b6ad9f17a802ee498c46e004a0eb49bc148f2fd230864601a86dcf6db/python-dateutil-2.9.0.post0.tar.gz", hash = "sha256:37dd54208da7e1cd875388217d5e00ebd4179249f90fb72437e91a35459a0ad3", size = 342432 } +wheels = [{ url = "https://files.pythonhosted.org/packages/ec/57/56b9bcc3c9c6a792fcbaf139543cee77261f3651ca9da0c93f5c1221264b/python_dateutil-2.9.0.post0-py2.py3-none-any.whl", hash = "sha256:a8b2bc7bffae282281c8140a97d3aa9c14da0b136dfe83f850eea9a5f7470427", size = 229892 }] + +[[distribution.dependencies]] +name = "six" + +[[distribution]] +name = "six" +version = "1.17.0" +source = "registry+https://pypi.org/simple" +sdist = { url = "https://files.pythonhosted.org/packages/94/e7/b2c673351809dca68a0e064b6af791aa332cf192da575fd474ed7d6f16a2/six-1.17.0.tar.gz", hash = "sha256:ff70335d468e7eb6ec65b95b99d3a2836546063f63acc5171de367e834932a81", size = 34031 } +wheels = [{ url = "https://files.pythonhosted.org/packages/b7/ce/149a00dd41f10bc29e5921b496af8b574d8413afcd5e30dfa0ed46c2cc5e/six-1.17.0-py2.py3-none-any.whl", hash = "sha256:4721f391ed90541fddacab5acf947aa0d3dc7d27b2e1e8eda2be8970586c3274", size = 11050 }] + +[[distribution]] +name = "urllib3" +version = "1.26.18" +source = "registry+https://pypi.org/simple" +sdist = { url = "https://files.pythonhosted.org/packages/0c/39/64487bf07df2ed854cc06078c27c0d0abc59bd27b32232876e403c333a08/urllib3-1.26.18.tar.gz", hash = "sha256:f8ecc1bba5667413457c529ab955bf8c67b45db799d159066261719e328580a0", size = 305687 } +wheels = [{ url = "https://files.pythonhosted.org/packages/b0/53/aa91e163dcfd1e5b82d8a890ecf13314e3e149c05270cc644581f77f17fd/urllib3-1.26.18-py2.py3-none-any.whl", hash = "sha256:34b97092d7e0a3a8cf7cd10e386f401b3737364026c45e622aa02903dffe0f07", size = 143835 }] diff --git a/crates/socket-patch-core/tests/fixtures/upstream/uv-0.2/0.2.18/pyproject.toml b/crates/socket-patch-core/tests/fixtures/upstream/uv-0.2/0.2.18/pyproject.toml new file mode 100644 index 000000000..189fa288e --- /dev/null +++ b/crates/socket-patch-core/tests/fixtures/upstream/uv-0.2/0.2.18/pyproject.toml @@ -0,0 +1,5 @@ +[project] +name = "proj" +version = "0.1.0" +requires-python = ">=3.9" +dependencies = ["urllib3==1.26.18", "python-dateutil==2.9.0.post0", "isodate==0.6.1"] diff --git a/crates/socket-patch-core/tests/fixtures/upstream/uv-0.2/0.2.18/uv.lock b/crates/socket-patch-core/tests/fixtures/upstream/uv-0.2/0.2.18/uv.lock new file mode 100644 index 000000000..ada843968 --- /dev/null +++ b/crates/socket-patch-core/tests/fixtures/upstream/uv-0.2/0.2.18/uv.lock @@ -0,0 +1,54 @@ +version = 1 +requires-python = ">=3.9" + +[[distribution]] +name = "isodate" +version = "0.6.1" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/db/7a/c0a56c7d56c7fa723988f122fa1f1ccf8c5c4ccc48efad0d214b49e5b1af/isodate-0.6.1.tar.gz", hash = "sha256:48c5881de7e8b0a0d648cb024c8062dc84e7b840ed81e864c7614fd3c127bde9", size = 28443 } +dependencies = [ + { name = "six" }, +] +wheels = [ + { url = "https://files.pythonhosted.org/packages/b6/85/7882d311924cbcfc70b1890780763e36ff0b140c7e51c110fc59a532f087/isodate-0.6.1-py2.py3-none-any.whl", hash = "sha256:0751eece944162659049d35f4f549ed815792b38793f07cf73381c1c87cbed96", size = 41722 }, +] + +[[distribution]] +name = "proj" +version = "0.1.0" +source = { editable = "." } +dependencies = [ + { name = "isodate" }, + { name = "python-dateutil" }, + { name = "urllib3" }, +] + +[[distribution]] +name = "python-dateutil" +version = "2.9.0.post0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/66/c0/0c8b6ad9f17a802ee498c46e004a0eb49bc148f2fd230864601a86dcf6db/python-dateutil-2.9.0.post0.tar.gz", hash = "sha256:37dd54208da7e1cd875388217d5e00ebd4179249f90fb72437e91a35459a0ad3", size = 342432 } +dependencies = [ + { name = "six" }, +] +wheels = [ + { url = "https://files.pythonhosted.org/packages/ec/57/56b9bcc3c9c6a792fcbaf139543cee77261f3651ca9da0c93f5c1221264b/python_dateutil-2.9.0.post0-py2.py3-none-any.whl", hash = "sha256:a8b2bc7bffae282281c8140a97d3aa9c14da0b136dfe83f850eea9a5f7470427", size = 229892 }, +] + +[[distribution]] +name = "six" +version = "1.17.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/94/e7/b2c673351809dca68a0e064b6af791aa332cf192da575fd474ed7d6f16a2/six-1.17.0.tar.gz", hash = "sha256:ff70335d468e7eb6ec65b95b99d3a2836546063f63acc5171de367e834932a81", size = 34031 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/b7/ce/149a00dd41f10bc29e5921b496af8b574d8413afcd5e30dfa0ed46c2cc5e/six-1.17.0-py2.py3-none-any.whl", hash = "sha256:4721f391ed90541fddacab5acf947aa0d3dc7d27b2e1e8eda2be8970586c3274", size = 11050 }, +] + +[[distribution]] +name = "urllib3" +version = "1.26.18" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/0c/39/64487bf07df2ed854cc06078c27c0d0abc59bd27b32232876e403c333a08/urllib3-1.26.18.tar.gz", hash = "sha256:f8ecc1bba5667413457c529ab955bf8c67b45db799d159066261719e328580a0", size = 305687 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/b0/53/aa91e163dcfd1e5b82d8a890ecf13314e3e149c05270cc644581f77f17fd/urllib3-1.26.18-py2.py3-none-any.whl", hash = "sha256:34b97092d7e0a3a8cf7cd10e386f401b3737364026c45e622aa02903dffe0f07", size = 143835 }, +] diff --git a/crates/socket-patch-core/tests/fixtures/upstream/uv-0.2/0.2.34/pyproject.toml b/crates/socket-patch-core/tests/fixtures/upstream/uv-0.2/0.2.34/pyproject.toml new file mode 100644 index 000000000..189fa288e --- /dev/null +++ b/crates/socket-patch-core/tests/fixtures/upstream/uv-0.2/0.2.34/pyproject.toml @@ -0,0 +1,5 @@ +[project] +name = "proj" +version = "0.1.0" +requires-python = ">=3.9" +dependencies = ["urllib3==1.26.18", "python-dateutil==2.9.0.post0", "isodate==0.6.1"] diff --git a/crates/socket-patch-core/tests/fixtures/upstream/uv-0.2/0.2.34/uv.lock b/crates/socket-patch-core/tests/fixtures/upstream/uv-0.2/0.2.34/uv.lock new file mode 100644 index 000000000..6b79fdfd8 --- /dev/null +++ b/crates/socket-patch-core/tests/fixtures/upstream/uv-0.2/0.2.34/uv.lock @@ -0,0 +1,54 @@ +version = 1 +requires-python = ">=3.9" + +[[distribution]] +name = "isodate" +version = "0.6.1" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "six" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/db/7a/c0a56c7d56c7fa723988f122fa1f1ccf8c5c4ccc48efad0d214b49e5b1af/isodate-0.6.1.tar.gz", hash = "sha256:48c5881de7e8b0a0d648cb024c8062dc84e7b840ed81e864c7614fd3c127bde9", size = 28443 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/b6/85/7882d311924cbcfc70b1890780763e36ff0b140c7e51c110fc59a532f087/isodate-0.6.1-py2.py3-none-any.whl", hash = "sha256:0751eece944162659049d35f4f549ed815792b38793f07cf73381c1c87cbed96", size = 41722 }, +] + +[[distribution]] +name = "proj" +version = "0.1.0" +source = { editable = "." } +dependencies = [ + { name = "isodate" }, + { name = "python-dateutil" }, + { name = "urllib3" }, +] + +[[distribution]] +name = "python-dateutil" +version = "2.9.0.post0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "six" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/66/c0/0c8b6ad9f17a802ee498c46e004a0eb49bc148f2fd230864601a86dcf6db/python-dateutil-2.9.0.post0.tar.gz", hash = "sha256:37dd54208da7e1cd875388217d5e00ebd4179249f90fb72437e91a35459a0ad3", size = 342432 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/ec/57/56b9bcc3c9c6a792fcbaf139543cee77261f3651ca9da0c93f5c1221264b/python_dateutil-2.9.0.post0-py2.py3-none-any.whl", hash = "sha256:a8b2bc7bffae282281c8140a97d3aa9c14da0b136dfe83f850eea9a5f7470427", size = 229892 }, +] + +[[distribution]] +name = "six" +version = "1.17.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/94/e7/b2c673351809dca68a0e064b6af791aa332cf192da575fd474ed7d6f16a2/six-1.17.0.tar.gz", hash = "sha256:ff70335d468e7eb6ec65b95b99d3a2836546063f63acc5171de367e834932a81", size = 34031 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/b7/ce/149a00dd41f10bc29e5921b496af8b574d8413afcd5e30dfa0ed46c2cc5e/six-1.17.0-py2.py3-none-any.whl", hash = "sha256:4721f391ed90541fddacab5acf947aa0d3dc7d27b2e1e8eda2be8970586c3274", size = 11050 }, +] + +[[distribution]] +name = "urllib3" +version = "1.26.18" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/0c/39/64487bf07df2ed854cc06078c27c0d0abc59bd27b32232876e403c333a08/urllib3-1.26.18.tar.gz", hash = "sha256:f8ecc1bba5667413457c529ab955bf8c67b45db799d159066261719e328580a0", size = 305687 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/b0/53/aa91e163dcfd1e5b82d8a890ecf13314e3e149c05270cc644581f77f17fd/urllib3-1.26.18-py2.py3-none-any.whl", hash = "sha256:34b97092d7e0a3a8cf7cd10e386f401b3737364026c45e622aa02903dffe0f07", size = 143835 }, +] diff --git a/crates/socket-patch-core/tests/fixtures/upstream/uv-0.2/0.2.5/pyproject.toml b/crates/socket-patch-core/tests/fixtures/upstream/uv-0.2/0.2.5/pyproject.toml new file mode 100644 index 000000000..189fa288e --- /dev/null +++ b/crates/socket-patch-core/tests/fixtures/upstream/uv-0.2/0.2.5/pyproject.toml @@ -0,0 +1,5 @@ +[project] +name = "proj" +version = "0.1.0" +requires-python = ">=3.9" +dependencies = ["urllib3==1.26.18", "python-dateutil==2.9.0.post0", "isodate==0.6.1"] diff --git a/crates/socket-patch-core/tests/fixtures/upstream/uv-0.2/0.2.5/uv.lock b/crates/socket-patch-core/tests/fixtures/upstream/uv-0.2/0.2.5/uv.lock new file mode 100644 index 000000000..32e682621 --- /dev/null +++ b/crates/socket-patch-core/tests/fixtures/upstream/uv-0.2/0.2.5/uv.lock @@ -0,0 +1,94 @@ +version = 1 + +[[distribution]] +name = "isodate" +version = "0.6.1" +source = "registry+https://pypi.org/simple" + +[distribution.sdist] +url = "https://files.pythonhosted.org/packages/db/7a/c0a56c7d56c7fa723988f122fa1f1ccf8c5c4ccc48efad0d214b49e5b1af/isodate-0.6.1.tar.gz" +hash = "sha256:48c5881de7e8b0a0d648cb024c8062dc84e7b840ed81e864c7614fd3c127bde9" +size = 28443 + +[[distribution.wheel]] +url = "https://files.pythonhosted.org/packages/b6/85/7882d311924cbcfc70b1890780763e36ff0b140c7e51c110fc59a532f087/isodate-0.6.1-py2.py3-none-any.whl" +hash = "sha256:0751eece944162659049d35f4f549ed815792b38793f07cf73381c1c87cbed96" +size = 41722 + +[[distribution.dependencies]] +name = "six" +version = "1.17.0" +source = "registry+https://pypi.org/simple" + +[[distribution]] +name = "proj" +version = "0.1.0" +source = "editable+file:///project" + +[distribution.sdist] +url = "file:///project" + +[[distribution.dependencies]] +name = "isodate" +version = "0.6.1" +source = "registry+https://pypi.org/simple" + +[[distribution.dependencies]] +name = "python-dateutil" +version = "2.9.0.post0" +source = "registry+https://pypi.org/simple" + +[[distribution.dependencies]] +name = "urllib3" +version = "1.26.18" +source = "registry+https://pypi.org/simple" + +[[distribution]] +name = "python-dateutil" +version = "2.9.0.post0" +source = "registry+https://pypi.org/simple" + +[distribution.sdist] +url = "https://files.pythonhosted.org/packages/66/c0/0c8b6ad9f17a802ee498c46e004a0eb49bc148f2fd230864601a86dcf6db/python-dateutil-2.9.0.post0.tar.gz" +hash = "sha256:37dd54208da7e1cd875388217d5e00ebd4179249f90fb72437e91a35459a0ad3" +size = 342432 + +[[distribution.wheel]] +url = "https://files.pythonhosted.org/packages/ec/57/56b9bcc3c9c6a792fcbaf139543cee77261f3651ca9da0c93f5c1221264b/python_dateutil-2.9.0.post0-py2.py3-none-any.whl" +hash = "sha256:a8b2bc7bffae282281c8140a97d3aa9c14da0b136dfe83f850eea9a5f7470427" +size = 229892 + +[[distribution.dependencies]] +name = "six" +version = "1.17.0" +source = "registry+https://pypi.org/simple" + +[[distribution]] +name = "six" +version = "1.17.0" +source = "registry+https://pypi.org/simple" + +[distribution.sdist] +url = "https://files.pythonhosted.org/packages/94/e7/b2c673351809dca68a0e064b6af791aa332cf192da575fd474ed7d6f16a2/six-1.17.0.tar.gz" +hash = "sha256:ff70335d468e7eb6ec65b95b99d3a2836546063f63acc5171de367e834932a81" +size = 34031 + +[[distribution.wheel]] +url = "https://files.pythonhosted.org/packages/b7/ce/149a00dd41f10bc29e5921b496af8b574d8413afcd5e30dfa0ed46c2cc5e/six-1.17.0-py2.py3-none-any.whl" +hash = "sha256:4721f391ed90541fddacab5acf947aa0d3dc7d27b2e1e8eda2be8970586c3274" +size = 11050 + +[[distribution]] +name = "urllib3" +version = "1.26.18" +source = "registry+https://pypi.org/simple" + +[distribution.sdist] +url = "https://files.pythonhosted.org/packages/0c/39/64487bf07df2ed854cc06078c27c0d0abc59bd27b32232876e403c333a08/urllib3-1.26.18.tar.gz" +hash = "sha256:f8ecc1bba5667413457c529ab955bf8c67b45db799d159066261719e328580a0" +size = 305687 + +[[distribution.wheel]] +url = "https://files.pythonhosted.org/packages/b0/53/aa91e163dcfd1e5b82d8a890ecf13314e3e149c05270cc644581f77f17fd/urllib3-1.26.18-py2.py3-none-any.whl" +hash = "sha256:34b97092d7e0a3a8cf7cd10e386f401b3737364026c45e622aa02903dffe0f07" +size = 143835 diff --git a/crates/socket-patch-core/tests/fixtures/upstream/uv-0.2/README.md b/crates/socket-patch-core/tests/fixtures/upstream/uv-0.2/README.md new file mode 100644 index 000000000..449cb01a4 --- /dev/null +++ b/crates/socket-patch-core/tests/fixtures/upstream/uv-0.2/README.md @@ -0,0 +1,7 @@ +Locks written by real `uv lock` binaries (uv 0.1.45, 0.2.5, 0.2.15, 0.2.16, +0.2.18, 0.2.34) for the `pyproject.toml` beside each, on macOS arm64 with +CPython 3.11 and `UV_NO_CONFIG=1`. The only edit is the editable root's +absolute path, replaced with `file:///project`. They cover the three uv 0.2 +`[[distribution]]` grammars and the uv 0.2.23 move of an entry's inline +`dependencies` ahead of its `sdist`; `tests/upstream_restore_golden.rs` +round-trips them through the hosted rewrite and the hosted unwind. diff --git a/crates/socket-patch-core/tests/fixtures/vex-discover-golden/upstream.json b/crates/socket-patch-core/tests/fixtures/vex-discover-golden/upstream.json index 0915c1e47..f33011697 100644 --- a/crates/socket-patch-core/tests/fixtures/vex-discover-golden/upstream.json +++ b/crates/socket-patch-core/tests/fixtures/vex-discover-golden/upstream.json @@ -1,4 +1,154 @@ { + "upstream/uv-0.2/0.1.45": { + "refs": [], + "diagnostics": [], + "recognized": [], + "unlocked_pins": [], + "elsewhere": [ + { + "purl": "pkg:pypi/isodate@0.6.1", + "file": "uv.lock" + }, + { + "purl": "pkg:pypi/python-dateutil@2.9.0.post0", + "file": "uv.lock" + }, + { + "purl": "pkg:pypi/six@1.17.0", + "file": "uv.lock" + }, + { + "purl": "pkg:pypi/urllib3@1.26.18", + "file": "uv.lock" + } + ], + "live_claims": [] + }, + "upstream/uv-0.2/0.2.15": { + "refs": [], + "diagnostics": [], + "recognized": [], + "unlocked_pins": [], + "elsewhere": [ + { + "purl": "pkg:pypi/isodate@0.6.1", + "file": "uv.lock" + }, + { + "purl": "pkg:pypi/python-dateutil@2.9.0.post0", + "file": "uv.lock" + }, + { + "purl": "pkg:pypi/six@1.17.0", + "file": "uv.lock" + }, + { + "purl": "pkg:pypi/urllib3@1.26.18", + "file": "uv.lock" + } + ], + "live_claims": [] + }, + "upstream/uv-0.2/0.2.16": { + "refs": [], + "diagnostics": [], + "recognized": [], + "unlocked_pins": [], + "elsewhere": [ + { + "purl": "pkg:pypi/isodate@0.6.1", + "file": "uv.lock" + }, + { + "purl": "pkg:pypi/python-dateutil@2.9.0.post0", + "file": "uv.lock" + }, + { + "purl": "pkg:pypi/six@1.17.0", + "file": "uv.lock" + }, + { + "purl": "pkg:pypi/urllib3@1.26.18", + "file": "uv.lock" + } + ], + "live_claims": [] + }, + "upstream/uv-0.2/0.2.18": { + "refs": [], + "diagnostics": [], + "recognized": [], + "unlocked_pins": [], + "elsewhere": [ + { + "purl": "pkg:pypi/isodate@0.6.1", + "file": "uv.lock" + }, + { + "purl": "pkg:pypi/python-dateutil@2.9.0.post0", + "file": "uv.lock" + }, + { + "purl": "pkg:pypi/six@1.17.0", + "file": "uv.lock" + }, + { + "purl": "pkg:pypi/urllib3@1.26.18", + "file": "uv.lock" + } + ], + "live_claims": [] + }, + "upstream/uv-0.2/0.2.34": { + "refs": [], + "diagnostics": [], + "recognized": [], + "unlocked_pins": [], + "elsewhere": [ + { + "purl": "pkg:pypi/isodate@0.6.1", + "file": "uv.lock" + }, + { + "purl": "pkg:pypi/python-dateutil@2.9.0.post0", + "file": "uv.lock" + }, + { + "purl": "pkg:pypi/six@1.17.0", + "file": "uv.lock" + }, + { + "purl": "pkg:pypi/urllib3@1.26.18", + "file": "uv.lock" + } + ], + "live_claims": [] + }, + "upstream/uv-0.2/0.2.5": { + "refs": [], + "diagnostics": [], + "recognized": [], + "unlocked_pins": [], + "elsewhere": [ + { + "purl": "pkg:pypi/isodate@0.6.1", + "file": "uv.lock" + }, + { + "purl": "pkg:pypi/python-dateutil@2.9.0.post0", + "file": "uv.lock" + }, + { + "purl": "pkg:pypi/six@1.17.0", + "file": "uv.lock" + }, + { + "purl": "pkg:pypi/urllib3@1.26.18", + "file": "uv.lock" + } + ], + "live_claims": [] + }, "upstream/uv-explicit-extra": { "refs": [], "diagnostics": [], diff --git a/crates/socket-patch-core/tests/upstream_restore_golden.rs b/crates/socket-patch-core/tests/upstream_restore_golden.rs index c07abed14..88adc6afd 100644 --- a/crates/socket-patch-core/tests/upstream_restore_golden.rs +++ b/crates/socket-patch-core/tests/upstream_restore_golden.rs @@ -1163,10 +1163,16 @@ async fn pypi_mock(releases: &[Release<'_>]) -> (MockServer, EnvGuard) { for (name, version, files) in releases { let urls: Vec = files .iter() - .map(|(filename, sha, size, uploaded)| { + .map(|(file, sha, size, uploaded)| { + // A full URL is served as is; a bare filename at + // `pypi_file_url`'s. + let (filename, url) = match file.rsplit_once('/') { + Some((_, leaf)) => (leaf.to_string(), file.to_string()), + None => (file.to_string(), pypi_file_url(file)), + }; serde_json::json!({ "filename": filename, - "url": pypi_file_url(filename), + "url": url, "digests": { "sha256": sha }, "size": size, "upload_time_iso_8601": uploaded, @@ -2123,6 +2129,329 @@ async fn uv_underscore_upload_time_locks_round_trip() { assert_pypi_round_trip("uv 0.6.17 script", &input, &[urllib3_dep()], None).await; } +// ── uv 0.2 `[[distribution]]` locks ────────────────────────────────────────── + +const DATEUTIL_UUID: &str = "44444444-4444-4444-4444-444444444444"; +const DATEUTIL_WHEEL: &str = "python_dateutil-2.9.0.post0-py2.py3-none-any.whl"; + +/// The real `uv lock` output in `tests/fixtures/upstream/uv-0.2/`: the +/// sub-table grammar (uv 0.1.45, 0.2.5), string sources with inline +/// artifacts (0.2.15 with source-qualified dependency tables, 0.2.16 with +/// bare ones) and inline sources (0.2.18, which writes an `sdist` before +/// inline `dependencies`, and 0.2.34, which writes it after them). +const UV_02_LOCKS: [&str; 6] = ["0.1.45", "0.2.5", "0.2.15", "0.2.16", "0.2.18", "0.2.34"]; + +/// A uv 0.2 fixture project: its `uv.lock` and `pyproject.toml`. +fn uv_02_project(version: &str) -> BTreeMap { + let file = |name: &str| fixture(&format!("upstream/uv-0.2/{version}/{name}")); + tree(&[ + ("uv.lock", file("uv.lock")), + ("pyproject.toml", file("pyproject.toml")), + ]) +} + +fn dateutil_dep() -> DepOverride { + pypi_dep( + "python-dateutil", + "2.9.0.post0", + DATEUTIL_WHEEL, + DATEUTIL_UUID, + ) +} + +/// A mock PyPI serving the files the uv 0.2 fixtures pin for urllib3 and +/// python-dateutil, at their real URLs. +async fn uv_02_pypi() -> (MockServer, EnvGuard) { + pypi_mock(&[ + ( + "urllib3", + "1.26.18", + vec![ + ( + "https://files.pythonhosted.org/packages/b0/53/aa91e163dcfd1e5b82d8a890ecf13314e3e149c05270cc644581f77f17fd/urllib3-1.26.18-py2.py3-none-any.whl", + URLLIB3_WHEEL_SHA, + 143835, + "2023-10-17T17:46:21.184066Z", + ), + ( + "https://files.pythonhosted.org/packages/0c/39/64487bf07df2ed854cc06078c27c0d0abc59bd27b32232876e403c333a08/urllib3-1.26.18.tar.gz", + URLLIB3_SDIST_SHA, + 305687, + "2023-10-17T17:46:24.000000Z", + ), + ], + ), + ( + "python-dateutil", + "2.9.0.post0", + vec![ + ( + "https://files.pythonhosted.org/packages/66/c0/0c8b6ad9f17a802ee498c46e004a0eb49bc148f2fd230864601a86dcf6db/python-dateutil-2.9.0.post0.tar.gz", + "37dd54208da7e1cd875388217d5e00ebd4179249f90fb72437e91a35459a0ad3", + 342432, + "2024-03-01T18:46:02.000000Z", + ), + ( + "https://files.pythonhosted.org/packages/ec/57/56b9bcc3c9c6a792fcbaf139543cee77261f3651ca9da0c93f5c1221264b/python_dateutil-2.9.0.post0-py2.py3-none-any.whl", + "a8b2bc7bffae282281c8140a97d3aa9c14da0b136dfe83f850eea9a5f7470427", + 229892, + "2024-03-01T18:46:03.000000Z", + ), + ], + ), + ]) + .await +} + +/// The hosted rewrite's warnings for `input`. +fn hosted_warnings( + input: &BTreeMap, + deps: &[DepOverride], +) -> Vec<(String, String)> { + rewrite_registry_redirect_with_pipenv_version(input, deps, &BTreeMap::new(), None, false) + .warnings + .into_iter() + .map(|w| (w.code, w.detail)) + .collect() +} + +/// Whatever the hosted rewrite writes into a real uv 0.2 lock, in each of +/// its grammars, the unwind restores byte for byte: the source in the +/// entry's spelling, the artifacts in the entry's grammar (sub-tables in +/// uv's order, an `sdist` placed beside inline `dependencies` as the +/// lock's other registry packages show), dependents' references, and the +/// paired pyproject's `[tool.uv.sources]`. +#[tokio::test] +#[serial] +async fn uv_distribution_locks_round_trip() { + let (_server, _env) = uv_02_pypi().await; + for version in UV_02_LOCKS { + let project = uv_02_project(version); + for deps in [vec![urllib3_dep()], vec![urllib3_dep(), dateutil_dep()]] { + let label = format!("uv {version} {} pins", deps.len()); + assert_eq!(hosted_warnings(&project, &deps), vec![], "{label}"); + let rewritten = hosted(&project, &deps, None); + assert!( + !rewritten["uv.lock"].contains("files.pythonhosted.org/packages/b0/53") + && rewritten["uv.lock"].contains("patch.socket.dev"), + "{label}: the hosted rewrite did not wire urllib3:\n{}", + rewritten["uv.lock"] + ); + for eol in ["\n", "\r\n"] { + let input = project + .iter() + .map(|(k, v)| (k.clone(), v.replace('\n', eol))) + .collect(); + assert_pypi_round_trip(&format!("{label} {eol:?}"), &input, &deps, None).await; + } + let lock_only = tree(&[("uv.lock", project["uv.lock"].clone())]); + assert_pypi_round_trip(&format!("{label} lock-only"), &lock_only, &deps, None).await; + } + } +} + +/// The entry `name` of a uv 0.2 lock, cut out with its sub-tables. +fn without_distribution(lock: &str, name: &str) -> String { + let start = lock + .find(&format!("[[distribution]]\nname = \"{name}\"")) + .unwrap(); + let end = lock[start + 1..] + .find("\n[[distribution]]\n") + .map_or(lock.len(), |i| start + 2 + i); + format!("{}{}", &lock[..start], &lock[end..]) +} + +/// uv 0.2.18–0.2.22 write an entry's `sdist` before its inline +/// `dependencies`, 0.2.23–0.2.34 after them. When no other registry entry +/// carries both, the rewrite refuses an entry the unwind could not place +/// its `sdist` in, and the unwind refuses an entry an earlier release +/// wired that way; the rest of the lock is still handled. +#[tokio::test] +#[serial] +async fn uv_distribution_sdist_order_without_evidence_is_refused() { + let (_server, _env) = uv_02_pypi().await; + for version in ["0.2.18", "0.2.34"] { + // isodate is the only other registry entry with both. + let mut project = uv_02_project(version); + let lock = without_distribution(&project["uv.lock"], "isodate"); + project.insert("uv.lock".into(), lock); + let deps = [urllib3_dep(), dateutil_dep()]; + let warnings = hosted_warnings(&project, &deps); + assert!( + matches!(warnings.as_slice(), [(code, detail)] + if code == "redirect_uv_lock_unsupported" + && detail.contains("python-dateutil==2.9.0.post0") + && detail.contains("0.2.23")), + "uv {version}: {warnings:?}" + ); + let rewritten = hosted(&project, &deps, None); + assert!(rewritten["uv.lock"].contains("files.pythonhosted.org/packages/66/c0")); + assert!(!rewritten["uv.lock"].contains("files.pythonhosted.org/packages/b0/53")); + + // A lock wired before the refusal existed. + let full = uv_02_project(version); + let mut wired = hosted(&full, &deps, None); + let lock = without_distribution(&wired["uv.lock"], "isodate"); + wired.insert("uv.lock".into(), lock); + let (after, statuses) = restore_tree(&wired, &RestoreOptions::default()).await; + let status = |purl: &str| { + statuses + .iter() + .find(|(p, _)| p.contains(purl)) + .map(|(_, s)| s.clone()) + .unwrap() + }; + assert_eq!(status("urllib3"), PinStatus::Restored, "uv {version}"); + assert!( + matches!(status("python-dateutil"), PinStatus::Refused(why) if why.contains("not derivable")), + "uv {version}: {statuses:?}" + ); + assert!(after["uv.lock"].contains("files.pythonhosted.org/packages/b0/53")); + assert!(after["uv.lock"].contains(&format!("/{DATEUTIL_UUID}/"))); + } +} + +/// Both directions read an entry's grammar off the entry itself before +/// any sibling: an entry with sub-table artifacts and a string source in a +/// lock whose first registry package is inline in both comes back as it +/// was, not in the first package's shape. +#[tokio::test] +#[serial] +async fn uv_distribution_entry_grammar_wins_over_the_first_sibling() { + let (_server, _env) = pypi_mock(&[urllib3_release()]).await; + let lock = format!( + "version = 1\n\n\ +[[distribution]]\nname = \"idna\"\nversion = \"3.4\"\nsource = {{ registry = \"https://pypi.org/simple\" }}\n\ +wheels = [{{ url = \"https://files.pythonhosted.org/packages/bb/idna-3.4-py3-none-any.whl\", hash = \"sha256:{b}\" }}]\n\n\ +[[distribution]]\nname = \"urllib3\"\nversion = \"1.26.18\"\nsource = \"registry+https://pypi.org/simple\"\n\n\ +[distribution.sdist]\nurl = \"{sdist}\"\nhash = \"sha256:{URLLIB3_SDIST_SHA}\"\n\n\ +[[distribution.wheel]]\nurl = \"{wheel}\"\nhash = \"sha256:{URLLIB3_WHEEL_SHA}\"\n", + b = "b".repeat(64), + sdist = pypi_file_url(URLLIB3_SDIST), + wheel = pypi_file_url(URLLIB3_WHEEL), + ); + let input = tree(&[("uv.lock", lock)]); + let rewritten = hosted(&input, &[urllib3_dep()], None); + assert!( + rewritten["uv.lock"].contains("[[distribution.wheel]]") + && rewritten["uv.lock"].contains("source = \"direct+"), + "{}", + rewritten["uv.lock"] + ); + assert_pypi_round_trip("entry grammar", &input, &[urllib3_dep()], None).await; +} + +/// uv 0.2.12–0.2.27 apply an `exclude-newer` cutoff from the project's or +/// the user's settings, or `UV_EXCLUDE_NEWER`, without recording it in the +/// lock, and 0.2.28–0.2.33 record it at the lock's top level: which release +/// files the lock kept is then not derivable. The rewrite refuses a lock +/// whose project sets one; the unwind refuses on any of them. uv ≤ 0.2.5 +/// (the sub-table grammar) apply no cutoff, so those locks round-trip. +#[tokio::test] +#[serial] +async fn uv_distribution_exclude_newer_is_refused() { + let (_server, _env) = uv_02_pypi().await; + let cutoff = "\n[tool.uv]\nexclude-newer = \"2024-06-01T00:00:00Z\"\n"; + for version in ["0.2.15", "0.2.34"] { + let mut project = uv_02_project(version); + let pyproject = project["pyproject.toml"].clone() + cutoff; + project.insert("pyproject.toml".into(), pyproject); + let warnings = hosted_warnings(&project, &[urllib3_dep()]); + assert!( + matches!(warnings.as_slice(), [(code, detail)] + if code == "redirect_uv_lock_unsupported" && detail.contains("exclude-newer")), + "uv {version}: {warnings:?}" + ); + assert_eq!(hosted(&project, &[urllib3_dep()], None), project); + + // The same cutoff from the settings file above the root (uv reads + // it, and so does the unwind), or recorded in the lock itself. + let plain = uv_02_project(version); + let mut ancestor = plain.clone(); + ancestor.insert( + socket_patch_core::patch::redirect::UV_ANCESTOR_CONFIG_KEY.into(), + "../uv.toml\nexclude-newer = \"2024-06-01\"\n".into(), + ); + let mut recorded = plain.clone(); + recorded.insert( + "uv.lock".into(), + plain["uv.lock"].replacen( + "requires-python = \">=3.9\"\n", + "requires-python = \">=3.9\"\nexclude-newer = \"2024-06-01 00:00:00 UTC\"\n", + 1, + ), + ); + for (label, input) in [("ancestor", &ancestor), ("recorded", &recorded)] { + let warnings = hosted_warnings(input, &[urllib3_dep()]); + assert!( + warnings + .iter() + .any(|(code, detail)| code == "redirect_uv_lock_unsupported" + && detail.contains("exclude-newer")), + "uv {version} {label}: {warnings:?}" + ); + assert_eq!(&hosted(input, &[urllib3_dep()], None), input, "{label}"); + } + + // Pins wired before the setting appeared. + let project = uv_02_project(version); + let opts = RestoreOptions::default(); + let (why, _, after) = pypi_refusal_after(&project, &[urllib3_dep()], &opts, |files| { + files.insert("uv.toml".into(), "exclude-newer = \"2024-06-01\"\n".into()); + }) + .await; + assert!(why.contains("uv.toml sets uv `exclude-newer`"), "{why}"); + assert!(after["uv.lock"].contains(&format!("/{PYPI_UUID}/"))); + let (why, _, _) = pypi_refusal_after(&project, &[urllib3_dep()], &opts, |files| { + let lock = files["uv.lock"].replacen( + "requires-python = \">=3.9\"\n", + "requires-python = \">=3.9\"\nexclude-newer = \"2024-06-01 00:00:00 UTC\"\n", + 1, + ); + files.insert("uv.lock".into(), lock); + }) + .await; + assert!(why.contains("the lock's `exclude-newer` option"), "{why}"); + let _cutoff = EnvGuard::set("UV_EXCLUDE_NEWER", "2024-06-01T00:00:00Z"); + let (why, _, _) = pypi_refusal(&project, &[urllib3_dep()], &opts).await; + assert!(why.contains("`UV_EXCLUDE_NEWER` is set"), "{why}"); + } + let mut project = uv_02_project("0.2.5"); + let pyproject = project["pyproject.toml"].clone() + cutoff; + project.insert("pyproject.toml".into(), pyproject); + assert_pypi_round_trip("uv 0.2.5 exclude-newer", &project, &[urllib3_dep()], None).await; +} + +/// A hosted entry whose source was respelled as a `[distribution.source]` +/// table (which uv never writes, but reads) is restored in that spelling. +#[tokio::test] +#[serial] +async fn uv_distribution_source_table_is_restored() { + let (_server, _env) = uv_02_pypi().await; + let project = tree(&[("uv.lock", uv_02_project("0.2.34")["uv.lock"].clone())]); + let mut wired = hosted(&project, &[urllib3_dep()], None); + let lock = &wired["uv.lock"]; + // urllib3 is the lock's last entry. + let start = lock.rfind("source = { url = ").unwrap(); + let end = start + lock[start..].find('\n').unwrap() + 1; + let url = lock[start..end].split('"').nth(1).unwrap().to_string(); + let lock = format!( + "{}{}\n[distribution.source]\nurl = \"{url}\"\n", + &lock[..start], + &lock[end..] + ); + wired.insert("uv.lock".into(), lock); + let (after, statuses) = restore_tree(&wired, &RestoreOptions::default()).await; + assert_eq!(statuses.len(), 1, "{statuses:?}"); + assert_eq!(statuses[0].1, PinStatus::Restored); + let want = project["uv.lock"].replacen( + "source = { registry = \"https://pypi.org/simple\" }\nsdist = { url = \"https://files.pythonhosted.org/packages/0c/39", + "sdist = { url = \"https://files.pythonhosted.org/packages/0c/39", + 1, + ) + "\n[distribution.source]\nregistry = \"https://pypi.org/simple\"\n"; + assert_eq!(after["uv.lock"], want); +} + #[tokio::test] #[serial] async fn pylock_round_trips() { @@ -2608,8 +2937,25 @@ async fn uv_refusals() { "requires-python = \">=3.8\"\n", "requires-python = \">=3.8\"\n\n[options]\nexclude-newer = \"2024-01-01T00:00:00Z\"\n", ); - let input = tree(&[("uv.lock", newer)]); - let (why, _, _) = pypi_refusal(&input, &[urllib3_dep()], &RestoreOptions::default()).await; + // The hosted rewrite refuses to wire such a lock; one wired before + // the cutoff was recorded comes back to the restore. + let input = tree(&[("uv.lock", newer.clone())]); + assert_eq!(hosted(&input, &[urllib3_dep()], None), input); + let input = tree(&[("uv.lock", direct.clone())]); + let (why, _, _) = pypi_refusal_after( + &input, + &[urllib3_dep()], + &RestoreOptions::default(), + |files| { + let lock = files["uv.lock"].replacen( + "requires-python = \">=3.8\"\n", + "requires-python = \">=3.8\"\n\n[options]\nexclude-newer = \"2024-01-01T00:00:00Z\"\n", + 1, + ); + files.insert("uv.lock".into(), lock); + }, + ) + .await; assert!(why.contains("exclude-newer"), "{why}"); // Offline. let input = tree(&[("uv.lock", direct.clone())]); @@ -2679,9 +3025,28 @@ async fn uv_exclude_newer_package_refuses_only_its_package() { // Exempting another package leaves the global cutoff on urllib3. "exclude-newer = \"2024-01-01T00:00:00Z\"\n\n[options.exclude-newer-package]\nidna = false\n", ] { + // The hosted rewrite refuses to wire it; a pin wired before + // the cutoff was recorded comes back to the restore. let input = input_with(options); - let (why, rewritten, after) = - pypi_refusal(&input, &[urllib3_dep()], &RestoreOptions::default()).await; + assert_eq!( + hosted(&input, &[urllib3_dep()], None), + input, + "{rel} {options}" + ); + let (why, rewritten, after) = pypi_refusal_after( + &input_with(""), + &[urllib3_dep()], + &RestoreOptions::default(), + |files| { + let lock = files[*rel].replacen( + "[options]\n\n", + &format!("[options]\n\n{options}"), + 1, + ); + files.insert(rel.to_string(), lock); + }, + ) + .await; assert_eq!(after, rewritten, "{rel} {options}: the lock stays hosted"); assert!(why.contains("exclude-newer"), "{rel} {options}: {why}"); } diff --git a/docs/testing/uv-compatibility.md b/docs/testing/uv-compatibility.md index 3ca0fd11e..575232ff5 100644 --- a/docs/testing/uv-compatibility.md +++ b/docs/testing/uv-compatibility.md @@ -17,7 +17,7 @@ managers. |-------|--------|----------| | `requirements.txt`, including uv-generated hash continuations | Exact version pins become direct artifact URLs with the patched SHA-256. Extras and markers are retained; hashes for the replaced artifact are removed. | Requirements refer to a committed wheel under `.socket/vendor/pypi/` with its hash. | | `uv.lock`, native `version = 1`, `[[package]]` | The package source and artifact entry agree on the hosted URL and hash. A paired `pyproject.toml` receives the corresponding uv source configuration. | The package source refers to the committed wheel. The paired `pyproject.toml` records that source. | -| `uv.lock`, experimental `[[distribution]]` (uv 0.1.45–0.2.34) | Follows the entry's own shape: `direct+` string or `{ url = … }` table source, `[[distribution.wheel]]` sub-table or inline `wheels` entry, and source-qualified dependency references where the lock carries them. | Refused (`pypi_uv_legacy_lock_unsupported`): the experimental grammar cannot carry a portable local wheel — through 0.2.6 a relative path source does not parse, and on 0.2.17–0.2.34 `--locked` rejects it while `uv lock` and plain `uv sync` absolutize it. Use uv 0.2.35 or newer. | +| `uv.lock`, experimental `[[distribution]]` (uv 0.1.45–0.2.34) | Follows the entry's own shape: `direct+` string or `{ url = … }` table source, `[[distribution.wheel]]` sub-table or inline `wheels` entry, and source-qualified dependency references where the lock carries them. Rollback restores the entry in the same shape (round-tripped on real uv 0.1.45, 0.2.5, 0.2.15, 0.2.16, 0.2.18 and 0.2.34 locks). Refused: an entry with an sdist and inline `dependencies` when no other registry entry shows their order (it changed at 0.2.23), and, for inline artifacts, a project `exclude-newer` setting, which uv 0.2.12–0.2.27 apply without recording. | Refused (`pypi_uv_legacy_lock_unsupported`): the experimental grammar cannot carry a portable local wheel — through 0.2.6 a relative path source does not parse, and on 0.2.17–0.2.34 `--locked` rejects it while `uv lock` and plain `uv sync` absolutize it. Use uv 0.2.35 or newer. | | `*.py.lock` with its PEP 723 `*.py` script | Rewrites the lock and the script's uv source metadata together. | Rewrites the lock and script metadata together and commits the patched wheel. | | `pylock.toml` and `pylock..toml`, PEP 751 `lock-version = "1.0"` | Uses one `archive` URL with the patched SHA-256. | Uses one `archive` path with the committed wheel's SHA-256. | @@ -60,8 +60,21 @@ frozen, locked, and ordinary installation outcomes separately where supported. string source and silently ignore the lock (`--frozen` / `--locked` fail, an ordinary `uv sync` still installs the patch via the pyproject source), and 0.2.6–0.2.17 ignore an unexpected `[[distribution.wheel]]` and try to - build the direct wheel URL as a source archive. The `[[distribution]]` - grammar is hosted-only: vendored native wiring is refused with + build the direct wheel URL as a source archive. The hosted unwind + (`rollback`, `remove`, and `scan` dropping a pin) restores the registry + entry in the shape the entry itself carries. Measured with real binaries + (macOS arm64, CPython 3.11): every release from 0.1.45 to 0.2.34 records + each file of a release that the project's `requires-python` admits, on + every platform (releases up to somewhere between 0.2.19 and 0.2.25 record + every wheel; later ones drop the wheels `requires-python` excludes), so a + pure-Python 3 release is re-derivable. uv 0.2.18–0.2.22 write an entry's `sdist` before its inline + `dependencies` and 0.2.23–0.2.34 after them, which only another registry + entry carrying both shows. `uv lock` takes no `exclude-newer` through + 0.2.5, takes `--exclude-newer` from 0.2.6, and from 0.2.12 also + `UV_EXCLUDE_NEWER` and `exclude-newer` in `[tool.uv]` or a project or user + `uv.toml`. It records the cutoff only from 0.2.28 (at the lock's top level, + under `[options]` from 0.2.34). The `[[distribution]]` grammar is + hosted-only: vendored native wiring is refused with `pypi_uv_legacy_lock_unsupported`. The refusal is kept because no spelling of a committed wheel is stable across that era: through 0.2.6 a relative path source cannot be parsed at all (`path+` is an invalid URL, `path+file:`