diff --git a/crates/socket-patch-cli/CLI_CONTRACT.md b/crates/socket-patch-cli/CLI_CONTRACT.md index a3ed909d6..3ea1822be 100644 --- a/crates/socket-patch-cli/CLI_CONTRACT.md +++ b/crates/socket-patch-cli/CLI_CONTRACT.md @@ -579,8 +579,8 @@ A reverted lockfile plus a leftover ledger or artifact therefore stops attesting | `patched_ref_invalid` | A Socket-shaped reference failed validation or is not live wiring: unsafe coordinates, a non-canonical uuid, a leaf naming another package, a path escaping the root, an inert or orphaned entry. | | `patched_ref_unattributable` | A Socket patch uuid cannot be tied to exactly one artifact / version. Examples: a maven repository no pin names, a nuget source without a mapping, a lock contested by another lock. | | `sbt_owned_file_modified` | `socket-patch.sbt` / `socket-patch-vendor.sbt` is not byte-for-byte what socket-patch generates; none of its pins is discovered. | -| `sbt_resolution_unverified` | An sbt pin the build's local resolution evidence (`target/`, after `sbt update`) does not verify: the reference stays, without the lockfile basis. | -| `vendored_tree_missing` | A `socket-patch-vendor.sbt` pin whose `.socket/vendor/maven2/` file is missing or modified. | +| `sbt_resolution_unverified` | An sbt pin the build's local resolution evidence (`target/`, after `sbt update`) does not verify, or a build-wide blocker applies (stale or partial evidence, project definitions that cannot be read statically, a `dependencyOverrides` / `resolvers` reassignment, a `build.sbt.lock`); the detail names the cause and its remedy, or that it has none. A hosted reference stays, without the lockfile basis; a vendored pin is not attested (`vex_sbt_resolution_unverified`). | +| `vendored_tree_missing` | A `socket-patch-vendor.sbt` pin whose `.socket/vendor/maven2/` file is missing or modified; it is not attested (`vex_sbt_vendored_tree_missing`). | Human mode also prints `Note:` lines: superseded records, fetch failures, `--offline` withholding fetches, conflict details, and why a claim is dead. While patch records are fetched, a terminal shows a transient `Fetching patch records... (n/N)` status line on stderr (never under `--json` / `--silent`). @@ -2678,6 +2678,8 @@ match it, withholds the statement: | `vex_deno_lock_copy` | run warning and `failed[].reason` | An npm pin is wired, but `deno.lock` locks the same `name@version`, which `deno install` installs without the wiring; no statement while it does. `vendor --check` and `scan` still treat the wiring as live. | | `vex_maven_reactor_root` | run warning and `failed[].reason` | A hosted Maven pin sits in a root `pom.xml` that declares `` / `` (a profile's included), where a module's own literal `` overrides it; only the root pom is read, so no statement (#261). `list`, `rollback` and `remove` still find the pin. | | `vex_maven_classifier_unpatched` | run warning and `failed[].reason` | A hosted Maven pin's GA also has a live executable `` copy (not `sources` / `javadoc`) at a version other than the pin, which the classifier-less pin does not retarget and the grant does not serve; no statement while it does. `list`, `rollback` and `remove` still find the pin. | +| `vex_sbt_resolution_unverified` | run warning and `failed[].reason` | A vendored sbt pin (`socket-patch-vendor.sbt`) whose resolution the build's local evidence does not verify, by the same check and build-wide blockers as a hosted sbt pin (no evidence or evidence older than a build source, a declared project with none, a `dependencyOverrides` / `resolvers` reassignment, a `build.sbt.lock`, project definitions that cannot be read statically, another version or other bytes resolved); no statement while it holds. The note names the cause and its remedy: `sbt update` clears stale or missing evidence, keeping the setting's existing entries (appending with `+=` / `++=` in place of a `:=`, `~=` or `--=`) clears a reassignment, removing what forces another version clears a version conflict, and a build whose projects cannot be read statically (a computed `Project(...)`, `projectMatrix`, `crossProject`) is never attested. `vendor --check` and `scan` still treat the wiring as live. | +| `vex_sbt_vendored_tree_missing` | run warning and `failed[].reason` | A vendored sbt pin whose committed `.socket/vendor/maven2/` file is missing or does not hash to the pinned sha256 (sbt's load-time check fails the build); no statement until the tree is restored. | | `vex_gradle_derived_cache_unchecked` | run warning | The derived-cache walk was cut short (a very large transforms cache); the statement is still emitted, the copies the walk did reach were checked. | A vendored Gradle entry attests only while its wiring is live (apply line, index rows, diff --git a/crates/socket-patch-cli/src/commands/vex_sources.rs b/crates/socket-patch-cli/src/commands/vex_sources.rs index 25e49273e..b39f5e21e 100644 --- a/crates/socket-patch-cli/src/commands/vex_sources.rs +++ b/crates/socket-patch-cli/src/commands/vex_sources.rs @@ -232,6 +232,14 @@ const NOTE_MAVEN_REACTOR_ROOT: &str = "vex_maven_reactor_root"; /// classifier copy at another version, which the pin does not reach /// (`vex::Unattested`). const NOTE_MAVEN_CLASSIFIER_UNPATCHED: &str = "vex_maven_classifier_unpatched"; +/// Omission tag and note: a vendored sbt pin the build's local resolution +/// evidence does not verify, or under a build-wide blocker +/// (`vex::Unattested`, `UnattestedKind::SbtResolutionUnverified`). +const NOTE_SBT_RESOLUTION_UNVERIFIED: &str = "vex_sbt_resolution_unverified"; +/// Omission tag and note: a vendored sbt pin's committed tree file is +/// missing or modified (`vex::Unattested`, +/// `UnattestedKind::SbtVendoredTreeMissing`). +const NOTE_SBT_VENDORED_TREE_MISSING: &str = "vex_sbt_vendored_tree_missing"; /// The omission tag of an [`Unattested`](socket_patch_core::vex::Unattested) /// ref, and the remedy its note ends with. @@ -268,9 +276,53 @@ fn unattested_note(kind: UnattestedKind) -> (&'static str, &'static str) { "not attested while that classifier copy resolves an unpatched version; the \ wiring itself is intact, so re-running `scan` does not change this", ), + UnattestedKind::SbtResolutionUnverified => ( + NOTE_SBT_RESOLUTION_UNVERIFIED, + "not attested while that holds (the detail names the remedy, or that there is \ + none); the wiring itself is intact, so `vendor --check` and `scan` still treat it \ + as live", + ), + UnattestedKind::SbtVendoredTreeMissing => ( + NOTE_SBT_VENDORED_TREE_MISSING, + "not attested while the committed tree differs from the pin (restore \ + `.socket/vendor/maven2` from version control, or re-run `socket-patch vendor`)", + ), } } +/// The [`Unattested`](socket_patch_core::vex::Unattested) mark discovery +/// put on patch `uuid` for `purl`'s package, if any. +fn unattested_mark<'d>( + discovery: &'d Discovery, + purl: &str, + uuid: &str, +) -> Option<&'d socket_patch_core::vex::Unattested> { + let pkg = canonical_base_purl(purl); + discovery + .unattested + .iter() + .find(|u| u.uuid == uuid && PurlKey::same(&u.purl, &pkg)) +} + +/// Omit `key` for the mark `u`: its `failed[]` entry and the note naming +/// why and the remedy. +fn gate_unattested( + key: &str, + u: &socket_patch_core::vex::Unattested, + gated: &mut Vec, + notes: &mut Vec, +) { + let (code, remedy) = unattested_note(u.kind); + gated.push(failed(key, code)); + notes.push(PlanNote { + code, + detail: format!( + "{key}: patch {} is wired, but {}; {remedy}", + u.uuid, u.detail + ), + }); +} + /// One hosted-basis view purl's wiring (see [`Plan::hosted`]). #[derive(Debug, Clone, PartialEq, Eq)] pub(crate) struct HostedWiring { @@ -422,23 +474,10 @@ pub(crate) async fn plan( // rollback, remove, list and the ledgers' liveness working, and the // patch is omitted. cands.retain(|c| { - let pkg = canonical_base_purl(&c.key); - let Some(u) = discovery - .unattested - .iter() - .find(|u| u.uuid == c.uuid && PurlKey::same(&u.purl, &pkg)) - else { + let Some(u) = unattested_mark(&discovery, &c.key, &c.uuid) else { return true; }; - let (code, remedy) = unattested_note(u.kind); - gated.push(failed(&c.key, code)); - notes.push(PlanNote { - code, - detail: format!( - "{}: patch {} is wired, but {}; {remedy}", - c.key, c.uuid, u.detail - ), - }); + gate_unattested(&c.key, u, &mut gated, &mut notes); false }); for (key, old_uuid, wired) in &superseded { @@ -471,6 +510,7 @@ pub(crate) async fn plan( continue; } } + let mut took_over = false; let basis = if let Some(vref) = cand .discovered .iter() @@ -507,6 +547,7 @@ pub(crate) async fn plan( cand.uuid = hosted.uuid.clone(); cand.record = Some(hosted); cand.vendor_entry = None; + took_over = true; Basis::Hosted { lockfile_basis: false, } @@ -542,6 +583,21 @@ pub(crate) async fn plan( } else { Basis::Installed }; + // The vendored basis attests the ENTRY's patch, which need not be + // the candidate's uuid (a manifest record under another uuid keeps + // the key; a vendored sbt pin has no ref to swap it in): its mark + // is checked against what is actually attested. + // Likewise the hosted-over-vendored takeover attests the redirect + // ledger's uuid, swapped in after the early gate ran. + let attested = match &basis { + Basis::Vendored(entry) => Some((entry.base_purl.as_str(), entry.uuid.as_str())), + Basis::Hosted { .. } if took_over => Some((cand.key.as_str(), cand.uuid.as_str())), + _ => None, + }; + if let Some(u) = attested.and_then(|(purl, uuid)| unattested_mark(&discovery, purl, uuid)) { + gate_unattested(&cand.key, u, &mut gated, &mut notes); + continue; + } based.push((cand, basis)); } @@ -1542,6 +1598,29 @@ mod tests { let dead = plan(&common(tmp.path()), mk(), &[], &RunApiClient::new()).await; assert!(dead.view.patches.is_empty()); assert_eq!(dead.gated, vec![failed(key, VENDOR_UNWIRED)]); + + // The takeover attests the redirect ledger's uuid, swapped in after + // the early mark gate ran: a mark on it still gates. + let mut marked = mk(); + marked + .discovery + .unattested + .push(socket_patch_core::vex::Unattested { + purl: key.into(), + uuid: U2.into(), + file: "package-lock.json".into(), + detail: "package-lock.json bypasses it".into(), + kind: UnattestedKind::LockAboveBase, + }); + let gated = plan( + &common(tmp.path()), + marked, + &[key.to_string()], + &RunApiClient::new(), + ) + .await; + assert!(gated.view.patches.is_empty(), "{:?}", gated.view.patches); + assert_eq!(gated.gated, vec![failed(key, NOTE_LOCK_ABOVE_BASE)]); } /// Core discover rule 11 through the plan: a vendor ledger entry and a diff --git a/crates/socket-patch-cli/tests/e2e_vex_lockfile/sbt_vendored.rs b/crates/socket-patch-cli/tests/e2e_vex_lockfile/sbt_vendored.rs index 535f44529..5e79b5e8c 100644 --- a/crates/socket-patch-cli/tests/e2e_vex_lockfile/sbt_vendored.rs +++ b/crates/socket-patch-cli/tests/e2e_vex_lockfile/sbt_vendored.rs @@ -5,8 +5,9 @@ //! //! The suffixed tree path carries no full uuid, so the generated file is no //! `.socket/vendor///` reference: a vendored sbt pin is attested -//! through the vendor ledger `vendor` writes, and the extractor diagnoses -//! what would make that attestation a lie. Each cell vendors for real +//! through the vendor ledger `vendor` writes, and only when the extractor +//! finds nothing that would make that attestation a lie (the hosted sbt +//! evidence gate). Each cell vendors for real //! (`vendor` over the sbt 1.13.0 probe build, the installed jar in an //! isolated Coursier cache), drops `.socket/manifest.json`, and runs //! `vex --offline` (the ledger embeds the record): @@ -14,12 +15,14 @@ //! | cell | state | expected | //! |---|---|---| //! | a | after `sbt update` (the evidence resolves the suffixed version from the tree) | attested, no sbt diagnostic | -//! | b | before `sbt update` (the evidence still resolves the base version) | `sbt_resolution_unverified` | +//! | b | before `sbt update` (the evidence still resolves the base version) | `sbt_resolution_unverified`, `vex_sbt_resolution_unverified`, not attested | //! | c | a tree jar tampered | `vendored_tree_missing`, not attested | //! | d | the generated file edited | `sbt_owned_file_modified` | //! | e | no ledger | nothing attested; the tree check still runs | +//! | f | after `sbt update`, then a `dependencyOverrides :=` | `vex_sbt_resolution_unverified` naming it, not attested | +//! | g | before `sbt update`, the manifest back (same or another uuid) | `vex_sbt_resolution_unverified`, not attested | -use crate::common::envelope::all_codes; +use crate::common::envelope::{all_codes, warning_codes}; use crate::common::git_sha256; use std::path::{Path, PathBuf}; @@ -42,6 +45,8 @@ const CENTRAL: &str = "repo1.maven.org/maven2"; const MEMBER: &str = "META-INF/NOTICE.txt"; const VULN: &str = "GHSA-sbtv-test-xxxx"; const VENDORED_FILE: &str = "socket-patch-vendor.sbt"; +/// A manifest uuid other than the vendored one. +const OTHER_UUID: &str = "9e8d7c6b-5a49-4382-9f0e-2a3b4c5d6e7f"; fn tree_jar(proj: &Path) -> PathBuf { proj.join(".socket/vendor/maven2/com/google/code/gson/gson") @@ -69,6 +74,8 @@ struct Fx { _tmp: tempfile::TempDir, proj: PathBuf, home: SbtHome, + /// The manifest `vendor` ran with. + manifest: Value, } impl Fx { @@ -111,6 +118,7 @@ impl Fx { _tmp: tmp, proj, home, + manifest, }; let (code, env) = fx.run(&["vendor"]); assert_eq!(code, Some(0), "vendor: {env}"); @@ -120,6 +128,17 @@ impl Fx { fx } + /// Put the manifest back, its gson record filed under `uuid`. + fn restore_manifest(&self, uuid: &str) { + let mut manifest = self.manifest.clone(); + manifest["patches"][GSON.purl()]["uuid"] = uuid.into(); + std::fs::write( + self.proj.join(".socket/manifest.json"), + serde_json::to_vec_pretty(&manifest).unwrap(), + ) + .unwrap(); + } + /// What `sbt update` records after vendoring. fn sbt_update(&self) { let changed = record_pinned_resolution( @@ -197,6 +216,17 @@ fn attested(doc: Option<&Value>) -> bool { }) } +/// The details of the top-level warnings coded `code`. +fn warning_details(env: &Value, code: &str) -> Vec { + env["warnings"] + .as_array() + .into_iter() + .flatten() + .filter(|w| w["code"] == code) + .filter_map(|w| w["detail"].as_str().map(str::to_string)) + .collect() +} + const SBT_DIAGS: &[&str] = &[ "sbt_resolution_unverified", "vendored_tree_missing", @@ -220,11 +250,66 @@ fn sbt_vendored_vex_attests_after_sbt_update() { #[test] fn sbt_vendored_vex_before_sbt_update_is_unverified() { let fx = Fx::vendored(); - let (_, env, _) = fx.vex(); + let (_, env, doc) = fx.vex(); + let got = all_codes(&env); + assert!( + got.contains(&"sbt_resolution_unverified".to_string()), + "{env}" + ); + assert!( + got.contains(&"vex_sbt_resolution_unverified".to_string()), + "{env}" + ); + assert!( + !attested(doc.as_ref()), + "an unverified resolution is never attested: {doc:?}" + ); +} + +#[test] +fn sbt_vendored_vex_withholds_under_an_overrides_reassignment() { + let fx = Fx::vendored(); + fx.sbt_update(); + // Dated back to its old time, the edit stays older than the evidence + // however slow the run (staleness would withhold the pin too). + let build = fx.proj.join("build.sbt"); + let dated = std::fs::metadata(&build).unwrap().modified().unwrap(); + let mut text = std::fs::read_to_string(&build).unwrap(); + text.push_str("ThisBuild / dependencyOverrides := Seq()\n"); + std::fs::write(&build, text).unwrap(); + std::fs::File::options() + .write(true) + .open(&build) + .unwrap() + .set_modified(dated) + .unwrap(); + let (_, env, doc) = fx.vex(); + let details = warning_details(&env, "vex_sbt_resolution_unverified"); assert!( - all_codes(&env).contains(&"sbt_resolution_unverified".to_string()), + details.iter().any(|d| d.contains("`dependencyOverrides`")), "{env}" ); + assert!( + !attested(doc.as_ref()), + "an overrides reassignment replaces the pin: {doc:?}" + ); +} + +/// #2008 review: the manifest still records gson, under the vendored uuid +/// or another one (the vendor entry, not the candidate's uuid, is what the +/// vendored basis attests). Before `sbt update` neither attests. +#[test] +fn sbt_vendored_vex_with_a_manifest_still_needs_evidence() { + let fx = Fx::vendored(); + for uuid in [UUID, OTHER_UUID] { + fx.restore_manifest(uuid); + let (_, env, doc) = fx.vex(); + assert!( + warning_codes(&env).contains(&"vex_sbt_resolution_unverified".to_string()), + "{uuid}: {env}" + ); + assert!(!attested(doc.as_ref()), "{uuid}: {doc:?}"); + } } #[test] diff --git a/crates/socket-patch-core/src/crawlers/sbt_evidence.rs b/crates/socket-patch-core/src/crawlers/sbt_evidence.rs index 009ce3929..9dd8c0987 100644 --- a/crates/socket-patch-core/src/crawlers/sbt_evidence.rs +++ b/crates/socket-patch-core/src/crawlers/sbt_evidence.rs @@ -634,3 +634,36 @@ pub(crate) fn stage_fixture(from: &Path, to: &Path) { #[cfg(test)] mod tests; + +/// Helpers for tests that stage sbt evidence on disk (this module's and +/// `vex::discover::sbt`'s). +#[cfg(test)] +pub(crate) mod testing { + use std::path::Path; + use std::time::SystemTime; + + /// Date `path` at `t`. + pub(crate) fn set_mtime(path: &Path, t: SystemTime) { + std::fs::OpenOptions::new() + .write(true) + .open(path) + .unwrap() + .set_modified(t) + .unwrap(); + } + + /// The `file:` URI sbt writes for `path`: `file:///C:/x` on Windows (a + /// backslash is no URI separator, and an invalid JSON escape). `path` + /// must not be a verbatim (`\\?\`) path, which sbt never records. + pub(crate) fn file_uri(path: &Path) -> String { + let text = path + .to_string_lossy() + .replace('\\', "/") + .replace(' ', "%20"); + if text.starts_with('/') { + format!("file://{text}") + } else { + format!("file:///{text}") + } + } +} diff --git a/crates/socket-patch-core/src/crawlers/sbt_evidence/tests.rs b/crates/socket-patch-core/src/crawlers/sbt_evidence/tests.rs index d2a78133b..44150e514 100644 --- a/crates/socket-patch-core/src/crawlers/sbt_evidence/tests.rs +++ b/crates/socket-patch-core/src/crawlers/sbt_evidence/tests.rs @@ -5,6 +5,7 @@ use std::fs; use std::path::{Path, PathBuf}; use std::time::{Duration, SystemTime}; +use super::testing::{file_uri, set_mtime}; use super::*; const VERSIONS: &[&str] = &["0.13.18", "1.2.8", "1.3.13", "1.9.9", "1.13.0", "2.0.9"]; @@ -32,15 +33,6 @@ fn write(root: &Path, rel: &str, body: &[u8]) -> PathBuf { path } -fn set_mtime(path: &Path, t: SystemTime) { - fs::OpenOptions::new() - .write(true) - .open(path) - .unwrap() - .set_modified(t) - .unwrap(); -} - const JSON: &str = r#"{"configurations":[{"configuration":{"name":"compile"},"modules":[{"module":{"organization":"g","name":"a","revision":"1"},"artifacts":[],"evicted":false}],"details":[]}]}"#; fn rels(e: &SbtEvidence) -> Vec<&str> { @@ -346,20 +338,6 @@ fn symlinks_are_never_followed() { assert_eq!(e.build_sources.len(), 1); } -/// The `file:` URI sbt writes for `path`: `file:///C:/x` on Windows (a -/// backslash is no URI separator, and an invalid JSON escape). -fn file_uri(path: &Path) -> String { - let text = path - .to_string_lossy() - .replace('\\', "/") - .replace(' ', "%20"); - if text.starts_with('/') { - format!("file://{text}") - } else { - format!("file:///{text}") - } -} - #[test] fn cache_roots_from_artifact_paths() { let caches = tempfile::tempdir().unwrap(); diff --git a/crates/socket-patch-core/src/vex/discover/mod.rs b/crates/socket-patch-core/src/vex/discover/mod.rs index 815d4644d..29b740e85 100644 --- a/crates/socket-patch-core/src/vex/discover/mod.rs +++ b/crates/socket-patch-core/src/vex/discover/mod.rs @@ -606,13 +606,25 @@ pub enum UnattestedKind { /// version: the classifier-less pin does not retarget it and the grant /// serves only the main jar, so the build runs the public classifier jar. MavenClassifierUnpatched, + /// A vendored sbt pin (`socket-patch-vendor.sbt`) whose resolution the + /// build's local evidence does not verify, or under a build-wide + /// blocker (stale or partial evidence, a `dependencyOverrides` / + /// `resolvers` reassignment, a `build.sbt.lock`): the same conditions + /// that withhold a hosted sbt ref's lock pin. The vendor ledger, not a + /// ref, carries this wiring, so the mark stands alone. + SbtResolutionUnverified, + /// A vendored sbt pin whose committed `.socket/vendor/maven2` tree file + /// is missing or does not hash to the pinned sha256: sbt's load-time + /// check fails the build closed, so no copy is the patch. + SbtVendoredTreeMissing, } /// A ref discovery emits (so rollback, remove and list find the wiring, /// and the ledgers' liveness gates still see it wired) that must not be /// attested: the files show a build that may run a copy the pin does not -/// reach ([`UnattestedKind`]). The CLI's VEX plan omits every candidate of -/// `(purl, uuid)` with `detail`. Unlike a contest +/// reach ([`UnattestedKind`]). A vendored sbt pin is marked with no ref +/// beside it (its vendor ledger entry is the claim). The CLI's VEX plan +/// omits every candidate of `(purl, uuid)` with `detail`. Unlike a contest /// ([`Discovery::unpatched_copy`], [`Discovery::contest_across_locks`]), /// this never drops the ref, so `vendor --check` and `scan` keep treating /// the wiring as live — the right answer when re-running `scan` / `vendor` @@ -694,7 +706,10 @@ pub struct Discovery { /// [`Discovery::contest_within_locks`] ([`Discovery::unidentified_copy`]); /// always empty once discovery returns (folded into `unpatched_copies`). pub unidentified_copies: Vec, - /// Refs in `refs` whose wiring a build bypasses ([`Unattested`]). + /// Wired patches that must not be attested ([`Unattested`]): a ref in + /// `refs` whose wiring a build bypasses, or a vendored sbt pin whose + /// claim is its vendor ledger entry (no ref at all). Consumers match a + /// mark by `(purl, uuid)`, never by a ref. pub unattested: Vec, /// Copies no wiring reaches, pending [`Discovery::unattest_unwired_copies`]; /// always empty once discovery returns (folded into `unattested`). @@ -1364,8 +1379,10 @@ impl Discovery { }); } - /// Record that the ref `(purl, uuid)` is wired but bypassed by the - /// build `file` shows ([`Unattested`]). Pushed beside the ref itself. + /// Record that patch `uuid` is wired for `purl` but must not be + /// attested, for the reason `file` shows ([`Unattested`]). Pushed + /// beside the ref itself, or, for a vendored sbt pin (whose claim is + /// the vendor ledger's, not a ref's), alone. pub(crate) fn unattested( &mut self, purl: &str, diff --git a/crates/socket-patch-core/src/vex/discover/sbt.rs b/crates/socket-patch-core/src/vex/discover/sbt.rs index f77b5f81e..ff7bcfd8d 100644 --- a/crates/socket-patch-core/src/vex/discover/sbt.rs +++ b/crates/socket-patch-core/src/vex/discover/sbt.rs @@ -19,9 +19,17 @@ //! first checkout's `file:` origin (`docs/design/sbt-template-probe.md`, //! case i), and a copy whose bytes are the pinned ones is the patch. //! -//! Vendored pins are attested by the vendor ledger; this extractor only -//! diagnoses what would make that attestation a lie: a tree file gone or -//! changed (`vendored_tree_missing`) or an unverified resolution. +//! Vendored pins are attested by the vendor ledger; this extractor holds +//! them to the same evidence a hosted pin needs. A tree file gone or +//! changed (`vendored_tree_missing`) or a resolution the evidence does not +//! verify (`sbt_resolution_unverified`, the hosted check and build-wide +//! blockers verbatim) is diagnosed AND marked [`Unattested`] for the pin's +//! `(purl, uuid)`, so `vex` omits the ledger entry. Its liveness is +//! untouched: the wiring is intact, and `scan --prune` must not revert a +//! pin only because `sbt update` has not run since. A memory view has no +//! committed tree to hash, so it reports none missing. +//! +//! [`Unattested`]: super::Unattested //! //! A generated file socket-patch cannot parse strictly is //! `sbt_owned_file_modified` (a foreign file under the name is not ours and @@ -30,7 +38,7 @@ use std::collections::BTreeMap; use std::path::{Path, PathBuf}; -use super::{DiscoverCtx, Discovery, PatchedRef, DIAG_REF_INVALID}; +use super::{DiscoverCtx, Discovery, PatchedRef, UnattestedKind, DIAG_REF_INVALID}; use crate::formats::sbt::owned_file::{ parse, OwnedFileError, SbtFileMode, SbtPin, HOSTED_FILE, VENDORED_FILE, VENDORED_REPO_REL, }; @@ -97,7 +105,9 @@ async fn evidence(root: &Path) -> (Option, Option) { /// declared project with no evidence (only part of the build resolved), a /// project definition that cannot be read statically, a build source /// reassigning `dependencyOverrides` (it replaces the generated override -/// where it applies) or `resolvers`, or a `build.sbt.lock`. +/// where it applies) or `resolvers`, or a `build.sbt.lock`. Each reason +/// names its own remedy (or that there is none), so a reader can relay it +/// as is. fn evidence_blocker( e: &crate::crawlers::sbt_evidence::SbtEvidence, res: &JvmResolution, @@ -110,8 +120,10 @@ fn evidence_blocker( } let Some(declared) = declared_projects(&e.build_sources) else { return Some( - "the build's project definitions cannot be read statically, so its evidence may \ - not cover every project" + "the build's project definitions cannot be read statically (a computed \ + `Project(...)`, `projectMatrix` or `crossProject`), so its evidence may not cover \ + every project; only a build whose projects are literal `project` / \ + `Project(\"\", file(\"\"))` definitions can be verified" .to_string(), ); }; @@ -129,19 +141,24 @@ fn evidence_blocker( let findings = scan_build_sources(&e.build_sources); if !findings.overrides_assignment.is_empty() { return Some(format!( - "{} reassign(s) `dependencyOverrides`, which replaces the pin where it applies", + "{} reassign(s) or filter(s) `dependencyOverrides` (`:=`, `~=` or `--=`), which can \ + replace or drop the pin where it applies; keep the setting's existing entries \ + (append with `+=` / `++=`)", findings.overrides_assignment.join(", ") )); } if !findings.resolvers_assignment.is_empty() { return Some(format!( - "{} reassign(s) `resolvers`, which can drop the pin's repository", + "{} reassign(s) or transform(s) a resolver setting (`resolvers`, \ + `externalResolvers` or `fullResolvers`, with `:=` or `~=`), which can drop the \ + pin's repository; keep the setting's existing entries (append with `+=` / `++=`)", findings.resolvers_assignment.join(", ") )); } if !findings.dependency_lock.is_empty() { return Some(format!( - "{} locks the build's dependencies, which rejects the pinned version", + "{} locks the build's dependencies, which rejects the pinned version; the pin \ + cannot be verified while it exists", findings.dependency_lock.join(", ") )); } @@ -217,7 +234,9 @@ async fn verify( } if let Some(other) = versions.iter().find(|v| **v != pin.sv) { return Err(format!( - "the build resolves {ga}:{other}, not the pinned {}", + "the build resolves {ga}:{other}, not the pinned {}; run `sbt update`, and if it \ + still does, something in the build forces that version (a `dependencyOverrides \ + +=` or `force()` of it, or a project the pin does not reach): remove it", pin.sv )); } @@ -229,7 +248,8 @@ async fn verify( .collect(); if paths.is_empty() { return Err(format!( - "the build's resolution evidence records no artifact file for {ga}:{}", + "the build's resolution evidence records no artifact file for {ga}:{}, so it \ + cannot show which copy the build uses; run `sbt update`", pin.sv )); } @@ -240,14 +260,17 @@ async fn verify( Some(got) if got == pin.pom_sha256 => {} Some(got) => { return Err(format!( - "{ga}:{} resolved to {} with sha256 {got}, which is not pinned", + "{ga}:{} resolved to {} with sha256 {got}, which is not pinned (a copy \ + another repository serves, such as `mavenLocal` or an Ivy-local publish, \ + or a modified file): remove or restore it, then run `sbt update`", pin.sv, path.display() )) } None => { return Err(format!( - "{ga}:{} resolved to {}, which cannot be read", + "{ga}:{} resolved to {}, which cannot be read; restore it or run \ + `sbt update`", pin.sv, path.display() )) @@ -256,7 +279,7 @@ async fn verify( } if !jar_seen { return Err(format!( - "no resolved artifact of {ga}:{} is the pinned jar", + "no resolved artifact of {ga}:{} is the pinned jar; run `sbt update`", pin.sv )); } @@ -326,37 +349,65 @@ async fn vendored_check( hashes: &mut Hashes, out: &mut Discovery, ) { - for (ext, want) in [("pom", &pin.pom_sha256), ("jar", &pin.jar_sha256)] { - let rel = format!("{VENDORED_REPO_REL}/{}", pin.repo_path(ext)); - let got = match ctx.disk_root() { - Some(root) => hashes.of(&root.join(&rel)).await, - None => None, - }; - if got.as_deref() != Some(want.as_str()) { - out.diag( - DIAG_VENDORED_TREE_MISSING, - VENDORED_FILE, - format!( - "{VENDORED_FILE}: patch {} pins {rel}, which is {}", - pin.uuid, - if got.is_some() { "modified" } else { "missing" } - ), - ); - return; - } - } - if let Err(why) = verify_with(pin, res, blocker, hashes).await { + // The vendor ledger records the pin under this purl (the planner builds + // it the same way); a coordinate it rejects was never vendored. `why` + // is both the diagnostic's text and the mark's detail, which completes + // "patch is wired, but …" (the VEX note). + let purl = maven_purl(&pin.group, &pin.artifact, &pin.base); + let withhold = |out: &mut Discovery, code, why: String, kind| { out.diag( - DIAG_SBT_RESOLUTION_UNVERIFIED, + code, VENDORED_FILE, format!("{VENDORED_FILE}: patch {}: {why}", pin.uuid), ); + if let Some(purl) = &purl { + out.unattested(purl, &pin.uuid, VENDORED_FILE, why, kind); + } + }; + // Only a disk holds the committed tree: a memory view has none to + // hash, which proves nothing missing. + if let Some(root) = ctx.disk_root() { + for (ext, want) in [("pom", &pin.pom_sha256), ("jar", &pin.jar_sha256)] { + let rel = format!("{VENDORED_REPO_REL}/{}", pin.repo_path(ext)); + let got = hashes.of(&root.join(&rel)).await; + if got.as_deref() != Some(want.as_str()) { + let state = if got.is_some() { "modified" } else { "missing" }; + withhold( + out, + DIAG_VENDORED_TREE_MISSING, + format!("{VENDORED_FILE} pins {rel}, which is {state}"), + UnattestedKind::SbtVendoredTreeMissing, + ); + return; + } + } + } + // Without a disk there is no evidence either: unverified, exactly as a + // hosted pin read from memory gets no lock pin (and `sbt update` cannot + // change that, so the detail says why instead). + let verdict = if ctx.disk_root().is_none() { + Err( + "the project is read from memory, which holds no sbt resolution evidence; \ + only a scan of the checkout on disk can verify the pin" + .to_string(), + ) + } else { + verify_with(pin, res, blocker, hashes).await + }; + if let Err(why) = verdict { + withhold( + out, + DIAG_SBT_RESOLUTION_UNVERIFIED, + why, + UnattestedKind::SbtResolutionUnverified, + ); } } #[cfg(test)] mod tests { use super::*; + use crate::crawlers::sbt_evidence::testing::{file_uri, set_mtime}; use sha2::{Digest as _, Sha256}; const UUID: &str = "abcdef12-0000-4000-8000-000000000001"; @@ -504,4 +555,229 @@ mod tests { Ok(()) ); } + + // ── vendored pins: the hosted evidence gate, end to end ──────────── + + const GSON_UUID: &str = "1d3c1fd2-5e6f-4a7b-8c9d-0e1f2a3b4c5d"; + const GSON_PURL: &str = "pkg:maven/com.google.code.gson/gson@2.8.9"; + const GSON_BUILD: &str = + "libraryDependencies += \"com.google.code.gson\" % \"gson\" % \"2.8.9\"\n"; + const EVIDENCE: &str = "target/update/update_cache/output"; + + fn later(secs: u64) -> std::time::SystemTime { + std::time::SystemTime::now() + std::time::Duration::from_secs(secs) + } + + fn earlier(secs: u64) -> std::time::SystemTime { + std::time::SystemTime::now() - std::time::Duration::from_secs(secs) + } + + /// An sbt 1.9.9 build with gson vendored for real (generated file, + /// committed tree, ledger) and, unless `evidence` is false, the record + /// `sbt update` writes after it: the suffixed version resolved from the + /// tree jar, newer than every build source. `(tmp, root, pin)`. + async fn vendored_gson(evidence: bool) -> (tempfile::TempDir, PathBuf, SbtPin) { + use crate::vendor::jvm::testing::{populate, vendor}; + use crate::vendor::jvm::{JvmPatch, Shape}; + // The raw tempdir path: a canonical Windows path is verbatim + // (`\\?\C:\…`), which sbt never records. + let tmp = tempfile::tempdir().unwrap(); + let root = tmp.path().to_path_buf(); + populate( + &root, + &[ + ("project/build.properties", "sbt.version=1.9.9\n"), + ("build.sbt", GSON_BUILD), + ], + ); + let patch = JvmPatch { + group_id: "com.google.code.gson", + artifact_id: "gson", + version: "2.8.9", + uuid: GSON_UUID, + jar: b"PK-gson", + upstream_pom: b"\n com.google.code.gson\n \ + gson\n 2.8.9\n\n", + upstream_module: None, + extra_artifacts: &[], + patched_members: &[], + }; + let mut ledger = BTreeMap::new(); + vendor(&root, Shape::Sbt, &patch, &mut ledger) + .await + .expect("vendor gson"); + let text = std::fs::read_to_string(root.join(VENDORED_FILE)).unwrap(); + let pin = parse(SbtFileMode::Vendored, &text) + .unwrap() + .pins + .remove(GSON_UUID) + .expect("gson pin"); + if evidence { + let jar = root.join(VENDORED_REPO_REL).join(pin.repo_path("jar")); + assert!(jar.is_file(), "{}", jar.display()); + let json = format!( + r#"{{"configurations":[{{"configuration":{{"name":"compile"}},"modules":[{{"module":{{"organization":"com.google.code.gson","name":"gson","revision":"{}"}},"artifacts":[[{{"name":"gson"}},"{}"]],"evicted":false}}],"details":[]}}]}}"#, + pin.sv, + file_uri(&jar) + ); + populate(&root, &[(EVIDENCE, json.as_str())]); + set_mtime(&root.join(EVIDENCE), later(60)); + } + (tmp, root, pin) + } + + /// The `(purl, uuid, kind)` of every [`super::super::Unattested`] mark. + fn marks(out: &Discovery) -> Vec<(String, String, UnattestedKind)> { + out.unattested + .iter() + .map(|u| (u.purl.clone(), u.uuid.clone(), u.kind)) + .collect() + } + + fn unverified() -> Vec<(String, String, UnattestedKind)> { + vec![( + GSON_PURL.to_string(), + GSON_UUID.to_string(), + UnattestedKind::SbtResolutionUnverified, + )] + } + + /// Fresh evidence resolving the pinned bytes from the tree: no + /// diagnostic, nothing withheld (the ledger attests it). + #[tokio::test] + async fn vendored_pin_with_verifying_evidence_is_attestable() { + let (_tmp, root, _) = vendored_gson(true).await; + let out = super::super::discover_patched_refs(&root).await; + assert!(out.diagnostics.is_empty(), "{:#?}", out.diagnostics); + assert!(out.unattested.is_empty(), "{:#?}", out.unattested); + } + + /// The owner rule: a vendored sbt pin needs the evidence a hosted one + /// does, so with none (no `sbt update` since vendoring) the ledger + /// entry is withheld from VEX, not only diagnosed. + #[tokio::test] + async fn vendored_pin_without_evidence_is_unattested() { + let (_tmp, root, _) = vendored_gson(false).await; + let out = super::super::discover_patched_refs(&root).await; + assert_eq!(marks(&out), unverified(), "{:#?}", out.diagnostics); + assert!( + out.unattested[0] + .detail + .contains("no sbt resolution evidence"), + "{:?}", + out.unattested[0] + ); + } + + /// Evidence showing the pinned bytes, but older than a build source + /// edited since: it proves nothing about the build as it stands. + #[tokio::test] + async fn vendored_pin_with_stale_evidence_is_unattested() { + let (_tmp, root, _) = vendored_gson(true).await; + set_mtime(&root.join("build.sbt"), later(120)); + let out = super::super::discover_patched_refs(&root).await; + assert_eq!(marks(&out), unverified(), "{:#?}", out.diagnostics); + assert!( + out.unattested[0].detail.contains("changed after the last"), + "{:?}", + out.unattested[0] + ); + } + + /// A project reassigning `dependencyOverrides` replaces the generated + /// override where it applies, whatever the (fresh) evidence shows. + #[tokio::test] + async fn vendored_pin_under_an_overrides_reassignment_is_unattested() { + let (_tmp, root, _) = vendored_gson(true).await; + std::fs::write( + root.join("build.sbt"), + format!( + "{GSON_BUILD}dependencyOverrides := Seq(\"com.google.code.gson\" % \"gson\" % \ + \"2.8.9\")\n" + ), + ) + .unwrap(); + // Every build source still older than the evidence, however slow + // the run. + for rel in ["build.sbt", VENDORED_FILE, "project/build.properties"] { + set_mtime(&root.join(rel), earlier(60)); + } + let out = super::super::discover_patched_refs(&root).await; + assert_eq!(marks(&out), unverified(), "{:#?}", out.diagnostics); + assert!( + out.unattested[0].detail.contains("`dependencyOverrides`"), + "{:?}", + out.unattested[0] + ); + assert!( + out.diagnostics + .iter() + .any(|d| d.code == DIAG_SBT_RESOLUTION_UNVERIFIED), + "{:#?}", + out.diagnostics + ); + } + + /// A tree file gone: sbt's load-time check fails the build, so no + /// copy is the patch. + #[tokio::test] + async fn vendored_pin_with_a_missing_tree_file_is_unattested() { + let (_tmp, root, pin) = vendored_gson(true).await; + std::fs::remove_file(root.join(VENDORED_REPO_REL).join(pin.repo_path("pom"))).unwrap(); + let out = super::super::discover_patched_refs(&root).await; + assert_eq!( + marks(&out), + vec![( + GSON_PURL.to_string(), + GSON_UUID.to_string(), + UnattestedKind::SbtVendoredTreeMissing, + )], + "{:#?}", + out.diagnostics + ); + assert!( + out.unattested[0].detail.ends_with(".pom, which is missing") + && out.unattested[0].detail.starts_with(VENDORED_FILE), + "{:?}", + out.unattested[0] + ); + // The diagnostic and the mark say the same thing. + let diag = out + .diagnostics + .iter() + .find(|d| d.code == DIAG_VENDORED_TREE_MISSING) + .expect("diagnosed"); + assert!(diag.detail.ends_with(&out.unattested[0].detail), "{diag:?}"); + } + + /// A memory view holds no committed tree to hash: no tree is reported + /// missing (a gating mark discovery cannot back), while the pin is + /// still unverified, as a hosted pin read from memory gets no lock pin. + #[tokio::test] + async fn vendored_pin_over_a_memory_view_reports_no_missing_tree() { + let (_tmp, root, _) = vendored_gson(true).await; + let mut project = crate::vendor::lock_inventory::MemoryProject::new(); + for rel in [VENDORED_FILE, "build.sbt", "project/build.properties"] { + project.insert_text(rel, std::fs::read_to_string(root.join(rel)).unwrap()); + } + let out = super::super::discover_patched_refs_view( + crate::vendor::lock_inventory::ProjectView::Memory(&project), + &super::super::DiscoverOptions::default(), + ) + .await; + assert!( + !out.diagnostics + .iter() + .any(|d| d.code == DIAG_VENDORED_TREE_MISSING), + "{:#?}", + out.diagnostics + ); + assert_eq!(marks(&out), unverified(), "{:#?}", out.diagnostics); + // `sbt update` cannot fix a memory view: the detail says why. + assert!( + out.unattested[0].detail.contains("read from memory"), + "{:?}", + out.unattested[0] + ); + } } diff --git a/docs/ecosystems.md b/docs/ecosystems.md index b03dc6665..6f40ca93a 100644 --- a/docs/ecosystems.md +++ b/docs/ecosystems.md @@ -1261,7 +1261,15 @@ the repository list, so it is left as a - The in-memory hosted engine (GitHub App) has no `target/` evidence, so it never wires sbt. - Vendored sbt VEX attests through the vendor ledger; the generated file alone - is not a VEX reference. + is not a VEX reference. The ledger entry needs the same `target/` evidence + as a hosted pin, and is omitted as `vex_sbt_resolution_unverified` while + that evidence does not show the build resolving the pinned bytes: before + `sbt update` has run since vendoring or since a build source changed, + while a build source reassigns `dependencyOverrides` / a resolver setting with + `:=`, `~=` or `--=`, or a `build.sbt.lock` exists, while something forces another + version, and always for a build whose projects cannot be read statically + (`projectMatrix`, `crossProject`, a computed `Project(...)`). The note + names the cause and its remedy. ## Cargo: shared registry cache