Repository navigation
Plan one lock model for every lockfile reader - #2001
Mikola Lysenko (mikolalysenko) wants to merge 9 commits into
Conversation
Lockfile discovery (vex/discover), the lock inventory (vendor/lock_inventory) and the hosted and vendored writers each read the same project files their own way: two Socket-origin classifiers, three Python precedence rules, second walks of package-lock, yarn, vlt and pnpm locks, a second parse of uv/pylock/script locks, five Cargo.toml dependency-table walks, three readers of Maven's checksums.sha256. The inventory also has no NuGet, JVM, Hatch or Deno models. docs/design/lock-model-unification.md describes the target, one per-project model whose entries carry an Origin (Registry, Hosted, Vendored, Other, Rejected), file selection by install tree (precedence within a tree, union across trees), and the stack of PRs that gets there. scripts/loc-budget.py counts implementation lines (no comments, no cfg(test) items, no test-only modules) for two groups in scripts/loc-budget.json: lock-readers (30,898 lines today) and lock-writers (88,363). The lint-ecosystems job runs it with --warn, which annotates a group over budget but never fails, so bug fixes in these files are not blocked; each PR in the stack lowers the budget as it deletes code. No product code changes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
formats/ models are guarded to stay free of I/O and of the host policy (formats/mod.rs architecture_tests bans hosted_patch_uuid there), so the classifier is a top-level `origin` module and a project layer applies it. Spell out the three layers and where each part of the classifier moves from. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Owner decisions: inconsistencies between discovery, the inventory and the two modes are bugs to remove, not behavior to preserve; hosted and vendored share one core and differ only in where a rewired entry points; when in doubt the hosted mode's behavior wins, and every such call is listed in the doc so it can be overruled. Behavior changes are allowed per PR, listed, pinned by tests and shown as golden diffs. First decision recorded: pnpm 11's env-lockfile document counts as its own install tree, as the hosted planner already treats it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
docs/design/lock-model-divergences.md lists every question on which discovery, the inventory, hosted and vendored mode answer differently today, across all ecosystems, with the resolution under the principles and flags for false attestations and target-specific differences. Owner decisions recorded: the modes are ground truth over the readers; vendored mode's Maven profile and reactor support and its scala-cli wiring are back-ported to hosted as the planners merge (the JVM PR splits into readers, Maven planner, and Gradle/sbt/scala-cli). One call is pending: whether hosted keeps rewriting a shrinkwrap.yaml that pnpm never reads. Corrects the install-tree section: npm-shrinkwrap.json does not shadow package-lock.json; every consumer but the inventory already reads both. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Owner decisions: a dependency the user patched themselves is never patched or attested by default, in every mode and package manager, unless the user explicitly overrides; and no reader or writer reads project files outside the git repository root. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review of #2001 found the counter miscounting in several shapes, each now pinned by a test in scripts/tests/test_loc_budget.py: - a cfg(test) struct field, enum variant, parameter or match arm drove the scan below the item's depth and swallowed the code after it; the item now ends at a top-level `,` or before the bracket closing its container - `#[cfg(test)] use a::{..};` left its trailing `;` line counted - cfg predicates are parsed: `all(unix, test)` and nested `all` count, `not(test)` doesn't, and a file opening with `#![cfg(test)]` is test-only - `#[path]` is found among any attributes around the item and resolves against the declaring file's directory, as rustc does - `mod x;` inside a test-only inline module (vex::discover's `testing { mod golden; }`) marks that file test-only - `git ls-files -z` keeps paths with spaces whole, and only the group files plus their module declarers are read - `--files` with an unknown group is a usage error, not a KeyError golden.rs (435 lines) and two `};` lines leave the counts, so the budget drops to 30462 / 88362. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review of #2001: the Decisions table said every read is bounded to the repository root, while G1, J13 and P4 still said to read an ancestor go.work, parent NuGet configs and every include. The owner narrowed the rule on 2026-11-05, and the table now records it with the rest of that day's calls: - the bound applies to workspace runs only; --global / -g / --global-prefix runs are unbounded, and without a git checkout upward lookups behave as before - a marker file above the repository root (below home) makes hosted and vendored refuse with a message naming it - out-of-repository -r includes stay bounded in workspace runs - N11: shrinkwrap.yaml beside pnpm-lock.yaml is shadowed, unless the project pins pnpm 1 / 2, where it governs and vendored refuses - discovery withholds pnpm root and member-lock attestations whenever a pnpm-lock.<branch>.yaml exists - user patches: the override is --allow-patch-conflicts, and vex never attests under a user patch G1, J9, J13, P4, P9, X12, N1, N2, N4, N5, N11, N12 and N14 name the open PRs that fix them, and the fixes-on-main table gains a PR column. The problem table now counts four checksums.sha256 readers, as J9 does. The stack table splits PR 2 into 2a / 2b with their numbers, and the size budget section says why CI only warns and gives the corrected counts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
bugbot run Generated by Claude Code |
…g code loc-budget.py only recognized `pub name: Type` as a field, so a #[cfg(test)] on a `crate name: Type` field or a `const N: usize` generic parameter was read as an item. _item_end then didn't stop at the comma and excluded the rest of the struct or parameter list from the implementation count. Treat `crate` followed by a name like `pub`, and read `const` as a parameter when the attribute follows `<` or `,`, where no item can start. A `crate::path` statement and a plain const item keep their current handling. Reported by Bugbot on #2001. Co-Authored-By: Claude <noreply@anthropic.com>
|
bugbot run Generated by Claude Code |
|
Ready for review at 4b1fc91.
Generated by Claude Code |
Follow-up PRs grep for the names this doc cites. OriginPolicy::hosted_uuid exists nowhere in the tree; the function is DiscoverCtx::hosted_uuid (its memoized fast path is the hosted_hosts cell). The npm flavor router lives at vendor/lock_inventory/npm_family.rs, since doc paths are relative to src/. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011f9aeNYq2GHyrsNnPBepcC
|
[final reviewer] I reviewed this design PR. Nothing blocks it. I pushed one small fix in d6ccde3, so the brief will follow on that head:
I removed Ready for review; the burn-down re-labels it once CI is green. Non-blocking notes for the approver:
bugbot run Generated by Claude Code |
|
bugbot run Generated by Claude Code |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit d6ccde3. Configure here.
|
Ready for review at
Generated by Claude Code |
Final review briefWhat it does. First PR of the Risk: low. Docs plus a CI step that runs in Look here
Verified
Changes I made. In the previous pass, d6ccde3: corrected two symbol/path names in the design doc ( Open questions. None blocking. Minor: the exclude entry Auto-merge isn't armed yet: Generated by Claude Code |
Brings in #2002's review fixes (OriginPolicy imported from crate::origin, the redirect re-export dropped, the shared host matcher, the stricter architecture guards) and #2001's later counter fixes. Restores crates/socket-patch-core/src/origin.rs to the lock-readers group, which the previous merge (1d27a20) inherited the loss of from 1-origin's 4242ebc. Budget at this branch's counts: lock-readers 30767, lock-writers 88315. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Brings in #2002's review fixes and #2001's later counter fixes through 2a. Restores crates/socket-patch-core/src/origin.rs to the lock-readers group (lost in the earlier 1-origin merge). Budget at this branch's counts: lock-readers 30766, lock-writers 88114, both below 2a's 30767 and 88315. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
First PR of the
lock-model/*stack. No product code changes.Why
Three groups of code read the same project files, each its own way:
vex/discover/) finds Socket wiring. It isn't only forvex:scan,vendor, the hosted engine, the hosted unwind, rollout and ledger liveness all call it.vendor/lock_inventory/) lists every package a lockfile resolves.patch/redirect/,hosted/,vendor/) parse the files again to plan their edits.A survey of every format found this duplicated work:
Cargo.tomldependency tables;checksums.sha256.The inventory also has no NuGet, JVM, Hatch or Deno models.
What this PR adds
docs/design/lock-model-unification.md: the target design and the full stack.Origin:Registry,Hosted,Vendored,OtherorRejected.docs/design/lock-model-divergences.md: every question the readers and modes answer differently, with its resolution and the open PR that fixes it.scripts/loc-budget.pywithscripts/loc-budget.json: counts implementation lines (no comments, no items whosecfgimpliestest, no test-only modules) for two groups.lock-readerslock-writersEach later PR in the stack lowers these numbers in the same commit.
CI:
lint-ecosystemsruns the counter with--warn. It adds an annotation when a group is over budget but never fails, so bug fixes landing onmainin these files aren't blocked while the stack is open (the doc's Size budget section explains why).ci.ymlforbids step-levelcontinue-on-error, which is why it's a separate mode.Tests:
scripts/tests/test_loc_budget.py(31 tests) covers comment andcfg(test)stripping, cfg predicates, strings, chars and lifetimes, test-only fields, variants, parameters and match arms, test-module declarations (#[path], inline modules,#[cfg(test)]), module resolution, file listing, and the check, warn, report and--filesmodes.Stack
A PR that changes behavior lists each change, pins it with a test and shows its golden diffs (principle 4 in the doc).
lock-model/0-design(#2001)lock-model/1-origin(#2002)originmodule (not part offormats/): discovery's classifier and the redirect's hosted-URL rule, moved verbatimlock-model/2a-pnpm-readers(#2009)lock-model/2b-pnpm-planner(#2029)shrinkwrap.yamlbesidepnpm-lock.yamlis shadowedlock-model/10-nugetlock-model/11a-jvm-readerslock-model/11b-maven-plannerlock-model/11c-gradle-sbt-scalalock-model/12-install-treeslock-model/13-cleanupReview findings
From the code review of this PR. Each loc-budget.py bug got a test in
scripts/tests/test_loc_budget.pythat failed before the fix (13 failed, then all 27 pass). A later commit (4b1fc91, from Bugbot's review) also keepscfg(test)crate name: Typefields and const generic parameters from swallowing code, so there are 31 tests now.cfg(test)struct field, enum variant, parameter or match arm drove the item scan below its own depth and swallowed the code after it. An item now ends at a top-level,when it is a field, variant, parameter or arm (with<…>generics tracked soHashMap<A, B>stays whole), and always just before a bracket that closes its container.#[cfg(test)] use a::{…};left its;line counted (vendor/lock_inventory/mod.rs, one line;vendor/npm_lock.rs, one line). A;right after the closing brace now belongs to the item.cfg(all(unix, test))and#[cfg(test)]were missed. cfg predicates are now parsed:test, anall(…)with such an argument, or anany(…)whose every argument is one.not(test)never counts, so "containstestanywhere" was narrowed to that. A file whose#[cfg(test)]sits outside any block is test-only, and so are its submodules.#[path]is now read from any attribute around the item, before or after#[cfg(test)], and an out-of-line#[path]resolves against the declaring file's own directory, as rustc does (inside inline modules, against their directory).mod x;inside a test-only inline module is recorded with its inline path. This was live:vex/discover/testing/golden.rs(435 lines, declared inside#[cfg(test)] mod testing { mod golden; }) counted as implementation.formats::originand had no 2a/2b or 11a/b/c split; it now matches the doc, which also gained the 2a/2b split and PR numbers.checksums.sha256readers everywhere (doc problem table, J9, this description).git ls-files -z, split on NUL. The counter now reads only the group files, their siblings and each ancestor directory's module file (2.9 s to 2.0 s locally); the summary line is nowall groups(the union of the groups) instead of the whole repository.--fileswith an unknown group is now a usage error (exit 2) that lists the groups.Budget after the counter fixes:
lock-readers30,898 to 30,462 (golden.rsand one};),lock-writers88,363 to 88,362, lowered with--lower.Also (owner decisions, 2026-11-05), now in the Decisions table:
--global/-g/--global-prefixruns are unbounded, and without a git checkout upward lookups behave as before (Never read or rewrite project files outside the git repository root #2025).shrinkwrap.yamlbesidepnpm-lock.yamlis shadowed (never rewritten, not in the gate) unless the project's pnpm evidence pins pnpm 1 / 2, whereshrinkwrap.yamlgoverns and vendored refuses (Plan pnpm rewires with one core for both modes (lock-model 2b) #2029).pnpm-lock.<branch>.yamlexists, regardless of settings or environment (Stop attesting npm-family pins pnpm or a user patch overrides #2010).-rincludes stay bounded in workspace runs (Never read or rewrite project files outside the git repository root #2025).--allow-patch-conflicts, andvexnever attests under a user patch even with it (Stop attesting npm-family pins pnpm or a user patch overrides #2010, Never patch a pnpm, yarn or vlt dependency the user patched #2018).The divergences rows G1, J9, J13, P4, P9, X12, N1, N2, N4, N5, N11, N12 and N14 now name the open PR that fixes them, and the fixes-on-main table has a PR column.
Testing
python3 -B -m unittest discover -s scripts/tests: 411 tests, OK (1 skipped).scripts/loc-budget.py --checkexits 0, with both groups at their budget.Note
Low Risk
Documentation and auxiliary Python tooling only; CI uses a non-blocking warn mode so normal merges are unaffected.
Overview
Design-only kickoff for the
lock-model/*stack: no changes to Rust product code.Adds
docs/design/lock-model-unification.md(single lock model,Origin, install trees, principles, owner decisions, JVM/NuGet scope, staged stack) anddocs/design/lock-model-divergences.md(matrix of reader/mode disagreements and resolutions).Introduces
scripts/loc-budget.pywithscripts/loc-budget.json: counts non-comment, non-cfg(test)-gated implementation lines forlock-readersandlock-writersgroups; supports--check,--warn,--lower, and--files.scripts/tests/test_loc_budget.pyexercises the counter and modes.CI: the
lint-ecosystemsjob runsloc-budget.py --warn, emitting GitHub annotations for over-budget groups without failing the job.Reviewed by Cursor Bugbot for commit d6ccde3. Configure here.
Generated by Claude Code