Skip to content

Read Maven checksums.sha256 one way and never drop user lines - #2016

Open
Mikola Lysenko (mikolalysenko) wants to merge 3 commits into
mainfrom
fix/maven-checksums-one-reader
Open

Mikola Lysenko (mikolalysenko) wants to merge 3 commits into
mainfrom
fix/maven-checksums-one-reader

Conversation

@mikolalysenko

@mikolalysenko Mikola Lysenko (mikolalysenko) commented Nov 4, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

Hosted Maven merged into .mvn/checksums/checksums.sha256 by splitting every line on two spaces, rebuilding the file from that map, and silently dropping any line it couldn't split that way. That deleted a user's single-space line, a * (binary marker) line, a tab-separated line, comments and stray text. It also rewrote a CRLF file as LF and re-sorted every line.

The file and the .mvn/maven.config settings were read five different ways (discovery, the hosted merge, the hosted drop filter, the unenforced-pin warning, the upstream unwind), and each judged differently whether Maven enforces the file.

Change

One pure module, formats::maven::trusted_checksums (no I/O, no host policy), used by all five sites.

  • Reading the checksums file (parse_line, entries): tolerant. It accepts what sha256sum-style tools write: one or more spaces or tabs, an optional *, CRLF, blanks around the line. # lines are comments. The first field doesn't have to be a valid hash, so a line for a path hosted mode owns is recognised whatever its hash. Each entry records maven_reads: true only for the exact <64 lowercase hex> <path> shape that maven-resolver's SummaryFileTrustedChecksumsSource checks. Maven splits on exactly two spaces, doesn't strip *, and compares against its own lowercase digest.
  • pinned: path → hash for the lines Maven checks. A later line for the same path wins.
  • edit:
    • Owned paths are rewritten in place in canonical form, keeping that line's ending, and collapsed to one line.
    • Dropped paths are removed whatever their hash.
    • Missing owned paths are inserted before the first later checksum line that sorts after them, ahead of the comment block directly above that line, or appended. New lines use the file's dominant line ending.
    • Every other line is kept byte for byte.
  • config_enforces: the one rule, documented in the code. maven.config is read the way Maven 3.9+ reads it, one argument per line (maven_config_line_properties, split out of maven_config_properties). It then requires every setting Maven reads this file by, judged with the resolver's defaults: post-processor on, checksumAlgorithms naming SHA-256, summary-file source on, basedir at .mvn/checksums, originAware off.
  • enforced: config_enforces plus the Maven Wrapper rule hosted already warned on. An Apache Maven older than 3.9.4 ignores the files; no wrapper, or another distribution, counts as enforcing.
  • CONFIG_ARGS: the maven.config lines hosted writes. They and the keys read are built from one literal per settings prefix.

How each site uses it:

Site Uses
Hosted merge and drop edit. A file is written only when its text changes. A grant hash that isn't 64 hex digits writes no checksum line and warns redirect_maven_trusted_checksums_invalid_hash; the -socket version pin still lands.
Hosted unenforced-pin warning config_enforces + pinned + the wrapper rule
Discovery enforced + pinned. VEX attests integrity only when Maven checks it, and hosted's judgment and discovery's now match.
Unwind edit. The "resolver lines left" warning uses config_has_written_settings (presence).

Golden change: in redirect-maven.json, mvn-config-merge/expected goes from locked_integrity: Sha256Hex(ccc…) to null. That fixture keeps the user's originAware=true, under which Maven reads checksums-<repoId>.sha256, not this file, so the old attestation was false.

Review findings (/code-review high)

Each finding was confirmed with a failing test on the previous head (26d9d3b) before it was fixed.

  1. Fixed: a -Dk=v token inside a longer line no longer counts. Only whole-line arguments do.
  2. Fixed: config_enforces now checks all five settings Maven reads the file by, not just the switch. failIfMissing doesn't decide whether a committed line is checked, so it isn't required. The unwind keeps presence semantics on purpose: its warning is about leftover lines hosted mode may have written, and a partial or switched-off set is still worth removing.
  3. Fixed: owned-path matching no longer depends on the hash being valid, so stale SRI or truncated lines are dropped on re-pin and unwind, and restore_mvn no longer returns early. Grant hashes are validated when written.
  4. Fixed: uppercase hex now gives maven_reads = false and doesn't attest.
  5. Fixed: duplicate lines for an owned path collapse to one.
  6. Fixed: inserted entries go ahead of the comment block attached to the next entry.
  7. Fixed: inserted lines use the file's dominant ending (a tie or no endings gives LF).
  8. Fixed: TRUSTED_CHECKSUMS_ON is gone. CONFIG_ARGS and the keys share one literal per prefix.
  9. Fixed: discovery applies the same wrapper rule as hosted, and hosted's "enforced" is discovery's.
  10. Partly fixed: edit writes into one buffer, with no per-line format!. It still scans once for which owned paths are present before writing. A true single pass can't know whether an owned path appears later without moving that line, and moving it would churn a file the user ordered. The scan is one cheap parse of a tens-of-lines file.

Tests

  • maven_checksums_merge_keeps_every_user_line: fails before the first commit, passes after. User lines in single-space, *, tab, comment and stray-text form survive a pin and a re-pin.
  • The CRLF re-pin test asserts the file stays CRLF.
  • maven_checksums_owned_lines_go_whatever_their_hash: hosted re-pin with stale hashes, and refusal of an invalid grant hash.
  • checksums_unwind_drops_owned_lines_whatever_their_hash and checksums_unwind_reads_every_spelling_and_keeps_crlf: the unwind.
  • checksums_attest_only_what_maven_enforces: discovery across config, wrapper and spelling cases.
  • Reader and rule round trips in formats::maven::trusted_checksums (12 tests).
  • every_fail_closed_rewriter_fixture_yields_the_patch_uuid_not_the_token now expects no integrity for mvn-config-merge, for the originAware=true reason above.
  • cargo test -p socket-patch-core --lib -- formats::maven vex::discover:: patch::redirect:: vendor::jvm::: 1591 passed, golden included. socket-patch-cli e2e_vex_lockfile + in_process_get_hosted_ecosystems filtered to maven: 20 passed.
  • cargo clippy --locked --workspace --all-features -- -D warnings and cargo fmt --all -- --check are clean.

Not done

  • Writing to maven.config (merge_mvn_config) is unchanged.
  • GNU escaped filenames and BSD-style SHA256 (path) = hash lines are not interpreted. They are kept verbatim.
  • A basedir spelled differently from what hosted writes (for example an absolute path to the same directory) counts as not enforced. That errs toward claiming less.

Review findings, round 2 (/code-review high on ec69ff3, cloud routine)

Fixed in a53a056. Each test below failed on ec69ff3.

  1. A broken checksum line for the current suffixed version survived: confirmed and fixed. An invalid grant hash writes no line, but a line an earlier release wrote mangled (lowercased base64) for the same path stayed, and Maven fails the build on a mismatch. Those paths are now dropped on a fresh pin and on a re-run (version pin already present). Test: maven_checksums_owned_lines_go_whatever_their_hash (extended).
  2. The writer parsed maven.config its own way: confirmed and fixed. merge_mvn_config keyed settings by the raw line[..=eq] prefix. A user's indented, --define k=v, --define=k=v or quoted setting wasn't seen, so hosted appended its own line, and that later line won. The user's value was silently overridden, against the documented contract. The writer now uses maven_config_line_properties, discovery's reader. Test: maven_config_conflict_in_any_spelling_keeps_the_user_value.
  3. The rollback's leftover-settings warning was narrowed: confirmed and fixed. config_has_written_settings is presence-based again (maven_config_properties, which also sees a -Dk=v token joined onto a longer line). This matches round 1's "the unwind keeps presence semantics". Test: config_enforces_needs_every_setting_maven_reads_the_file_by (the assertion is flipped).
  4. Docs were stale: fixed. The discovery module doc and docs/ecosystems.md now describe the five-setting and wrapper rule and the redirect_maven_trusted_checksums_invalid_hash warning.

Declined:

  • Other basedir spellings (${maven.multiModuleProjectDirectory}/..., trailing /). These still count as not enforced. That errs toward claiming less (owner decision 4), as the "Not done" section already says.
  • Header comment placement. An inserted line can land above a leading comment block, but that block is attached to the entry after it. The order is deterministic and no line is lost.
  • Decoding an SRI-base64 grant hash. The patch server sends hex. Refusing and warning is the fail-closed choice.
  • maven.config read twice in discovery, and the trusted_checksums fn/module name clash. Both are minor and left as is.

Tests (cloud run): cargo test -p socket-patch-core --lib -- formats::maven vex::discover:: patch::redirect:: vendor::jvm:: gives 1591 passed. The one failure, vlt_heal::an_unremovable_hidden_lock_keeps_every_store_entry, fails only because the container runs as root, where chmod 0555 doesn't block. CLI e2e_vex_lockfile and in_process_get_hosted_ecosystems filtered to maven pass. clippy and fmt are clean.


Note

Medium Risk
Touches hosted Maven wiring, rollback/unwind, and VEX attestation for checksum pins; behavior is more conservative (fewer false attestations) but changes long-standing merge semantics.

Overview
Hosted Maven used to merge .mvn/checksums/checksums.sha256 by splitting on two spaces, re-sorting, and dropping lines it could not parse (single-space/sha256sum-style lines, comments, CRLF quirks). That logic lived separately from discovery, unwind, and the “unenforced pin” warning, so the same file was judged inconsistently.

This PR adds formats::maven::trusted_checksums, a pure module shared by the hosted rewriter, rollback/unwind, and VEX discovery. It tolerantly parses user lines but only treats exact <64 lowercase hex> <path> entries as Maven-enforced pins; edit rewrites owned paths in canonical form, drops superseded pins, inserts with the file’s dominant line ending, and keeps every other line byte-for-byte. config_enforces / enforced align on Maven 3.9+ whole-line maven.config arguments (via new maven_config_line_properties) plus wrapper ≥ 3.9.4.

Behavior changes: invalid grant SHA-256 (not 64 hex) skips checksum lines with redirect_maven_trusted_checksums_invalid_hash (version suffix still applies); VEX no longer attests checksum integrity when config/wrapper/spelling means Maven would not enforce (e.g. mvn-config-merge with originAware=true → locked_integrity: null in golden).

Reviewed by Cursor Bugbot for commit ec69ff3. Configure here.


Generated by Claude Code

The hosted Maven merge re-parsed .mvn/checksums/checksums.sha256 on a
two-space separator, rebuilt the file from that map and silently dropped
every line it could not split that way: a user's single-space or `*`
(binary marker) sha256sum line, a tab-separated line, a comment, any
stray text. It also normalized a CRLF file to LF and re-sorted it.

The file and the maven.config switch were read four different ways
(discovery, the hosted merge, its drop filter, the unenforced-pin
warning) plus a fifth in the upstream unwind. They now share one pure
reader in formats::maven::trusted_checksums:

- parse_line / entries: tolerant read of what sha256sum-style tools
  emit (any run of spaces/tabs, an optional `*`, CRLF, surrounding
  blanks; a 64-digit hex hash). Each entry records whether it is in the
  exact `<hash>  <path>` shape maven-resolver's own reader honours.
- pinned: path -> hash of the lines Maven enforces, a later line
  winning (as in SummaryFileTrustedChecksumsSource).
- edit: rewrites owned paths in place, drops named paths, inserts new
  entries in sorted position (else appends) with the file's own line
  ending, and keeps every other line byte for byte.
- switch_on: the trusted-checksums switch rule, decided once: read
  maven.config with the existing shared maven_config_properties reader
  (one argument per line, `#` comments, -Dk / --define, later wins) and
  judge the value as the resolver does (Boolean.parseBoolean).

Discovery, the hosted merge/drop/warning and the unwind all use it.
Discovery now attests integrity only from lines Maven itself enforces;
a tolerated spelling is recognised (and replaced or dropped when it names
a path hosted owns) but pins nothing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@mikolalysenko
Mikola Lysenko (mikolalysenko) marked this pull request as ready for review November 5, 2026 16:48
@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

bugbot run


Generated by Claude Code

@mikolalysenko Mikola Lysenko (mikolalysenko) added the Ready for review Agent-verified: mergeable, CI green, Bugbot clean — awaiting human review label Nov 5, 2026
@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

Labeled Ready for review (burn-down agent).

  • Head: 26d9d3b777f4e96fe7f38b19f6931d36c3310462
  • CI: 80/82 checks green; the only red is the e2e_vex_build pdm leg (+ its ci-ok rollup), which is main-wide: hishel 2.0.0 (released 2026-11-05) breaks pdm lock on every PR. Fix pdm e2e red on every PR: pin hishel<2 #2028 fixes it; merge main in after it lands.
  • Bugbot: reviewed this head, no issues found.
  • Mergeable, no CHANGELOG.md change.
  • Slack announcement not sent this run (no Slack send tool available); the next run will retry.

Generated by Claude Code

Review fixes on the shared trusted_checksums reader:

- The enforcement rule (config_enforces) now needs every setting Maven
  reads .mvn/checksums/checksums.sha256 by, with the resolver's defaults:
  the post-processor on, checksumAlgorithms naming SHA-256, the
  summary-file source on, its basedir at .mvn/checksums and originAware
  off. Settings are whole-line arguments only (Maven 3.9+, the only line
  that enforces), never a -Dk=v token inside a longer line; the per-line
  reading is split out of maven_config_properties as
  maven_config_line_properties.
- enforced() adds the wrapper rule the hosted rewriter already warned on
  (an Apache Maven < 3.9.4 ignores the files). Discovery attests only
  through it, so VEX never claims integrity Maven does not check: the
  mvn-config-merge fixture keeps the user's originAware=true and its
  golden loses its integrity.
- An uppercase-hex line no longer counts as read by Maven (it compares
  to its own lowercase digest).
- Lines for a path hosted mode owns are recognised whatever their hash,
  so a stale SRI or truncated line is replaced, dropped on re-pin and
  dropped by the unwind; a grant hash that is not 64 hex digits writes no
  checksum line (redirect_maven_trusted_checksums_invalid_hash).
- edit collapses an owned path to one line, inserts ahead of the comment
  block directly above the line it lands before, and inserts with the
  file's dominant line ending; it writes into one buffer.
- The written maven.config args and the keys read come from one literal
  per settings prefix (CONFIG_ARGS, re-exported as MVN_CONFIG_ARGS).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

Final review brief (for the human approver)

What it does: adds one pure module, formats::maven::trusted_checksums, that reads and edits .mvn/checksums/checksums.sha256 and decides whether .mvn/maven.config plus the Maven Wrapper version make Maven enforce it. Hosted merge, hosted drop, the unenforced-pin warning, VEX discovery and the upstream unwind all use it now. The old merge re-sorted the file, turned CRLF into LF and dropped user lines. The new edit keeps every line it doesn't own byte-for-byte. A grant hash that isn't 64 hex digits is no longer written and raises redirect_maven_trusted_checksums_invalid_hash.

Risk: low to medium. The logic is pure, unit-tested and idempotent. VEX output changes on purpose: integrity is no longer attested for originAware=true projects (golden file updated), wrapper Maven below 3.9.4, single-space/*/uppercase spellings, or configs with only the switch line. Configs that earlier CLI versions wrote still attest, because the six args are unchanged.

Look here

  • trusted_checksums.rs:186-206:`` parse_line and the "Maven reads this" rule
  • trusted_checksums.rs:247-306:``` edit (presence scan, keeps each line's ending, dominant ending for new lines)
  • trusted_checksums.rs:82-91:```` config_enforces, the single enforcement rule
  • redirect/mod.rs:9210-9246: hash validation, invalid-hash warning, write only on a text change
  • discover/maven.rs:403-416: discovery reads maven-wrapper.properties and calls enforced

Verified

  • Read the full diff (7 files, +915/-211). CHANGELOG.md untouched; no debug leftovers; no stale references to the removed helpers.
  • cargo test -p socket-patch-core --lib -- formats::maven vex::discover:: patch::redirect:: vendor::jvm::: 1629 passed. 1 failed: vlt_heal::...an_unremovable_hidden_lock_keeps_every_store_entry, in a file this PR doesn't touch. It relies on chmod blocking writes, and the review sandbox ran as root. All 12 trusted_checksums tests pass.
  • Probed CRLF, a lone \r, a comment-only file, duplicate owned lines with mixed endings, a BOM, and a missing final newline. All behave sensibly.
  • CI: clippy green. ci-ok is red only because of the pdm e2e_vex_build job (pdm lock fails since hishel 2.0 came out). Fix pdm e2e red on every PR: pin hishel<2 #2028 fixes that, and the same job is red on every open PR. No open Bugbot threads.

Changes I made: none.

Open questions (nits, none blocking)

  • pinned() (:217-222)``````` filters out invalid two-space lines before "later line wins". With A x.jar followed by ` x.jar`, VEX attests A while Maven checks the later line, so the build fails. The failure is closed (no unchecked artifact runs), but VEX claims more than is true.
  • docs/ecosystems.md:860-866 doesn't mention the invalid-hash refusal or the stricter attestation rule. CLI_CONTRACT.md was updated.
  • Cosmetic: edit line 279 could use digest::is_hex(e.hash, 64) instead of allocating, and dropping a line leaves any comment above it in place.

Auto-merge is not armed yet because ci-ok is red. The merge steward arms it once CI is green on this head, so approving then sends it straight to the merge queue.


Generated by Claude Code

@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

bugbot run


Generated by Claude Code

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit ec69ff3. Configure here.

@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

Ready for review re-confirmed at the new head (burn-down agent).

  • Head: ec69ff32 (commit Judge Maven Trusted Checksums enforcement one way in every mode, pushed after the earlier label)
  • CI: 28 passed, 19 skipped, 2 failed. Both failures are the e2e_vex_build pdm leg and its ci-ok rollup: hishel 2.0.0 broke pdm lock on every PR. Fix pdm e2e red on every PR: pin hishel<2 #2028 fixes it.
  • Bugbot: reviewed ec69ff32 and found no issues.
  • Mergeable, no CHANGELOG.md change, auto-merge not armed.
  • Slack: not announced. This run had no Slack send tool.

Generated by Claude Code

Second review round on the Trusted Checksums change.

- The hosted writer (`merge_mvn_config`) still keyed settings by the raw
  `line[..=eq]` prefix, so a user's indented, `--define k=v` or quoted
  setting was not seen, hosted appended its own line after it, and that
  later line won, silently overriding the user's value. It now reads the
  file with `maven_config_line_properties`, the reader discovery judges
  enforcement with, and reports such a setting as
  `redirect_maven_trusted_checksums_conflict`.
- A grant whose sha256 is not 64 hex digits writes no checksum line, but
  a line an earlier release wrote for the same suffixed version (its
  hash mangled) stayed, and Maven fails the build on a mismatched line.
  Those paths are now dropped, on a fresh pin and on a re-run alike.
- The rollback's leftover-settings warning keys on presence again
  (`maven_config_properties`, which also sees a `-Dk=v` token joined onto
  a longer line), not on Maven 3.9's enforcement reading.
- The discovery module doc and docs/ecosystems.md describe the stricter
  attestation rule and the invalid-hash warning.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Ready for review Agent-verified: mergeable, CI green, Bugbot clean — awaiting human review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants