Skip to content

publish.yml: optional static-asset upload before the image push - #9

Merged
DeviaVir merged 4 commits into
mainfrom
publish-static-assets
Nov 14, 2026
Merged

DeviaVir merged 4 commits into
mainfrom
publish-static-assets

Conversation

@DeviaVir

@DeviaVir DeviaVir commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

Summary

New optional inputs for publish.yml:

static-assets-path: /app/.next/static
static-assets-destination: gs://example-static-bucket/_next/static

When both are set, the publish job uploads that directory from the built image to GCS before it pushes the image. Use case: content-hashed web assets (for example Next.js .next/static with assetPrefix) stay available for every build. During a rolling deploy, a browser can no longer get HTML from one build and then a 404 for a chunk that only the other build's pods have.

Both inputs empty (the default): the step is skipped, with no behaviour change.

Design

  • Trust model unchanged. The build job still runs caller code without cloud credentials. The new step runs in the publish job on the already digest-verified local OCI layout, and executes no caller content.
  • Extraction. It selects linux/amd64 (or the single image), applies the layers in order, and honours OCI whiteouts and opaque directories.
    • Rejected: absolute paths, .., links and special files in the path or its ancestors, sparse entries, zstd layers, and oversized extension headers.
    • Limits: 50 MiB per file, 2 GiB total, 20,000 files.
  • Upload. GCS JSON API multipart, create-only (ifGenerationMatch=0).
    • An existing object must match the content hash, the stored-byte md5 and the serving metadata. Otherwise the job fails, because content-hashed names must never change.
    • Text types are stored gzip-encoded, so GCS transcodes for clients without gzip. All objects get public, max-age=31536000, immutable.
    • 8 parallel uploads; retries on 429, 5xx and connection errors; a 600 s hard deadline.
  • Safety. Inputs reach the scripts only as environment variables, and are validated in prepare. The token is never printed. TLS is verified; redirects and ambient proxies are refused. Errors do not echo transport or tar-controlled text.
  • IAM (README): storage.objects.create, plus storage.objects.get for the existing-object comparison (a publicly readable bucket covers that).

Validation

  • A harness that extracts the steps from publish.yml at run time: 62 tests pass on Python 3.12 and 3.13. It covers:
    • Raw and gzip layers, overwrites, whiteouts, opaque directories, traversal, links and special files, a missing or empty path, zstd refusal, and size limits (with reduced thresholds).
    • The real unpack step.
    • A fake GCS API: create, identical 412, different 412, 429/500/connection retries, refused redirects, content types, gzip correctness, object-name encoding.
  • actionlint: clean. zizmor --offline: no findings.
  • Not yet run on a real runner against a real bucket. The first consumer is a private repository that will verify it on its staging publish after release.

Release

After merge: release PR main → v1, tag v1.2.0.

Security review

Three independent read-only review rounds of this PR:

  • Round 1: gate fail-open on dropped outputs, collision diagnostics, attacker text in errors, PAX sparse memory, gzip byte stability, queued uploads after a failure, timeout.
  • Round 2: Solaris X and global PAX header limits, multi-platform assets, private tarfile hooks.
  • Round 3: merge; the remaining optional items are also applied:
    • The 35-minute timeout applies only when static assets are used.
    • Content-Disposition and Content-Language are now compared.
    • Control characters are rejected in the path.
    • The README documents that the whole-image limits apply to every layer.

The harness now has 83 tests, all passing on Python 3.12 and 3.13. actionlint is clean, and zizmor --offline has no findings.

New optional inputs static-assets-path and static-assets-destination.
When set, the publish job reads that directory from the validated local
OCI layout (layers applied in order, OCI whiteouts honored, links and
special files rejected, size and count limits) without executing caller
content, and uploads it to the GCS destination with the publisher's
access token before the image is pushed.

Uploads are create-only (ifGenerationMatch=0). An existing object must
match byte for byte and in serving metadata, otherwise the job fails.
Text assets are stored gzip-encoded; all objects get
"public, max-age=31536000, immutable".

Use case: content-hashed web assets (for example Next.js .next/static
with assetPrefix) stay available for every build, so rolling deploys
cannot serve HTML that references chunks missing from the other pods.
Both inputs empty: no behaviour change.
@DeviaVir
DeviaVir requested a review from a team as a code owner November 14, 2026 08:57
chase (AI) added 3 commits November 14, 2026 10:20
- Gate the step on the raw inputs and fail closed when the prepare
  outputs are missing (GitHub drops outputs that contain a secret value,
  which would otherwise skip the upload and push the image anyway).
- Name the colliding object in the error; README: separate bucket or
  prefix per trust level, and how to recover from a collision.
- Own AssetError for the step's messages; every other error is reported
  generically, so build-controlled text never reaches the log.
- Reject PAX sparse maps before tarfile parses them.
- On a gzip md5 mismatch with matching content hash, compare the
  decompressed stored object (bounded) instead of failing, so a runner
  zlib change cannot block unchanged assets.
- Cancel queued uploads after the first failure.
- Publish job timeout 35 minutes; document amd64-only asset upload.
…assets

Second review round:
- Include Solaris extended headers (type X) in the 1 MiB extension-header
  limit and cap the total of global PAX headers per layer at 1 MiB.
- Fail closed at startup if the private tarfile hooks the limits rely
  on no longer exist.
- Reject static assets with more than one platform: separate builds can
  produce different asset names and only one platform would be uploaded.
- README: one-line Cache-Control.
- Publish job timeout 35 minutes only when static assets are used;
  other callers keep 20.
- An existing object with Content-Disposition or Content-Language is a
  collision.
- static-assets-path rejects all control characters.
- README: whole-image parsing limits apply to every layer.
@DeviaVir
DeviaVir requested a balanced review from Copilot November 14, 2026 09:38
@DeviaVir
DeviaVir enabled auto-merge (squash) November 14, 2026 09:38

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Custom parsing of caller-controlled OCI layers and GCS upload handling in a credentialed job warrant human security review and real-runner/bucket validation beyond the local harness.

No open issues 🎉

What changed in this PR

Adds optional static-asset uploads to the reusable publish workflow so content-hashed assets remain available across rolling deployments. Uploads complete before image publishing, while callers that omit the new inputs retain existing behavior.

Changes:

  • Adds paired asset-path and GCS-destination inputs, with validation and single-platform enforcement.
  • Extracts assets from verified OCI layers and uploads them with create-only writes, collision checks, retries, and resource limits.
  • Documents configuration, permissions, retention requirements, and a Next.js example.
File Description
README.md Documents static-asset setup, safety constraints, and usage.
.github/​workflows/​publish.yml Adds input validation, bounded asset extraction, and GCS uploads before image publishing.

Review effort: Balanced

@DeviaVir
DeviaVir merged commit 44a5501 into main Nov 14, 2026
2 checks passed
@DeviaVir
DeviaVir deleted the publish-static-assets branch November 14, 2026 09:56
DeviaVir added a commit that referenced this pull request Nov 14, 2026
…11)

* Merge pull request #6 from Blockstream/security-hardening

publish.yml: security fixes for explicit tags, artifact hand-off and OCI checks

* publish.yml: optional static-asset upload before the image push (#9)

* publish.yml: optional static-asset upload before the image push

New optional inputs static-assets-path and static-assets-destination.
When set, the publish job reads that directory from the validated local
OCI layout (layers applied in order, OCI whiteouts honored, links and
special files rejected, size and count limits) without executing caller
content, and uploads it to the GCS destination with the publisher's
access token before the image is pushed.

Uploads are create-only (ifGenerationMatch=0). An existing object must
match byte for byte and in serving metadata, otherwise the job fails.
Text assets are stored gzip-encoded; all objects get
"public, max-age=31536000, immutable".

Use case: content-hashed web assets (for example Next.js .next/static
with assetPrefix) stay available for every build, so rolling deploys
cannot serve HTML that references chunks missing from the other pods.
Both inputs empty: no behaviour change.

* publish.yml: harden static-asset upload after security review

- Gate the step on the raw inputs and fail closed when the prepare
  outputs are missing (GitHub drops outputs that contain a secret value,
  which would otherwise skip the upload and push the image anyway).
- Name the colliding object in the error; README: separate bucket or
  prefix per trust level, and how to recover from a collision.
- Own AssetError for the step's messages; every other error is reported
  generically, so build-controlled text never reaches the log.
- Reject PAX sparse maps before tarfile parses them.
- On a gzip md5 mismatch with matching content hash, compare the
  decompressed stored object (bounded) instead of failing, so a runner
  zlib change cannot block unchanged assets.
- Cancel queued uploads after the first failure.
- Publish job timeout 35 minutes; document amd64-only asset upload.

* publish.yml: bound all tar extension headers; single-platform static assets

Second review round:
- Include Solaris extended headers (type X) in the 1 MiB extension-header
  limit and cap the total of global PAX headers per layer at 1 MiB.
- Fail closed at startup if the private tarfile hooks the limits rely
  on no longer exist.
- Reject static assets with more than one platform: separate builds can
  produce different asset names and only one platform would be uploaded.
- README: one-line Cache-Control.

* publish.yml: final review items for static-asset upload

- Publish job timeout 35 minutes only when static assets are used;
  other callers keep 20.
- An existing object with Content-Disposition or Content-Language is a
  collision.
- static-assets-path rejects all control characters.
- README: whole-image parsing limits apply to every layer.

---------

Co-authored-by: chase (AI) <[email protected]>

* build(deps): bump sigstore/cosign-installer (#8)

Bumps the actions group with 1 update: [sigstore/cosign-installer](https://github.com/sigstore/cosign-installer).


Updates `sigstore/cosign-installer` from 4.1.2 to 4.1.3
- [Release notes](https://github.com/sigstore/cosign-installer/releases)
- [Commits](sigstore/cosign-installer@6f9f177...756fe2d)

---
updated-dependencies:
- dependency-name: sigstore/cosign-installer
  dependency-version: 4.1.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: actions
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: chase (AI) <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants