Skip to content

[Docker] Install Go toolchain matching the image architecture - #41203

Open
ayush-builds-stuff wants to merge 1 commit into
apache:masterfrom
ayush-builds-stuff:fix/20983-docker-go-targetarch
Open

ayush-builds-stuff wants to merge 1 commit into
apache:masterfrom
ayush-builds-stuff:fix/20983-docker-go-targetarch

Conversation

@ayush-builds-stuff

Copy link
Copy Markdown

Summary

The dev build environment image is built FROM ubuntu:24.04, which is multi-arch, so on
arm64 hosts the image is arm64 — but the Go toolchain installed into it was always
linux-amd64. The result is an image carrying a toolchain that does not match the image it
lives in:

  • With binfmt emulation available (Docker Desktop + Rosetta), go silently runs emulated:
    go env GOARCH reports amd64 inside an arm64 container, and every Go step is slow.
  • Without emulation (arm64 Linux hosts, minimal Docker setups), it fails outright with
    /lib64/ld-linux-x86-64.so.2: cannot open shared object file, as reported in [Bug]: local-env-setup.sh fails on M1 mac #20983.

This derives the Go download from the architecture of the image being built, and asserts
go version in the same layer so a future arch mismatch fails the build instead of
producing a silently broken image.

fixes #20983

Implementation

  • arch comes from BuildKit's TARGETARCH (values amd64/arm64, the same naming Go uses),
    with dpkg --print-architecture as the fallback for legacy-builder builds where
    TARGETARCH is unset.
  • Download host switched from golang.org/dl (which 301s to go.dev) to go.dev/dl, and the
    tarball is removed from the layer after extraction.
  • Grepped dev-support/: this was the only hardcoded amd64 left.

Testing

Both platforms were built end-to-end from this Dockerfile on an M-series Mac, and both
produce a toolchain that matches the image:

$ docker build --platform linux/arm64 -f dev-support/docker/Dockerfile -t beam-build:arm64 dev-support/docker
#16 go version go1.26.2 linux/arm64                      # PASS

$ docker run --rm beam-build:arm64 sh -c 'dpkg --print-architecture; go version; go env GOARCH'
arm64
go version go1.26.2 linux/arm64
arm64

$ docker build --platform linux/amd64 -f dev-support/docker/Dockerfile -t beam-build:amd64 dev-support/docker
#16 go version go1.26.2 linux/amd64                      # PASS (emulated, no amd64 hardware available)

$ docker run --rm --platform linux/amd64 beam-build:amd64 sh -c 'dpkg --print-architecture; go version;go env GOARCH'
amd64
go version go1.26.2 linux/amd64
amd64

In both images a trivial program compiles and runs with the matching architecture
(go build && ./prog → ok arm64 / ok amd64).

Architecture selection was also probed directly on both platforms:

--platform linux/arm64  →  TARGETARCH=arm64  dpkg=arm64
--platform linux/amd64  →  TARGETARCH=amd64  dpkg=amd64

Not exercised: the dpkg --print-architecture fallback used when TARGETARCH is unset
(legacy builder, DOCKER_BUILDKIT=0). BuildKit is the default, and the fallback resolves to
the same value as TARGETARCH on both architectures above.

Also note the LegacyKeyValueFormat build warnings are pre-existing, on untouched ENV lines.

No CHANGES.md entry: internal build tooling, no user-facing behavior change.

AI disclosure

Investigated, implemented and tested with an AI coding agent (Pi-Bolt). The account owner
reviewed the change, owns the content, and is responsible for it.

Credit to @rokadepiyush49-rgb for the diagnosis in #20983.

@github-actions github-actions Bot added the size/s S: 10-49 lines changed label Nov 11, 2026

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📋 Review Summary

This PR cleanly resolves #20983 by replacing the hardcoded linux-amd64 Go toolchain download in dev-support/docker/Dockerfile with dynamic architecture detection via BuildKit's TARGETARCH (falling back to dpkg --print-architecture). It also cleans up the downloaded tarball in the same layer and adds a build-time go version sanity check.

🔍 General Feedback

  • The implementation is idiomatic, defensive (set -eux), and properly supports both BuildKit and legacy Docker builders across amd64 and arm64 platforms.
  • Removing /tmp/go.tar.gz in the same RUN instruction is a good improvement that reduces final image layer size.

💡 Tip: Comment /gemini review on this pull request to trigger a new Gemini code review.

@github-actions

Copy link
Copy Markdown
Contributor

Assigning reviewers:

R: @shunping added as fallback since no labels match configuration
Backup reviewer: Abacn (if the main reviewer is unresponsive, you can escalate to them by tagging their username)

Note: If you would like to opt out of this review, comment assign to next reviewer.

Available commands:

  • stop reviewer notifications - opt out of the automated review tooling
  • remind me after tests pass - tag the comment author after tests pass
  • waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

The PR bot will only process comments in the main thread (not review comments).

@shunping shunping left a comment •

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM on the fix. Could you sync to the latest master and resolve the conflict? Then it will be good to merge.

   The Go download was hardcoded to linux-amd64 while the image is built from
   multi-arch ubuntu:24.04, so arm64 hosts ended up with a toolchain that could
   not run natively (silently emulated where binfmt is registered, failing with
   "ld-linux-x86-64.so.2: cannot open shared object file" where it is not).

   Take the architecture from BuildKit's TARGETARCH, falling back to
   dpkg --print-architecture, and assert `go version` in the same layer so a
   mismatch fails the build instead of producing a broken image.

   Fixes apache#20983
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Next Action: Reviewers size/s S: 10-49 lines changed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: local-env-setup.sh fails on M1 mac

2 participants