Docker Exec Format Error: Fix Arch Mismatch

You built the image on your new MacBook, pushed it, and the cluster answered: "exec /app/server: exec format error." The binary is right there — you can see it in the image — yet the kernel won't touch it. Or the reverse: CI built it, your laptop won't run it. This error spikes every time a team mixes Apple Silicon laptops with Intel servers or Intel CI, and it always looks like a corrupt binary until you learn what the kernel is actually saying.

Exec format error (ENOEXEC) is the kernel reporting that a file isn't executable code for this machine: wrong CPU architecture, a script with no shebang line, or a shebang pointing at an interpreter that doesn't exist. The container assembled fine — mounts, namespaces, cgroups all worked — and then PID 1 failed at the first instruction. That's why it differs from OCI create failures: the runtime did its job, the binary couldn't do its.

This guide covers the two families in order: architecture mismatch (confirm, --platform, buildx multi-arch) and broken entrypoint scripts (shebang, exec bit, CRLF). You'll get the inspect commands that prove which family you're in and the build patterns that end it permanently.

What the Kernel Is Telling You

Exec format error is errno ENOEXEC, returned by the execve syscall when the kernel can't parse a file as runnable code for this CPU. Two triggers, no others: an ELF binary whose machine field (x86-64, AArch64) doesn't match the processor, or a text file without a valid #! interpreter line. The container runtime did everything right — namespaces, mounts, cgroups all assembled — and PID 1 died at the kernel boundary before executing anything. Your application code is provably innocent: it never ran.

This placement in the startup sequence is the diagnostic gift. OCI create failures happen before exec (assembly broke); immediate exits happen after exec (code ran and died). Exec format error is exactly at exec — so you check exactly two things: binary architecture vs host CPU, and script header vs interpreter. Nothing else in the stack can produce this errno, which makes the error one of Docker's most precise once you know the vocabulary.

The modern backdrop is Apple Silicon meeting Intel fleets. docker build defaults to the builder's native arch, so M-series laptops emit arm64 images that amd64 clusters reject, while CI on amd64 emits images that run emulated (slowly) or fail on arm64-only edge nodes. Tags don't record architecture visibly, tests don't check it, and dashboards don't show it — so the mismatch travels silently from laptop to registry to cluster, detonating only at exec.

Confirm the Mismatch: Host vs Image Architecture

Diagnosis is a two-command comparison. uname -m prints the host CPU: x86_64 (amd64) or aarch64 (arm64). docker image inspect --format '{{.Os}}/{{.Architecture}}' prints what the image was built for. Different answers is the whole diagnosis — no logs to read, no config to parse. On Kubernetes, get the node side with kubectl get nodes -o jsonpath='{...status.nodeInfo.architecture}' and the deployed side from the pod's imageID digest, since the tag may have moved since the failing pull.

Watch for the partial cases. An image can be multi-arch at the manifest level while the node pulled the wrong variant — rare, but check with buildx imagetools inspect. A binary inside a right-arch image can still be wrong-arch: a Go binary cross-compiled with GOARCH unset in a Dockerfile that assumed the builder's arch, or a vendored amd64 sidecar copied into an arm64 image. When image and host agree but exec still fails, exec into the image and run file /app/server — it reports the ELF machine type per file, catching the binary that disagrees with its own image.

Record both values in the incident thread before fixing. Architecture bugs recur across teams (laptop fleets, CI migrations, new node pools), and a written pair of values — host aarch64, image amd64 — makes the next occurrence a pattern match instead of a fresh mystery. The file command on the exact failing path is the tiebreaker when image-level checks pass.

# The two-command diagnosis: host CPU vs image arch
uname -m
docker image inspect --format '{{.Os}}/{{.Architecture}}' myapp:latest

# Kubernetes: node arch vs what the pod actually pulled
kubectl get nodes -o jsonpath='{range .items[*]}{.metadata.name} {.status.nodeInfo.architecture}{"\n"}{end}'
kubectl get pod -l app=myapp -o jsonpath='{..imageID}' | tr ' ' '\n' | sort -u

# Right image, wrong binary inside? Check the exact failing path
docker run --rm --entrypoint sh myapp:latest -c 'file /app/server || head -c 4 /app/server | od -An -tx1'
# Want: ELF 7f 45 4c 46 + matching machine; scripts must start with #!

Run It Now: the --platform Flag

When you need the container running in the next five minutes, --platform overrides architecture selection at pull and run time. docker run --platform linux/amd64 pulls the amd64 variant of a multi-arch tag (or fails loudly on single-arch tags built for the other CPU) and runs it — natively on amd64 hosts, emulated on arm64 via Docker Desktop's Rosetta or QEMU. The same flag works on pull, build, and compose (platform: in the service), so the override follows your normal workflow instead of requiring special commands.

Understand emulation's limits before leaning on it. QEMU user-mode emulation runs amd64 code on arm64 at a fraction of native speed — fine for a smoke test, painful for a database, and unavailable on Kubernetes nodes entirely. Rosetta on Docker Desktop is faster but Desktop-only. So --platform is triage and local development convenience, not a production strategy: it gets the container up now while you build the correct native image next.

Use --platform deliberately in heterogeneous workflows even without errors. CI that must test the production arch from arm64 runners, developers verifying the amd64 artifact on M-series laptops, and migration periods with mixed node pools all benefit from explicit flags. Explicit beats ambient: a flag in the command documents the intent, while silent arch selection is how wrong-arch tags get pushed in the first place.

# Pull and run the production arch explicitly (triage + verification)
docker pull --platform linux/amd64 myapp:latest
docker run --rm --platform linux/amd64 myapp:latest --version

# Compose equivalent for local dev against the prod arch
# services:
#   app:
#     image: myapp:latest
#     platform: linux/amd64

# Check what a tag actually offers before overriding blindly
docker buildx imagetools inspect myapp:latest | grep -E 'Name|Platform'

# Emulation check on Desktop: expect slower startup, full function
docker run --rm --platform linux/amd64 myapp:latest sh -c 'uname -m && echo EMULATED-OK'

Ship Multi-Arch: buildx Once, Run Anywhere

The permanent fix is one tag containing native code for every CPU you run: a multi-arch manifest list built with docker buildx. Create a builder once (docker buildx create --use, with the container driver for multi-platform), then build with --platform linux/amd64,linux/arm64 and push. The registry stores one tag pointing at two images; each machine pulls its native variant automatically — M-series laptops get arm64, Intel clusters get amd64, and nobody passes flags. Exec format error becomes structurally impossible across those arches.

Dockerfiles need modest discipline to build cleanly per arch. Base images must themselves be multi-arch (official images are), RUN steps must not assume the builder's arch (use BUILDARCH-aware conditionals or TARGETARCH args when downloading arch-specific binaries), and compiled languages should build natively per platform rather than cross-compiling blindly. The classic failure is curl-ing an amd64 tarball in a RUN step: it works on the amd64 leg and dies on arm64. Parameterize with $TARGETARCH and both legs stay green.

Verify the artifact, not the intent. buildx imagetools inspect must show both platforms, and a smoke run per arch (native runners or --platform pulls) must pass before the tag ships. In CI, build both legs on every release and gate promotion on the dual smoke. Teams that adopt this stop thinking about architecture entirely — the manifest absorbs the laptop-vs-cluster difference that used to page people.

# One-time: a builder that can do multi-platform
docker buildx create --name multi --driver docker-container --use
docker buildx inspect --bootstrap

# Build both arches, push one tag
# (Dockerfile must honor $TARGETARCH for arch-specific downloads)
docker buildx build --platform linux/amd64,linux/arm64 -t myapp:latest --push .

# Verify the manifest really holds both
docker buildx imagetools inspect myapp:latest | grep -E 'Platform|Digest'

# Smoke each arch natively (or via --platform on mixed runners)
docker run --rm --platform linux/amd64 myapp:latest --version
docker run --rm --platform linux/arm64 myapp:latest --version

Shebang, Exec Bit, CRLF: the Script Family

When architectures match and exec still fails, the entrypoint script is broken in one of three ways. Missing shebang: the kernel has no interpreter to invoke, so ENOEXEC even though sh could run the file fine — add #!/bin/sh (or the real interpreter) as byte one. Lost exec bit: COPY from some contexts (notably Windows checkouts) strips +x, so chmod +x in the Dockerfile with an explicit RUN chmod. CRLF line endings: the shebang becomes #!/bin/sh<CR>, naming an interpreter with a carriage return that doesn't exist — invisible in editors, fatal to the kernel.

Diagnose the bytes, not the text. head -1 file | od -c shows the truth: # ! /bin/sh is healthy, missing #! is obvious, and \r is the CRLF smoking gun. test -x confirms the exec bit inside the image (not on your laptop — the image is what runs). And verify the interpreter ships in the base: Alpine has /bin/sh but no /bin/bash, slim images drop shells you assumed, distroless has no shell at all — a #!/bin/bash shebang in those images fails exactly like a missing file.

Fix at every layer. Dockerfile: COPY then RUN chmod +x, with the shebang as the file's first line. Repo: .gitattributes forcing text=auto eol=lf on *.sh so Windows checkouts can't reintroduce CRLF. CI: a smoke that boots the real entrypoint (not sh -c) so script breakage fails builds. The script family is entirely preventable — each guard is one line, and together they close it forever.

# Read the bytes, not the text: want '# ! /bin/sh \n', no \r
docker run --rm --entrypoint sh myapp:latest -c 'head -1 /app/start.sh | od -c | head -3'

# Exec bit inside the image (laptop perms don't matter)
docker run --rm --entrypoint sh myapp:latest -c 'test -x /app/start.sh && echo EXEC-OK || echo MISSING-X-BIT'

# Interpreter must exist in the base image
docker run --rm --entrypoint sh myapp:latest -c 'ls -l /bin/sh /bin/bash 2>&1'

# Fixes (Dockerfile + repo):
# COPY start.sh /app/start.sh
# RUN chmod +x /app/start.sh        # restore exec bit
# first line of start.sh: #!/bin/sh # real interpreter only
# .gitattributes: *.sh text=auto eol=lf  # CRLF can never return

Lock It In: Gates That End This Error

Two gates close both families permanently. The architecture gate runs in CI after the push: inspect the pushed image's .Architecture (and manifest platforms for multi-arch) and compare against the architectures of your clusters and laptop fleet. Mismatch fails the pipeline before any rollout starts — the Thursday hand-push would have died right there with a message naming arm64 vs amd64. The entrypoint gate boots the image with its real entrypoint and runs the health check: no sh overrides, no --platform crutches, the exact command production will exec.

Back these with access hygiene. Humans shouldn't push release tags — restrict registry writes to the CI identity so laptop builds can't overwrite release bytes. Pin base images by digest so upstream arch changes arrive as deliberate PRs, not surprises. And document the fleet's architectures in one place: which clusters are amd64, which edge nodes are arm64, what developers run — so build platform lists stay correct as hardware changes.

Verify the whole chain after adopting multi-arch: imagetools shows both platforms, per-arch smokes pass, the arch gate is green, and a canary pod on each node type starts clean. Exec format error is a build-time fact meeting a run-time CPU; checking the fact before the meeting means they never disagree again. That's the entire strategy: prove arch and entrypoint in CI, and production only ever execs what the kernel can run.

# Gate 1: pushed arch must match the cluster (fail pipeline otherwise)
IMG_ARCH=$(docker image inspect --format '{{.Architecture}}' myapp:${TAG})
CLUSTER_ARCHES=$(kubectl get nodes -o jsonpath='{range .items[*]}{.status.nodeInfo.architecture}{"\n"}{end}' | sort -u)
echo "image=$IMG_ARCH cluster: $CLUSTER_ARCHES"
echo "$CLUSTER_ARCHES" | grep -qx "$IMG_ARCH" || { echo 'ARCH MISMATCH — blocking rollout'; exit 1; }

# Gate 2: boot the REAL entrypoint, no overrides
# docker run --rm myapp:${TAG} /healthcheck

# Post-adoption proof: manifest + per-arch smokes + canary
# docker buildx imagetools inspect myapp:${TAG} | grep Platform
kubectl rollout status deploy/myapp --timeout=120s
kubectl logs -l app=myapp --tail=3