qemu-amd64-docker-slow-rebuilds

Docker buildx rebuild on arm64 host via QEMU amd64 takes 25-40 minutes per iteration when the image includes torch + sentence-transformers. Any Python source change invalidates the pip-install layer, triggering full re-download under QEMU emulation. Default PyPI torch wheel on linux/amd64 also pulls 1.2GB of CUDA libs that are dead weight for CPU-only deployments like AWS App Runner.

1. Standard Dockerfile with COPY . . before pip install -e . — worked once, but every Python edit invalidates the big pip-install layer. 25+ minute rebuilds on every iteration.

1. Using PyPI default torch wheel — pulled nvidia_cudnn_cu13, nvidia_nccl_cu13, cuda_toolkit (~1.2GB) even though we only need CPU. No GPU at runtime on App Runner.

Two-stage Dockerfile that decouples deps from source:

FROM python:3.13-slim
WORKDIR /app

# Stage 1: heavy deps — cached independently of source
RUN pip install --no-cache-dir \
    --extra-index-url https://download.pytorch.org/whl/cpu torch

COPY pyproject.toml ./
COPY README.md ./

# Editable install needs package dirs to exist. Placeholders satisfy that contract
# without copying real source yet, so this layer only invalidates when pyproject changes.
RUN mkdir -p central_api local_mcp \
 && touch central_api/__init__.py local_mcp/__init__.py \
 && pip install --no-cache-dir -e "."

# Pre-download model so container cold-start doesn't stall on a 130MB HF fetch
RUN python -c "from sentence_transformers import SentenceTransformer; \
             SentenceTransformer('BAAI/bge-small-en-v1.5')"

# Stage 2: source — this layer rebuilds in seconds when only Python changes
COPY central_api ./central_api
COPY local_mcp ./local_mcp

EXPOSE 8080
CMD ["uvicorn", "--factory", "central_api.main:create_app", "--host", "0.0.0.0", "--port", "8080"]

Key insights:

• --extra-index-url https://download.pytorch.org/whl/cpu BEFORE other installs — pip prefers CPU wheel over CUDA one.
• Placeholder __init__.py files let pip install -e . resolve the package without the real source, caching the entire deps layer.
• Real COPY central_api happens LAST — invalidates nothing expensive.

First build still slow (~25min for the one-time deps layer build under QEMU). Every subsequent Python-only rebuild: <2 minutes including push to ECR.

• docker buildx with --platform linux/amd64
• Colima running on arm64 Mac with tonistiigi/binfmt --install amd64 for QEMU emulation
• --extra-index-url https://download.pytorch.org/whl/cpu for CPU-only torch

• You're building natively for the target arch (no QEMU) — cache invalidation still matters but wall-clock is much lower.
• Your deps layer legitimately changes every build (unlikely for production services).
• You need CUDA at runtime — obviously don't force the CPU wheel then.