Engineering Blog

The Requirement and the Problem

At cloudscale, we aim to run our coding agents in containers or (micro)VMs that restrict filesystem access to the project working directory and network egress to the inference API. Everything else is deny-by-default and only granted temporarily when required. Our current go-to tool for this use case is Docker Sandboxes, which runs each agent in its own Linux microVM and is driven by the sbx CLI.

When the agent has no way of reaching PyPI, npm or GitHub at runtime, everything has to be there when it starts. Unfortunately, this is more complicated than one would think, especially on a Mac: the project directory is shared with the Linux microVM, and the host's .venv and node_modules folders contain a bunch of native macOS binaries that cannot be used there. It is, however, quite funny watching a coding agent lose its mind over why jest cannot be run, until it discovers the architecture mismatch of unrs-resolver.

Why We Were Not Able to Use v2 Kits

Docker maintains images for a selected number of agents. These agent templates, as they are called, include some common development tools and runtimes such as Python and Node. But what if you need another tool or another version? Earlier this year, Docker introduced kits, which answered that question. Essentially, they allowed running some scripts right after the sandbox was created.

schemaVersion: "2"
kind: mixin
name: ruff
permissions:
  network:
    allow:
      - github.com
      - release-assets.githubusercontent.com
setup:
  install:
    - command: |
        curl -fsSL https://github.com/astral-sh/ruff/releases/download/0.17.3/ruff-$(uname -m)-unknown-linux-musl.tar.gz \
          | tar -xz --no-same-owner -C /usr/local/bin --strip-components=1

Unfortunately, the network permissions granted were not limited to the runtime of the install script, but remained active for the complete lifetime of the sandbox. This made v2 kits essentially useless for us, since the goal is to only allow access to the inference API. And github.com is about as good an exfiltration channel as it gets.

Another minor annoyance was the absence of any caching mechanism: more complicated hooks consume quite some time on each startup, and there is no way to store, share, sign and attest the result of the install hook.

We were able to overcome both limitations by extending the agent template, which is just a Dockerfile. Naturally, egress is not limited when building it with Docker, and we get cached layers.

FROM docker.io/sbx/claude-image:latest

USER root
RUN curl -fsSL https://github.com/astral-sh/ruff/releases/download/0.17.3/ruff-$(uname -m)-unknown-linux-musl.tar.gz \
  | tar -xz --no-same-owner -C /usr/local/bin --strip-components=1

USER agent

But now we need to build this for each agent that we want to support, and when we work on different software projects, we might want to select a different set of tools. Both problems can be overcome with just a Dockerfile and some layering trickery:

ARG BASE
ARG RUFF=scratch

FROM alpine AS ruff-build
RUN wget -qO- https://github.com/astral-sh/ruff/releases/download/0.17.3/ruff-$(uname -m)-unknown-linux-musl.tar.gz \
  | tar -xz --no-same-owner --strip-components=1 -C /usr/local/bin

FROM scratch AS ruff-enabled
COPY --from=ruff-build /usr/local/bin/ruff /usr/local/bin/ruff

FROM ${RUFF} AS ruff

FROM ${BASE}
COPY --from=ruff / /

This allows us to mix and match agents and tools from a single Dockerfile:

$ docker buildx build --no-cache --build-arg BASE=docker.io/sbx/claude-image:latest --build-arg RUFF=ruff-enabled -t claude-ruff .
$ docker buildx build --no-cache --build-arg BASE=docker.io/sbx/opencode-image:latest -t opencode .

Note: We use --no-cache because BuildKit's cache doesn't notice when RUFF switches back to scratch, so without it, a rebuild can quietly keep the ruff layer from an earlier build.

The v3 Paradigm Shift

At the end of September, Docker announced the Docker Sandbox Kit Specification v3, which addresses both problems mentioned above. First of all, network policies became much more elaborate and can now be restricted to the sandbox startup using the install keyword:

capabilities:
  - type: com.docker.sandbox/network-policy@2
    config:
      install:
        allow:
          - hosts: [github.com]
            methods: [GET]
            paths: [/astral-sh/ruff/releases/download/0.17.3/*]
          - hosts: [release-assets.githubusercontent.com]
            methods: [GET]
            paths: ["/**"]

As you can see, capabilities such as network-policy@2 carry their own version, separate from the kit spec version.

But with Kits v3, this is not the recommended way to install tools. Kits can now include their own Dockerfile (referred to as a build recipe) that can be built beforehand and that sbx stacks at sandbox create time, similar to our Dockerfile above. This also makes our own layering Dockerfile unnecessary: nothing is downloaded inside the sandbox, so the kit needs no network access at all, and the resulting image can be cached, pushed, signed and attested just like any other image.

# syntax=docker/sandbox-kit:3
schemaVersion: "3"
kind: mixin

build: |
  FROM alpine AS fetch
  RUN wget -qO- https://github.com/astral-sh/ruff/releases/download/0.17.3/ruff-$(uname -m)-unknown-linux-musl.tar.gz \
    | tar -xz --strip-components=1

  FROM scratch
  COPY --from=fetch /ruff /usr/local/bin/ruff
  ENTRYPOINT ["/usr/local/bin/ruff"]

The exciting thing is that this kit can be built using regular docker buildx. The # syntax= line instructs BuildKit to use the docker/sandbox-kit:3 BuildKit frontend. This frontend validates the descriptor data (the YAML part), builds the embedded Dockerfile just like any other Dockerfile, and adds the descriptor as an annotation to the image manifest as vnd.docker.sandbox.kit.descriptor.

Since regular Docker ignores the annotation, we can even run the kit as a regular image:

# The local registry and --provenance=false are only there to make the annotation easy to inspect
# (with provenance, the tag points to an index and the annotation sits one level down on the image manifest)

$ docker buildx build . -f ruff.yaml --provenance=false -t localhost:5000/ruff-kit --push
$ docker run --rm localhost:5000/ruff-kit --version
ruff 0.17.3
$ docker buildx imagetools inspect --raw localhost:5000/ruff-kit \
    | jq -r '.annotations["vnd.docker.sandbox.kit.descriptor"] | fromjson'
{
  "schemaVersion": "3",
  "kind": "mixin",
  "build": "FROM alpine AS fetch\nRUN wget -qO- https://github.com/astral-sh/ruff/...\nENTRYPOINT [\"/usr/local/bin/ruff\"]\n"
}

When we launch a sandbox with sbx run docker.io/docker/sbx-kit-claude:2.1.311 --kit ./ruff-kit, it first takes the Claude image and then adds the ruff kit layers on top, similar to what crane append would do.

The key insight here is that a built kit artifact is an annotated OCI image. If, for example, the annotation contains a network allow list, this metadata now travels with the image, and any compliant runtime (currently sbx) can adhere to it. Others (e.g. Docker) just ignore the annotation, but can still run it as a regular OCI image. Docker is bringing the spec to the CNCF, which means that in the future, we could see other compliant runtimes.

Deep Dive: Nix as the Perfect Mixin

Of course, stacking self-contained binaries on top of a base image is boring. But when you want to stack runtimes such as Node, Python or PostgreSQL onto a base image where you have no idea about, and no influence over, which libraries are available, you have no choice but to bring everything yourself. A handy tool to achieve this is Nix, and it has a handful of useful properties for this use case:

• Nix-built binaries are fully self-contained in /nix/store/ and load all their libraries from there. This makes them agnostic to any libraries in the base layer.
• There are no file conflicts with the base image. Apart from symlinks in /usr/local/bin, everything lives under /nix/store/<hash>-<name>.

So the suggested pattern is a FROM scratch layer holding just /nix and symlinks.

# syntax=docker/dockerfile:1
FROM --platform=$BUILDPLATFORM nixos/nix:2.35.2 AS build
ARG TARGETARCH PKG
# nixos-26.05 head on 2026-10-30
ARG NIXPKGS=github:NixOS/nixpkgs/6f06d204d8f2d8761d9da78778745974a3216381
RUN sys=$([ $TARGETARCH = arm64 ] && echo aarch64 || echo x86_64)-linux \
 && nix --extra-experimental-features 'nix-command flakes' \
      build "$NIXPKGS#legacyPackages.$sys.$PKG" \
 && mkdir -p /out/nix/store /out/usr/local/bin \
 && cp -a $(nix-store -qR result) /out/nix/store/ \
 && ln -s $(readlink -f result)/bin/* /out/usr/local/bin/

FROM scratch
COPY --from=build /out /

Then build with:

$ R=registry.example.com/sbx-mixins   # any registry repository you can push to
$ docker buildx build --build-arg PKG=nodejs_24 -t $R:nodejs_24-mixin --push .
$ docker buildx build --build-arg PKG=python313 -t $R:python313-mixin --push .

Let's see which versions the kits include:

$ for i in nodejs_24 python313; do
    echo "# $i"
    crane export $R:$i-mixin - | tar -t | grep -oE '^nix/store/[^/]*-(glibc|openssl)-[0-9][^/]*' | sort -u
  done
# nodejs_24
nix/store/dphsh3jff8w6n9p3xbfmkia5krqwv52f-glibc-2.42-103
nix/store/q1fgk31hkgl8a7znhmvb0bq8bbcaf64y-openssl-3.6.5
nix/store/fya253x5ky7rhfz6rkgdwjr1fzf7fy71-openssl-3.6.5-bin
nix/store/f0lcjigadrvilgw2fprzdd96fw055q0b-openssl-3.6.5-dev
# python313
nix/store/dphsh3jff8w6n9p3xbfmkia5krqwv52f-glibc-2.42-103
nix/store/q1fgk31hkgl8a7znhmvb0bq8bbcaf64y-openssl-3.6.5

The final useful property of Nix is that a Nix store path starts with a hash of every input that went into building it, so two images containing the same path contain the same package. Overlaying one copy on another therefore can't conflict. And anything that needs a different version gets a different path that sits alongside it instead of replacing it.

So when we stack the two images above, we get:

# stack
nix/store/dphsh3jff8w6n9p3xbfmkia5krqwv52f-glibc-2.42-103
nix/store/q1fgk31hkgl8a7znhmvb0bq8bbcaf64y-openssl-3.6.5
nix/store/fya253x5ky7rhfz6rkgdwjr1fzf7fy71-openssl-3.6.5-bin
nix/store/f0lcjigadrvilgw2fprzdd96fw055q0b-openssl-3.6.5-dev

Our Current Setup

The last problem we had to solve is that in the default mode of Docker Sandboxes, the workspace is shared between the host and the microVM. In other words, the host's .venv and node_modules are visible to, but not usable by, the agent, and there is no way to exclude them from the sync yet. Luckily, python -m venv takes a path argument, which allowed us to put the Linux venv somewhere else inside the microVM.

For node_modules, there seems to be no equivalent option. So the best thing we came up with was to bind-mount another directory over $WORKSPACE_DIR/node_modules (the project directory inside the sandbox) in a kit. Since the Docker-provided agent images do not include the mount command, we had to pivot to a raw mount syscall from Python: ctypes.CDLL(None, use_errno=True).mount(src.encode(), dst.encode(), None, 4096, None) (4096 is MS_BIND; this runs as root at sandbox startup).

Using the v3 kit volume capability was a dead end, since the volume was mounted before $WORKSPACE_DIR.

So our current setup looks roughly like this:

• A PostgreSQL mixin.
• A Python mixin with the interpreter from Nix and the venv created at startup time.
• A Node mixin with the interpreter from Nix and the node_modules populated at startup time.

The venv and node_modules are still populated at startup time, during the install phase with an install-scoped allow list for PyPI and npm that is revoked before the agent starts. This is necessary since the kit's build context does not include the project files. In a way, this makes sense: it should not matter whether you build the kit locally or pull it from a registry, where your local project was obviously not available. But we lose the startup time benefit.

Rough Edges

The Docker-built OpenCode kit comes with a broad allow list, and even a global deny-all policy does not override it. The same goes for the plain shell kit, which allows deb.debian.org and dhi.io. While there is a deny clause in the network-policy spec, and deny wins across the merged allows, there is as of now no "deny everything except X". So we have to either keep a deny list in sync with every host in the published allow list, knowing that upstream can add new hosts at any time, or create a new image with our own set of descriptors.

Another quirk to be aware of is that a mixin build recipe can set a PATH variable, and when sbx stacks the layers, the paths of the mixin get appended. This means that if the agent kit already contains a command with the same name, the agent kit's version wins.

What This Protects Against, and What It Doesn't

No internet for the agent is not a silver bullet. Agent security follows the Swiss cheese model: every layer has holes, and only stacked together do they stop most attacks. This is what our layer covers and where its holes are:

✅ Protects against	❌ Doesn't protect against
Exfiltration to arbitrary hosts. A prompt-injected agent can't curl your code or secrets to an attacker's server, push to a random repo, or post to a pastebin.	The inference API itself. Everything in the agent's context goes to the model. If that model is hosted, your prompts leave by design. If the provider runs tools like web search on its own servers, those sit outside your egress policy too.
Reading outside the project. No ~/.ssh, no browser profile, no cloud credentials from the host.	Secrets inside the project. A .env in the working directory is readable by the agent, and from there it ends up in the prompt.
Installing things at runtime. A hallucinated or malicious package can't be pulled in mid-session.	The install phase. Dependencies are fetched at startup, and their install scripts run while the registry is reachable. Use lockfiles with hashes, skip install scripts where you can, and restrict registry access to GET.
Chatty clients. Telemetry and crash reports can't reach vendor hosts, unless they go to the same host as the inference API.	Writing back to the host, unless you use clone mode. In the default mode, the workspace is shared, so files the agent writes land on your machine immediately: Git hooks or a .venv run unsandboxed the next time you use them, and since Git doesn't track them, they never show up in a diff. Clone mode (which I had asked for) gives the agent its own Git clone, so only commits you fetch come back.
Tool drift. Every tool can be pinned, built ahead of time, and can be signed and attested.	Bad code. A manipulated agent can still write subtly wrong or malicious code into the project. The sandbox contains the agent, not its output.

For the other slices, see our post on navigating AI coding agents.

Conclusion

Does security come at a cost? Yes. Do we think it's worth it? Yes. Did we over-engineer compared to people who just run claude? Probably yes. Was it a fun learning journey? YES!

---

Wenn du uns Kommentare oder Korrekturen mitteilen möchtest, kannst du unsere Engineers unter engineering-blog@cloudscale.ch erreichen.