> For the complete documentation index, see [llms.txt](https://docs.instruqt.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.instruqt.com/sandboxes/hosts/custom-container-images.md).

# Custom container images

Create custom container images for your sandboxes.

## Background

Before creating your custom Docker image, ensure that the image you need is not already available on the [Docker Hub](https://hub.docker.com/). Read the [Docker documentation](https://docs.docker.com/get-started/introduction/build-and-push-first-image/) to build your own Docker image.

## Create images

You have two options for host container images:

* Host publicly on Docker Hub
* Host privately in a Google Cloud Artifact Registry (see steps below)

### Docker Hub

Host publicly on Docker Hub. When using public Docker hub there is not any required integration or security changes required to run containers within your Instruqt sandbox.

1. Within your track sandbox configuration select add host\
   ![](https://2094212015-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MGJDYBXyftBAZb1Wq0e%2Fuploads%2FzQVEX0kRjC5nOhPVXR2K%2Fimage.png?alt=media\&token=5547d1ee-fbfa-455d-91c2-bdf005fcc679)
2. Provide your hostname and container name. For example public docker hub container "busybox" would just require "busybox" to be put in for the container image name. If you have your own namespace you would prefix the name of your repository with that namespace (ie. namespace/repository)\
   ![](https://2094212015-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MGJDYBXyftBAZb1Wq0e%2Fuploads%2FbjpvGjVEXQSaAdfJaeGI%2Fimage.png?alt=media\&token=ef6394fc-4487-4f58-a523-f384593828ff)
3. Click save to add the host to the sandbox

### Google Cloud Artifact Registry

Host privately in a Google Cloud Artifact Registry (see steps below)

{% hint style="warning" %}
Instruqt only supports Linux *x86/amd64* based images. If you are using an Apple Silicon Macbook, use the `--platform=linux/amd64` flag for `docker build command.`
{% endhint %}

If you opt for private images, you need to authorize Instruqt's service accounts access to your repository. You can do so by granting the role `roles/storage.objectViewer` on the backing storage and `roles/artifactregistry.reader` role to the following Instruqt service accounts:

* `serviceAccount:instruqt-participants-nodepool@instruqt-prod.iam.gserviceaccount.com`
* `serviceAccount:instruqt-backend@instruqt-prod.iam.gserviceaccount.com`

### Image streaming (caching) compatibility

Instruqt pulls sandbox container images onto GKE nodes that have [image streaming](https://cloud.google.com/kubernetes-engine/docs/how-to/image-streaming) enabled. With streaming, a container starts before its image is fully downloaded, which is the difference between a host that is ready in seconds and one that stalls on a multi-gigabyte pull. On average, this cuts image pull time by up to 63%. The biggest gains are on large, complex images such as a full IDE environment or a database engine. Small, lightweight images see less difference because they were already quick to start.

Caches are per region. The first pull of an image in a region runs at normal speed. Later pulls in the same region use the cached copy and benefit from the reduced pull time.

Streaming is best-effort. When an image is not eligible, GKE falls back to a full pull without reporting an error: the container still starts, only much slower. Empty layers and duplicate layers are the most common reasons an image becomes ineligible.

#### Requirements

* Store the image in a standard or remote Artifact Registry repository, or in a public registry on Docker Hub. Images hosted anywhere else, such as Quay, GitHub Container Registry, or Amazon ECR, are not eligible.
* Use an OCI or Docker V2 schema 2 manifest. Images built with the V2 manifest schema 1 format are not eligible; rebuilding with any current version of Docker or BuildKit produces a supported manifest.
* Build for `linux/amd64`.
* Keep the image free of empty and duplicate layers.
* Size must be greater than 10 MB.

#### Avoid empty layers

An empty layer is a layer whose filesystem diff contains no changes. Empty layers always carry the same well-known digest, `sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4`, so two of them in one image are duplicates as well: a single stray `RUN` repeated twice creates both problems at once.

Common sources of empty layers:

* `RUN` steps that change nothing on disk, such as `RUN true`, `RUN echo "building app"`, or `RUN ls -la /opt`.
* `COPY` or `ADD` of a directory that contains no files.
* `VOLUME` declarations, which create a mount point but no content.

Collapse related commands into a single layer instead:

```dockerfile
RUN apt-get update \
 && apt-get install -y --no-install-recommends curl jq \
 && rm -rf /var/lib/apt/lists/*
```

Metadata instructions such as `ENV`, `LABEL`, `WORKDIR`, `USER`, `EXPOSE`, `CMD`, and `ENTRYPOINT` do not create filesystem layers when you build with BuildKit, so they are safe to use freely.

#### Avoid duplicate layers

A duplicate layer is the same layer digest appearing more than once in an image manifest. Common sources:

* Copying identical content into the final image twice, for example two `COPY --from=builder` steps that resolve to the same files.
* Multi-stage builds where two stages run the same install commands and both are copied into the final image.
* Rebasing or appending layers onto a base image that already contains those exact layers.
* More than one empty layer.

Give each layer distinct content, and copy any given artifact into the final stage exactly once.

#### Verify an image before you use it

Inspect a local image with `docker`. Duplicate layers print a digest; a clean image prints nothing:

```bash
docker image inspect IMAGE --format '{{json .RootFS.Layers}}' | jq -r '.[]' | sort | uniq -d
```

Empty layers show up as `0B` steps in the build history:

```bash
docker history --no-trunc --format '{{.Size}}\t{{.CreatedBy}}' IMAGE | grep -E '^0B'
```

For an image already pushed to a registry, inspect the manifest with [crane](https://github.com/google/go-containerregistry/blob/main/cmd/crane/README.md):

```bash
IMAGE=LOCATION-docker.pkg.dev/PROJECT/REPO/IMAGE:TAG

# Duplicate layers
crane manifest "$IMAGE" | jq -r '.layers[].digest' | sort | uniq -d

# Empty layers
crane manifest "$IMAGE" \
  | jq -r '.layers[] | select(.digest == "sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4") | .digest'

# Manifest format; expect an OCI or Docker manifest v2 media type
crane manifest "$IMAGE" | jq -r '.mediaType, .schemaVersion'
```

#### Fix an existing image

If you cannot rebuild from the Dockerfile, flatten the image into a single layer. This removes every empty and duplicate layer:

```bash
crane flatten "$IMAGE" -t "${IMAGE}-flat"
```

{% hint style="info" %}
Flattening trades layer reuse for compatibility. A flattened image shares no layers with other images in the repository, so it costs more storage and takes longer to pull on the fallback path. Fixing the Dockerfile is the better long-term option.
{% endhint %}