handbook/docs/deploy.md

Deployment Guide

How a project gets onto rethink-net — our Ubuntu 26.x servers. Get your container to follow a few consistent rules and deploying is mostly handing us a compose.yaml.

!!! tip "What can run here" APIs, websites, applets, bots, monitors.

!!! danger "Your compose names nothing repo-specific" No container_name. No hardcoded names, workspace, or /srv paths. The deploy layer injects identity and host paths at deploy time — your compose stays generic so it can't collide with any other service on the fleet.

- service key is always **`svc`**
- named volumes use **bare names** (`cache`, not `myapp_cache`)
- host paths come from **`${LOGS_DIR}` / `${CONFIG_DIR}`**

Naming things after your repo used to cause **container-name collisions** at
fleet scale — two repos shipping the same name clashed. The generic compose
below fixes that; copy it verbatim.

Docker — the services account

Every service runs containerized as the shared services account: uid/gid 1337, fixed fleet-wide. Build your image to be uid-agnostic so it runs cleanly as that account.

FROM python:3.12-slim
ENV HOME=/tmp                                  # (1)!

WORKDIR /app
RUN apt-get update \
    && apt-get install -y --no-install-recommends git \
    && rm -rf /var/lib/apt/lists/*            # (2)!

COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt   # (3)!

COPY . .
RUN chmod -R a+rwX /app                        # (4)!
CMD ["python", "-m", "yourapp"]

1. HOME=/tmp — the services account has no home dir; anything writing to $HOME (caches, configs) needs a writable target.
2. Include git only if the container itself needs it — e.g. you pip install from git, or the app shells out to git at runtime. The build and host always have git; this line is about what's inside the image.
3. Install deps before copying the code (see layer caching).
4. chmod -R a+rwX /app makes the app tree writable by any uid — that's what "uid-agnostic" means.

!!! tip "Faster builds with uv (optional)" uv is a drop-in for pip that reads the same pyproject.toml — no lockfile needed in the image. Swap the deps layer and add UV_COMPILE_BYTECODE so containers don't pay the first-import .pyc compile cost:

```dockerfile
FROM python:3.12-slim
COPY --from=ghcr.io/astral-sh/uv:latest /uv /uvx /bin/   # (1)!
ENV HOME=/tmp
ENV UV_COMPILE_BYTECODE=1                              # (2)!

WORKDIR /app
COPY pyproject.toml .
RUN uv pip install --system .                          # (3)!
COPY . .
RUN chmod -R a+rwX /app
CMD ["python", "-m", "yourapp"]
```

1.  Pull the `uv` binary from its published image — no pip-installing uv itself.
2.  Compile bytecode at build time so the container doesn't eat the first-import
    `.pyc` compile cost on every cold start.
3.  `--system` installs into the image's Python (no venv needed — the container
    *is* the isolation); reads `pyproject.toml`, no `uv.lock` required.

Getting your files into the image

COPY . . grabs the whole repo, but be explicit about anything that needs its own place — assets, templates, a config the app reads at runtime. COPY <src> <dest>: <src> is relative to the build context (your repo), <dest> is a path in the image.

=== "A single file"

```dockerfile
COPY ./config.toml /app/config.toml   # (1)!
```

1.  One file into a specific path. The app reads it at `/app/config.toml`.

=== "A directory"

```dockerfile
COPY ./assets /app/assets   # (1)!
```

1.  A whole tree. Trailing paths are dirs — `/app/assets/` mirrors `./assets/`.

=== "Deps first (layer caching)"

```dockerfile
COPY requirements.txt .     # (1)!
RUN pip install --no-cache-dir -r requirements.txt
COPY . .                    # (2)!
```

1.  Copy just the deps file first, install, **then** copy the code.
2.  See [layer caching](#layer-caching) — this ordering is why.

!!! warning "Don't COPY secrets into the image" Anything sensitive stays out of the image — no COPY ./secrets.env. Secrets live on the host and are injected read-only at runtime (see Secrets). Add them to .dockerignore so a blanket COPY . . can't sweep them in.

Your compose.yaml

Copy this verbatim. It names nothing repo-specific — the deploy layer fills in identity and host paths.

services:
  svc:                                       # (1)!
    build: .
    user: "1337:1337"                        # (2)!
    restart: unless-stopped                  # (3)!
    environment:
      HOME: /tmp
    volumes:
      - ${LOGS_DIR:-./logs}:/app/logs        # (4)!
      - ${CONFIG_DIR:-./config}:/app/config  # (5)!
      # - ${MOUNTS_DIR:-./mounts}:/app/data  # (6)!
      - cache:/app/cache                     # (7)!

volumes:
  cache:                                     # (8)!

1. The service key is always svc — never a repo-specific name, never a container_name. The ops tooling keys off svc.
2. Run as the shared services account (uid/gid 1337, fixed fleet-wide). No in-container user/useradd — set it here. This is the one identity line that stays; it's not repo-specific.
3. Required. The host-side service is oneshot; Docker's own restart policy is what recovers a crashed container.
4. Logs → host (output), path injected by the deploy layer. The :-./logs default lets you docker compose up locally with nothing set and still work.
5. Config → host (input you edit), likewise injected.
6. Optional, commented by default. A host dir for arbitrary read/write data. The deploy layer injects MOUNTS_DIR and auto-creates the dir — but you uncomment the line and pick the container path (/app/data here), since that's app-specific and can't be auto-mounted.
7. Named volume with a bare name — ephemeral, auto-namespaced per service at deploy time so it can't collide.
8. Declare bare (cache, not myapp_cache). Deploy auto-prefixes it.

!!! tip "Three tiers of storage" - logs (output) and config (input you edit) — we handle these: injected and managed at deploy time. - mounts (optional) — a host dir for arbitrary read/write data. The deploy layer injects MOUNTS_DIR and auto-creates the dir, but the mount line is opt-in: uncomment it and choose the container path yourself (it's app-specific, so it can't be auto-mounted). - Named volumes (cache, …) — yours: anything else your service persists. Docker owns them, no host paths to manage.

!!! tip "The ${VAR:-./default} pattern" Host paths use a variable with a local fallback. Locally, docker compose up needs nothing set — it uses ./logs, ./config (and ./mounts if you enable it). When deployed, the ops tooling sets the real values, so the same file works both places.

Deployment is handled centrally

!!! note "You don't run any deploy commands" All services are deployed centrally by the ops tooling. You don't pick a host, manage keys, or run anything — just make your repo follow the compose convention above and it's deployable.

At deploy time the tooling generates the environment your compose reads, so the ${...} variables resolve without anything from you. You can rely on these being set — that's why your compose works unchanged on the fleet:

LOGS_DIR      # your host log dir      -> ${LOGS_DIR}
CONFIG_DIR    # your host config dir   -> ${CONFIG_DIR}
MOUNTS_DIR    # optional host data dir -> ${MOUNTS_DIR} (if you enable the mount)

Write your logs and config to those mount points and you're set. If a service won't come up or its logs aren't persisting, that's a host-side detail on our end — flag it and we'll sort it.

Layer caching

Copy the deps file and install before COPY . .. Docker caches layers in order, so deps only reinstall when the deps file changes — not on every code edit. Get this backwards and every one-line change triggers a full dependency reinstall. Same principle whether you use pip or uv:

COPY requirements.txt .                              # pip baseline
RUN pip install --no-cache-dir -r requirements.txt   # cached until deps change
COPY . .                                             # changes every build

COPY pyproject.toml .                                # uv path
RUN uv pip install --system .                        # cached until deps change
COPY . .                                             # changes every build

Subprocess and browser workloads

Bots that spawn Chrome, Xvfb, ffmpeg, or other child processes need three extra knobs in compose:

services:
  svc:
    build: .
    user: "1337:1337"
    restart: unless-stopped
    init: true              # (1)!
    shm_size: "2gb"         # (2)!
    mem_limit: "4g"         # (3)!

1. Runs tini as PID 1 to reap zombie subprocesses and forward signals. Without it, spawned Chrome/Xvfb processes leak as zombies.
2. Chrome and most headless browsers crash on Docker's default 64 MB /dev/shm. Bump it for any browser workload.
3. Bound memory — especially when each worker spawns a browser. Raise it as worker count grows.

!!! warning "The PID-1 gotcha with shell-wrapper CMDs" If your CMD is a shell-script wrapper (e.g. xvfb-run ...), it must not be PID 1, or the real process dies on startup. init: true is exactly what fixes this — tini takes PID 1, and your wrapper runs as a normal child.

What your compose / Dockerfile needs

Compose

• service key svc — no container_name, no repo-specific names
• user: "1337:1337" — the shared services account
• restart: unless-stopped
• host paths via ${LOGS_DIR} / ${CONFIG_DIR} (and optional ${MOUNTS_DIR}) — never hardcoded
• named volumes with bare names (cache, not myapp_cache)
• for browser/subprocess workloads: init: true, shm_size, mem_limit

Dockerfile

• HOME=/tmp
• chmod -R a+rwX /app (uid-agnostic; runs as the services account, 1337)
• deps installed before the code copy (layer caching) — pip or uv pip install
• git in the image if the container needs it
• using uv? add ENV UV_COMPILE_BYTECODE=1

Secrets

!!! warning "Secrets never go in the image" We do not commit secrets (usually, lol). They stay gitignored and live on the host in your config dir, reaching your container read-only via ${CONFIG_DIR}. Add them to .dockerignore so a COPY . . can't sweep them into a layer.

Rotating a secret is a host-side edit — update the file and the service picks it up on restart. No rebuild, and nothing you run: flag it and we handle the restart.