handbook/docs/deploy.md

# Deployment Guide

> How a project gets onto **rethink-net** — our Ubuntu 26.x servers. Get your
> container to follow a few consistent rules and deploying is mostly handing us a
> `compose.yaml`.

!!! tip "What can run here"
    APIs, websites, applets, bots, monitors.

## Preparing for Deploy

Get these three right and your repo is deployable — you don't run any deploy commands
yourself, the ops tooling takes it from git. Each tab is one piece.

!!! danger "Your compose names nothing repo-specific"
    **No `container_name`. No hardcoded names, workspace, or `/srv` paths.** The
    deploy layer injects identity and host paths at deploy time — your compose stays
    generic so it can't collide with any other service on the fleet.

    - the service key is **always `svc`** — it never changes, in any repo
    - named volumes use **bare names** (`cache`, not `myapp_cache`)
    - host paths come from **`${LOGS_DIR}` / `${CONFIG_DIR}`**

    Naming things after your repo used to cause **container-name collisions** at
    fleet scale — two repos shipping the same name clashed. The generic compose in
    the **Compose** tab fixes that.

=== "Compose"

    The `compose.yaml` your repo ships. Copy it verbatim — it names nothing
    repo-specific, so the deploy layer can inject identity and host paths without
    collisions.

    ```yaml
    services:
      svc:                                     # generic service key — ALWAYS svc, no container_name
        build: .
        user: "1337:1337"                      # the shared services account (required)
        restart: unless-stopped                # required — host unit is oneshot; this recovers crashes
        environment:
          HOME: /tmp
        volumes:
          - ${LOGS_DIR:-./logs}:/app/logs        # output   — host log dir, injected at deploy
          - ${CONFIG_DIR:-./config}:/app/config  # input     — host config dir, injected at deploy
          # - ${MOUNTS_DIR:-./mounts}:/app/data  # optional  — arbitrary host data (opt-in)
          - cache:/app/cache                     # your data — ephemeral named volume

    volumes:
      cache:                                   # bare name — deploy auto-prefixes it per service
    ```

    **The rules:**

    - **Service key is always `svc`** — never a repo-specific name, never a
      `container_name`. The ops tooling keys off `svc`; this is what makes services
      collision-proof on the fleet.
    - **`user: "1337:1337"` and `restart: unless-stopped` are required** — the shared
      `services` account (uid/gid 1337, fixed fleet-wide), and Docker's restart policy
      recovering a crashed container (the host unit is oneshot).
    - **Host paths come from `${...}` variables, never hardcoded.** Write to
      `${LOGS_DIR}` and `${CONFIG_DIR}`; `${MOUNTS_DIR}` is optional.
    - **Named volumes use bare names** (`cache`, not `myapp_cache`) — deploy
      auto-prefixes them per service so they can't collide.

    !!! tip "The container path is where your code reads and writes"
        `WORKDIR` is `/app`, so the **right-hand side** of each volume line is the path
        your code targets. If your app writes to `./cache` (i.e. `/app/cache`), that's
        the `cache` mount; logs go to `/app/logs`, config is read from `/app/config`.

    **Three tiers of storage:**

    - **`logs`** (output) and **`config`** (input you edit) — **we handle these**:
      injected and managed at deploy time.
    - **`mounts`** (optional) — a host dir for arbitrary read/write data. The deploy
      layer **injects `MOUNTS_DIR` and auto-creates the dir**, but the mount line is
      **opt-in**: uncomment it and choose the container path yourself (it's
      app-specific, so it can't be auto-mounted).
    - **Named volumes** (`cache`, …) — **yours**: anything else your service persists.
      Docker owns them, no host paths to manage.

    Locally, `docker compose up` needs nothing set — the `${VAR:-./default}` fallbacks
    use `./logs`, `./config` (and `./mounts` if you enable it). When deployed, the ops
    tooling sets the real values, so the same file works both places.

    **Subprocess and browser workloads** (bots that spawn Chrome, Xvfb, ffmpeg) need
    three extra knobs:

    ```yaml
    services:
      svc:
        build: .
        user: "1337:1337"
        restart: unless-stopped
        init: true              # tini as PID 1 — reaps zombie subprocesses, forwards signals
        shm_size: "2gb"         # Chrome/headless browsers crash on Docker's default 64 MB /dev/shm
        mem_limit: "4g"         # bound memory — raise as worker/browser count grows
    ```

    !!! warning "The PID-1 gotcha with shell-wrapper CMDs"
        If your `CMD` is a shell-script wrapper (e.g. `xvfb-run ...`), it must **not**
        be PID 1, or the real process dies on startup. `init: true` fixes this — tini
        takes PID 1, your wrapper runs as a normal child.

=== "Dockerfile"

    Every service runs containerized as the shared **`services`** account: **uid/gid
    1337**, fixed fleet-wide. Build the image to be **uid-agnostic** so it runs cleanly
    as that account.

    ```dockerfile
    FROM python:3.12-slim
    ENV HOME=/tmp                                  # services account has no home dir; $HOME must be writable

    WORKDIR /app
    RUN apt-get update \
        && apt-get install -y --no-install-recommends git \
        && rm -rf /var/lib/apt/lists/*             # include git ONLY if the container itself needs it

    COPY requirements.txt .
    RUN pip install --no-cache-dir -r requirements.txt   # deps before code — layer caching

    COPY . .
    RUN chmod -R a+rwX /app                         # writable by any uid — this is "uid-agnostic"
    CMD ["python", "-m", "yourapp"]
    ```

    **Layer caching:** copy the deps file and install **before** `COPY . .` — Docker
    caches layers in order, so deps only reinstall when the deps file changes, not on
    every code edit.

    **Getting files in:** `COPY . .` grabs the whole repo; be explicit about anything
    that needs its own place. In `COPY <src> <dest>`, `<src>` is relative to the build
    context (your repo), `<dest>` is a path in the image.

    ```dockerfile
    COPY ./config.toml /app/config.toml   # a single file into a specific path
    COPY ./assets /app/assets             # a whole directory (tree mirrors ./assets/)
    ```

    !!! tip "Faster builds with uv (optional)"
        [uv](https://docs.astral.sh/uv/) is a drop-in for pip that reads the same
        `pyproject.toml` — no lockfile needed in the image. Add `UV_COMPILE_BYTECODE`
        so containers don't pay the first-import `.pyc` compile cost:

        ```dockerfile
        FROM python:3.12-slim
        COPY --from=ghcr.io/astral-sh/uv:latest /uv /uvx /bin/   # pull the uv binary from its image
        ENV HOME=/tmp
        ENV UV_COMPILE_BYTECODE=1                                # compile bytecode at build, not cold start

        WORKDIR /app
        COPY pyproject.toml .
        RUN uv pip install --system .                            # into the image's Python, no venv/lock
        COPY . .
        RUN chmod -R a+rwX /app
        CMD ["python", "-m", "yourapp"]
        ```

=== "Secrets"

    !!! warning "Secrets never go in the image"
        We do **not** commit secrets (usually, lol). They stay **gitignored** and live
        on the host in your config dir, reaching your container read-only via
        `${CONFIG_DIR}`. Add them to `.dockerignore` so a `COPY . .` can't sweep them
        into a layer.

    - Secrets live on the **host**, in your config dir — never in git, never in the
      image.
    - They reach the container **read-only** via the injected `${CONFIG_DIR}` mount.
    - Keep them out of the build context: list them in `.dockerignore` so a blanket
      `COPY . .` can't pull them into a layer.

    **Rotating a secret** is a host-side edit — update the file and the service picks
    it up on restart. No rebuild, and nothing you run: flag it and we handle the
    restart.

## The path to deploy

You build and test locally — leaning on our libraries, AI, and this handbook — then
push to git. From there it's staging (if you're running gitflow) and, once it's good,
deployment: the ops tooling builds the Docker image and starts it on the fleet.

```mermaid
flowchart LR
    subgraph build ["you build"]
        direction TB
        local["local testing"]
        libs["our libraries<br/>(rethink-public)"]
        ai["AI-assisted<br/>(Claude Code)"]
        docs["reading the<br/>handbook"]
        local --- libs
        libs --- ai
        ai --- docs
        docs --- local
    end

    build -->|push| git["git<br/>(Gitea)"]
    git -.->|develop, if gitflow| stage["staging"]
    git --> deploy["deployment<br/>(ops tooling)"]
    stage --> deploy
    deploy --> docker["docker<br/>(image built + run on the fleet)"]

    classDef ship fill:#061541,stroke:#569bcc,color:#eef1f6;
    classDef work fill:#0e1530,stroke:#294274,color:#eef1f6;
    class git,stage,deploy,docker ship;
    class local,libs,ai,docs work;
```