handbook/docs/deploy.md

# Deployment Guide

> How a project gets onto **rethink-net** — our Ubuntu 26.x servers. Get your
> container to follow a few consistent rules and deploying is mostly handing us a
> `compose.yaml`.

!!! success "What can run here"
    APIs, websites, applets, bots, monitors.

## Preparing for Deploy

You build and test locally — leaning on our libraries, AI, and this handbook — then
push to git. From there the ops tooling takes over: staging (if you run gitflow), then
it builds the Docker image and runs it on the fleet. **You don't run any deploy
commands** — your only job is the three pieces in the tabs below.

```mermaid
flowchart LR
    subgraph build ["you build"]
        direction TB
        local["local testing"]
        libs["libs<br/>(rethink-public, pip)"]
        ai["AI-assisted<br/>(Claude Code)"]
        docs["reading the<br/>handbook"]
        local --- libs
        libs --- ai
        ai --- docs
        docs --- local
    end

    build -->|push| git["git<br/>(Gitea)"]
    git -.->|develop, if gitflow| stage["staging"]
    git --> deploy["deployment<br/>(ops tooling)"]
    stage --> deploy
    deploy --> docker["docker<br/>(built + run<br/>on the fleet)"]

    classDef ship fill:#061541,stroke:#569bcc,color:#eef1f6;
    classDef work fill:#0e1530,stroke:#294274,color:#eef1f6;
    class git,stage,deploy,docker ship;
    class local,libs,ai,docs work;
```

!!! danger "Your compose names nothing repo-specific"
    **No `container_name`, no hardcoded names/paths.** The deploy layer injects
    identity and host paths so your compose can't collide with any other service on
    the fleet:

    - the service key is **always `svc`** — it never changes, in any repo
    - named volumes use **bare names** (`cache`, not `myapp_cache`)
    - host paths come from **`${LOGS_DIR}` / `${CONFIG_DIR}`**

=== "Compose"

    The `compose.yaml` your repo ships. Copy it verbatim — it names nothing
    repo-specific, so the deploy layer can inject identity and host paths without
    collisions.

    ```yaml
    services:
      svc:                                     # generic service key — ALWAYS svc, no container_name
        build: .
        user: "1337:1337"                      # the shared services account (required)
        restart: unless-stopped                # required — host unit is oneshot; this recovers crashes
        environment:
          HOME: /tmp
        volumes:
          - ${LOGS_DIR:-./logs}:/app/logs        # output   — host log dir, injected at deploy
          - ${CONFIG_DIR:-./config}:/app/config  # input     — host config dir, injected at deploy
          # - ${MOUNTS_DIR:-./mounts}:/app/data  # optional  — arbitrary host data (opt-in)
          - cache:/app/cache                     # your data — ephemeral named volume

    volumes:
      cache:                                   # bare name — deploy auto-prefixes it per service
    ```

    !!! example "The container path is where your code reads and writes"
        `WORKDIR` is `/app`, so the **right-hand side** of each volume line is the path
        your code targets. Write to `./cache` (i.e. `/app/cache`) → the `cache` volume;
        logs go to `/app/logs`, config is read from `/app/config`.

    **Storage, three tiers:**

    - **`logs` + `config`** — we inject and manage these at deploy time.
    - **`mounts`** (optional) — host dir for arbitrary data. We inject `MOUNTS_DIR` and
      create the dir; you **uncomment** the line and pick the container path.
    - **Named volumes** (`cache`, …) — yours; Docker owns them, no host paths to manage.

    !!! warning "Two kinds of logs — and crashes go to the other one"
        `${LOGS_DIR}` holds **only** what your app writes to disk through its logger
        (e.g. `log_setup` writing a file) — your own structured logging. It does **not**
        capture the process's stdout/stderr, and that's where **startup crashes and
        uncaught exceptions land** — a traceback from a failed import or a missing file
        never reaches your logger. So if a service dies on startup, or you don't see the
        error in your log files, it's in the process output, not `${LOGS_DIR}`. Make
        fatal errors visible — and to route uncaught exceptions into your log file too,
        install a top-level hook:

        ```python
        import sys, logging
        sys.excepthook = lambda *exc: logging.getLogger().critical("uncaught", exc_info=exc)
        ```

    !!! success "Same file, both places"
        Locally, `docker compose up` needs nothing set — the `${VAR:-./default}`
        fallbacks use `./logs` / `./config`. When deployed, the ops tooling sets the
        real values. One `compose.yaml` works everywhere.

    **Subprocess and browser workloads** (bots that spawn Chrome, Xvfb, ffmpeg) need
    three extra knobs:

    ```yaml
    services:
      svc:
        build: .
        user: "1337:1337"
        restart: unless-stopped
        init: true              # tini as PID 1 — reaps zombie subprocesses, forwards signals
        shm_size: "2gb"         # Chrome/headless browsers crash on Docker's default 64 MB /dev/shm
        mem_limit: "4g"         # bound memory — raise as worker/browser count grows
    ```

    !!! warning "The PID-1 gotcha with shell-wrapper CMDs"
        If your `CMD` is a shell-script wrapper (e.g. `xvfb-run ...`), it must **not**
        be PID 1, or the real process dies on startup. `init: true` fixes this — tini
        takes PID 1, your wrapper runs as a normal child.

=== "Dockerfile"

    Every service runs containerized as the shared **`services`** account: **uid/gid
    1337**, fixed fleet-wide. Build the image to be **uid-agnostic** so it runs cleanly
    as that account.

    ```dockerfile
    FROM python:3.12-slim
    ENV HOME=/tmp                                  # services account has no home dir; $HOME must be writable

    WORKDIR /app
    RUN apt-get update \
        && apt-get install -y --no-install-recommends git \
        && rm -rf /var/lib/apt/lists/*             # include git ONLY if the container itself needs it

    COPY requirements.txt .
    RUN pip install --no-cache-dir -r requirements.txt   # deps before code — layer caching

    COPY . .
    RUN chmod -R a+rwX /app                         # writable by any uid — this is "uid-agnostic"
    CMD ["python", "-m", "yourapp"]
    ```

    **Layer caching:** copy the deps file and install **before** `COPY . .` — Docker
    caches layers in order, so deps only reinstall when the deps file changes, not on
    every code edit.

    **Getting files in:** `COPY . .` grabs the whole repo; be explicit about anything
    that needs its own place. In `COPY <src> <dest>`, `<src>` is relative to the build
    context (your repo), `<dest>` is a path in the image.

    ```dockerfile
    COPY ./config.toml /app/config.toml   # a single file into a specific path
    COPY ./assets /app/assets             # a whole directory (tree mirrors ./assets/)
    ```

    !!! tip "Faster builds with uv (optional)"
        [uv](https://docs.astral.sh/uv/) is a drop-in for pip that reads the same
        `pyproject.toml` — no lockfile needed in the image. Add `UV_COMPILE_BYTECODE`
        so containers don't pay the first-import `.pyc` compile cost:

        ```dockerfile
        FROM python:3.12-slim
        COPY --from=ghcr.io/astral-sh/uv:latest /uv /uvx /bin/   # pull the uv binary from its image
        ENV HOME=/tmp
        ENV UV_COMPILE_BYTECODE=1                                # compile bytecode at build, not cold start

        WORKDIR /app
        COPY pyproject.toml .
        RUN uv pip install --system .                            # into the image's Python, no venv/lock
        COPY . .
        RUN chmod -R a+rwX /app
        CMD ["python", "-m", "yourapp"]
        ```

=== "Secrets"

    !!! warning "Secrets never go in the image"
        We do **not** commit secrets (usually, lol). They stay **gitignored** and live
        on the host in your config dir, reaching your container read-only via
        `${CONFIG_DIR}`. Add them to `.dockerignore` so a `COPY . .` can't sweep them
        into a layer.

    - Secrets live on the **host**, in your config dir — never in git, never in the
      image.
    - They reach the container **read-only** via the injected `${CONFIG_DIR}` mount.
    - Keep them out of the build context: list them in `.dockerignore` so a blanket
      `COPY . .` can't pull them into a layer.

    **Rotating a secret** is a host-side edit — update the file and the service picks
    it up on restart. No rebuild, and nothing you run: flag it and we handle the
    restart.

## Supporting services live in your compose, not the fleet

Need Redis? **Declare it in your own `compose.yaml`, as a sidecar.** There is no shared
Redis — nothing fleet-wide, nothing per-workspace, nothing ops provisions for you. A repo
that needs Redis brings its own; a repo that doesn't adds nothing.

That's the whole point of the deploy model. Every service already runs in its own compose
project on its own network so that one service falling over can't touch another. A shared
Redis puts that coupling straight back: one process everything depends on, whose OOM, stray
`FLUSHALL`, single-threaded stall, or restart becomes *everyone's* outage. A sidecar shares
its owning repo's fate and nobody else's — and the isolation is free, because it rides the
per-project network you already get. No ACLs, no key-prefix discipline, no shared
credentials to manage.

Your app talks to it with the [`redis` lib](libraries.md) from the suite (async,
config-free, kv/hash/ttl/pubsub), pointed at **`redis://redis:6379`** — the compose
**service name**, not a host port. The sidecar comes up auto-namespaced on your project's
network like every other container, exactly as the naming convention above describes.

### Ephemeral or persistent — pick deliberately

A sidecar Redis is **ephemeral by default**: restart it and the data is gone. That's
correct for some workloads and quietly destructive for others, so make the call on purpose.
Ask one question — *if this data vanished on a restart, would anything be lost?*

- **No → ephemeral.** A scratch cache, a dedupe set, rate-limit counters, transient data
  you can just re-fetch. Nothing to back up, nothing to grow.
- **Yes → persistent.** An outbound webhook or notification queue, a job queue, anything
  that could be mid-flight when the process dies. Losing it drops real work.

=== "Ephemeral (cache / throwaway)"

    Fine to lose on restart — no volume, no persistence, by design.

    ```yaml
    redis:
      image: redis:7-alpine
      restart: unless-stopped
      command: redis-server --save "" --maxmemory 256mb --maxmemory-policy allkeys-lru
      # no volume: throwaway by design
    ```

=== "Persistent (durable queue / state)"

    Survives restart, rebuild, and reboot — the append-only file lives on the host mounts
    dir injected at deploy, the same `${MOUNTS_DIR}` mechanism described above. Redis just
    uses it as its backing store.

    ```yaml
    redis:
      image: redis:7-alpine
      restart: unless-stopped
      command: redis-server --appendonly yes --appendfsync everysec
      volumes:
        - ${MOUNTS_DIR:-./mounts}/redis:/data   # AOF persists on the host mounts dir
    ```

The app connects the same way in both modes — `REDIS_URL: redis://redis:6379`. Only the
durability changes.

!!! warning "Two things to know before you rely on a persistent sidecar"
    **`--appendfsync everysec` can lose ~1 second of the newest entries** on a hard crash.
    For a webhook queue that's an acceptable trade — just know it's there. Use
    `--appendfsync always` if you genuinely cannot drop a single entry (safer, slower).

    **Data under the mounts dir is not backed up.** Mounts are excluded from the backup
    pipeline, and that's *right* for a queue: a lost queue means some notifications didn't
    fire, not that business data is gone. **That's the dividing line.** If losing this data
    would actually hurt, it isn't queue or cache state — it's a system of record, and it
    belongs in Postgres (which *is* backed up), not a local mount.

### The rules

!!! danger "Don't do these"
    - **Don't map Redis to a host port.** No `ports: - "6379:6379"`. Two repos both
      grabbing host 6379 on the same box collide. Keep it internal to the compose network —
      nothing exposed, nothing to collide.
    - **Don't stand up a shared or fleet-wide Redis.** Per-repo means per-need. One Redis
      per *project*, shared by that project's containers if a repo runs several — never one
      per fleet.
    - **Don't treat persistent Redis as a database.** Queues and caches, yes. A durable
      system of record, no — that's Postgres, and unlike a mount it's backed up.

**You can't reach another repo's Redis** — different project, different network. That's not
a restriction you have to work around; it's the isolation working *for* you. Nobody else's
service can touch your cache or drain your queue either, and you never have to think about
whose keys are whose.

!!! quote "What about ACLs?"
    A shared Redis *can* be secured — Redis 6+ ACLs scope users by command, key pattern, and
    channel. But ACLs don't solve resource contention or the noisy-neighbour problem, and
    they add real management burden, so the fleet uses per-repo sidecars instead. Reserve
    ACLs for the rare case of a deliberately shared, durable, backed-up Redis run as actual
    infrastructure.