handbook/docs/deploy.md

# Deployment Guide

> How a project gets onto **rethink-net** — our Ubuntu 26.x servers. Get your
> container to follow a few consistent rules and deploying is mostly handing us a
> `compose.yaml`.

!!! tip "What can run here"
    APIs, websites, applets, bots, monitors.

!!! danger "One name everywhere — repo = container = volumes"
    The **repo name is the name.** Your service (the compose service key), the
    container, and every named volume **must all use that same name**. If your repo
    is `myrepo`:

    - service → `myrepo`
    - container → `myrepo`
    - volumes → `myrepo-data`, `myrepo-cache`, … (the repo name, then a suffix)

    Mismatched names break our deploy, monitoring, and log scraping — they key off
    the repo name. **Do not** name the service one thing and the repo another.

## Docker — the services account

Every service runs containerized as the shared **`services`** account:
**uid/gid 1337**, fixed fleet-wide. Build your image to be **uid-agnostic** so it
runs cleanly as that account.

```dockerfile
FROM python:3.12-slim
ENV HOME=/tmp                                  # (1)!

WORKDIR /app
RUN apt-get update \
    && apt-get install -y --no-install-recommends git \
    && rm -rf /var/lib/apt/lists/*            # (2)!

COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt   # (3)!

COPY . .
RUN chmod -R a+rwX /app                        # (4)!
CMD ["python", "-m", "yourapp"]
```

1.  `HOME=/tmp` — the `services` account has no home dir; anything writing to
    `$HOME` (caches, configs) needs a writable target.
2.  Include `git` **only if the container itself needs it** — e.g. you
    `pip install` from git, or the app shells out to git at runtime. The build and
    host always have git; this line is about what's *inside* the image.
3.  Install deps **before** copying the code (see [layer
    caching](#layer-caching)).
4.  `chmod -R a+rwX /app` makes the app tree writable by **any** uid — that's what
    "uid-agnostic" means.

!!! tip "Faster builds with uv (optional)"
    [uv](https://docs.astral.sh/uv/) is a drop-in for pip that reads the same
    `pyproject.toml` — no lockfile needed in the image. Swap the deps layer and add
    `UV_COMPILE_BYTECODE` so containers don't pay the first-import `.pyc` compile
    cost:

    ```dockerfile
    FROM python:3.12-slim
    COPY --from=ghcr.io/astral-sh/uv:latest /uv /uvx /bin/   # (1)!
    ENV HOME=/tmp
    ENV UV_COMPILE_BYTECODE=1                              # (2)!

    WORKDIR /app
    COPY pyproject.toml .
    RUN uv pip install --system .                          # (3)!
    COPY . .
    RUN chmod -R a+rwX /app
    CMD ["python", "-m", "yourapp"]
    ```

    1.  Pull the `uv` binary from its published image — no pip-installing uv itself.
    2.  Compile bytecode at build time so the container doesn't eat the first-import
        `.pyc` compile cost on every cold start.
    3.  `--system` installs into the image's Python (no venv needed — the container
        *is* the isolation); reads `pyproject.toml`, no `uv.lock` required.

The service key, `container_name`, and volume names below are all **`myrepo`** —
the repo name. Match yours to your repo.

```yaml
services:
  myrepo:                                      # (1)!
    build: .
    container_name: myrepo                     # (2)!
    user: "1337:1337"                          # (3)!
    environment:
      HOME: /tmp
    volumes:
      - /srv/config/<workspace>/<project>:/app/config:ro  # (4)!
      - /srv/logs/<workspace>/<project>:/app/logs          # (5)!
      - myrepo-data:/app/data                              # (6)!

volumes:
  myrepo-data:                                 # (7)!
```

1.  Service key = **the repo name**.
2.  Container name = **the repo name** — same as the service.
3.  Run as the shared account. **No** in-container `user`/`useradd` — don't bake a
    user into the image; set it here.
4.  Config: host-managed bind mount, mounted **read-only**.
5.  Logs: bind mount — live and rolled, scraped for monitoring.
6.  Named volume = **repo name + suffix** (`myrepo-data`). Docker owns it, so there
    are no host permissions to fiddle with.
7.  Declare the volume under the same **repo-name-prefixed** key.

## Paths and mounts

Everything host-side follows one shape: `/srv/<kind>/<workspace>/<project>/`.

!!! warning "What `<workspace>` is"
    `<workspace>` is the owning bucket for a project.

    - **An individual dev?** It's your **lowercase username** — `ricky`, `xattam`, …
    - **A shared / official project?** It's the **workspace** it belongs to —
      `bots`, `web`, `apis`, …

| What | Where | How |
| --- | --- | --- |
| Repo + compose | `/srv/docker/<workspace>/<project>/` | created by the git clone, not pre-provisioned |
| Config | `/srv/config/<workspace>/<project>/` | bind mount, host-managed, read-only |
| Logs | `/srv/logs/<workspace>/<project>/` | bind mount; live + rolled, scraped |
| Mounts (other host-visible data) | `/srv/mounts/<workspace>/<project>/` | bind mount, host-managed |
| Caches, profiles, scratch | named volume | Docker manages ownership |

All `/srv/...` paths are owned by the `services` user (uid/gid **1337**).

!!! note "If your service won't start or its logs aren't persisting"
    That's usually a host-side bind-mount **ownership** thing — the kind of detail
    **we sort out at deploy time**, not something you need to chown or provision.
    If a bot won't come up or logs/caches keep vanishing, flag it and we'll fix
    the mount perms. Stick to a clean `compose.yaml` and let us handle the host.

## Layer caching

Copy the deps file and install **before** `COPY . .`. Docker caches layers in
order, so deps only reinstall when the deps file changes — not on every code edit.
Get this backwards and every one-line change triggers a full dependency reinstall.
Same principle whether you use pip or uv:

```dockerfile
COPY requirements.txt .                              # pip baseline
RUN pip install --no-cache-dir -r requirements.txt   # cached until deps change
COPY . .                                             # changes every build
```

```dockerfile
COPY pyproject.toml .                                # uv path
RUN uv pip install --system .                        # cached until deps change
COPY . .                                             # changes every build
```

## Subprocess and browser workloads

Bots that spawn Chrome, Xvfb, ffmpeg, or other child processes need three extra
knobs in compose:

```yaml
services:
  yourbot:
    build: .
    user: "1337:1337"
    init: true              # (1)!
    shm_size: "2gb"         # (2)!
    mem_limit: "4g"         # (3)!
```

1.  Runs **tini** as PID 1 to reap zombie subprocesses and forward signals.
    Without it, spawned Chrome/Xvfb processes leak as zombies.
2.  Chrome and most headless browsers crash on Docker's default **64 MB**
    `/dev/shm`. Bump it for any browser workload.
3.  Bound memory — especially when each worker spawns a browser. Raise it as
    worker count grows.

!!! warning "The PID-1 gotcha with shell-wrapper CMDs"
    If your `CMD` is a shell-script wrapper (e.g. `xvfb-run ...`), it must **not**
    be PID 1, or the real process dies on startup. `init: true` is exactly what
    fixes this — tini takes PID 1, and your wrapper runs as a normal child.

## What your compose / Dockerfile needs

- `user: "1337:1337"`
- bind mounts for **config + logs**
- named volumes for **the rest**
- secrets bind-mounted **`:ro`**
- `HOME=/tmp`
- `chmod -R a+rwX /app`
- deps installed **before** the code copy (layer caching) — pip or `uv pip install`
- `git` in the image **if the container needs it**
- for browser/subprocess workloads: `init: true`, `shm_size`, `mem_limit`
- using uv? add `ENV UV_COMPILE_BYTECODE=1`

## Secrets

!!! warning "Secrets never go in the image"
    We do **not** commit secrets (usually, lol). They stay **gitignored**, live on
    the host at `/srv/config/<workspace>/<project>/`, and are bind-mounted
    **read-only** at runtime. Add them to `.dockerignore` so a `COPY . .` can't
    sweep them into a layer.

Rotating a secret = edit the host file and restart. No rebuild.

```bash
vim /srv/config/<workspace>/<project>/secrets.env   # edit on the host
docker compose restart yourapp                       # pick up the change — no rebuild
```