handbook/docs/deploy.md

Deployment Guide

How a project gets onto rethink-net — our Ubuntu 26.x servers. Get your container to follow a few consistent rules and deploying is mostly handing us a compose.yaml.

!!! tip "What can run here" APIs, websites, applets, bots, monitors.

!!! danger "One name everywhere — repo = container = volumes" The repo name is the name. Your service (the compose service key), the container, and every named volume must all use that same name. If your repo is myrepo:

- service → `myrepo`
- container → `myrepo`
- volumes → `myrepo-data`, `myrepo-cache`, … (the repo name, then a suffix)

Mismatched names break our deploy, monitoring, and log scraping — they key off
the repo name. **Do not** name the service one thing and the repo another.

Docker — the services account

Every service runs containerized as the shared services account: uid/gid 1337, fixed fleet-wide. Build your image to be uid-agnostic so it runs cleanly as that account.

FROM python:3.12-slim
ENV HOME=/tmp                                  # (1)!

WORKDIR /app
RUN apt-get update \
    && apt-get install -y --no-install-recommends git \
    && rm -rf /var/lib/apt/lists/*            # (2)!

COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt   # (3)!

COPY . .
RUN chmod -R a+rwX /app                        # (4)!
CMD ["python", "-m", "yourapp"]

1. HOME=/tmp — the services account has no home dir; anything writing to $HOME (caches, configs) needs a writable target.
2. Include git only if the container itself needs it — e.g. you pip install from git, or the app shells out to git at runtime. The build and host always have git; this line is about what's inside the image.
3. Install deps before copying the code (see layer caching).
4. chmod -R a+rwX /app makes the app tree writable by any uid — that's what "uid-agnostic" means.

!!! tip "Faster builds with uv (optional)" uv is a drop-in for pip that reads the same pyproject.toml — no lockfile needed in the image. Swap the deps layer and add UV_COMPILE_BYTECODE so containers don't pay the first-import .pyc compile cost:

```dockerfile
FROM python:3.12-slim
COPY --from=ghcr.io/astral-sh/uv:latest /uv /uvx /bin/   # (1)!
ENV HOME=/tmp
ENV UV_COMPILE_BYTECODE=1                              # (2)!

WORKDIR /app
COPY pyproject.toml .
RUN uv pip install --system .                          # (3)!
COPY . .
RUN chmod -R a+rwX /app
CMD ["python", "-m", "yourapp"]
```

1.  Pull the `uv` binary from its published image — no pip-installing uv itself.
2.  Compile bytecode at build time so the container doesn't eat the first-import
    `.pyc` compile cost on every cold start.
3.  `--system` installs into the image's Python (no venv needed — the container
    *is* the isolation); reads `pyproject.toml`, no `uv.lock` required.

The service key, container_name, and volume names below are all myrepo — the repo name. Match yours to your repo.

services:
  myrepo:                                      # (1)!
    build: .
    container_name: myrepo                     # (2)!
    user: "1337:1337"                          # (3)!
    environment:
      HOME: /tmp
    volumes:
      - /srv/config/<workspace>/<project>:/app/config:ro  # (4)!
      - /srv/logs/<workspace>/<project>:/app/logs          # (5)!
      - myrepo-data:/app/data                              # (6)!

volumes:
  myrepo-data:                                 # (7)!

1. Service key = the repo name.
2. Container name = the repo name — same as the service.
3. Run as the shared account. No in-container user/useradd — don't bake a user into the image; set it here.
4. Config: host-managed bind mount, mounted read-only.
5. Logs: bind mount — live and rolled, scraped for monitoring.
6. Named volume = repo name + suffix (myrepo-data). Docker owns it, so there are no host permissions to fiddle with.
7. Declare the volume under the same repo-name-prefixed key.

Paths and mounts

Everything host-side follows one shape: /srv/<kind>/<workspace>/<project>/.

!!! warning "What <workspace> is" <workspace> is the owning bucket for a project.

- **An individual dev?** It's your **lowercase username** — `ricky`, `xattam`, …
- **A shared / official project?** It's the **workspace** it belongs to —
  `bots`, `web`, `apis`, …

What	Where	How
Repo + compose	/srv/docker/<workspace>/<project>/	created by the git clone, not pre-provisioned
Config	/srv/config/<workspace>/<project>/	bind mount, host-managed, read-only
Logs	/srv/logs/<workspace>/<project>/	bind mount; live + rolled, scraped
Mounts (other host-visible data)	/srv/mounts/<workspace>/<project>/	bind mount, host-managed
Caches, profiles, scratch	named volume	Docker manages ownership

All /srv/... paths are owned by the services user (uid/gid 1337).

!!! note "If your service won't start or its logs aren't persisting" That's usually a host-side bind-mount ownership thing — the kind of detail we sort out at deploy time, not something you need to chown or provision. If a bot won't come up or logs/caches keep vanishing, flag it and we'll fix the mount perms. Stick to a clean compose.yaml and let us handle the host.

Layer caching

Copy the deps file and install before COPY . .. Docker caches layers in order, so deps only reinstall when the deps file changes — not on every code edit. Get this backwards and every one-line change triggers a full dependency reinstall. Same principle whether you use pip or uv:

COPY requirements.txt .                              # pip baseline
RUN pip install --no-cache-dir -r requirements.txt   # cached until deps change
COPY . .                                             # changes every build

COPY pyproject.toml .                                # uv path
RUN uv pip install --system .                        # cached until deps change
COPY . .                                             # changes every build

Subprocess and browser workloads

Bots that spawn Chrome, Xvfb, ffmpeg, or other child processes need three extra knobs in compose:

services:
  yourbot:
    build: .
    user: "1337:1337"
    init: true              # (1)!
    shm_size: "2gb"         # (2)!
    mem_limit: "4g"         # (3)!

1. Runs tini as PID 1 to reap zombie subprocesses and forward signals. Without it, spawned Chrome/Xvfb processes leak as zombies.
2. Chrome and most headless browsers crash on Docker's default 64 MB /dev/shm. Bump it for any browser workload.
3. Bound memory — especially when each worker spawns a browser. Raise it as worker count grows.

!!! warning "The PID-1 gotcha with shell-wrapper CMDs" If your CMD is a shell-script wrapper (e.g. xvfb-run ...), it must not be PID 1, or the real process dies on startup. init: true is exactly what fixes this — tini takes PID 1, and your wrapper runs as a normal child.

What your compose / Dockerfile needs

• user: "1337:1337"
• bind mounts for config + logs
• named volumes for the rest
• secrets bind-mounted :ro
• HOME=/tmp
• chmod -R a+rwX /app
• deps installed before the code copy (layer caching) — pip or uv pip install
• git in the image if the container needs it
• for browser/subprocess workloads: init: true, shm_size, mem_limit
• using uv? add ENV UV_COMPILE_BYTECODE=1

Secrets

!!! warning "Secrets never go in the image" We do not commit secrets (usually, lol). They stay gitignored, live on the host at /srv/config/<workspace>/<project>/, and are bind-mounted read-only at runtime. Add them to .dockerignore so a COPY . . can't sweep them into a layer.

Rotating a secret = edit the host file and restart. No rebuild.

vim /srv/config/<workspace>/<project>/secrets.env   # edit on the host
docker compose restart yourapp                       # pick up the change — no rebuild