diff --git a/docs/deploy.md b/docs/deploy.md index a89e8fc..9d1c6f3 100644 --- a/docs/deploy.md +++ b/docs/deploy.md @@ -18,23 +18,29 @@ runs cleanly as that account. ```dockerfile FROM python:3.12-slim ENV HOME=/tmp # (1)! + WORKDIR /app RUN apt-get update \ && apt-get install -y --no-install-recommends git \ && rm -rf /var/lib/apt/lists/* # (2)! + +COPY requirements.txt . +RUN pip install --no-cache-dir -r requirements.txt # (3)! + COPY . . -RUN pip install --no-cache-dir . \ - && chmod -R a+rwX /app # (3)! +RUN chmod -R a+rwX /app # (4)! CMD ["python", "-m", "yourapp"] ``` 1. `HOME=/tmp` — the `services` account has no home dir; anything writing to `$HOME` (caches, configs) needs a writable target. -2. Install `git` **only if** you `pip install` from git, then clean the apt lists - to keep the image small. -3. `chmod -R a+rwX /app` makes the app tree writable by **any** uid — this is - what "uid-agnostic" means. Note it only covers **non-mounted** dirs (see the - [footgun](#permissions-the-bind-mount-footgun)). +2. Include `git` **only if the container itself needs it** — e.g. you + `pip install` from git, or the app shells out to git at runtime. The build and + host always have git; this line is about what's *inside* the image. +3. Install deps **before** copying the code (see [layer + caching](#layer-caching)). +4. `chmod -R a+rwX /app` makes the app tree writable by **any** uid — that's what + "uid-agnostic" means. ```yaml services: @@ -45,7 +51,7 @@ services: HOME: /tmp volumes: - /srv/configs/:/app/config:ro # (2)! - - /srv//:/app/logs # (3)! + - /srv/logs//:/app/logs # (3)! - yourapp-data:/app/data # (4)! volumes: @@ -63,29 +69,55 @@ volumes: | What | Where | How | | --- | --- | --- | -| Configs | `/srv/configs//` | bind mount, host-managed | -| Logs | `/srv///` | bind mount; live + rolled, scraped | +| Configs | `/srv/configs//` | bind mount, host-managed, read-only | +| Logs | `/srv/logs///` | bind mount; live + rolled, scraped | | Caches, profiles, scratch | named volume | Docker manages ownership | -## Permissions — the bind-mount footgun +!!! note "If your service won't start or its logs aren't persisting" + That's usually a host-side bind-mount **ownership** thing — the kind of detail + **we sort out at deploy time**, not something you need to chown or provision. + If a bot won't come up or logs/caches keep vanishing, flag it and we'll fix + the mount perms. Stick to a clean `compose.yaml` and let us handle the host. -!!! danger "Bind-mount sources must exist and be 1337-owned *before* `up`" - `docker compose up` does **not** create bind-mount directories as you. If a - bind-mount source is missing, the Docker daemon (**root**) creates it **as - root** — and your container (**1337**) then can't write it. Logs silently fall - back to console-only; caches re-download every run. +## Layer caching -So: +Copy `requirements.txt` and `pip install` **before** `COPY . .`. Docker caches +layers in order, so deps only reinstall when `requirements.txt` changes — not on +every code edit. Get this backwards and every one-line change triggers a full +dependency reinstall. -- **Bind-mount sources (configs, logs) must EXIST and be 1337-owned before `up`** - — handled at provisioning, not a per-deploy chown hook. -- **Named volumes avoid this entirely** — use them for anything that doesn't need - host visibility. +```dockerfile +COPY requirements.txt . +RUN pip install --no-cache-dir -r requirements.txt # cached until deps change +COPY . . # changes every build +``` -!!! note "Why `chmod a+rwX /app` doesn't save you here" - The Dockerfile `chmod` only covers **non-mounted** dirs. A bind mount - overrides the image directory with the host directory, so for mounted paths - the **host-side ownership wins** — the image's permissions are irrelevant. +## Subprocess and browser workloads + +Bots that spawn Chrome, Xvfb, ffmpeg, or other child processes need three extra +knobs in compose: + +```yaml +services: + yourbot: + build: . + user: "1337:1337" + init: true # (1)! + shm_size: "2gb" # (2)! + mem_limit: "4g" # (3)! +``` + +1. Runs **tini** as PID 1 to reap zombie subprocesses and forward signals. + Without it, spawned Chrome/Xvfb processes leak as zombies. +2. Chrome and most headless browsers crash on Docker's default **64 MB** + `/dev/shm`. Bump it for any browser workload. +3. Bound memory — especially when each worker spawns a browser. Raise it as + worker count grows. + +!!! warning "The PID-1 gotcha with shell-wrapper CMDs" + If your `CMD` is a shell-script wrapper (e.g. `xvfb-run ...`), it must **not** + be PID 1, or the real process dies on startup. `init: true` is exactly what + fixes this — tini takes PID 1, and your wrapper runs as a normal child. ## What your compose / Dockerfile needs @@ -95,7 +127,9 @@ So: - secrets bind-mounted **`:ro`** - `HOME=/tmp` - `chmod -R a+rwX /app` -- `git` in the build if you `pip install` from git +- deps installed **before** the code copy (layer caching) +- `git` in the image **if the container needs it** +- for browser/subprocess workloads: `init: true`, `shm_size`, `mem_limit` ## Secrets diff --git a/docs/standards.md b/docs/standards.md index 0067ffc..3143bdb 100644 --- a/docs/standards.md +++ b/docs/standards.md @@ -51,6 +51,12 @@ $ flake8 - Every library and project has a **README** with the install line, what it does, and a usage example. +!!! note "Licensing — no per-file headers" + Don't prepend license/copyright boilerplate to source files. If a repo needs a + license, it's a single top-level **`LICENSE`** file — never repeated per file. + Most internal repos don't carry one; add it only when a repo is meant for + outside use and the terms are decided. + === "Do" ```python