diff --git a/docs/deploy.md b/docs/deploy.md index d924762..d3122d2 100644 --- a/docs/deploy.md +++ b/docs/deploy.md @@ -7,17 +7,18 @@ !!! tip "What can run here" APIs, websites, applets, bots, monitors. -!!! danger "One name everywhere — repo = container = volumes" - The **repo name is the name.** Your service (the compose service key), the - container, and every named volume **must all use that same name**. If your repo - is `myrepo`: +!!! danger "Your compose names nothing repo-specific" + **No `container_name`. No hardcoded names, workspace, or `/srv` paths.** The + deploy layer injects identity and host paths at deploy time — your compose stays + generic so it can't collide with any other service on the fleet. - - service → `myrepo` - - container → `myrepo` - - volumes → `myrepo-data`, `myrepo-cache`, … (the repo name, then a suffix) + - service key is always **`svc`** + - named volumes use **bare names** (`cache`, not `myapp_cache`) + - host paths come from **`${LOGS_DIR}` / `${CONFIG_DIR}` / `${MOUNTS_DIR}`** - Mismatched names break our deploy, monitoring, and log scraping — they key off - the repo name. **Do not** name the service one thing and the repo another. + Naming things after your repo used to cause **container-name collisions** at + fleet scale — two repos shipping the same name clashed. The generic compose + below fixes that; copy it verbatim. ## Docker — the services account @@ -78,62 +79,99 @@ CMD ["python", "-m", "yourapp"] 3. `--system` installs into the image's Python (no venv needed — the container *is* the isolation); reads `pyproject.toml`, no `uv.lock` required. -The service key, `container_name`, and volume names below are all **`myrepo`** — -the repo name. Match yours to your repo. +## Your `compose.yaml` + +Copy this verbatim. It names nothing repo-specific — the deploy layer fills in +identity and host paths. ```yaml services: - myrepo: # (1)! + svc: # (1)! build: . - container_name: myrepo # (2)! - user: "1337:1337" # (3)! + user: "1337:1337" # (2)! + restart: unless-stopped # (3)! environment: HOME: /tmp volumes: - - /srv/config//:/app/config:ro # (4)! - - /srv/logs//:/app/logs # (5)! - - myrepo-data:/app/data # (6)! + - ${LOGS_DIR:-./logs}:/app/logs # (4)! + - ${CONFIG_DIR:-./config}:/app/config # (5)! + - ${MOUNTS_DIR:-./mounts}:/app/mounts + - cache:/app/cache # (6)! volumes: - myrepo-data: # (7)! + cache: # (7)! ``` -1. Service key = **the repo name**. -2. Container name = **the repo name** — same as the service. -3. Run as the shared account. **No** in-container `user`/`useradd` — don't bake a - user into the image; set it here. -4. Config: host-managed bind mount, mounted **read-only**. -5. Logs: bind mount — live and rolled, scraped for monitoring. -6. Named volume = **repo name + suffix** (`myrepo-data`). Docker owns it, so there - are no host permissions to fiddle with. -7. Declare the volume under the same **repo-name-prefixed** key. +1. The service key is always **`svc`** — never a repo-specific name, never a + `container_name`. Our tooling keys off `svc` (the `svc` command, the + `svc--` unit). +2. Run as the shared **`services`** account (**uid/gid 1337**, fixed fleet-wide). + **No** in-container `user`/`useradd` — set it here. This is the one identity + line that stays; it's not repo-specific. +3. **Required.** The host-side service is oneshot; Docker's own restart policy is + what recovers a crashed container. +4. Logs → host, path **injected** by the deploy layer. The `:-./logs` default lets + you `docker compose up` locally with nothing set and still work. +5. Config → host, likewise injected. Same pattern for `${MOUNTS_DIR}`. +6. Named volume with a **bare name** — ephemeral, auto-namespaced per service at + deploy time so it can't collide. +7. Declare bare (`cache`, not `myapp_cache`). Deploy auto-prefixes it. -## Paths and mounts +!!! tip "The `${VAR:-./default}` pattern" + Every host path uses a variable with a local fallback. Locally, `docker compose + up` needs nothing set — it uses `./logs`, `./config`, `./mounts`. On the fleet, + the deploy layer sets the real values (below), so the same file works both + places. -Everything host-side follows one shape: `/srv////`. +## How deploy fills it in -!!! warning "What `` is" - `` is the owning bucket for a project. +At deploy time we set the variables your compose reads, so identity and host paths +are injected — not baked into your file. You can rely on these being present: - - **An individual dev?** It's your **lowercase username** — `ricky`, `xattam`, … - - **A shared / official project?** It's the **workspace** it belongs to — - `bots`, `web`, `apis`, … +``` +COMPOSE_PROJECT_NAME=- +LOGS_DIR=/srv/logs// +CONFIG_DIR=/srv/config// +MOUNTS_DIR=/srv/mounts// +``` -| What | Where | How | -| --- | --- | --- | -| Repo + compose | `/srv/docker///` | created by the git clone, not pre-provisioned | -| Config | `/srv/config///` | bind mount, host-managed, read-only | -| Logs | `/srv/logs///` | bind mount; live + rolled, scraped | -| Mounts (other host-visible data) | `/srv/mounts///` | bind mount, host-managed | -| Caches, profiles, scratch | named volume | Docker manages ownership | +Which makes ownership obvious and collisions impossible: -All `/srv/...` paths are owned by the `services` user (uid/gid **1337**). +| item | value | +| --- | --- | +| container | `--svc-1` | +| volume | `-_cache` | +| network | `-_default` | +| logs | `/srv/logs///` | + +!!! warning "What `` and `` are" + - **``** — who owns it: an individual dev (`ricky`, `xattam`, …) or a + category (`tpv`, `web`, `bots`, …). + - **``** — the git repo name, lowercased, by default. Overridable at + deploy time. + +Host paths live under `/srv////` (config, logs, mounts), all +owned by the `services` user (uid/gid **1337**). You never write these paths in your +compose — you read the injected `${...}` variables. + +## Onboarding a service + +One command registers and starts a service: + +``` +deploy [name] +``` + +It registers the service, generates a per-repo **read-only deploy key on the host** +(the private key never leaves the box), pushes the public key to the repo's Gitea +deploy keys automatically, clones the code, and installs and starts the service's +unit. No manual key paste, no host setup on your end. !!! note "If your service won't start or its logs aren't persisting" That's usually a host-side bind-mount **ownership** thing — the kind of detail **we sort out at deploy time**, not something you need to chown or provision. - If a bot won't come up or logs/caches keep vanishing, flag it and we'll fix - the mount perms. Stick to a clean `compose.yaml` and let us handle the host. + If a bot won't come up or logs/caches keep vanishing, flag it and we'll fix it. + Stick to the generic `compose.yaml` above and let us handle the host. ## Layer caching @@ -161,9 +199,10 @@ knobs in compose: ```yaml services: - yourbot: + svc: build: . user: "1337:1337" + restart: unless-stopped init: true # (1)! shm_size: "2gb" # (2)! mem_limit: "4g" # (3)! @@ -183,28 +222,30 @@ services: ## What your compose / Dockerfile needs -- `user: "1337:1337"` -- bind mounts for **config + logs** -- named volumes for **the rest** -- secrets bind-mounted **`:ro`** +**Compose** + +- service key `svc` — **no `container_name`**, no repo-specific names +- `user: "1337:1337"` — the shared `services` account +- `restart: unless-stopped` +- host paths via `${LOGS_DIR}` / `${CONFIG_DIR}` / `${MOUNTS_DIR}` — never hardcoded +- named volumes with **bare names** (`cache`, not `myapp_cache`) +- for browser/subprocess workloads: `init: true`, `shm_size`, `mem_limit` + +**Dockerfile** + - `HOME=/tmp` -- `chmod -R a+rwX /app` +- `chmod -R a+rwX /app` (uid-agnostic; runs as the `services` account, 1337) - deps installed **before** the code copy (layer caching) — pip or `uv pip install` - `git` in the image **if the container needs it** -- for browser/subprocess workloads: `init: true`, `shm_size`, `mem_limit` - using uv? add `ENV UV_COMPILE_BYTECODE=1` ## Secrets !!! warning "Secrets never go in the image" - We do **not** commit secrets (usually, lol). They stay **gitignored**, live on - the host at `/srv/config///`, and are bind-mounted - **read-only** at runtime. Add them to `.dockerignore` so a `COPY . .` can't - sweep them into a layer. + We do **not** commit secrets (usually, lol). They stay **gitignored** and live + on the host under the config dir (`/srv/config///`), reaching + your container read-only via `${CONFIG_DIR}`. Add them to `.dockerignore` so a + `COPY . .` can't sweep them into a layer. -Rotating a secret = edit the host file and restart. No rebuild. - -```bash -vim /srv/config///secrets.env # edit on the host -docker compose restart yourapp # pick up the change — no rebuild -``` +Rotating a secret is a host-side edit — update the file and the service picks it up +on restart. No rebuild, and nothing you run: flag it and we handle the restart. diff --git a/docs/standards.md b/docs/standards.md index 3143bdb..06e9ea3 100644 --- a/docs/standards.md +++ b/docs/standards.md @@ -129,3 +129,43 @@ TimeoutError: request timed out after 30s Libraries **emit only** — `log = logging.getLogger(__name__)` and nothing else. Handlers, levels, and formatting are configured once at the application entry point, so a lib never dictates how its host logs. + +## Service compose + +A deployable service ships a `compose.yaml` that names **nothing repo-specific** — +the deploy layer injects identity and host paths. See the +[Deploy guide](deploy.md#your-composeyaml) for the full convention and the variables +you can rely on. The short version: + +=== "Right" + + ```yaml + services: + svc: + user: "1337:1337" + restart: unless-stopped + volumes: + - ${LOGS_DIR:-./logs}:/app/logs + - profile:/app/profile + + volumes: + profile: + ``` + +=== "Wrong (causes collisions)" + + ```yaml + services: + nova: + container_name: nova # repo-specific name -> collides + volumes: + - /srv/logs/ricky/nova:/app/logs # hardcoded host path + - nova_profile:/app/profile # repo-prefixed volume + + volumes: + nova_profile: + ``` + +Generic service key `svc`, no `container_name`, host paths from `${...}` variables, +and **bare** volume names — that's what makes a service collision-proof on the +fleet.