split Deploy into a hub + 3 sub-pages; inline comments over annotations
The Deploy page got long and the code annotations rendered inconsistently
(some as (n) numbers, some as clickable +, and a marker on a fully-commented
line made that line vanish). Fix both:
- Deploy is now a HUB (docs/deploy/index.md): the intro + collision rule +
central-deploy note, then card links to three focused sub-pages. Deploy
stays the single global-nav entry; sub-pages are not_in_nav, reached from
the hub cards.
- deploy/compose.md — compose convention, storage tiers, how deploy fills
it in, subprocess/browser knobs, checklist
- deploy/dockerfile.md — services-account image, COPY, layer caching, uv
- deploy/secrets.md — keeping secrets out of the image
- Replace code annotations with INLINE COMMENTS on the compose/Dockerfile
examples: everything visible at once, no + to click, and the commented
MOUNTS_DIR line no longer disappears.
- Update inbound links (index card, standards, workflow, environments) to
deploy/ and deploy/compose.md; nav Deploy -> deploy/index.md with
not_in_nav for the sub-pages.
Verified in-browser: hub cards link correctly, sub-pages render with visible
inline comments (0 annotation markers), left nav shows only Deploy;
mkdocs build --strict clean (validates not_in_nav + all cross-links).
Signed-off-by: disqualifier <dev@disqualifier.me>
This commit is contained in:
-280
@@ -1,280 +0,0 @@
|
|||||||
# Deployment Guide
|
|
||||||
|
|
||||||
> How a project gets onto **rethink-net** — our Ubuntu 26.x servers. Get your
|
|
||||||
> container to follow a few consistent rules and deploying is mostly handing us a
|
|
||||||
> `compose.yaml`.
|
|
||||||
|
|
||||||
!!! tip "What can run here"
|
|
||||||
APIs, websites, applets, bots, monitors.
|
|
||||||
|
|
||||||
!!! danger "Your compose names nothing repo-specific"
|
|
||||||
**No `container_name`. No hardcoded names, workspace, or `/srv` paths.** The
|
|
||||||
deploy layer injects identity and host paths at deploy time — your compose stays
|
|
||||||
generic so it can't collide with any other service on the fleet.
|
|
||||||
|
|
||||||
- service key is always **`svc`**
|
|
||||||
- named volumes use **bare names** (`cache`, not `myapp_cache`)
|
|
||||||
- host paths come from **`${LOGS_DIR}` / `${CONFIG_DIR}`**
|
|
||||||
|
|
||||||
Naming things after your repo used to cause **container-name collisions** at
|
|
||||||
fleet scale — two repos shipping the same name clashed. The generic compose
|
|
||||||
below fixes that; copy it verbatim.
|
|
||||||
|
|
||||||
## Docker — the services account
|
|
||||||
|
|
||||||
Every service runs containerized as the shared **`services`** account:
|
|
||||||
**uid/gid 1337**, fixed fleet-wide. Build your image to be **uid-agnostic** so it
|
|
||||||
runs cleanly as that account.
|
|
||||||
|
|
||||||
```dockerfile
|
|
||||||
FROM python:3.12-slim
|
|
||||||
ENV HOME=/tmp # (1)!
|
|
||||||
|
|
||||||
WORKDIR /app
|
|
||||||
RUN apt-get update \
|
|
||||||
&& apt-get install -y --no-install-recommends git \
|
|
||||||
&& rm -rf /var/lib/apt/lists/* # (2)!
|
|
||||||
|
|
||||||
COPY requirements.txt .
|
|
||||||
RUN pip install --no-cache-dir -r requirements.txt # (3)!
|
|
||||||
|
|
||||||
COPY . .
|
|
||||||
RUN chmod -R a+rwX /app # (4)!
|
|
||||||
CMD ["python", "-m", "yourapp"]
|
|
||||||
```
|
|
||||||
|
|
||||||
1. `HOME=/tmp` — the `services` account has no home dir; anything writing to
|
|
||||||
`$HOME` (caches, configs) needs a writable target.
|
|
||||||
2. Include `git` **only if the container itself needs it** — e.g. you
|
|
||||||
`pip install` from git, or the app shells out to git at runtime. The build and
|
|
||||||
host always have git; this line is about what's *inside* the image.
|
|
||||||
3. Install deps **before** copying the code (see [layer
|
|
||||||
caching](#layer-caching)).
|
|
||||||
4. `chmod -R a+rwX /app` makes the app tree writable by **any** uid — that's what
|
|
||||||
"uid-agnostic" means.
|
|
||||||
|
|
||||||
!!! tip "Faster builds with uv (optional)"
|
|
||||||
[uv](https://docs.astral.sh/uv/) is a drop-in for pip that reads the same
|
|
||||||
`pyproject.toml` — no lockfile needed in the image. Swap the deps layer and add
|
|
||||||
`UV_COMPILE_BYTECODE` so containers don't pay the first-import `.pyc` compile
|
|
||||||
cost:
|
|
||||||
|
|
||||||
```dockerfile
|
|
||||||
FROM python:3.12-slim
|
|
||||||
COPY --from=ghcr.io/astral-sh/uv:latest /uv /uvx /bin/ # (1)!
|
|
||||||
ENV HOME=/tmp
|
|
||||||
ENV UV_COMPILE_BYTECODE=1 # (2)!
|
|
||||||
|
|
||||||
WORKDIR /app
|
|
||||||
COPY pyproject.toml .
|
|
||||||
RUN uv pip install --system . # (3)!
|
|
||||||
COPY . .
|
|
||||||
RUN chmod -R a+rwX /app
|
|
||||||
CMD ["python", "-m", "yourapp"]
|
|
||||||
```
|
|
||||||
|
|
||||||
1. Pull the `uv` binary from its published image — no pip-installing uv itself.
|
|
||||||
2. Compile bytecode at build time so the container doesn't eat the first-import
|
|
||||||
`.pyc` compile cost on every cold start.
|
|
||||||
3. `--system` installs into the image's Python (no venv needed — the container
|
|
||||||
*is* the isolation); reads `pyproject.toml`, no `uv.lock` required.
|
|
||||||
|
|
||||||
### Getting your files into the image
|
|
||||||
|
|
||||||
`COPY . .` grabs the whole repo, but be explicit about anything that needs its own
|
|
||||||
place — assets, templates, a config the app reads at runtime. `COPY <src> <dest>`:
|
|
||||||
`<src>` is relative to the build context (your repo), `<dest>` is a path in the
|
|
||||||
image.
|
|
||||||
|
|
||||||
=== "A single file"
|
|
||||||
|
|
||||||
```dockerfile
|
|
||||||
COPY ./config.toml /app/config.toml # (1)!
|
|
||||||
```
|
|
||||||
|
|
||||||
1. One file into a specific path. The app reads it at `/app/config.toml`.
|
|
||||||
|
|
||||||
=== "A directory"
|
|
||||||
|
|
||||||
```dockerfile
|
|
||||||
COPY ./assets /app/assets # (1)!
|
|
||||||
```
|
|
||||||
|
|
||||||
1. A whole tree. Trailing paths are dirs — `/app/assets/` mirrors `./assets/`.
|
|
||||||
|
|
||||||
=== "Deps first (layer caching)"
|
|
||||||
|
|
||||||
```dockerfile
|
|
||||||
COPY requirements.txt . # (1)!
|
|
||||||
RUN pip install --no-cache-dir -r requirements.txt
|
|
||||||
COPY . . # (2)!
|
|
||||||
```
|
|
||||||
|
|
||||||
1. Copy just the deps file first, install, **then** copy the code.
|
|
||||||
2. See [layer caching](#layer-caching) — this ordering is why.
|
|
||||||
|
|
||||||
!!! warning "Don't `COPY` secrets into the image"
|
|
||||||
Anything sensitive stays **out** of the image — no `COPY ./secrets.env`. Secrets
|
|
||||||
live on the host and are injected read-only at runtime (see [Secrets](#secrets)).
|
|
||||||
Add them to `.dockerignore` so a blanket `COPY . .` can't sweep them in.
|
|
||||||
|
|
||||||
## Your `compose.yaml`
|
|
||||||
|
|
||||||
Copy this verbatim. It names nothing repo-specific — the deploy layer fills in
|
|
||||||
identity and host paths.
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
services:
|
|
||||||
svc: # (1)!
|
|
||||||
build: .
|
|
||||||
user: "1337:1337" # (2)!
|
|
||||||
restart: unless-stopped # (3)!
|
|
||||||
environment:
|
|
||||||
HOME: /tmp
|
|
||||||
volumes:
|
|
||||||
- ${LOGS_DIR:-./logs}:/app/logs # (4)!
|
|
||||||
- ${CONFIG_DIR:-./config}:/app/config # (5)!
|
|
||||||
# - ${MOUNTS_DIR:-./mounts}:/app/data # optional — see below
|
|
||||||
- cache:/app/cache # (6)!
|
|
||||||
|
|
||||||
volumes:
|
|
||||||
cache: # (7)!
|
|
||||||
```
|
|
||||||
|
|
||||||
1. The service key is always **`svc`** — never a repo-specific name, never a
|
|
||||||
`container_name`. The ops tooling keys off `svc`.
|
|
||||||
2. Run as the shared **`services`** account (**uid/gid 1337**, fixed fleet-wide).
|
|
||||||
**No** in-container `user`/`useradd` — set it here. This is the one identity
|
|
||||||
line that stays; it's not repo-specific.
|
|
||||||
3. **Required.** The host-side service is oneshot; Docker's own restart policy is
|
|
||||||
what recovers a crashed container.
|
|
||||||
4. Logs → host (**output**), path **injected** by the deploy layer. The `:-./logs`
|
|
||||||
default lets you `docker compose up` locally with nothing set and still work.
|
|
||||||
5. Config → host (**input you edit**), likewise injected.
|
|
||||||
6. Named volume mounted at **`/app/cache`** — ephemeral scratch. The **right-hand
|
|
||||||
path is where your code writes**: `WORKDIR` is `/app`, so if your app writes to
|
|
||||||
`./cache` (or `/app/cache`), that's this mount. Same rule for logs (`/app/logs`)
|
|
||||||
and config (`/app/config`) — match the container path to where your code reads
|
|
||||||
and writes. The volume name is a bare name, auto-namespaced per service at deploy
|
|
||||||
so it can't collide.
|
|
||||||
7. Declare bare (`cache`, not `myapp_cache`). Deploy auto-prefixes it.
|
|
||||||
|
|
||||||
The commented **`${MOUNTS_DIR}`** line is the optional third tier — a host dir for
|
|
||||||
arbitrary read/write data. The deploy layer injects `MOUNTS_DIR` and auto-creates the
|
|
||||||
dir, but the mount line is **opt-in**: uncomment it and pick the container path
|
|
||||||
(`/app/data` above) yourself, since that's app-specific and can't be auto-mounted.
|
|
||||||
|
|
||||||
!!! tip "Three tiers of storage"
|
|
||||||
- **`logs`** (output) and **`config`** (input you edit) — **we handle these**:
|
|
||||||
injected and managed at deploy time.
|
|
||||||
- **`mounts`** (optional) — a host dir for arbitrary read/write data. The deploy
|
|
||||||
layer **injects `MOUNTS_DIR` and auto-creates the dir**, but the mount line is
|
|
||||||
**opt-in**: uncomment it and choose the container path yourself (it's
|
|
||||||
app-specific, so it can't be auto-mounted).
|
|
||||||
- **Named volumes** (`cache`, …) — **yours**: anything else your service
|
|
||||||
persists. Docker owns them, no host paths to manage.
|
|
||||||
|
|
||||||
!!! tip "The `${VAR:-./default}` pattern"
|
|
||||||
Host paths use a variable with a local fallback. Locally, `docker compose up`
|
|
||||||
needs nothing set — it uses `./logs`, `./config` (and `./mounts` if you enable
|
|
||||||
it). When deployed, the ops tooling sets the real values, so the same file works
|
|
||||||
both places.
|
|
||||||
|
|
||||||
## Deployment is handled centrally
|
|
||||||
|
|
||||||
!!! note "You don't run any deploy commands"
|
|
||||||
All services are deployed centrally by the ops tooling. You don't pick a host,
|
|
||||||
manage keys, or run anything — just make your repo follow the compose convention
|
|
||||||
above and it's deployable.
|
|
||||||
|
|
||||||
At deploy time the tooling **generates** the environment your compose reads, so the
|
|
||||||
`${...}` variables resolve without anything from you. You can rely on these being
|
|
||||||
set — that's why your compose works unchanged on the fleet:
|
|
||||||
|
|
||||||
```
|
|
||||||
LOGS_DIR # your host log dir -> ${LOGS_DIR}
|
|
||||||
CONFIG_DIR # your host config dir -> ${CONFIG_DIR}
|
|
||||||
MOUNTS_DIR # optional host data dir -> ${MOUNTS_DIR} (if you enable the mount)
|
|
||||||
```
|
|
||||||
|
|
||||||
Write your logs and config to those mount points and you're set. If a service won't
|
|
||||||
come up or its logs aren't persisting, that's a host-side detail on our end — flag it
|
|
||||||
and we'll sort it.
|
|
||||||
|
|
||||||
## Layer caching
|
|
||||||
|
|
||||||
Copy the deps file and install **before** `COPY . .`. Docker caches layers in
|
|
||||||
order, so deps only reinstall when the deps file changes — not on every code edit.
|
|
||||||
Get this backwards and every one-line change triggers a full dependency reinstall.
|
|
||||||
Same principle whether you use pip or uv:
|
|
||||||
|
|
||||||
```dockerfile
|
|
||||||
COPY requirements.txt . # pip baseline
|
|
||||||
RUN pip install --no-cache-dir -r requirements.txt # cached until deps change
|
|
||||||
COPY . . # changes every build
|
|
||||||
```
|
|
||||||
|
|
||||||
```dockerfile
|
|
||||||
COPY pyproject.toml . # uv path
|
|
||||||
RUN uv pip install --system . # cached until deps change
|
|
||||||
COPY . . # changes every build
|
|
||||||
```
|
|
||||||
|
|
||||||
## Subprocess and browser workloads
|
|
||||||
|
|
||||||
Bots that spawn Chrome, Xvfb, ffmpeg, or other child processes need three extra
|
|
||||||
knobs in compose:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
services:
|
|
||||||
svc:
|
|
||||||
build: .
|
|
||||||
user: "1337:1337"
|
|
||||||
restart: unless-stopped
|
|
||||||
init: true # (1)!
|
|
||||||
shm_size: "2gb" # (2)!
|
|
||||||
mem_limit: "4g" # (3)!
|
|
||||||
```
|
|
||||||
|
|
||||||
1. Runs **tini** as PID 1 to reap zombie subprocesses and forward signals.
|
|
||||||
Without it, spawned Chrome/Xvfb processes leak as zombies.
|
|
||||||
2. Chrome and most headless browsers crash on Docker's default **64 MB**
|
|
||||||
`/dev/shm`. Bump it for any browser workload.
|
|
||||||
3. Bound memory — especially when each worker spawns a browser. Raise it as
|
|
||||||
worker count grows.
|
|
||||||
|
|
||||||
!!! warning "The PID-1 gotcha with shell-wrapper CMDs"
|
|
||||||
If your `CMD` is a shell-script wrapper (e.g. `xvfb-run ...`), it must **not**
|
|
||||||
be PID 1, or the real process dies on startup. `init: true` is exactly what
|
|
||||||
fixes this — tini takes PID 1, and your wrapper runs as a normal child.
|
|
||||||
|
|
||||||
## What your compose / Dockerfile needs
|
|
||||||
|
|
||||||
**Compose**
|
|
||||||
|
|
||||||
- service key `svc` — **no `container_name`**, no repo-specific names
|
|
||||||
- `user: "1337:1337"` — the shared `services` account
|
|
||||||
- `restart: unless-stopped`
|
|
||||||
- host paths via `${LOGS_DIR}` / `${CONFIG_DIR}` (and optional `${MOUNTS_DIR}`) —
|
|
||||||
never hardcoded
|
|
||||||
- named volumes with **bare names** (`cache`, not `myapp_cache`)
|
|
||||||
- for browser/subprocess workloads: `init: true`, `shm_size`, `mem_limit`
|
|
||||||
|
|
||||||
**Dockerfile**
|
|
||||||
|
|
||||||
- `HOME=/tmp`
|
|
||||||
- `chmod -R a+rwX /app` (uid-agnostic; runs as the `services` account, 1337)
|
|
||||||
- deps installed **before** the code copy (layer caching) — pip or `uv pip install`
|
|
||||||
- `git` in the image **if the container needs it**
|
|
||||||
- using uv? add `ENV UV_COMPILE_BYTECODE=1`
|
|
||||||
|
|
||||||
## Secrets
|
|
||||||
|
|
||||||
!!! warning "Secrets never go in the image"
|
|
||||||
We do **not** commit secrets (usually, lol). They stay **gitignored** and live
|
|
||||||
on the host in your config dir, reaching your container read-only via
|
|
||||||
`${CONFIG_DIR}`. Add them to `.dockerignore` so a `COPY . .` can't sweep them
|
|
||||||
into a layer.
|
|
||||||
|
|
||||||
Rotating a secret is a host-side edit — update the file and the service picks it up
|
|
||||||
on restart. No rebuild, and nothing you run: flag it and we handle the restart.
|
|
||||||
@@ -0,0 +1,106 @@
|
|||||||
|
# Compose convention
|
||||||
|
|
||||||
|
The `compose.yaml` your repo ships. Copy it verbatim — it names nothing
|
||||||
|
repo-specific, so the deploy layer can inject identity and host paths without
|
||||||
|
collisions.
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
services:
|
||||||
|
svc: # generic service key — ALWAYS svc, no container_name
|
||||||
|
build: .
|
||||||
|
user: "1337:1337" # the shared services account (required)
|
||||||
|
restart: unless-stopped # required — the host unit is oneshot; this recovers crashes
|
||||||
|
environment:
|
||||||
|
HOME: /tmp
|
||||||
|
volumes:
|
||||||
|
- ${LOGS_DIR:-./logs}:/app/logs # output — host log dir, injected at deploy
|
||||||
|
- ${CONFIG_DIR:-./config}:/app/config # input — host config dir, injected at deploy
|
||||||
|
# - ${MOUNTS_DIR:-./mounts}:/app/data # optional — arbitrary host data (opt-in, see below)
|
||||||
|
- cache:/app/cache # your data — ephemeral named volume
|
||||||
|
|
||||||
|
volumes:
|
||||||
|
cache: # bare name — deploy auto-prefixes it per service
|
||||||
|
```
|
||||||
|
|
||||||
|
## The rules
|
||||||
|
|
||||||
|
- **Service key is always `svc`.** Never a repo-specific name, never a
|
||||||
|
`container_name` — the ops tooling keys off `svc`. This is what makes services
|
||||||
|
collision-proof on the fleet.
|
||||||
|
- **`user: "1337:1337"` and `restart: unless-stopped` are required.** The first runs
|
||||||
|
as the shared `services` account (uid/gid 1337, fixed fleet-wide); the second lets
|
||||||
|
Docker recover a crashed container (the host-side unit is oneshot).
|
||||||
|
- **Host paths come from `${...}` variables, never hardcoded.** Write to `${LOGS_DIR}`
|
||||||
|
and `${CONFIG_DIR}`; `${MOUNTS_DIR}` is optional.
|
||||||
|
- **Named volumes use bare names** (`cache`, not `myapp_cache`) — deploy auto-prefixes
|
||||||
|
them per service so they can't collide.
|
||||||
|
|
||||||
|
!!! tip "The container path is where your code reads and writes"
|
||||||
|
`WORKDIR` is `/app`, so the **right-hand side** of each volume line is the path
|
||||||
|
your code targets. If your app writes to `./cache` (i.e. `/app/cache`), that's the
|
||||||
|
`cache` mount; logs go to `/app/logs`, config is read from `/app/config`. Match
|
||||||
|
the container path to what your code actually uses.
|
||||||
|
|
||||||
|
## Three tiers of storage
|
||||||
|
|
||||||
|
- **`logs`** (output) and **`config`** (input you edit) — **we handle these**:
|
||||||
|
injected and managed at deploy time.
|
||||||
|
- **`mounts`** (optional) — a host dir for arbitrary read/write data. The deploy layer
|
||||||
|
**injects `MOUNTS_DIR` and auto-creates the dir**, but the mount line is **opt-in**:
|
||||||
|
uncomment it and choose the container path yourself (it's app-specific, so it can't
|
||||||
|
be auto-mounted).
|
||||||
|
- **Named volumes** (`cache`, …) — **yours**: anything else your service persists.
|
||||||
|
Docker owns them, no host paths to manage.
|
||||||
|
|
||||||
|
!!! tip "The `${VAR:-./default}` pattern"
|
||||||
|
Host paths use a variable with a local fallback. Locally, `docker compose up` needs
|
||||||
|
nothing set — it uses `./logs`, `./config` (and `./mounts` if you enable it). When
|
||||||
|
deployed, the ops tooling sets the real values, so the same file works both places.
|
||||||
|
|
||||||
|
## How deploy fills it in
|
||||||
|
|
||||||
|
At deploy time the tooling **generates** the environment your compose reads, so the
|
||||||
|
`${...}` variables resolve without anything from you:
|
||||||
|
|
||||||
|
```
|
||||||
|
LOGS_DIR # your host log dir -> ${LOGS_DIR}
|
||||||
|
CONFIG_DIR # your host config dir -> ${CONFIG_DIR}
|
||||||
|
MOUNTS_DIR # optional host data dir -> ${MOUNTS_DIR} (if you enable the mount)
|
||||||
|
```
|
||||||
|
|
||||||
|
Write your logs and config to those mount points and you're set. If a service won't
|
||||||
|
come up or its logs aren't persisting, that's a host-side detail on our end — flag it
|
||||||
|
and we'll sort it.
|
||||||
|
|
||||||
|
## Subprocess and browser workloads
|
||||||
|
|
||||||
|
Bots that spawn Chrome, Xvfb, ffmpeg, or other child processes need three extra knobs:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
services:
|
||||||
|
svc:
|
||||||
|
build: .
|
||||||
|
user: "1337:1337"
|
||||||
|
restart: unless-stopped
|
||||||
|
init: true # tini as PID 1 — reaps zombie subprocesses, forwards signals
|
||||||
|
shm_size: "2gb" # Chrome/headless browsers crash on Docker's default 64 MB /dev/shm
|
||||||
|
mem_limit: "4g" # bound memory — raise as worker/browser count grows
|
||||||
|
```
|
||||||
|
|
||||||
|
!!! warning "The PID-1 gotcha with shell-wrapper CMDs"
|
||||||
|
If your `CMD` is a shell-script wrapper (e.g. `xvfb-run ...`), it must **not** be
|
||||||
|
PID 1, or the real process dies on startup. `init: true` is exactly what fixes this
|
||||||
|
— tini takes PID 1, and your wrapper runs as a normal child.
|
||||||
|
|
||||||
|
## Checklist
|
||||||
|
|
||||||
|
- service key `svc` — **no `container_name`**, no repo-specific names
|
||||||
|
- `user: "1337:1337"` — the shared `services` account
|
||||||
|
- `restart: unless-stopped`
|
||||||
|
- host paths via `${LOGS_DIR}` / `${CONFIG_DIR}` (and optional `${MOUNTS_DIR}`) — never
|
||||||
|
hardcoded
|
||||||
|
- named volumes with **bare names** (`cache`, not `myapp_cache`)
|
||||||
|
- for browser/subprocess workloads: `init: true`, `shm_size`, `mem_limit`
|
||||||
|
|
||||||
|
See **[Dockerfile & build](dockerfile.md)** for the image, and **[Secrets](secrets.md)**
|
||||||
|
for keeping credentials out of it.
|
||||||
@@ -0,0 +1,91 @@
|
|||||||
|
# Dockerfile & build
|
||||||
|
|
||||||
|
Every service runs containerized as the shared **`services`** account: **uid/gid
|
||||||
|
1337**, fixed fleet-wide. Build your image to be **uid-agnostic** so it runs cleanly
|
||||||
|
as that account.
|
||||||
|
|
||||||
|
```dockerfile
|
||||||
|
FROM python:3.12-slim
|
||||||
|
ENV HOME=/tmp # services account has no home dir; $HOME must be writable
|
||||||
|
|
||||||
|
WORKDIR /app
|
||||||
|
RUN apt-get update \
|
||||||
|
&& apt-get install -y --no-install-recommends git \
|
||||||
|
&& rm -rf /var/lib/apt/lists/* # include git ONLY if the container itself needs it
|
||||||
|
|
||||||
|
COPY requirements.txt .
|
||||||
|
RUN pip install --no-cache-dir -r requirements.txt # deps before code — see layer caching
|
||||||
|
|
||||||
|
COPY . .
|
||||||
|
RUN chmod -R a+rwX /app # writable by any uid — this is "uid-agnostic"
|
||||||
|
CMD ["python", "-m", "yourapp"]
|
||||||
|
```
|
||||||
|
|
||||||
|
- **`HOME=/tmp`** — the `services` account has no home dir; anything writing to `$HOME`
|
||||||
|
(caches, configs) needs a writable target.
|
||||||
|
- **`git`** — include it **only if the container itself needs it** (e.g. you
|
||||||
|
`pip install` from git, or the app shells out to git). The build and host always have
|
||||||
|
git; this is about what's *inside* the image.
|
||||||
|
- **`chmod -R a+rwX /app`** — makes the app tree writable by any uid, so it runs as
|
||||||
|
1337.
|
||||||
|
|
||||||
|
## Layer caching
|
||||||
|
|
||||||
|
Copy the deps file and install **before** `COPY . .`. Docker caches layers in order, so
|
||||||
|
deps only reinstall when the deps file changes — not on every code edit. Get this
|
||||||
|
backwards and every one-line change triggers a full dependency reinstall.
|
||||||
|
|
||||||
|
```dockerfile
|
||||||
|
COPY requirements.txt . # pip baseline
|
||||||
|
RUN pip install --no-cache-dir -r requirements.txt # cached until deps change
|
||||||
|
COPY . . # changes every build
|
||||||
|
```
|
||||||
|
|
||||||
|
## Getting your files into the image
|
||||||
|
|
||||||
|
`COPY . .` grabs the whole repo, but be explicit about anything that needs its own
|
||||||
|
place — assets, templates, a config the app reads at runtime. In `COPY <src> <dest>`,
|
||||||
|
`<src>` is relative to the build context (your repo) and `<dest>` is a path in the
|
||||||
|
image.
|
||||||
|
|
||||||
|
```dockerfile
|
||||||
|
COPY ./config.toml /app/config.toml # a single file into a specific path
|
||||||
|
COPY ./assets /app/assets # a whole directory (tree mirrors ./assets/)
|
||||||
|
```
|
||||||
|
|
||||||
|
!!! warning "Don't `COPY` secrets into the image"
|
||||||
|
Anything sensitive stays **out** of the image — no `COPY ./secrets.env`. Secrets
|
||||||
|
live on the host and are injected read-only at runtime (see
|
||||||
|
**[Secrets](secrets.md)**). Add them to `.dockerignore` so a blanket `COPY . .`
|
||||||
|
can't sweep them in.
|
||||||
|
|
||||||
|
## Faster builds with uv (optional)
|
||||||
|
|
||||||
|
[uv](https://docs.astral.sh/uv/) is a drop-in for pip that reads the same
|
||||||
|
`pyproject.toml` — no lockfile needed in the image. Swap the deps layer and add
|
||||||
|
`UV_COMPILE_BYTECODE` so containers don't pay the first-import `.pyc` compile cost:
|
||||||
|
|
||||||
|
```dockerfile
|
||||||
|
FROM python:3.12-slim
|
||||||
|
COPY --from=ghcr.io/astral-sh/uv:latest /uv /uvx /bin/ # pull the uv binary from its image
|
||||||
|
ENV HOME=/tmp
|
||||||
|
ENV UV_COMPILE_BYTECODE=1 # compile bytecode at build, not first cold start
|
||||||
|
|
||||||
|
WORKDIR /app
|
||||||
|
COPY pyproject.toml .
|
||||||
|
RUN uv pip install --system . # --system: into the image's Python, no venv/lock
|
||||||
|
COPY . .
|
||||||
|
RUN chmod -R a+rwX /app
|
||||||
|
CMD ["python", "-m", "yourapp"]
|
||||||
|
```
|
||||||
|
|
||||||
|
## Checklist
|
||||||
|
|
||||||
|
- `HOME=/tmp`
|
||||||
|
- `chmod -R a+rwX /app` (uid-agnostic; runs as the `services` account, 1337)
|
||||||
|
- deps installed **before** the code copy (layer caching) — pip or `uv pip install`
|
||||||
|
- `git` in the image **if the container needs it**
|
||||||
|
- using uv? add `ENV UV_COMPILE_BYTECODE=1`
|
||||||
|
|
||||||
|
See **[Compose convention](compose.md)** for the `compose.yaml`, and
|
||||||
|
**[Secrets](secrets.md)** for credentials.
|
||||||
@@ -0,0 +1,54 @@
|
|||||||
|
# Deployment Guide
|
||||||
|
|
||||||
|
> How a project gets onto **rethink-net** — our Ubuntu 26.x servers. Get your
|
||||||
|
> container to follow a few consistent rules and deploying is mostly handing us a
|
||||||
|
> `compose.yaml`.
|
||||||
|
|
||||||
|
!!! tip "What can run here"
|
||||||
|
APIs, websites, applets, bots, monitors.
|
||||||
|
|
||||||
|
!!! danger "Your compose names nothing repo-specific"
|
||||||
|
**No `container_name`. No hardcoded names, workspace, or `/srv` paths.** The
|
||||||
|
deploy layer injects identity and host paths at deploy time — your compose stays
|
||||||
|
generic so it can't collide with any other service on the fleet.
|
||||||
|
|
||||||
|
- service key is always **`svc`**
|
||||||
|
- named volumes use **bare names** (`cache`, not `myapp_cache`)
|
||||||
|
- host paths come from **`${LOGS_DIR}` / `${CONFIG_DIR}`**
|
||||||
|
|
||||||
|
Naming things after your repo used to cause **container-name collisions** at
|
||||||
|
fleet scale — two repos shipping the same name clashed. The generic compose
|
||||||
|
convention fixes that.
|
||||||
|
|
||||||
|
!!! note "You don't run any deploy commands"
|
||||||
|
All services are deployed centrally by the ops tooling. You don't pick a host,
|
||||||
|
manage keys, or run anything — just make your repo follow the compose convention
|
||||||
|
and it's deployable. At deploy time the tooling **generates** the environment your
|
||||||
|
compose reads (`${LOGS_DIR}`, `${CONFIG_DIR}`, optional `${MOUNTS_DIR}`), so the
|
||||||
|
same file works locally and on the fleet.
|
||||||
|
|
||||||
|
## The three things your repo needs
|
||||||
|
|
||||||
|
<div class="grid cards" markdown>
|
||||||
|
|
||||||
|
- :material-file-cog: __[Compose convention](compose.md)__
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
The `compose.yaml` your repo ships — generic `svc` service, the `${...}` host
|
||||||
|
mounts, storage tiers, and how deploy fills it in.
|
||||||
|
|
||||||
|
- :material-docker: __[Dockerfile & build](dockerfile.md)__
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
The uid-agnostic image (services account 1337), getting files in with `COPY`,
|
||||||
|
layer caching, uv, and subprocess/browser workloads.
|
||||||
|
|
||||||
|
- :material-key: __[Secrets](secrets.md)__
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
How secrets stay out of the image and reach the container read-only at runtime.
|
||||||
|
|
||||||
|
</div>
|
||||||
@@ -0,0 +1,19 @@
|
|||||||
|
# Secrets
|
||||||
|
|
||||||
|
!!! warning "Secrets never go in the image"
|
||||||
|
We do **not** commit secrets (usually, lol). They stay **gitignored** and live on
|
||||||
|
the host in your config dir, reaching your container read-only via `${CONFIG_DIR}`.
|
||||||
|
Add them to `.dockerignore` so a `COPY . .` can't sweep them into a layer.
|
||||||
|
|
||||||
|
## How it works
|
||||||
|
|
||||||
|
- Secrets live on the **host**, in your config dir — never in git, never in the image.
|
||||||
|
- They reach the container **read-only** via the injected `${CONFIG_DIR}` mount (see
|
||||||
|
**[Compose convention](compose.md)**).
|
||||||
|
- Keep them out of the build context: list them in `.dockerignore` so a blanket
|
||||||
|
`COPY . .` can't pull them into a layer.
|
||||||
|
|
||||||
|
## Rotating a secret
|
||||||
|
|
||||||
|
A host-side edit — update the file and the service picks it up on restart. No rebuild,
|
||||||
|
and nothing you run: flag it and we handle the restart.
|
||||||
@@ -67,7 +67,7 @@ happens, depending on where the project runs:
|
|||||||
COPY . .
|
COPY . .
|
||||||
```
|
```
|
||||||
|
|
||||||
This is how things run in production — see the [Deploy guide](deploy.md) for the
|
This is how things run in production — see the [Deploy guide](deploy/) for the
|
||||||
full container standard (uid 1337, the compose convention, layer caching, and the
|
full container standard (uid 1337, the compose convention, layer caching, and the
|
||||||
uv image setup).
|
uv image setup).
|
||||||
|
|
||||||
|
|||||||
+1
-1
@@ -66,7 +66,7 @@ out of it; examples use placeholders like `<workspace>`, `<project>`, and
|
|||||||
Project-based Python isolation — local `.venv`, Makefile, or Docker — and
|
Project-based Python isolation — local `.venv`, Makefile, or Docker — and
|
||||||
local version management with pyenv.
|
local version management with pyenv.
|
||||||
|
|
||||||
- :material-rocket-launch: __[Deploy](deploy.md)__
|
- :material-rocket-launch: __[Deploy](deploy/)__
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|||||||
+1
-1
@@ -134,7 +134,7 @@ TimeoutError: request timed out after 30s
|
|||||||
|
|
||||||
A deployable service ships a `compose.yaml` that names **nothing repo-specific** —
|
A deployable service ships a `compose.yaml` that names **nothing repo-specific** —
|
||||||
the deploy layer injects identity and host paths. See the
|
the deploy layer injects identity and host paths. See the
|
||||||
[Deploy guide](deploy.md#your-composeyaml) for the full convention and the variables
|
[Deploy guide](deploy/compose.md) for the full convention and the variables
|
||||||
you can rely on. The short version:
|
you can rely on. The short version:
|
||||||
|
|
||||||
=== "Right"
|
=== "Right"
|
||||||
|
|||||||
+1
-1
@@ -44,7 +44,7 @@ Our code lives on **Gitea** at
|
|||||||
2. Add your **SSH public key** under *Settings → SSH / GPG Keys* so you can clone
|
2. Add your **SSH public key** under *Settings → SSH / GPG Keys* so you can clone
|
||||||
and push over SSH.
|
and push over SSH.
|
||||||
3. For servers, we use a **per-repo deploy-key** model rather than your personal
|
3. For servers, we use a **per-repo deploy-key** model rather than your personal
|
||||||
key — see the [Deploy guide](deploy.md) for how a box gets read access to just
|
key — see the [Deploy guide](deploy/) for how a box gets read access to just
|
||||||
the repos it needs.
|
the repos it needs.
|
||||||
|
|
||||||
## Our git vs. public git (GitHub / GitLab)
|
## Our git vs. public git (GitHub / GitLab)
|
||||||
|
|||||||
+7
-1
@@ -55,4 +55,10 @@ nav:
|
|||||||
- Standards: standards.md
|
- Standards: standards.md
|
||||||
- Workflow: workflow.md
|
- Workflow: workflow.md
|
||||||
- Virtual environments: environments.md
|
- Virtual environments: environments.md
|
||||||
- Deploy: deploy.md
|
- Deploy: deploy/index.md
|
||||||
|
|
||||||
|
# Deploy sub-pages are reached from the Deploy hub's cards, not the global nav.
|
||||||
|
not_in_nav: |
|
||||||
|
/deploy/compose.md
|
||||||
|
/deploy/dockerfile.md
|
||||||
|
/deploy/secrets.md
|
||||||
|
|||||||
Reference in New Issue
Block a user