Compare commits
21
Commits
78e3c34d8d
..
main
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
9fb5067124 | ||
|
|
a248879451 | ||
|
|
b3acd57e32 | ||
|
|
6151cb2e4e | ||
|
|
350884c2b3 | ||
|
|
5797248931 | ||
|
|
823aa7708a | ||
|
|
f9e5e4f24d | ||
|
|
c335887dd6 | ||
|
|
fe707d5f24 | ||
|
|
0ebd6f86b0 | ||
|
|
08ad37eddc | ||
|
|
8c56fd98a8 | ||
|
|
7ed388c69c | ||
|
|
17b2888c1f | ||
|
|
c53d67da2f | ||
|
|
dafc1dcacd | ||
|
|
6498c5b6d5 | ||
|
|
5f91d73e1f | ||
|
|
75ad61b06a | ||
|
|
ed4e558dca |
@@ -9,6 +9,7 @@ __pycache__/
|
|||||||
*.py[cod]
|
*.py[cod]
|
||||||
.venv/
|
.venv/
|
||||||
venv/
|
venv/
|
||||||
|
uv.lock
|
||||||
|
|
||||||
# Playwright MCP run artifacts
|
# Playwright MCP run artifacts
|
||||||
.playwright-mcp/
|
.playwright-mcp/
|
||||||
|
|||||||
File diff suppressed because one or more lines are too long
|
After Width: | Height: | Size: 9.0 KiB |
+262
-116
@@ -1,147 +1,293 @@
|
|||||||
# Deployment Guide
|
# Deployment Guide
|
||||||
|
|
||||||
> Ready for your project to see the light? You may be eligible for deployment on
|
> How a project gets onto **rethink-net** — our Ubuntu 26.x servers. Get your
|
||||||
> **rethink-net** — our fleet of Ubuntu 26.x servers, ready to host whatever
|
> container to follow a few consistent rules and deploying is mostly handing us a
|
||||||
> you've built.
|
> `compose.yaml`.
|
||||||
|
|
||||||
!!! tip "Eligible"
|
!!! success "What can run here"
|
||||||
APIs, websites, applets, bots, monitors. The whole network runs on a few
|
APIs, websites, applets, bots, monitors.
|
||||||
simple, consistent rules — get your container to follow them and deploying is
|
|
||||||
mostly handing us a `compose.yaml`.
|
|
||||||
|
|
||||||
## Docker — the services account
|
## Preparing for Deploy
|
||||||
|
|
||||||
Every service runs containerized as the shared **`services`** account:
|
You build and test locally — leaning on our libraries, AI, and this handbook — then
|
||||||
**uid/gid 1337**, fixed fleet-wide. Build your image to be **uid-agnostic** so it
|
push to git. From there the ops tooling takes over: staging (if you run gitflow), then
|
||||||
runs cleanly as that account.
|
it builds the Docker image and runs it on the fleet. **You don't run any deploy
|
||||||
|
commands** — your only job is the three pieces in the tabs below.
|
||||||
|
|
||||||
```dockerfile
|
```mermaid
|
||||||
FROM python:3.12-slim
|
flowchart LR
|
||||||
ENV HOME=/tmp # (1)!
|
subgraph build ["you build"]
|
||||||
|
direction TB
|
||||||
|
local["local testing"]
|
||||||
|
libs["libs<br/>(rethink-public, pip)"]
|
||||||
|
ai["AI-assisted<br/>(Claude Code)"]
|
||||||
|
docs["reading the<br/>handbook"]
|
||||||
|
local --- libs
|
||||||
|
libs --- ai
|
||||||
|
ai --- docs
|
||||||
|
docs --- local
|
||||||
|
end
|
||||||
|
|
||||||
WORKDIR /app
|
build -->|push| git["git<br/>(Gitea)"]
|
||||||
RUN apt-get update \
|
git -.->|develop, if gitflow| stage["staging"]
|
||||||
&& apt-get install -y --no-install-recommends git \
|
git --> deploy["deployment<br/>(ops tooling)"]
|
||||||
&& rm -rf /var/lib/apt/lists/* # (2)!
|
stage --> deploy
|
||||||
|
deploy --> docker["docker<br/>(built + run<br/>on the fleet)"]
|
||||||
|
|
||||||
COPY requirements.txt .
|
classDef ship fill:#061541,stroke:#569bcc,color:#eef1f6;
|
||||||
RUN pip install --no-cache-dir -r requirements.txt # (3)!
|
classDef work fill:#0e1530,stroke:#294274,color:#eef1f6;
|
||||||
|
class git,stage,deploy,docker ship;
|
||||||
COPY . .
|
class local,libs,ai,docs work;
|
||||||
RUN chmod -R a+rwX /app # (4)!
|
|
||||||
CMD ["python", "-m", "yourapp"]
|
|
||||||
```
|
```
|
||||||
|
|
||||||
1. `HOME=/tmp` — the `services` account has no home dir; anything writing to
|
!!! danger "Your compose names nothing repo-specific"
|
||||||
`$HOME` (caches, configs) needs a writable target.
|
**No `container_name`, no hardcoded names/paths.** The deploy layer injects
|
||||||
2. Include `git` **only if the container itself needs it** — e.g. you
|
identity and host paths so your compose can't collide with any other service on
|
||||||
`pip install` from git, or the app shells out to git at runtime. The build and
|
the fleet:
|
||||||
host always have git; this line is about what's *inside* the image.
|
|
||||||
3. Install deps **before** copying the code (see [layer
|
- the service key is **always `svc`** — it never changes, in any repo
|
||||||
caching](#layer-caching)).
|
- named volumes use **bare names** (`cache`, not `myapp_cache`)
|
||||||
4. `chmod -R a+rwX /app` makes the app tree writable by **any** uid — that's what
|
- host paths come from **`${LOGS_DIR}` / `${CONFIG_DIR}`**
|
||||||
"uid-agnostic" means.
|
|
||||||
|
=== "Compose"
|
||||||
|
|
||||||
|
The `compose.yaml` your repo ships. Copy it verbatim — it names nothing
|
||||||
|
repo-specific, so the deploy layer can inject identity and host paths without
|
||||||
|
collisions.
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
services:
|
||||||
|
svc: # generic service key — ALWAYS svc, no container_name
|
||||||
|
build: .
|
||||||
|
user: "1337:1337" # the shared services account (required)
|
||||||
|
restart: unless-stopped # required — host unit is oneshot; this recovers crashes
|
||||||
|
environment:
|
||||||
|
HOME: /tmp
|
||||||
|
volumes:
|
||||||
|
- ${LOGS_DIR:-./logs}:/app/logs # output — host log dir, injected at deploy
|
||||||
|
- ${CONFIG_DIR:-./config}:/app/config # input — host config dir, injected at deploy
|
||||||
|
# - ${MOUNTS_DIR:-./mounts}:/app/data # optional — arbitrary host data (opt-in)
|
||||||
|
- cache:/app/cache # your data — ephemeral named volume
|
||||||
|
|
||||||
```yaml
|
|
||||||
services:
|
|
||||||
yourapp:
|
|
||||||
build: .
|
|
||||||
user: "1337:1337" # (1)!
|
|
||||||
environment:
|
|
||||||
HOME: /tmp
|
|
||||||
volumes:
|
volumes:
|
||||||
- /srv/configs/<project>:/app/config:ro # (2)!
|
cache: # bare name — deploy auto-prefixes it per service
|
||||||
- /srv/logs/<dev>/<project>:/app/logs # (3)!
|
```
|
||||||
- yourapp-data:/app/data # (4)!
|
|
||||||
|
|
||||||
volumes:
|
!!! example "The container path is where your code reads and writes"
|
||||||
yourapp-data:
|
`WORKDIR` is `/app`, so the **right-hand side** of each volume line is the path
|
||||||
```
|
your code targets. Write to `./cache` (i.e. `/app/cache`) → the `cache` volume;
|
||||||
|
logs go to `/app/logs`, config is read from `/app/config`.
|
||||||
|
|
||||||
1. Run as the shared account. **No** in-container `user`/`useradd` — don't bake a
|
**Storage, three tiers:**
|
||||||
user into the image; set it here.
|
|
||||||
2. Configs: host-managed bind mount, mounted **read-only**.
|
|
||||||
3. Logs: bind mount — live and rolled, scraped for monitoring.
|
|
||||||
4. Everything else: a **named volume**. Docker owns it, so there are no host
|
|
||||||
permissions to fiddle with.
|
|
||||||
|
|
||||||
## Paths and mounts
|
- **`logs` + `config`** — we inject and manage these at deploy time.
|
||||||
|
- **`mounts`** (optional) — host dir for arbitrary data. We inject `MOUNTS_DIR` and
|
||||||
|
create the dir; you **uncomment** the line and pick the container path.
|
||||||
|
- **Named volumes** (`cache`, …) — yours; Docker owns them, no host paths to manage.
|
||||||
|
|
||||||
| What | Where | How |
|
!!! warning "Two kinds of logs — and crashes go to the other one"
|
||||||
| --- | --- | --- |
|
`${LOGS_DIR}` holds **only** what your app writes to disk through its logger
|
||||||
| Configs | `/srv/configs/<project>/` | bind mount, host-managed, read-only |
|
(e.g. `log_setup` writing a file) — your own structured logging. It does **not**
|
||||||
| Logs | `/srv/logs/<dev>/<project>/` | bind mount; live + rolled, scraped |
|
capture the process's stdout/stderr, and that's where **startup crashes and
|
||||||
| Caches, profiles, scratch | named volume | Docker manages ownership |
|
uncaught exceptions land** — a traceback from a failed import or a missing file
|
||||||
|
never reaches your logger. So if a service dies on startup, or you don't see the
|
||||||
|
error in your log files, it's in the process output, not `${LOGS_DIR}`. Make
|
||||||
|
fatal errors visible — and to route uncaught exceptions into your log file too,
|
||||||
|
install a top-level hook:
|
||||||
|
|
||||||
!!! note "If your service won't start or its logs aren't persisting"
|
```python
|
||||||
That's usually a host-side bind-mount **ownership** thing — the kind of detail
|
import sys, logging
|
||||||
**we sort out at deploy time**, not something you need to chown or provision.
|
sys.excepthook = lambda *exc: logging.getLogger().critical("uncaught", exc_info=exc)
|
||||||
If a bot won't come up or logs/caches keep vanishing, flag it and we'll fix
|
```
|
||||||
the mount perms. Stick to a clean `compose.yaml` and let us handle the host.
|
|
||||||
|
|
||||||
## Layer caching
|
!!! success "Same file, both places"
|
||||||
|
Locally, `docker compose up` needs nothing set — the `${VAR:-./default}`
|
||||||
|
fallbacks use `./logs` / `./config`. When deployed, the ops tooling sets the
|
||||||
|
real values. One `compose.yaml` works everywhere.
|
||||||
|
|
||||||
Copy `requirements.txt` and `pip install` **before** `COPY . .`. Docker caches
|
**Subprocess and browser workloads** (bots that spawn Chrome, Xvfb, ffmpeg) need
|
||||||
layers in order, so deps only reinstall when `requirements.txt` changes — not on
|
three extra knobs:
|
||||||
every code edit. Get this backwards and every one-line change triggers a full
|
|
||||||
dependency reinstall.
|
|
||||||
|
|
||||||
```dockerfile
|
```yaml
|
||||||
COPY requirements.txt .
|
services:
|
||||||
RUN pip install --no-cache-dir -r requirements.txt # cached until deps change
|
svc:
|
||||||
COPY . . # changes every build
|
build: .
|
||||||
```
|
user: "1337:1337"
|
||||||
|
restart: unless-stopped
|
||||||
|
init: true # tini as PID 1 — reaps zombie subprocesses, forwards signals
|
||||||
|
shm_size: "2gb" # Chrome/headless browsers crash on Docker's default 64 MB /dev/shm
|
||||||
|
mem_limit: "4g" # bound memory — raise as worker/browser count grows
|
||||||
|
```
|
||||||
|
|
||||||
## Subprocess and browser workloads
|
!!! warning "The PID-1 gotcha with shell-wrapper CMDs"
|
||||||
|
If your `CMD` is a shell-script wrapper (e.g. `xvfb-run ...`), it must **not**
|
||||||
|
be PID 1, or the real process dies on startup. `init: true` fixes this — tini
|
||||||
|
takes PID 1, your wrapper runs as a normal child.
|
||||||
|
|
||||||
Bots that spawn Chrome, Xvfb, ffmpeg, or other child processes need three extra
|
=== "Dockerfile"
|
||||||
knobs in compose:
|
|
||||||
|
|
||||||
```yaml
|
Every service runs containerized as the shared **`services`** account: **uid/gid
|
||||||
services:
|
1337**, fixed fleet-wide. Build the image to be **uid-agnostic** so it runs cleanly
|
||||||
yourbot:
|
as that account.
|
||||||
build: .
|
|
||||||
user: "1337:1337"
|
|
||||||
init: true # (1)!
|
|
||||||
shm_size: "2gb" # (2)!
|
|
||||||
mem_limit: "4g" # (3)!
|
|
||||||
```
|
|
||||||
|
|
||||||
1. Runs **tini** as PID 1 to reap zombie subprocesses and forward signals.
|
```dockerfile
|
||||||
Without it, spawned Chrome/Xvfb processes leak as zombies.
|
FROM python:3.12-slim
|
||||||
2. Chrome and most headless browsers crash on Docker's default **64 MB**
|
ENV HOME=/tmp # services account has no home dir; $HOME must be writable
|
||||||
`/dev/shm`. Bump it for any browser workload.
|
|
||||||
3. Bound memory — especially when each worker spawns a browser. Raise it as
|
|
||||||
worker count grows.
|
|
||||||
|
|
||||||
!!! warning "The PID-1 gotcha with shell-wrapper CMDs"
|
WORKDIR /app
|
||||||
If your `CMD` is a shell-script wrapper (e.g. `xvfb-run ...`), it must **not**
|
RUN apt-get update \
|
||||||
be PID 1, or the real process dies on startup. `init: true` is exactly what
|
&& apt-get install -y --no-install-recommends git \
|
||||||
fixes this — tini takes PID 1, and your wrapper runs as a normal child.
|
&& rm -rf /var/lib/apt/lists/* # include git ONLY if the container itself needs it
|
||||||
|
|
||||||
## What your compose / Dockerfile needs
|
COPY requirements.txt .
|
||||||
|
RUN pip install --no-cache-dir -r requirements.txt # deps before code — layer caching
|
||||||
|
|
||||||
- `user: "1337:1337"`
|
COPY . .
|
||||||
- bind mounts for **configs + logs**
|
RUN chmod -R a+rwX /app # writable by any uid — this is "uid-agnostic"
|
||||||
- named volumes for **the rest**
|
CMD ["python", "-m", "yourapp"]
|
||||||
- secrets bind-mounted **`:ro`**
|
```
|
||||||
- `HOME=/tmp`
|
|
||||||
- `chmod -R a+rwX /app`
|
|
||||||
- deps installed **before** the code copy (layer caching)
|
|
||||||
- `git` in the image **if the container needs it**
|
|
||||||
- for browser/subprocess workloads: `init: true`, `shm_size`, `mem_limit`
|
|
||||||
|
|
||||||
## Secrets
|
**Layer caching:** copy the deps file and install **before** `COPY . .` — Docker
|
||||||
|
caches layers in order, so deps only reinstall when the deps file changes, not on
|
||||||
|
every code edit.
|
||||||
|
|
||||||
!!! warning "Secrets never go in the image"
|
**Getting files in:** `COPY . .` grabs the whole repo; be explicit about anything
|
||||||
We do **not** commit secrets (usually, lol). They stay **gitignored**, live on
|
that needs its own place. In `COPY <src> <dest>`, `<src>` is relative to the build
|
||||||
the host at `/srv/configs/<project>/`, and are bind-mounted **read-only** at
|
context (your repo), `<dest>` is a path in the image.
|
||||||
runtime. Add them to `.dockerignore` so a `COPY . .` can't sweep them into a
|
|
||||||
layer.
|
|
||||||
|
|
||||||
Rotating a secret = edit the host file and restart. No rebuild.
|
```dockerfile
|
||||||
|
COPY ./config.toml /app/config.toml # a single file into a specific path
|
||||||
|
COPY ./assets /app/assets # a whole directory (tree mirrors ./assets/)
|
||||||
|
```
|
||||||
|
|
||||||
```bash
|
!!! tip "Faster builds with uv (optional)"
|
||||||
vim /srv/configs/<project>/secrets.env # edit on the host
|
[uv](https://docs.astral.sh/uv/) is a drop-in for pip that reads the same
|
||||||
docker compose restart yourapp # pick up the change — no rebuild
|
`pyproject.toml` — no lockfile needed in the image. Add `UV_COMPILE_BYTECODE`
|
||||||
```
|
so containers don't pay the first-import `.pyc` compile cost:
|
||||||
|
|
||||||
|
```dockerfile
|
||||||
|
FROM python:3.12-slim
|
||||||
|
COPY --from=ghcr.io/astral-sh/uv:latest /uv /uvx /bin/ # pull the uv binary from its image
|
||||||
|
ENV HOME=/tmp
|
||||||
|
ENV UV_COMPILE_BYTECODE=1 # compile bytecode at build, not cold start
|
||||||
|
|
||||||
|
WORKDIR /app
|
||||||
|
COPY pyproject.toml .
|
||||||
|
RUN uv pip install --system . # into the image's Python, no venv/lock
|
||||||
|
COPY . .
|
||||||
|
RUN chmod -R a+rwX /app
|
||||||
|
CMD ["python", "-m", "yourapp"]
|
||||||
|
```
|
||||||
|
|
||||||
|
=== "Secrets"
|
||||||
|
|
||||||
|
!!! warning "Secrets never go in the image"
|
||||||
|
We do **not** commit secrets (usually, lol). They stay **gitignored** and live
|
||||||
|
on the host in your config dir, reaching your container read-only via
|
||||||
|
`${CONFIG_DIR}`. Add them to `.dockerignore` so a `COPY . .` can't sweep them
|
||||||
|
into a layer.
|
||||||
|
|
||||||
|
- Secrets live on the **host**, in your config dir — never in git, never in the
|
||||||
|
image.
|
||||||
|
- They reach the container **read-only** via the injected `${CONFIG_DIR}` mount.
|
||||||
|
- Keep them out of the build context: list them in `.dockerignore` so a blanket
|
||||||
|
`COPY . .` can't pull them into a layer.
|
||||||
|
|
||||||
|
**Rotating a secret** is a host-side edit — update the file and the service picks
|
||||||
|
it up on restart. No rebuild, and nothing you run: flag it and we handle the
|
||||||
|
restart.
|
||||||
|
|
||||||
|
## Supporting services live in your compose, not the fleet
|
||||||
|
|
||||||
|
Need Redis? **Declare it in your own `compose.yaml`, as a sidecar.** There is no shared
|
||||||
|
Redis — nothing fleet-wide, nothing per-workspace, nothing ops provisions for you. A repo
|
||||||
|
that needs Redis brings its own; a repo that doesn't adds nothing.
|
||||||
|
|
||||||
|
That's the whole point of the deploy model. Every service already runs in its own compose
|
||||||
|
project on its own network so that one service falling over can't touch another. A shared
|
||||||
|
Redis puts that coupling straight back: one process everything depends on, whose OOM, stray
|
||||||
|
`FLUSHALL`, single-threaded stall, or restart becomes *everyone's* outage. A sidecar shares
|
||||||
|
its owning repo's fate and nobody else's — and the isolation is free, because it rides the
|
||||||
|
per-project network you already get. No ACLs, no key-prefix discipline, no shared
|
||||||
|
credentials to manage.
|
||||||
|
|
||||||
|
Your app talks to it with the [`redis` lib](libraries.md) from the suite (async,
|
||||||
|
config-free, kv/hash/ttl/pubsub), pointed at **`redis://redis:6379`** — the compose
|
||||||
|
**service name**, not a host port. The sidecar comes up auto-namespaced on your project's
|
||||||
|
network like every other container, exactly as the naming convention above describes.
|
||||||
|
|
||||||
|
### Ephemeral or persistent — pick deliberately
|
||||||
|
|
||||||
|
A sidecar Redis is **ephemeral by default**: restart it and the data is gone. That's
|
||||||
|
correct for some workloads and quietly destructive for others, so make the call on purpose.
|
||||||
|
Ask one question — *if this data vanished on a restart, would anything be lost?*
|
||||||
|
|
||||||
|
- **No → ephemeral.** A scratch cache, a dedupe set, rate-limit counters, transient data
|
||||||
|
you can just re-fetch. Nothing to back up, nothing to grow.
|
||||||
|
- **Yes → persistent.** An outbound webhook or notification queue, a job queue, anything
|
||||||
|
that could be mid-flight when the process dies. Losing it drops real work.
|
||||||
|
|
||||||
|
=== "Ephemeral (cache / throwaway)"
|
||||||
|
|
||||||
|
Fine to lose on restart — no volume, no persistence, by design.
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
redis:
|
||||||
|
image: redis:7-alpine
|
||||||
|
restart: unless-stopped
|
||||||
|
command: redis-server --save "" --maxmemory 256mb --maxmemory-policy allkeys-lru
|
||||||
|
# no volume: throwaway by design
|
||||||
|
```
|
||||||
|
|
||||||
|
=== "Persistent (durable queue / state)"
|
||||||
|
|
||||||
|
Survives restart, rebuild, and reboot — the append-only file lives on the host mounts
|
||||||
|
dir injected at deploy, the same `${MOUNTS_DIR}` mechanism described above. Redis just
|
||||||
|
uses it as its backing store.
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
redis:
|
||||||
|
image: redis:7-alpine
|
||||||
|
restart: unless-stopped
|
||||||
|
command: redis-server --appendonly yes --appendfsync everysec
|
||||||
|
volumes:
|
||||||
|
- ${MOUNTS_DIR:-./mounts}/redis:/data # AOF persists on the host mounts dir
|
||||||
|
```
|
||||||
|
|
||||||
|
The app connects the same way in both modes — `REDIS_URL: redis://redis:6379`. Only the
|
||||||
|
durability changes.
|
||||||
|
|
||||||
|
!!! warning "Two things to know before you rely on a persistent sidecar"
|
||||||
|
**`--appendfsync everysec` can lose ~1 second of the newest entries** on a hard crash.
|
||||||
|
For a webhook queue that's an acceptable trade — just know it's there. Use
|
||||||
|
`--appendfsync always` if you genuinely cannot drop a single entry (safer, slower).
|
||||||
|
|
||||||
|
**Data under the mounts dir is not backed up.** Mounts are excluded from the backup
|
||||||
|
pipeline, and that's *right* for a queue: a lost queue means some notifications didn't
|
||||||
|
fire, not that business data is gone. **That's the dividing line.** If losing this data
|
||||||
|
would actually hurt, it isn't queue or cache state — it's a system of record, and it
|
||||||
|
belongs in Postgres (which *is* backed up), not a local mount.
|
||||||
|
|
||||||
|
### The rules
|
||||||
|
|
||||||
|
!!! danger "Don't do these"
|
||||||
|
- **Don't map Redis to a host port.** No `ports: - "6379:6379"`. Two repos both
|
||||||
|
grabbing host 6379 on the same box collide. Keep it internal to the compose network —
|
||||||
|
nothing exposed, nothing to collide.
|
||||||
|
- **Don't stand up a shared or fleet-wide Redis.** Per-repo means per-need. One Redis
|
||||||
|
per *project*, shared by that project's containers if a repo runs several — never one
|
||||||
|
per fleet.
|
||||||
|
- **Don't treat persistent Redis as a database.** Queues and caches, yes. A durable
|
||||||
|
system of record, no — that's Postgres, and unlike a mount it's backed up.
|
||||||
|
|
||||||
|
**You can't reach another repo's Redis** — different project, different network. That's not
|
||||||
|
a restriction you have to work around; it's the isolation working *for* you. Nobody else's
|
||||||
|
service can touch your cache or drain your queue either, and you never have to think about
|
||||||
|
whose keys are whose.
|
||||||
|
|
||||||
|
!!! quote "What about ACLs?"
|
||||||
|
A shared Redis *can* be secured — Redis 6+ ACLs scope users by command, key pattern, and
|
||||||
|
channel. But ACLs don't solve resource contention or the noisy-neighbour problem, and
|
||||||
|
they add real management burden, so the fleet uses per-repo sidecars instead. Reserve
|
||||||
|
ACLs for the rare case of a deliberately shared, durable, backed-up Redis run as actual
|
||||||
|
infrastructure.
|
||||||
|
|||||||
+57
-4
@@ -21,7 +21,14 @@ happens, depending on where the project runs:
|
|||||||
```bash
|
```bash
|
||||||
python -m venv .venv # create it (once)
|
python -m venv .venv # create it (once)
|
||||||
source .venv/bin/activate # activate for this shell
|
source .venv/bin/activate # activate for this shell
|
||||||
pip install -r requirements.txt
|
pip install -e . # install the project from pyproject.toml
|
||||||
|
```
|
||||||
|
|
||||||
|
Or with [uv](#uv-optional-faster) — same `pyproject.toml`, much faster:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
uv venv # create .venv
|
||||||
|
uv pip install -e . # install from pyproject.toml
|
||||||
```
|
```
|
||||||
|
|
||||||
Keep `.venv/` **gitignored** — it's per-machine, never committed.
|
Keep `.venv/` **gitignored** — it's per-machine, never committed.
|
||||||
@@ -55,13 +62,14 @@ happens, depending on where the project runs:
|
|||||||
|
|
||||||
```dockerfile
|
```dockerfile
|
||||||
FROM python:3.12-slim
|
FROM python:3.12-slim
|
||||||
COPY requirements.txt .
|
COPY pyproject.toml .
|
||||||
RUN pip install --no-cache-dir -r requirements.txt
|
RUN pip install --no-cache-dir . # or: uv pip install .
|
||||||
COPY . .
|
COPY . .
|
||||||
```
|
```
|
||||||
|
|
||||||
This is how things run in production — see the [Deploy guide](deploy.md) for the
|
This is how things run in production — see the [Deploy guide](deploy.md) for the
|
||||||
full container standard (uid 1337, mounts, layer caching).
|
full container standard (uid 1337, the compose convention, layer caching, and the
|
||||||
|
uv image setup).
|
||||||
|
|
||||||
!!! tip "Which one?"
|
!!! tip "Which one?"
|
||||||
**Local `.venv`** for quick iteration, **Makefile** when you want repeatable
|
**Local `.venv`** for quick iteration, **Makefile** when you want repeatable
|
||||||
@@ -69,6 +77,51 @@ happens, depending on where the project runs:
|
|||||||
exclusive — a project often has a `.venv` for local dev *and* a Dockerfile for
|
exclusive — a project often has a `.venv` for local dev *and* a Dockerfile for
|
||||||
deploy.
|
deploy.
|
||||||
|
|
||||||
|
## uv (optional, faster)
|
||||||
|
|
||||||
|
!!! tip "uv is the recommended fast path; pip stays the baseline"
|
||||||
|
[uv](https://docs.astral.sh/uv/) is a faster, standards-compliant drop-in for
|
||||||
|
pip. It reads the **same `pyproject.toml`** — no workflow change required, and
|
||||||
|
`pip` keeps working exactly as before. Use it wherever you'd reach for pip; the
|
||||||
|
rest of this handbook shows the pip command with the uv equivalent beside it.
|
||||||
|
|
||||||
|
Install deps straight from `pyproject.toml` (no `requirements.txt` needed):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
uv pip install . # install the project + its deps
|
||||||
|
uv pip install -e . # editable (dev) install
|
||||||
|
uv pip install '.[dev]' # with an extras group, e.g. dev
|
||||||
|
```
|
||||||
|
|
||||||
|
If you'd rather have uv manage the venv for you, use the managed-venv flow:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
uv sync # create/refresh .venv from pyproject.toml + uv.lock
|
||||||
|
uv run python -m yourapp # run inside the managed env, no manual activate
|
||||||
|
```
|
||||||
|
|
||||||
|
!!! info "`uv.lock` is local-only — never committed"
|
||||||
|
`uv sync` writes a `uv.lock` for your machine's resolved environment. It is
|
||||||
|
**gitignored**, not committed — we don't ship a lockfile. Pinning happens in
|
||||||
|
`pyproject.toml` (below), not the lock.
|
||||||
|
|
||||||
|
### Pinning deps
|
||||||
|
|
||||||
|
Pin **direct** dependencies in `pyproject.toml` with `==` or a git `@ref`:
|
||||||
|
|
||||||
|
```toml
|
||||||
|
[project]
|
||||||
|
dependencies = [
|
||||||
|
"requests==2.31.0",
|
||||||
|
"mylib @ git+https://git.rethinkstudios.io/rethink-public/mylib.git@<sha>",
|
||||||
|
]
|
||||||
|
```
|
||||||
|
|
||||||
|
!!! warning "This pins direct deps only — transitive deps still float"
|
||||||
|
`==` / `@ref` pins the packages **you** list. Their dependencies still resolve
|
||||||
|
fresh at build time. That's an accepted tradeoff — documented on purpose — not
|
||||||
|
an oversight: we pin what we depend on directly and let the rest float.
|
||||||
|
|
||||||
## Local dev with pyenv
|
## Local dev with pyenv
|
||||||
|
|
||||||
For local work you also need the right **Python version**, not just isolated deps.
|
For local work you also need the right **Python version**, not just isolated deps.
|
||||||
|
|||||||
+33
-6
@@ -1,11 +1,38 @@
|
|||||||
# rethink development
|
# rethink development
|
||||||
|
|
||||||
The public reference for building and shipping with Rethink Studios: our shared
|
The reference for building and shipping on our network: the shared libraries, the
|
||||||
libraries, our coding standards, and how to deploy a project on our network.
|
coding standards, and how to get a project deployed.
|
||||||
|
|
||||||
This is a public site — it documents generic patterns and conventions. Real
|
This is a public site — it documents generic patterns and conventions. Real
|
||||||
infrastructure specifics (hostnames, internal IPs, exact topology, secrets) stay
|
infrastructure specifics (hostnames, internal IPs, exact topology, secrets) stay
|
||||||
out of it; examples use placeholders like `<dev>`, `<project>`, and `/srv/...`.
|
out of it; examples use placeholders like `<workspace>`, `<project>`, and
|
||||||
|
`/srv/...`.
|
||||||
|
|
||||||
|
!!! example "Point your coding agent here"
|
||||||
|
Want your agent aware of our libraries, standards, and deploy rules before it
|
||||||
|
writes a line? Tell it to read this handbook — so it reaches for an existing
|
||||||
|
`rethink-public` lib instead of reinventing it, follows our conventions, and
|
||||||
|
builds a deploy-ready container.
|
||||||
|
|
||||||
|
=== "From the live site"
|
||||||
|
|
||||||
|
```text
|
||||||
|
Read https://docs.rethinkstudios.io and follow it: prefer our
|
||||||
|
rethink-public libraries, match our coding standards, and make anything
|
||||||
|
deployable per the deploy guide.
|
||||||
|
```
|
||||||
|
|
||||||
|
=== "From git"
|
||||||
|
|
||||||
|
Point it straight at the repo — no need to clone into your project:
|
||||||
|
|
||||||
|
```text
|
||||||
|
Read the markdown under docs/ in
|
||||||
|
https://git.rethinkstudios.io/rethink-public/handbook and follow it:
|
||||||
|
prefer our rethink-public libraries, match our coding standards, and make
|
||||||
|
anything deployable per the deploy guide. Clone to /tmp if you need it
|
||||||
|
local.
|
||||||
|
```
|
||||||
|
|
||||||
## Sections
|
## Sections
|
||||||
|
|
||||||
@@ -29,7 +56,7 @@ out of it; examples use placeholders like `<dev>`, `<project>`, and `/srv/...`.
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
How we actually work day to day — our Gitea, git habits, and the
|
Get hands on with how we dev — our Gitea, git habits, and the
|
||||||
plan-in-chat / build-in-Claude-Code flow, plus shell setup.
|
plan-in-chat / build-in-Claude-Code flow, plus shell setup.
|
||||||
|
|
||||||
- :material-language-python: __[Virtual environments](environments.md)__
|
- :material-language-python: __[Virtual environments](environments.md)__
|
||||||
@@ -43,7 +70,7 @@ out of it; examples use placeholders like `<dev>`, `<project>`, and `/srv/...`.
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
How to get a project running on **rethink-net** — containers, paths and
|
How to get a project running on **rethink-net** — the compose convention,
|
||||||
mounts, permissions, and secrets.
|
the one-command deploy, and secrets.
|
||||||
|
|
||||||
</div>
|
</div>
|
||||||
|
|||||||
@@ -0,0 +1,29 @@
|
|||||||
|
// render mermaid diagrams (emitted as <div class="mermaid">SOURCE</div>).
|
||||||
|
// render(id, src) is used directly and each block is processed once.
|
||||||
|
(function () {
|
||||||
|
var inited = false;
|
||||||
|
var seq = 0;
|
||||||
|
function boot() {
|
||||||
|
if (typeof mermaid === "undefined") return;
|
||||||
|
if (!inited) {
|
||||||
|
mermaid.initialize({ startOnLoad: false, theme: "dark", securityLevel: "loose" });
|
||||||
|
inited = true;
|
||||||
|
}
|
||||||
|
document.querySelectorAll("div.mermaid").forEach(function (el) {
|
||||||
|
if (el.dataset.mmdDone) return;
|
||||||
|
var src = el.textContent.trim();
|
||||||
|
if (!src) return;
|
||||||
|
el.dataset.mmdDone = "1";
|
||||||
|
mermaid.render("mmd-" + seq++, src).then(function (out) {
|
||||||
|
el.innerHTML = out.svg;
|
||||||
|
}).catch(function () {
|
||||||
|
delete el.dataset.mmdDone;
|
||||||
|
});
|
||||||
|
});
|
||||||
|
}
|
||||||
|
if (window.document$ && typeof window.document$.subscribe === "function") {
|
||||||
|
window.document$.subscribe(boot);
|
||||||
|
} else {
|
||||||
|
document.addEventListener("DOMContentLoaded", boot);
|
||||||
|
}
|
||||||
|
})();
|
||||||
Vendored
+3587
File diff suppressed because one or more lines are too long
+41
-1
@@ -125,7 +125,47 @@ TimeoutError: request timed out after 30s
|
|||||||
2026-06-29 14:03:11,204 WARNING aioweb.session fetch timed out: https://example.test/feed
|
2026-06-29 14:03:11,204 WARNING aioweb.session fetch timed out: https://example.test/feed
|
||||||
```
|
```
|
||||||
|
|
||||||
!!! note "Logging belongs to the app, not the library"
|
!!! info "Logging belongs to the app, not the library"
|
||||||
Libraries **emit only** — `log = logging.getLogger(__name__)` and nothing
|
Libraries **emit only** — `log = logging.getLogger(__name__)` and nothing
|
||||||
else. Handlers, levels, and formatting are configured once at the
|
else. Handlers, levels, and formatting are configured once at the
|
||||||
application entry point, so a lib never dictates how its host logs.
|
application entry point, so a lib never dictates how its host logs.
|
||||||
|
|
||||||
|
## Service compose
|
||||||
|
|
||||||
|
A deployable service ships a `compose.yaml` that names **nothing repo-specific** —
|
||||||
|
the deploy layer injects identity and host paths. See the
|
||||||
|
[Deploy guide](deploy.md) for the full convention and the variables
|
||||||
|
you can rely on. The short version:
|
||||||
|
|
||||||
|
=== "Right"
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
services:
|
||||||
|
svc:
|
||||||
|
user: "1337:1337"
|
||||||
|
restart: unless-stopped
|
||||||
|
volumes:
|
||||||
|
- ${LOGS_DIR:-./logs}:/app/logs
|
||||||
|
- profile:/app/profile
|
||||||
|
|
||||||
|
volumes:
|
||||||
|
profile:
|
||||||
|
```
|
||||||
|
|
||||||
|
=== "Wrong (causes collisions)"
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
services:
|
||||||
|
nova:
|
||||||
|
container_name: nova # repo-specific name -> collides
|
||||||
|
volumes:
|
||||||
|
- /srv/logs/ricky/nova:/app/logs # hardcoded host path
|
||||||
|
- nova_profile:/app/profile # repo-prefixed volume
|
||||||
|
|
||||||
|
volumes:
|
||||||
|
nova_profile:
|
||||||
|
```
|
||||||
|
|
||||||
|
Generic service key `svc`, no `container_name`, host paths from `${...}` variables,
|
||||||
|
and **bare** volume names — that's what makes a service collision-proof on the
|
||||||
|
fleet.
|
||||||
|
|||||||
+51
-6
@@ -1,10 +1,10 @@
|
|||||||
# Workflow
|
# Workflow
|
||||||
|
|
||||||
How we actually work day to day at Rethink Studios — where code lives, how we use
|
Get hands on with how we dev — where code lives, how we use git, the AI-assisted
|
||||||
git, the AI-assisted dev flow we recommend, and the shell setup that ties it
|
flow we recommend, and the shell setup that ties it together. This is the
|
||||||
together. This is the **our-flavored** version: why *we* do it this way and how
|
**our-flavored** version: why *we* do it this way and how *our* setup is wired. For
|
||||||
*our* setup is wired. For the truly generic parts (installing WSL, learning git),
|
the truly generic parts (installing WSL, learning git), we link the official docs
|
||||||
we link the official docs rather than reteach them.
|
rather than reteach them.
|
||||||
|
|
||||||
!!! info "Public, sanitized"
|
!!! info "Public, sanitized"
|
||||||
Examples use placeholders — `<you>`, `<key>`, `dev@<you>`, `/mnt/c/<your>/...`.
|
Examples use placeholders — `<you>`, `<key>`, `dev@<you>`, `/mnt/c/<your>/...`.
|
||||||
@@ -56,7 +56,7 @@ with the right one per project.
|
|||||||
|
|
||||||
We solve that with **per-repo local git config** — run a small alias inside a repo
|
We solve that with **per-repo local git config** — run a small alias inside a repo
|
||||||
to set its local user and the SSH key it pushes with (see
|
to set its local user and the SSH key it pushes with (see
|
||||||
[per-project git identity](#per-project-git-identity) below). No global identity
|
[per-project git identity](#handy-shell-setup) below). No global identity
|
||||||
juggling.
|
juggling.
|
||||||
|
|
||||||
Our conventions, in short:
|
Our conventions, in short:
|
||||||
@@ -241,3 +241,48 @@ gitcs() {
|
|||||||
- **`gl`** — a readable branch graph for understanding history at a glance.
|
- **`gl`** — a readable branch graph for understanding history at a glance.
|
||||||
- **`pyenv`** — per-project Python versions, so each repo builds against the
|
- **`pyenv`** — per-project Python versions, so each repo builds against the
|
||||||
version it targets.
|
version it targets.
|
||||||
|
|
||||||
|
## Paste service
|
||||||
|
|
||||||
|
A shared, self-hosted pastebin at
|
||||||
|
[paste.rethinkstudios.io](https://paste.rethinkstudios.io) — for quickly sharing
|
||||||
|
logs, snippets, or command output when you're pairing, filing an issue, or handing
|
||||||
|
output to a coding agent. Pastes are **unlisted** (random URL), **expire** by
|
||||||
|
default, and support **burn-after-read**. Anonymous — no login.
|
||||||
|
|
||||||
|
The easy way: pipe anything into it and get back a URL. Drop this into your
|
||||||
|
`.zshrc` / `.bashrc` — a **function** is preferred over a plain alias because it
|
||||||
|
reads stdin cleanly and has room to grow options (needs `jq` + `curl`):
|
||||||
|
|
||||||
|
=== "Function (recommended)"
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# paste stdin to the rethink paste service, print the URL. usage: cat file | pb
|
||||||
|
pb() {
|
||||||
|
jq -Rns '{text: inputs, expires: 259200}' \
|
||||||
|
| curl -s -H 'Content-Type: application/json' --data-binary @- https://paste.rethinkstudios.io/ \
|
||||||
|
| jq -r '"https://paste.rethinkstudios.io" + .path'
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
=== "Alias (alternative)"
|
||||||
|
|
||||||
|
Same behaviour as a one-liner — note the extra escaping the alias form needs:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
alias pb="jq -Rns '{text: inputs, expires: 259200}' | curl -s -H 'Content-Type: application/json' --data-binary @- https://paste.rethinkstudios.io/ | jq -r '\"https://paste.rethinkstudios.io\" + .path'"
|
||||||
|
```
|
||||||
|
|
||||||
|
Usage — pipe any file or command output straight in:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cat latest.log | pb # -> https://paste.rethinkstudios.io/xxxxxxx
|
||||||
|
mycommand 2>&1 | pb # pipe any command's output (stderr too)
|
||||||
|
```
|
||||||
|
|
||||||
|
`expires` is in **seconds** — `259200` = 72h (the default). Change it (e.g.
|
||||||
|
`86400` for 24h) or drop the field entirely.
|
||||||
|
|
||||||
|
!!! info "Rate-limited on creation"
|
||||||
|
The service rate-limits paste **creation** (not viewing), so it's built for
|
||||||
|
occasional shares — not bulk or automated posting.
|
||||||
|
|||||||
+10
-2
@@ -6,11 +6,15 @@ copyright: rethink development (handbook)
|
|||||||
extra_css:
|
extra_css:
|
||||||
- stylesheets/extra.css
|
- stylesheets/extra.css
|
||||||
|
|
||||||
|
extra_javascript:
|
||||||
|
- javascripts/mermaid.min.js
|
||||||
|
- javascripts/mermaid-init.js
|
||||||
|
|
||||||
theme:
|
theme:
|
||||||
name: material
|
name: material
|
||||||
language: en
|
language: en
|
||||||
logo: assets/logo.svg
|
logo: assets/logo.svg
|
||||||
favicon: assets/logo.svg
|
favicon: assets/favicon.svg
|
||||||
palette:
|
palette:
|
||||||
scheme: slate
|
scheme: slate
|
||||||
primary: custom
|
primary: custom
|
||||||
@@ -35,7 +39,11 @@ markdown_extensions:
|
|||||||
- toc:
|
- toc:
|
||||||
permalink: true
|
permalink: true
|
||||||
- pymdownx.details
|
- pymdownx.details
|
||||||
- pymdownx.superfences
|
- pymdownx.superfences:
|
||||||
|
custom_fences:
|
||||||
|
- name: mermaid
|
||||||
|
class: mermaid
|
||||||
|
format: !!python/name:pymdownx.superfences.fence_div_format
|
||||||
- pymdownx.tabbed:
|
- pymdownx.tabbed:
|
||||||
alternate_style: true
|
alternate_style: true
|
||||||
- pymdownx.highlight:
|
- pymdownx.highlight:
|
||||||
|
|||||||
Reference in New Issue
Block a user