log_setup/README.md

# log_setup

Stdlib, sync, **zero-dependency** logging setup an application calls **once** at its
entry point: a live `run.log`, rotation (daily / size / on-start), gzip of rolled
files, retention, console output, and a consistent `time | module | level | message`
format.

It **configures** logging (handlers, rotation, format) — which reusable libraries here
must never do. That's fine because `log_setup` is the *application's* entry-point
setup, not library-internal config. Libraries still only `logging.getLogger(__name__)`
and emit; their records flow into the handlers `log_setup` wired.

## Install

```
log_setup @ git+ssh://git@git.rethinkstudios.io/rethink-public/log_setup.git@v0.3.2
```

No dependencies — stdlib only.

## Quick start

```python
import logging
from log_setup import setup_logging

setup_logging(name="run", level="INFO")     # daily rotation, logs/ dir, gzip (file only)

log = logging.getLogger(__name__)
log.info("started")                          # -> ./run.log  (add console=True for stdout too)
```

Call it once, at the app's entry point — before the rest of the app runs. Every module
(yours and the libraries you import) then just does `logging.getLogger(__name__)` and
emits; the records land in the configured root.

## What you get

- **Live file** at a stable path: `./run.log` — always `tail -f run.log`, no dated name
  to chase. Rolled/compressed copies go into `log_dir` (default `logs/`).
- **Format:** `2026-06-27 19:55:05 | module.name | INFO | message`. `%(name)s` is the
  `getLogger` name each module used, so you see which lib/module logged.
- **Rotation** (`rotate=`):
  - `"daily"` (default) — rolls at midnight, dated name into `log_dir`, keeps
    `backup_count` days.
  - `"size"` — rolls at `max_bytes`, numbered backups in `log_dir`.
  - `"on_start"` — on startup, moves an existing `run.log` into `log_dir`
    (`run.<timestamp>.log[.gz]`) and starts fresh; prunes to `backup_count`.
  - `None` — single file, no rotation.
- **compress=True** (default) gzips each rolled file (`run.log.2026-06-27.gz`).
- **Retention** = `backup_count` (default 14) for every mode.
- **console=True** (off by default) also logs to stdout in the same format — opt in when
  you want live terminal output alongside the file.

## Output format (`output=`)

Two formats, two needs. Default is `"text"`; the live-file name is the same either way
(`run.log`, never auto-renamed), so a service can switch text↔json without breaking the
Promtail glob, bind-mount path, or your `tail` command.

- **`output="text"`** (default) — human-readable
  `2026-06-27 19:55:05 | module.name | INFO | message`, **local time**. The
  single-machine `tail -f` path. `fmt`/`datefmt` override it. Unchanged from v0.1.x.
- **`output="json"`** — structured **one JSON object per line** (JSON Lines) for the
  Grafana/Loki pipeline (Promtail → Loki → Grafana); Loki parses JSON fields into labels
  natively, no regex.

```python
setup_logging(name="run", output="json")
logging.getLogger("bot.core").info("ready", extra={"monitor": "heartbeat"})
# -> {"time": "2026-06-28T14:03:11Z", "level": "INFO", "module": "bot.core",
#     "message": "ready", "monitor": "heartbeat"}
```

- **Fields:** `time`, `level`, `module`, `message` always; any `extra={...}` keys land
  as **top-level** fields (stamp `monitor`/`service`/request-id for Loki labels — the lib
  stays domain-agnostic); error records carry the traceback in `exc_info` (never dropped).
- **Time is UTC ISO-8601 with a `Z`** (`2026-06-28T14:03:11Z`), not local. json is the
  aggregation path — logs from many servers/containers sort unambiguously only in UTC;
  Grafana converts to local for display. (Text mode stays local — that's a human on one
  box.)
- Both file and console use the chosen format. `fmt`/`datefmt` apply to text only (json
  builds fields, not a format string). An unknown `output` falls back to text + warns,
  never crashes. **Zero new deps** — stdlib `json` only.

## Signature

```python
setup_logging(
    name="run",               # base -> run.log (the live file at cwd)
    log_dir="logs",           # rotated/compressed copies live here (created if absent)
    level="INFO",             # root level everything inherits (str name or logging constant)
    module_levels=None,       # {logger_name: level} per-logger overrides (exact name match)
    rotate="daily",           # "daily" | "size" | "on_start" | None
    backup_count=14,          # rotated files to keep (older auto-deleted)
    max_bytes=10_000_000,     # only for rotate="size"
    compress=True,            # gzip rolled files
    console=False,            # also log to stdout (off by default; opt in)
    queue=False,              # route through a background QueueListener (async-friendly)
    output="text",            # "text" (human, local time) | "json" (structured, UTC)
    fmt=None,                 # override the text format string (text mode only)
    datefmt=None,             # override the text date format (text mode only)
) -> logging.Logger           # returns the configured root logger
```

## Quieting noisy dependencies (`module_levels`)

`level` is the **root default** — every logger inherits it. `module_levels` is an
optional `{logger_name: level}` map of **per-logger overrides** applied at setup, the
standard "turn down the chatty dependency while my own code stays at INFO" case:

```python
setup_logging(
    name="run",
    level="INFO",                 # our code logs at INFO
    module_levels={
        "motor": "WARNING",       # quiet the driver
        "pymongo": "WARNING",
        "aiohttp": "WARNING",     # also quiets aiohttp.client / aiohttp.access (hierarchy)
    },
)
```

- **Exact-name match — names are NOT discovered.** It calls
  `logging.getLogger(name).setLevel(level)` for exactly the name you give. There's no
  smart find of noisy modules; you name the loggers. A typo (`"moter"`) silently
  configures a logger nothing uses — no error, no effect. Get the names right.
- **Hierarchy applies** (the one "smart" part, and it's just stdlib): naming a **parent**
  quiets its whole subtree. `"aiohttp"` also quiets `aiohttp.client`, `aiohttp.access`,
  etc. — the way to catch sub-loggers without listing each.
- **str or int** per entry (`"WARNING"` or `logging.WARNING`) — same normalization as the
  root `level`.
- **Never crashes:** a bad level for one entry is **skipped with a warning**; the other
  entries and the rest of setup still apply. Consistent with the never-crash-over-logging
  rule.
- `None`/`{}` (default) → no overrides; existing callers are unaffected.

Common noisy library logger names: `motor`, `pymongo`, `aiohttp` (parent quiets
`aiohttp.client`/`aiohttp.access`), `discord` / `discord.*`, `asyncio`, `urllib3`. Check
a lib's *actual* logger name — some log under a name different from their package.

This already works without the lib (`logging.getLogger("motor").setLevel(WARNING)` after
setup does the same via stdlib hierarchy). The param's value is ergonomic: it keeps the
overrides in the **one** `setup_logging` call at the entry point instead of scattering
`setLevel` calls afterward — which is the whole point of `log_setup`.

## Async-friendly (`queue=True`)

For async-heavy apps, `queue=True` routes records through a stdlib `QueueHandler` to a
background `QueueListener` that owns the file/console handlers, so the event loop never
blocks on file I/O. The API stays sync (`log.info()` as usual); the queue is internal.
The listener is stopped (and flushed) cleanly at process exit, so no records are lost.

```python
setup_logging(name="run", queue=True)
```

## Safety

- **Idempotent:** calling `setup_logging` again clears only the handlers it added (no
  duplicate lines) and leaves handlers your app added itself alone.
- **Never crashes the app over logging:** if `log_dir` isn't writable, it falls back to
  console-only with a warning instead of raising.

## Scope — what this is NOT

`log_setup` produces clean, rotating, compressed, retention-managed, consistently
formatted **files**. It does **not** ship logs anywhere — no Loki/ELK/syslog/network
handlers. Getting files to a backend is a separate concern (e.g. Promtail tails
`run.log` → Loki → Grafana panels + alerting). Keeping shipping out means the log
backend can change without touching any app, and the consistent format here is what
makes downstream parsing and alerting easy.

Structured/JSON output is **in** as of v0.2.0 (`output="json"`) — text and json only.
Still deliberately out: logfmt or other formats, a format DSL, per-handler formats,
color formatting, per-logger filters, remote handlers.

## Versioning

Tagged `vX.Y.Z`. Pin the tag.