Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
8bf1866ca2 | ||
|
|
1207c53742 | ||
|
|
b52c1d37fa | ||
|
|
6a10f3acc0 | ||
|
|
ece9a6b9ca | ||
|
|
595f0363b3 | ||
|
|
efb35195f1 | ||
|
|
fc0898d70e | ||
|
|
011588a712 | ||
|
|
ddc81dd8fe | ||
|
|
74c5a42c5a | ||
|
|
ff29e05322 | ||
|
|
73007fe900 | ||
|
|
33d61633af | ||
|
|
871471dd58 | ||
|
|
54151b9835 | ||
|
|
84e1744d6f |
+1
-1
@@ -1,5 +1,5 @@
|
|||||||
# claude
|
# claude
|
||||||
CLAUDE.md
|
.claude/
|
||||||
|
|
||||||
# python
|
# python
|
||||||
__pycache__/
|
__pycache__/
|
||||||
|
|||||||
@@ -13,11 +13,13 @@ and emit; their records flow into the handlers `log_setup` wired.
|
|||||||
## Install
|
## Install
|
||||||
|
|
||||||
```
|
```
|
||||||
log_setup @ git+ssh://git@git.rethinkstudios.io/rethink-public/log_setup.git@v0.1.0
|
log_setup @ git+ssh://git@git.rethinkstudios.io/rethink-public/log_setup.git@v0.5.1
|
||||||
```
|
```
|
||||||
|
|
||||||
No dependencies — stdlib only.
|
No dependencies — stdlib only.
|
||||||
|
|
||||||
|
Drop the `@v0.5.1` suffix from the line above to install the latest unpinned.
|
||||||
|
|
||||||
## Quick start
|
## Quick start
|
||||||
|
|
||||||
```python
|
```python
|
||||||
@@ -41,35 +43,178 @@ emits; the records land in the configured root.
|
|||||||
- **Format:** `2026-06-27 19:55:05 | module.name | INFO | message`. `%(name)s` is the
|
- **Format:** `2026-06-27 19:55:05 | module.name | INFO | message`. `%(name)s` is the
|
||||||
`getLogger` name each module used, so you see which lib/module logged.
|
`getLogger` name each module used, so you see which lib/module logged.
|
||||||
- **Rotation** (`rotate=`):
|
- **Rotation** (`rotate=`):
|
||||||
- `"daily"` (default) — rolls at midnight, dated name into `log_dir`, keeps
|
- `"daily"` (default) — rolls at midnight into `log_dir`, keeps `backup_count` days.
|
||||||
`backup_count` days.
|
- `"size"` — rolls at `max_bytes` into `log_dir`, keeps `backup_count`. `backup_count=0`
|
||||||
- `"size"` — rolls at `max_bytes`, numbered backups in `log_dir`.
|
means **keep no rolled history**: the live file still rolls at `max_bytes` (size is
|
||||||
- `"on_start"` — on startup, moves an existing `run.log` into `log_dir`
|
always bounded), each rolled file is deleted immediately after landing — it does not
|
||||||
(`run.<timestamp>.log[.gz]`) and starts fresh; prunes to `backup_count`.
|
disable rotation (see **Retention** below).
|
||||||
|
- `"on_start"` — on startup, moves an existing live file into `log_dir` and starts fresh;
|
||||||
|
prunes to `backup_count`.
|
||||||
- `None` — single file, no rotation.
|
- `None` — single file, no rotation.
|
||||||
- **compress=True** (default) gzips each rolled file (`run.log.2026-06-27.gz`).
|
- **Historic files are named off the project** — see below. Every rolled file is
|
||||||
- **Retention** = `backup_count` (default 14) for every mode.
|
`<project>.<timestamp>.log[.gz]`; the live file keeps its own name.
|
||||||
|
- **compress=True** (default) gzips each rolled file.
|
||||||
|
- **Retention** = `backup_count` (default 14) for every mode — unless tiered retention is
|
||||||
|
enabled (below). For `rotate="size"`, `backup_count=0` is "keep none" (not "disable
|
||||||
|
rotation") — see the `size` bullet above and the note at the bottom of this section.
|
||||||
- **console=True** (off by default) also logs to stdout in the same format — opt in when
|
- **console=True** (off by default) also logs to stdout in the same format — opt in when
|
||||||
you want live terminal output alongside the file.
|
you want live terminal output alongside the file.
|
||||||
|
|
||||||
|
The `name` you pass is normalized so it produces exactly one `.log`: `name="latest"` and
|
||||||
|
`name="latest.log"` both yield the live file `latest.log` (never `latest.log.log`).
|
||||||
|
|
||||||
|
## Historic files are named off the project (`history_name`)
|
||||||
|
|
||||||
|
The **live** file keeps its defined `name` (`latest.log`). The **historic** (rolled/gz)
|
||||||
|
files are named off the **project namespace** — by default the current directory's basename
|
||||||
|
— so you can tell at a glance which service a log came from:
|
||||||
|
|
||||||
|
```python
|
||||||
|
# app run from bestbuy/run.py , with name="latest":
|
||||||
|
setup_logging(name="latest", rotate="daily")
|
||||||
|
# logs/
|
||||||
|
# latest.log <- live (the tail -f target)
|
||||||
|
# bestbuy.2026-07-01_02-00-00.log <- historic, named off the project dir
|
||||||
|
# bestbuy.2026-06-30_02-00-00.log.gz
|
||||||
|
```
|
||||||
|
|
||||||
|
- **Default** = `os.path.basename(os.getcwd())` (the project directory). Zero config.
|
||||||
|
- Override with **`history_name="foo"`** → historic files become `foo.<timestamp>.log[.gz]`.
|
||||||
|
- This changed in **v0.5.0**: historic files used to reuse the live `name`. To keep the old
|
||||||
|
behavior, pass `history_name=name`.
|
||||||
|
- Retention (tier counts / `backup_count`) is unchanged — it's just keyed to the project
|
||||||
|
stem now.
|
||||||
|
|
||||||
|
## Tiered retention (`keep_uncompressed` / `keep_compressed`)
|
||||||
|
|
||||||
|
The default is a flat `backup_count`: every rolled file is gzipped on roll and the oldest
|
||||||
|
are deleted past the count. If instead you want the recent logs **uncompressed** (read them
|
||||||
|
without `zcat`) and older ones **gzipped**, pass the two tier knobs:
|
||||||
|
|
||||||
|
```python
|
||||||
|
setup_logging(
|
||||||
|
name="latest",
|
||||||
|
rotate="on_start", # works for on_start, daily, and size
|
||||||
|
keep_uncompressed=3, # newest 3 rolled logs kept PLAIN
|
||||||
|
keep_compressed=7, # next 7 kept GZIPPED; total retained = 10
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
Result in `log_dir` (newest → oldest):
|
||||||
|
|
||||||
|
```
|
||||||
|
latest.log <- live "latest" (stable, tail -f)
|
||||||
|
latest.<t1>.log latest.<t2>.log latest.<t3>.log <- 3 newest: plain
|
||||||
|
latest.<t4>.log.gz ... latest.<t10>.log.gz <- next 7: gzipped
|
||||||
|
(anything past 10 deleted)
|
||||||
|
```
|
||||||
|
|
||||||
|
- Each restart (`on_start`) or roll (`daily`/`size`) moves the live file into `log_dir`,
|
||||||
|
then re-tiers: newest `keep_uncompressed` stay plain, the next `keep_compressed` are
|
||||||
|
gzipped in place, the rest deleted. Total kept = `keep_uncompressed + keep_compressed`.
|
||||||
|
- **Opt-in by presence** — pass either knob to enable tiering. Pass **neither** and
|
||||||
|
rotation behaves exactly as before (`backup_count` + gzip-on-roll), so existing callers
|
||||||
|
are unaffected.
|
||||||
|
- In tiered mode `backup_count` and the gzip-on-roll behavior of `compress` are **ignored**
|
||||||
|
— the tier counts bound retention instead.
|
||||||
|
- `keep_uncompressed=0` → everything gzipped; `keep_compressed=0` → only the plain tier.
|
||||||
|
Retention is count-based (not time-based).
|
||||||
|
|
||||||
|
## Output format (`output=`)
|
||||||
|
|
||||||
|
Two formats, two needs. Default is `"text"`; the live-file name is the same either way
|
||||||
|
(`run.log`, never auto-renamed), so a service can switch text↔json without breaking the
|
||||||
|
Promtail glob, bind-mount path, or your `tail` command.
|
||||||
|
|
||||||
|
- **`output="text"`** (default) — human-readable
|
||||||
|
`2026-06-27 19:55:05 | module.name | INFO | message`, **local time**. The
|
||||||
|
single-machine `tail -f` path. `fmt`/`datefmt` override it. Unchanged from v0.1.x.
|
||||||
|
- **`output="json"`** — structured **one JSON object per line** (JSON Lines) for the
|
||||||
|
Grafana/Loki pipeline (Promtail → Loki → Grafana); Loki parses JSON fields into labels
|
||||||
|
natively, no regex.
|
||||||
|
|
||||||
|
```python
|
||||||
|
setup_logging(name="run", output="json")
|
||||||
|
logging.getLogger("bot.core").info("ready", extra={"monitor": "heartbeat"})
|
||||||
|
# -> {"time": "2026-06-28T14:03:11Z", "level": "INFO", "module": "bot.core",
|
||||||
|
# "message": "ready", "monitor": "heartbeat"}
|
||||||
|
```
|
||||||
|
|
||||||
|
- **Fields:** `time`, `level`, `module`, `message` always; any `extra={...}` keys land
|
||||||
|
as **top-level** fields (stamp `monitor`/`service`/request-id for Loki labels — the lib
|
||||||
|
stays domain-agnostic); error records carry the traceback in `exc_info` (never dropped).
|
||||||
|
- **Time is UTC ISO-8601 with a `Z`** (`2026-06-28T14:03:11Z`), not local. json is the
|
||||||
|
aggregation path — logs from many servers/containers sort unambiguously only in UTC;
|
||||||
|
Grafana converts to local for display. (Text mode stays local — that's a human on one
|
||||||
|
box.)
|
||||||
|
- Both file and console use the chosen format. `fmt`/`datefmt` apply to text only (json
|
||||||
|
builds fields, not a format string). An unknown `output` falls back to text + warns,
|
||||||
|
never crashes. **Zero new deps** — stdlib `json` only.
|
||||||
|
|
||||||
## Signature
|
## Signature
|
||||||
|
|
||||||
```python
|
```python
|
||||||
setup_logging(
|
setup_logging(
|
||||||
name="run", # base -> run.log (the live file at cwd)
|
name="run", # base -> run.log (the live file at cwd)
|
||||||
log_dir="logs", # rotated/compressed copies live here (created if absent)
|
log_dir="logs", # rotated/compressed copies live here (created if absent)
|
||||||
level="INFO", # root level (str name or logging constant)
|
level="INFO", # root level everything inherits (str name or logging constant)
|
||||||
|
module_levels=None, # {logger_name: level} per-logger overrides (exact name match)
|
||||||
rotate="daily", # "daily" | "size" | "on_start" | None
|
rotate="daily", # "daily" | "size" | "on_start" | None
|
||||||
backup_count=14, # rotated files to keep (older auto-deleted)
|
history_name=None, # stem for rolled/historic files; None -> cwd basename (project)
|
||||||
|
backup_count=14, # rotated files to keep (flat retention; ignored if tiered)
|
||||||
|
keep_uncompressed=None, # tiered: newest N rolled logs kept PLAIN (opt-in)
|
||||||
|
keep_compressed=None, # tiered: next M rolled logs kept GZIPPED (opt-in)
|
||||||
max_bytes=10_000_000, # only for rotate="size"
|
max_bytes=10_000_000, # only for rotate="size"
|
||||||
compress=True, # gzip rolled files
|
compress=True, # gzip rolled files
|
||||||
console=False, # also log to stdout (off by default; opt in)
|
console=False, # also log to stdout (off by default; opt in)
|
||||||
queue=False, # route through a background QueueListener (async-friendly)
|
queue=False, # route through a background QueueListener (async-friendly)
|
||||||
fmt=None, # override the format string
|
output="text", # "text" (human, local time) | "json" (structured, UTC)
|
||||||
datefmt=None, # override the date format
|
fmt=None, # override the text format string (text mode only)
|
||||||
|
datefmt=None, # override the text date format (text mode only)
|
||||||
) -> logging.Logger # returns the configured root logger
|
) -> logging.Logger # returns the configured root logger
|
||||||
```
|
```
|
||||||
|
|
||||||
|
## Quieting noisy dependencies (`module_levels`)
|
||||||
|
|
||||||
|
`level` is the **root default** — every logger inherits it. `module_levels` is an
|
||||||
|
optional `{logger_name: level}` map of **per-logger overrides** applied at setup, the
|
||||||
|
standard "turn down the chatty dependency while my own code stays at INFO" case:
|
||||||
|
|
||||||
|
```python
|
||||||
|
setup_logging(
|
||||||
|
name="run",
|
||||||
|
level="INFO", # our code logs at INFO
|
||||||
|
module_levels={
|
||||||
|
"motor": "WARNING", # quiet the driver
|
||||||
|
"pymongo": "WARNING",
|
||||||
|
"aiohttp": "WARNING", # also quiets aiohttp.client / aiohttp.access (hierarchy)
|
||||||
|
},
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
- **Exact-name match — names are NOT discovered.** It calls
|
||||||
|
`logging.getLogger(name).setLevel(level)` for exactly the name you give. There's no
|
||||||
|
smart find of noisy modules; you name the loggers. A typo (`"moter"`) silently
|
||||||
|
configures a logger nothing uses — no error, no effect. Get the names right.
|
||||||
|
- **Hierarchy applies** (the one "smart" part, and it's just stdlib): naming a **parent**
|
||||||
|
quiets its whole subtree. `"aiohttp"` also quiets `aiohttp.client`, `aiohttp.access`,
|
||||||
|
etc. — the way to catch sub-loggers without listing each.
|
||||||
|
- **str or int** per entry (`"WARNING"` or `logging.WARNING`) — same normalization as the
|
||||||
|
root `level`.
|
||||||
|
- **Never crashes:** a bad level for one entry is **skipped with a warning**; the other
|
||||||
|
entries and the rest of setup still apply. Consistent with the never-crash-over-logging
|
||||||
|
rule.
|
||||||
|
- `None`/`{}` (default) → no overrides; existing callers are unaffected.
|
||||||
|
|
||||||
|
Common noisy library logger names: `motor`, `pymongo`, `aiohttp` (parent quiets
|
||||||
|
`aiohttp.client`/`aiohttp.access`), `discord` / `discord.*`, `asyncio`, `urllib3`. Check
|
||||||
|
a lib's *actual* logger name — some log under a name different from their package.
|
||||||
|
|
||||||
|
This already works without the lib (`logging.getLogger("motor").setLevel(WARNING)` after
|
||||||
|
setup does the same via stdlib hierarchy). The param's value is ergonomic: it keeps the
|
||||||
|
overrides in the **one** `setup_logging` call at the entry point instead of scattering
|
||||||
|
`setLevel` calls afterward — which is the whole point of `log_setup`.
|
||||||
|
|
||||||
## Async-friendly (`queue=True`)
|
## Async-friendly (`queue=True`)
|
||||||
|
|
||||||
For async-heavy apps, `queue=True` routes records through a stdlib `QueueHandler` to a
|
For async-heavy apps, `queue=True` routes records through a stdlib `QueueHandler` to a
|
||||||
@@ -87,6 +232,21 @@ setup_logging(name="run", queue=True)
|
|||||||
duplicate lines) and leaves handlers your app added itself alone.
|
duplicate lines) and leaves handlers your app added itself alone.
|
||||||
- **Never crashes the app over logging:** if `log_dir` isn't writable, it falls back to
|
- **Never crashes the app over logging:** if `log_dir` isn't writable, it falls back to
|
||||||
console-only with a warning instead of raising.
|
console-only with a warning instead of raising.
|
||||||
|
- **`rotate="size"` always bounds the live file (v0.5.1+).** Previously, `backup_count=0`
|
||||||
|
with `rotate="size"` silently disabled rotation entirely (the live file grew forever,
|
||||||
|
ignoring `max_bytes`). As of v0.5.1, the live file always rolls at `max_bytes`
|
||||||
|
regardless of `backup_count`; `backup_count=0` means "keep zero rolled files" (each roll
|
||||||
|
is deleted right after it lands) rather than "never roll." `backup_count>=1` behaves as
|
||||||
|
documented (keeps that many rolled files). This does not change `"daily"`/`"on_start"`,
|
||||||
|
where `backup_count=0` still means "roll, but don't prune the rolled files" (unbounded
|
||||||
|
`log_dir` growth) — that is a separate, pre-existing knob, not this fix's scope.
|
||||||
|
- **Gzip writes are crash-safe (v0.5.1+).** `_gzip_file` now writes to a `.tmp` sibling and
|
||||||
|
atomically `os.replace`s it onto the final `.gz` path, so a crash/OOM/power-loss mid-write
|
||||||
|
can never leave a truncated `.gz` at the path retention logic trusts. Tiered retention's
|
||||||
|
plain/gz dedupe additionally verifies a `.gz` decompresses cleanly before deleting its
|
||||||
|
plain twin — a corrupt `.gz` (from before this fix, or an external cause) is never
|
||||||
|
preferred over an intact plain copy; the plain is kept and the `.gz` gets rewritten
|
||||||
|
cleanly on the next retier pass instead of being deleted.
|
||||||
|
|
||||||
## Scope — what this is NOT
|
## Scope — what this is NOT
|
||||||
|
|
||||||
@@ -97,9 +257,10 @@ handlers. Getting files to a backend is a separate concern (e.g. Promtail tails
|
|||||||
backend can change without touching any app, and the consistent format here is what
|
backend can change without touching any app, and the consistent format here is what
|
||||||
makes downstream parsing and alerting easy.
|
makes downstream parsing and alerting easy.
|
||||||
|
|
||||||
Also out of v0.1.0 (possible later additions): structured/JSON logging, color
|
Structured/JSON output is **in** as of v0.2.0 (`output="json"`) — text and json only.
|
||||||
formatting, per-logger filters, remote handlers.
|
Still deliberately out: logfmt or other formats, a format DSL, per-handler formats,
|
||||||
|
color formatting, per-logger filters, remote handlers.
|
||||||
|
|
||||||
## Versioning
|
## Versioning
|
||||||
|
|
||||||
Tagged `vX.Y.Z`. Pin the tag.
|
Releases are tagged `vX.Y.Z`. The install line above pins a release; drop the `@vX.Y.Z` suffix to install the latest unpinned. Pin deliberately for reproducible installs.
|
||||||
|
|||||||
+1
-1
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
|
|||||||
|
|
||||||
[project]
|
[project]
|
||||||
name = "log_setup"
|
name = "log_setup"
|
||||||
version = "0.1.0"
|
version = "0.5.1"
|
||||||
description = "stdlib app-entry-point logging setup: live run.log, rotation, gzip, retention, consistent format"
|
description = "stdlib app-entry-point logging setup: live run.log, rotation, gzip, retention, consistent format"
|
||||||
requires-python = ">=3.10"
|
requires-python = ">=3.10"
|
||||||
dependencies = []
|
dependencies = []
|
||||||
|
|||||||
@@ -19,4 +19,4 @@ from .setup import setup_logging
|
|||||||
|
|
||||||
__all__ = ["setup_logging"]
|
__all__ = ["setup_logging"]
|
||||||
|
|
||||||
__version__ = "0.1.0"
|
__version__ = "0.5.0"
|
||||||
|
|||||||
@@ -1,15 +1,76 @@
|
|||||||
"""default log format + datefmt for the app-wide setup.
|
"""log formats for the app-wide setup: human-readable text + structured JSON lines.
|
||||||
|
|
||||||
one format for v0.1.0, used on both console and file. `%(name)s` is the getLogger
|
two output formats, two proven needs. `text` (default) is the human `tail -f` format
|
||||||
name the emitting module used, so each library/module shows in the line.
|
(`time | module | level | message`, local time). `json` is the Grafana/Loki path —
|
||||||
|
one JSON object per line (JSON Lines), fields parsed into labels natively, UTC
|
||||||
|
timestamps so logs aggregated from many machines/containers sort unambiguously.
|
||||||
|
`%(name)s` is the getLogger name the emitting module used, so each module shows.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
|
import datetime
|
||||||
|
import json
|
||||||
import logging
|
import logging
|
||||||
|
|
||||||
DEFAULT_FORMAT = "%(asctime)s | %(name)s | %(levelname)s | %(message)s"
|
DEFAULT_FORMAT = "%(asctime)s | %(name)s | %(levelname)s | %(message)s"
|
||||||
DEFAULT_DATEFMT = "%Y-%m-%d %H:%M:%S"
|
DEFAULT_DATEFMT = "%Y-%m-%d %H:%M:%S"
|
||||||
|
|
||||||
|
_RESERVED = frozenset(vars(logging.makeLogRecord({})).keys()) | {"message", "asctime"}
|
||||||
|
# this formatter's own canonical output keys — stdlib's LogRecord rejects `extra` keys
|
||||||
|
# colliding with real attribute names (e.g. `module`), but `time`/`level` are NOT
|
||||||
|
# LogRecord attrs, so a caller's extra={"time":...}/{"level":...} would otherwise
|
||||||
|
# overwrite the UTC timestamp / levelname. guard them explicitly
|
||||||
|
_OUTPUT_KEYS = frozenset({"time", "level", "module", "message"})
|
||||||
|
|
||||||
def build_formatter(fmt=None, datefmt=None) -> logging.Formatter:
|
|
||||||
"""build a logging.Formatter from overrides, falling back to the defaults"""
|
class JsonLinesFormatter(logging.Formatter):
|
||||||
|
"""format each record as a single-line JSON object (JSON Lines / .jsonl)
|
||||||
|
|
||||||
|
emits at minimum time/level/module/message. time is UTC ISO-8601 with a `Z`
|
||||||
|
suffix (e.g. 2026-06-28T14:03:11Z) so logs aggregated across machines and
|
||||||
|
containers sort unambiguously — Grafana converts to local for display. any
|
||||||
|
field passed via logging `extra={...}` lands as a top-level JSON field, which
|
||||||
|
is how a caller stamps monitor/service/request-id for Loki labels without the
|
||||||
|
lib knowing those domain concepts. a traceback (exc_info) is rendered into an
|
||||||
|
`exc_info` string field rather than dropped.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def format(self, record: logging.LogRecord) -> str:
|
||||||
|
when = datetime.datetime.fromtimestamp(record.created, datetime.timezone.utc)
|
||||||
|
payload = {
|
||||||
|
"time": when.strftime("%Y-%m-%dT%H:%M:%SZ"),
|
||||||
|
"level": record.levelname,
|
||||||
|
"module": record.name,
|
||||||
|
"message": record.getMessage(),
|
||||||
|
}
|
||||||
|
for key, value in record.__dict__.items():
|
||||||
|
if key not in _RESERVED and key not in _OUTPUT_KEYS and not key.startswith("_"):
|
||||||
|
payload[key] = value
|
||||||
|
if record.exc_info:
|
||||||
|
# cache the rendered traceback on the record (as stdlib Formatter does) so a
|
||||||
|
# second handler/format() of the same record doesn't re-render it
|
||||||
|
if not record.exc_text:
|
||||||
|
record.exc_text = self.formatException(record.exc_info)
|
||||||
|
payload["exc_info"] = record.exc_text
|
||||||
|
elif record.exc_text:
|
||||||
|
payload["exc_info"] = record.exc_text
|
||||||
|
if record.stack_info:
|
||||||
|
payload["stack_info"] = self.formatStack(record.stack_info)
|
||||||
|
return json.dumps(payload, default=str)
|
||||||
|
|
||||||
|
|
||||||
|
def build_formatter(output: str = "text", fmt=None, datefmt=None) -> logging.Formatter:
|
||||||
|
"""build the formatter for the chosen output format
|
||||||
|
|
||||||
|
`output="text"` (default) returns the human-readable text formatter, honoring
|
||||||
|
the raw `fmt`/`datefmt` format-string overrides. `output="json"` returns the
|
||||||
|
structured `JsonLinesFormatter` (which ignores `fmt`/`datefmt` — it builds
|
||||||
|
fields, not a format string). an unrecognized `output` falls back to text and
|
||||||
|
warns, never raising — a bad format arg must not take the app down.
|
||||||
|
"""
|
||||||
|
if output == "json":
|
||||||
|
return JsonLinesFormatter()
|
||||||
|
if output != "text":
|
||||||
|
logging.getLogger(__name__).warning(
|
||||||
|
"log_setup: unknown output %r; falling back to 'text'", output
|
||||||
|
)
|
||||||
return logging.Formatter(fmt or DEFAULT_FORMAT, datefmt or DEFAULT_DATEFMT)
|
return logging.Formatter(fmt or DEFAULT_FORMAT, datefmt or DEFAULT_DATEFMT)
|
||||||
|
|||||||
+349
-25
@@ -4,13 +4,107 @@ the stdlib rotating handlers roll a file next to the live file; these helpers
|
|||||||
override the namer/rotator so rolled files land in `log_dir` and are gzipped when
|
override the namer/rotator so rolled files land in `log_dir` and are gzipped when
|
||||||
asked, keep the live file at its stable path, and handle the on-start and prune
|
asked, keep the live file at its stable path, and handle the on-start and prune
|
||||||
paths the handlers don't manage themselves.
|
paths the handlers don't manage themselves.
|
||||||
|
|
||||||
|
gzip writes are crash-safe: `_gzip_file` writes to a `.tmp` sibling and atomically
|
||||||
|
`os.replace`s it onto the final `.gz` path, so a crash/OOM/power-loss mid-write never
|
||||||
|
leaves a truncated `.gz` where retention logic would trust it. `retier`'s plain/gz
|
||||||
|
dedupe additionally verifies a `.gz` decompresses cleanly (`_gz_intact`) before removing
|
||||||
|
its plain twin, so a corrupt `.gz` (pre-existing or externally caused) is never preferred
|
||||||
|
over an intact plain copy.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
import gzip
|
import gzip
|
||||||
import os
|
import os
|
||||||
import shutil
|
import shutil
|
||||||
import time
|
import time
|
||||||
from typing import Callable, Tuple
|
from typing import Callable, Optional, Tuple
|
||||||
|
|
||||||
|
|
||||||
|
def _move(source: str, dest: str) -> None:
|
||||||
|
"""rename source to dest, falling back to copy+unlink across filesystems
|
||||||
|
|
||||||
|
os.replace is atomic but raises OSError(EXDEV) when source and dest are on
|
||||||
|
different filesystems — exactly the container bind-mount / separate-logs-volume
|
||||||
|
case this lib targets. fall back to shutil.move (copy+unlink) so the roll still
|
||||||
|
lands instead of failing every rotation via the handler's silent handleError.
|
||||||
|
|
||||||
|
precondition: `dest` is a free, non-directory path (all call sites generate a unique
|
||||||
|
timestamped/dated dest). os.replace and shutil.move differ on a dest that already
|
||||||
|
exists as a directory, so this helper is not safe for arbitrary dests — only the
|
||||||
|
rotation paths that guarantee a fresh file dest.
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
os.replace(source, dest)
|
||||||
|
except OSError:
|
||||||
|
shutil.move(source, dest)
|
||||||
|
|
||||||
|
|
||||||
|
def _free_dest(dest: str) -> str:
|
||||||
|
"""return `dest`, or a `.N`-suffixed variant if it (or its .gz twin) already exists
|
||||||
|
|
||||||
|
used by the tiered rotator so a second roll landing on the same dated/stamped name
|
||||||
|
(e.g. two daily rolls in one day) doesn't clobber the earlier file. the suffix goes
|
||||||
|
before nothing here (dest is already the plain path) — checks both the plain and .gz
|
||||||
|
forms of each candidate.
|
||||||
|
"""
|
||||||
|
if not os.path.exists(dest) and not os.path.exists(dest + ".gz"):
|
||||||
|
return dest
|
||||||
|
counter = 1
|
||||||
|
while True:
|
||||||
|
candidate = f"{dest}.{counter}"
|
||||||
|
if not os.path.exists(candidate) and not os.path.exists(candidate + ".gz"):
|
||||||
|
return candidate
|
||||||
|
counter += 1
|
||||||
|
|
||||||
|
|
||||||
|
def _gzip_file(source: str, dest: str) -> None:
|
||||||
|
"""gzip source into dest then remove source (the rolled-file compression idiom)
|
||||||
|
|
||||||
|
writes to `dest + ".tmp"` first and `os.replace`s it onto `dest` once the gzip
|
||||||
|
write is complete, so a crash/OOM/power-loss mid-write never leaves a truncated
|
||||||
|
`.gz` at `dest` — the partial write stays quarantined in the `.tmp` name and the
|
||||||
|
source is untouched (safe to retry). os.replace is atomic on the same filesystem,
|
||||||
|
which the `.tmp` sibling always is.
|
||||||
|
|
||||||
|
the source mtime is carried onto dest so a file keeps its position when it crosses
|
||||||
|
the plain->gz tier boundary — retier ranks by mtime, and a fresh write would
|
||||||
|
otherwise make a just-compressed file look like the newest one and reshuffle tiers.
|
||||||
|
"""
|
||||||
|
mtime = _safe_mtime(source)
|
||||||
|
tmp_dest = dest + ".tmp"
|
||||||
|
try:
|
||||||
|
with open(source, "rb") as src, gzip.open(tmp_dest, "wb") as dst:
|
||||||
|
shutil.copyfileobj(src, dst)
|
||||||
|
except BaseException:
|
||||||
|
try:
|
||||||
|
os.remove(tmp_dest)
|
||||||
|
except OSError:
|
||||||
|
pass
|
||||||
|
raise
|
||||||
|
os.replace(tmp_dest, dest)
|
||||||
|
os.remove(source)
|
||||||
|
try:
|
||||||
|
os.utime(dest, (mtime, mtime))
|
||||||
|
except OSError:
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
def _gz_intact(path: str) -> bool:
|
||||||
|
"""return True if the gzip file at path decompresses cleanly end to end
|
||||||
|
|
||||||
|
belt-and-suspenders check before a dedupe site removes a plain twin in favor of
|
||||||
|
its .gz — a truncated/corrupt .gz (partial write, bad copy, disk error) must never
|
||||||
|
be trusted over an intact plain copy. reads the whole stream since gzip.open only
|
||||||
|
validates the end-of-stream marker on a full read; any failure (BadGzipFile, EOFError,
|
||||||
|
OSError, zlib error) is treated as "not intact" so the caller keeps the plain source.
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
with gzip.open(path, "rb") as handle:
|
||||||
|
while handle.read(1 << 20):
|
||||||
|
pass
|
||||||
|
return True
|
||||||
|
except Exception:
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
def make_namer(log_dir: str, compress: bool) -> Callable[[str], str]:
|
def make_namer(log_dir: str, compress: bool) -> Callable[[str], str]:
|
||||||
@@ -26,48 +120,255 @@ def make_namer(log_dir: str, compress: bool) -> Callable[[str], str]:
|
|||||||
return namer
|
return namer
|
||||||
|
|
||||||
|
|
||||||
def make_rotator(compress: bool) -> Callable[[str, str], None]:
|
def make_history_namer(
|
||||||
"""rotator: move (or gzip) the source live file to the destination rolled path"""
|
stem: str, log_dir: str, compress: bool = False, plain: bool = False,
|
||||||
|
clock=time.localtime,
|
||||||
|
) -> Callable[[str], str]:
|
||||||
|
"""namer minting historic rolled files `<stem>.<Y-m-d_H-M-S>.log[.gz]` in log_dir
|
||||||
|
|
||||||
|
used by size and daily (and their tiered variants). `stem` is the HISTORY stem (the
|
||||||
|
project namespace), independent of the live file's name — the returned rolled files
|
||||||
|
are keyed off it, and prune/retier glob the same stem.
|
||||||
|
|
||||||
|
the stdlib handler's own rolled name (`.N` for size, `.log.<date>` for daily) is
|
||||||
|
ignored — we mint our own uniform timestamped name so all modes converge on one shape
|
||||||
|
and retier can rank/tier them. `plain=True` (tiered mode) always lands `.log` and lets
|
||||||
|
retier decide compression; otherwise `.gz` is appended when `compress`. same-second
|
||||||
|
collisions are disambiguated with a counter, checking both .log and .log.gz forms.
|
||||||
|
"""
|
||||||
|
def namer(default_name: str) -> str:
|
||||||
|
stamp = time.strftime("%Y-%m-%d_%H-%M-%S", clock())
|
||||||
|
base = os.path.join(log_dir, f"{stem}.{stamp}")
|
||||||
|
candidate = base
|
||||||
|
counter = 1
|
||||||
|
while os.path.exists(candidate + ".log") or os.path.exists(candidate + ".log.gz"):
|
||||||
|
candidate = f"{base}.{counter}"
|
||||||
|
counter += 1
|
||||||
|
suffix = ".log.gz" if (compress and not plain) else ".log"
|
||||||
|
return candidate + suffix
|
||||||
|
return namer
|
||||||
|
|
||||||
|
|
||||||
|
def make_rotator(
|
||||||
|
compress: bool, log_dir: Optional[str] = None,
|
||||||
|
prune_stem: Optional[str] = None, backup_count: int = 0,
|
||||||
|
keep_uncompressed: Optional[int] = None, keep_compressed: Optional[int] = None,
|
||||||
|
) -> Callable[[str, str], None]:
|
||||||
|
"""rotator: move (or gzip) the source live file to the destination rolled path
|
||||||
|
|
||||||
|
legacy mode (default): gzip on roll when `compress`, then prune `log_dir` to
|
||||||
|
`backup_count` newest rolled files. the stdlib handler's own retention
|
||||||
|
(`getFilesToDelete`) only scans the live file's directory, so it never sees the
|
||||||
|
rolled files we redirect into `log_dir` — pruning here is what bounds retention for
|
||||||
|
the daily and size rolling modes. `backup_count <= 0` means "keep no rolled history":
|
||||||
|
`prune()` itself no-ops at `backup_count <= 0` (it means "don't touch history"), so a
|
||||||
|
zero-retention roll is deleted by the rotator directly right after landing, rather than
|
||||||
|
relying on prune to do it.
|
||||||
|
|
||||||
|
tiered mode (when `keep_uncompressed`/`keep_compressed` are given): land the rolled
|
||||||
|
file PLAIN and re-tier `log_dir` — newest `keep_uncompressed` stay uncompressed, the
|
||||||
|
next `keep_compressed` are gzipped, the rest deleted. `compress`/`backup_count` are
|
||||||
|
ignored in this mode (the tier counts bound retention instead).
|
||||||
|
"""
|
||||||
|
tiered = keep_uncompressed is not None or keep_compressed is not None
|
||||||
|
|
||||||
def rotator(source: str, dest: str) -> None:
|
def rotator(source: str, dest: str) -> None:
|
||||||
if not os.path.exists(source):
|
if not os.path.exists(source):
|
||||||
return
|
return
|
||||||
|
if tiered:
|
||||||
|
# dest carries the namer's .gz suffix in compress mode; strip it so the
|
||||||
|
# freshly-rolled file lands plain and retier decides its tier. disambiguate a
|
||||||
|
# dest that already exists (a second same-interval daily roll reuses the same
|
||||||
|
# dated name) with a counter, checking both .log and .log.gz forms, so the
|
||||||
|
# earlier roll isn't clobbered.
|
||||||
|
plain_dest = _free_dest(dest[:-3] if dest.endswith(".gz") else dest)
|
||||||
|
_move(source, plain_dest)
|
||||||
|
if log_dir is not None and prune_stem is not None:
|
||||||
|
retier(log_dir, prune_stem, keep_uncompressed or 0, keep_compressed or 0)
|
||||||
|
return
|
||||||
if compress:
|
if compress:
|
||||||
with open(source, "rb") as src, gzip.open(dest, "wb") as dst:
|
_gzip_file(source, dest)
|
||||||
shutil.copyfileobj(src, dst)
|
|
||||||
os.remove(source)
|
|
||||||
else:
|
else:
|
||||||
os.replace(source, dest)
|
_move(source, dest)
|
||||||
|
if backup_count <= 0:
|
||||||
|
# "keep no history": prune() no-ops at backup_count <= 0 (that's its "leave
|
||||||
|
# history alone" sentinel, not "delete everything"), so a zero-retention roll
|
||||||
|
# deletes its own just-landed file directly instead of relying on prune.
|
||||||
|
try:
|
||||||
|
os.remove(dest)
|
||||||
|
except OSError:
|
||||||
|
pass
|
||||||
|
elif log_dir is not None and prune_stem is not None:
|
||||||
|
prune(log_dir, prune_stem, backup_count)
|
||||||
return rotator
|
return rotator
|
||||||
|
|
||||||
|
|
||||||
def rotate_on_start(live_path: str, log_dir: str, compress: bool, clock=time.localtime) -> None:
|
def rotate_on_start(
|
||||||
|
live_path: str, log_dir: str, compress: bool, clock=time.localtime,
|
||||||
|
keep_uncompressed: Optional[int] = None, keep_compressed: Optional[int] = None,
|
||||||
|
history_stem: Optional[str] = None,
|
||||||
|
) -> None:
|
||||||
"""move an existing live file into log_dir with a timestamp, gzipped if asked
|
"""move an existing live file into log_dir with a timestamp, gzipped if asked
|
||||||
|
|
||||||
|
the rolled file is named off `history_stem` (the project namespace) when given, so
|
||||||
|
historic files carry the project name independent of the live file's stem; falls back
|
||||||
|
to the live file's own stem when history_stem is None/empty.
|
||||||
|
|
||||||
no-op if the live file doesn't exist. used by rotate="on_start" before the fresh
|
no-op if the live file doesn't exist. used by rotate="on_start" before the fresh
|
||||||
handler opens a new live file. the timestamp form is run.<%Y-%m-%d_%H-%M-%S>.log.
|
handler opens a new live file. the timestamp form is run.<%Y-%m-%d_%H-%M-%S>.log.
|
||||||
|
|
||||||
|
tiered mode (when `keep_uncompressed`/`keep_compressed` are given): the rolled file
|
||||||
|
always lands PLAIN (so it can occupy the newest uncompressed tier) and `retier`
|
||||||
|
decides compression/deletion across the whole stem — `compress` is ignored for the
|
||||||
|
just-rolled file.
|
||||||
"""
|
"""
|
||||||
if not os.path.exists(live_path):
|
if not os.path.exists(live_path):
|
||||||
return
|
return
|
||||||
stem = os.path.splitext(os.path.basename(live_path))[0]
|
tiered = keep_uncompressed is not None or keep_compressed is not None
|
||||||
|
live_stem = os.path.splitext(os.path.basename(live_path))[0]
|
||||||
|
stem = os.path.basename(history_stem) if history_stem else live_stem
|
||||||
stamp = time.strftime("%Y-%m-%d_%H-%M-%S", clock())
|
stamp = time.strftime("%Y-%m-%d_%H-%M-%S", clock())
|
||||||
dest = os.path.join(log_dir, f"{stem}.{stamp}.log")
|
suffix = ".log.gz" if (compress and not tiered) else ".log"
|
||||||
if compress:
|
# the stamp is 1-second resolution; two starts in the same second would collide
|
||||||
dest += ".gz"
|
# and the second clobber the first. disambiguate with a numeric counter so a rapid
|
||||||
with open(live_path, "rb") as src, gzip.open(dest, "wb") as dst:
|
# crash-restart loop doesn't lose the earlier rolled file. check BOTH the .log and
|
||||||
shutil.copyfileobj(src, dst)
|
# .log.gz forms of each candidate: in tiered mode an earlier same-stamp roll may have
|
||||||
os.remove(live_path)
|
# already been compressed to .log.gz, and reusing its bare stem would create a second
|
||||||
|
# file for the same logical roll and break the tier counts
|
||||||
|
|
||||||
|
def _taken(path: str) -> bool:
|
||||||
|
base = path[:-3] if path.endswith(".gz") else path
|
||||||
|
return os.path.exists(base) or os.path.exists(base + ".gz")
|
||||||
|
|
||||||
|
dest = os.path.join(log_dir, f"{stem}.{stamp}{suffix}")
|
||||||
|
counter = 1
|
||||||
|
while _taken(dest):
|
||||||
|
dest = os.path.join(log_dir, f"{stem}.{stamp}.{counter}{suffix}")
|
||||||
|
counter += 1
|
||||||
|
if compress and not tiered:
|
||||||
|
_gzip_file(live_path, dest)
|
||||||
else:
|
else:
|
||||||
os.replace(live_path, dest)
|
_move(live_path, dest)
|
||||||
|
if tiered:
|
||||||
|
retier(log_dir, stem, keep_uncompressed or 0, keep_compressed or 0)
|
||||||
|
|
||||||
|
|
||||||
|
def retier(log_dir: str, stem: str, keep_uncompressed: int, keep_compressed: int) -> None:
|
||||||
|
"""re-tier rolled files for stem: newest plain, next gzipped, rest deleted
|
||||||
|
|
||||||
|
newest-first by mtime: the first `keep_uncompressed` stay uncompressed, the next
|
||||||
|
`keep_compressed` are gzipped in place (a still-plain file in that band is compressed
|
||||||
|
to <name>.gz and the plain source removed), and everything beyond
|
||||||
|
keep_uncompressed+keep_compressed is deleted. the live <stem>.log is never touched.
|
||||||
|
fail-soft per file (skip on OSError) so retention never crashes setup.
|
||||||
|
|
||||||
|
`stem` is reduced to its basename: rolled files land in log_dir under the basename
|
||||||
|
(the namer/rotate_on_start basename them), so a `name` containing a directory (e.g.
|
||||||
|
"sub/run") must be matched by "run." here or nothing matches and retention silently
|
||||||
|
never fires (unbounded pileup).
|
||||||
|
|
||||||
|
ordering is by mtime, then by the roll counter parsed from the name so a same-second
|
||||||
|
burst (tied mtimes, counter-disambiguated stamps like run.<t>.log / run.<t>.1.log)
|
||||||
|
still tiers newest-first correctly rather than falling back to arbitrary listdir order.
|
||||||
|
"""
|
||||||
|
stem = os.path.basename(stem)
|
||||||
|
try:
|
||||||
|
names = [
|
||||||
|
name for name in os.listdir(log_dir)
|
||||||
|
if name.startswith(f"{stem}.") and name != f"{stem}.log"
|
||||||
|
]
|
||||||
|
except OSError:
|
||||||
|
return
|
||||||
|
entries = [os.path.join(log_dir, name) for name in names]
|
||||||
|
# dedupe plain/gz twins FIRST (a crash between _gzip_file's write and its os.remove can
|
||||||
|
# leave <x>.log beside <x>.log.gz). drop the redundant plain copy up front so the
|
||||||
|
# phantom twin never occupies a retention slot and evicts a distinct older roll — but
|
||||||
|
# ONLY once the .gz is verified to decompress cleanly. _gzip_file now writes atomically
|
||||||
|
# (temp+os.replace) so a fresh truncated twin can't occur, but a pre-existing corrupt
|
||||||
|
# .gz (older data, bad copy, disk error) must never win over an intact plain copy: that
|
||||||
|
# would delete the only good copy and keep garbage, strictly worse than doing nothing.
|
||||||
|
present = set(entries)
|
||||||
|
kept = []
|
||||||
|
for p in entries:
|
||||||
|
if not p.endswith(".gz") and (p + ".gz") in present:
|
||||||
|
if _gz_intact(p + ".gz"):
|
||||||
|
try:
|
||||||
|
os.remove(p)
|
||||||
|
except OSError:
|
||||||
|
kept.append(p) # couldn't remove — keep it in the accounting
|
||||||
|
continue
|
||||||
|
# .gz twin is corrupt/truncated — keep the intact plain, don't touch either
|
||||||
|
# file here (the second dedupe pass below or a future retier will retry the
|
||||||
|
# compress once the plain is re-gzipped cleanly)
|
||||||
|
kept.append(p)
|
||||||
|
files = [(p, _safe_mtime(p), _roll_counter(p)) for p in kept if os.path.isfile(p)]
|
||||||
|
# newest-first: higher mtime first, and within a tied second the higher roll counter
|
||||||
|
# (a later same-second roll) is newer
|
||||||
|
files.sort(key=lambda t: (t[1], t[2]), reverse=True)
|
||||||
|
|
||||||
|
keep = keep_uncompressed + keep_compressed
|
||||||
|
for index, (path, _, _) in enumerate(files):
|
||||||
|
if index >= keep:
|
||||||
|
try:
|
||||||
|
os.remove(path)
|
||||||
|
except OSError:
|
||||||
|
pass
|
||||||
|
elif index >= keep_uncompressed and not path.endswith(".gz"):
|
||||||
|
dest = path + ".gz"
|
||||||
|
if os.path.exists(dest):
|
||||||
|
# a crash/power-loss between _gzip_file's write and its os.remove can leave
|
||||||
|
# a plain source beside a fresh .gz. don't keep both (they'd double-count
|
||||||
|
# toward retention and evict a distinct older roll) — drop the redundant
|
||||||
|
# plain twin, but ONLY once the .gz is verified intact (see _gz_intact):
|
||||||
|
# a corrupt/truncated .gz here must never win over the last good copy.
|
||||||
|
if _gz_intact(dest):
|
||||||
|
try:
|
||||||
|
os.remove(path)
|
||||||
|
except OSError:
|
||||||
|
pass
|
||||||
|
continue
|
||||||
|
# .gz is corrupt — fall through and re-gzip the plain over the bad dest
|
||||||
|
# (_gzip_file writes atomically, so the corrupt dest is only replaced once
|
||||||
|
# a fully valid archive exists)
|
||||||
|
try:
|
||||||
|
_gzip_file(path, dest)
|
||||||
|
except OSError:
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
def _roll_counter(path: str) -> int:
|
||||||
|
"""parse the same-second disambiguation counter out of a rolled filename
|
||||||
|
|
||||||
|
only the on_start / size-namer shape carries a counter: `<stem>.<stamp>[.<counter>].log`
|
||||||
|
(optionally `.gz`), where a colliding same-second roll gets `.1`, `.2`, ... and a higher
|
||||||
|
counter is the later (newer) roll. the first roll of a second has no counter (0).
|
||||||
|
|
||||||
|
daily's dated names (`<stem>.log.<Y-m-d>`) do NOT end in `.log` and are second+-granular
|
||||||
|
(distinct mtimes), so they never need the counter tie-break — return 0 for them rather
|
||||||
|
than misparsing the trailing date component as a counter.
|
||||||
|
"""
|
||||||
|
base = path[:-3] if path.endswith(".gz") else path
|
||||||
|
if not base.endswith(".log"):
|
||||||
|
return 0
|
||||||
|
base = base[:-4]
|
||||||
|
tail = base.rsplit(".", 1)[-1]
|
||||||
|
return int(tail) if tail.isdigit() else 0
|
||||||
|
|
||||||
|
|
||||||
def prune(log_dir: str, stem: str, backup_count: int) -> None:
|
def prune(log_dir: str, stem: str, backup_count: int) -> None:
|
||||||
"""keep only the newest `backup_count` rolled files for a given stem in log_dir
|
"""keep only the newest `backup_count` rolled files for a given stem in log_dir
|
||||||
|
|
||||||
matches files beginning with `<stem>.` (e.g. run.*), sorted by mtime, deleting the
|
matches files beginning with `<stem>.` (e.g. run.*), sorted by mtime (then by the
|
||||||
oldest beyond the count. used for on_start, which the handlers don't auto-prune.
|
roll counter parsed from the name, mirroring retier — see _roll_counter), deleting
|
||||||
|
the oldest beyond the count. used for on_start, which the handlers don't auto-prune.
|
||||||
|
|
||||||
|
`stem` is reduced to its basename so a `name` containing a directory (e.g. "sub/run")
|
||||||
|
still matches the basenamed rolled files in log_dir (else nothing matches and old
|
||||||
|
files pile up forever).
|
||||||
"""
|
"""
|
||||||
if backup_count <= 0:
|
if backup_count <= 0:
|
||||||
return
|
return
|
||||||
|
stem = os.path.basename(stem)
|
||||||
try:
|
try:
|
||||||
entries = [
|
entries = [
|
||||||
os.path.join(log_dir, name)
|
os.path.join(log_dir, name)
|
||||||
@@ -76,9 +377,11 @@ def prune(log_dir: str, stem: str, backup_count: int) -> None:
|
|||||||
]
|
]
|
||||||
except OSError:
|
except OSError:
|
||||||
return
|
return
|
||||||
files = [(p, _safe_mtime(p)) for p in entries if os.path.isfile(p)]
|
files = [(p, _safe_mtime(p), _roll_counter(p)) for p in entries if os.path.isfile(p)]
|
||||||
files.sort(key=lambda pair: pair[1], reverse=True)
|
# newest-first: higher mtime first, and within a tied second the higher roll counter
|
||||||
for path, _ in files[backup_count:]:
|
# (a later same-second roll) is newer — mirrors retier's ordering (line ~289)
|
||||||
|
files.sort(key=lambda t: (t[1], t[2]), reverse=True)
|
||||||
|
for path, _, _ in files[backup_count:]:
|
||||||
try:
|
try:
|
||||||
os.remove(path)
|
os.remove(path)
|
||||||
except OSError:
|
except OSError:
|
||||||
@@ -93,10 +396,31 @@ def _safe_mtime(path: str) -> float:
|
|||||||
return 0.0
|
return 0.0
|
||||||
|
|
||||||
|
|
||||||
def attach_rolling(handler, log_dir: str, compress: bool) -> Tuple[Callable, Callable]:
|
def attach_rolling(
|
||||||
"""wire the custom namer + rotator onto a rotating handler; return them"""
|
handler, log_dir: str, compress: bool,
|
||||||
namer = make_namer(log_dir, compress)
|
prune_stem: Optional[str] = None, backup_count: int = 0,
|
||||||
rotator = make_rotator(compress)
|
keep_uncompressed: Optional[int] = None, keep_compressed: Optional[int] = None,
|
||||||
|
tiered: bool = False,
|
||||||
|
) -> Tuple[Callable, Callable]:
|
||||||
|
"""wire the custom namer + rotator onto a rotating handler; return them
|
||||||
|
|
||||||
|
rolled files are named off `prune_stem` (the HISTORY stem — the project namespace),
|
||||||
|
independent of the live file name, via make_history_namer: `<stem>.<stamp>.log[.gz]`
|
||||||
|
uniform across size and daily. this replaces the stdlib handler's own rolled-name
|
||||||
|
scheme (`.N` for size, `.log.<date>` for daily), which can't inject a project stem and
|
||||||
|
(for size) can't be managed once files are redirected into log_dir.
|
||||||
|
|
||||||
|
pass `prune_stem`/`backup_count` so the rotator prunes `log_dir` after each roll (the
|
||||||
|
handler's own retention can't see the redirected rolled files). pass
|
||||||
|
`keep_uncompressed`/`keep_compressed` for tiered retention (newest plain, next gzipped,
|
||||||
|
rest deleted) — see make_rotator. `tiered=True` lands rolls plain (retier compresses).
|
||||||
|
"""
|
||||||
|
namer = make_history_namer(
|
||||||
|
os.path.basename(prune_stem or ""), log_dir, compress, plain=tiered,
|
||||||
|
)
|
||||||
|
rotator = make_rotator(
|
||||||
|
compress, log_dir, prune_stem, backup_count, keep_uncompressed, keep_compressed,
|
||||||
|
)
|
||||||
handler.namer = namer
|
handler.namer = namer
|
||||||
handler.rotator = rotator
|
handler.rotator = rotator
|
||||||
return namer, rotator
|
return namer, rotator
|
||||||
|
|||||||
+198
-23
@@ -6,14 +6,18 @@ of rolled files, retention, console output, and a consistent format. it is calle
|
|||||||
the APPLICATION, not by reusable libraries (those stay emit-only). it is idempotent
|
the APPLICATION, not by reusable libraries (those stay emit-only). it is idempotent
|
||||||
(no duplicate handlers on repeat calls), never crashes the app over logging, and can
|
(no duplicate handlers on repeat calls), never crashes the app over logging, and can
|
||||||
route through a background queue so an async event loop doesn't block on file I/O.
|
route through a background queue so an async event loop doesn't block on file I/O.
|
||||||
|
|
||||||
|
`rotate="size"` always bounds the live file: the roll fires at `max_bytes` regardless
|
||||||
|
of `backup_count`, including `backup_count=0` (which means "keep zero rolled files",
|
||||||
|
not "never roll" — each roll is deleted right after landing).
|
||||||
"""
|
"""
|
||||||
|
|
||||||
import atexit
|
import atexit
|
||||||
import logging
|
import logging
|
||||||
import logging.handlers
|
import logging.handlers
|
||||||
import os
|
import os
|
||||||
import queue
|
import queue as _queue
|
||||||
from typing import Optional, Union
|
from typing import Dict, Optional, Union
|
||||||
|
|
||||||
from .formats import build_formatter
|
from .formats import build_formatter
|
||||||
from .rotation import attach_rolling, prune, rotate_on_start
|
from .rotation import attach_rolling, prune, rotate_on_start
|
||||||
@@ -22,13 +26,58 @@ log = logging.getLogger(__name__)
|
|||||||
|
|
||||||
_MARKER = "_log_setup_owned"
|
_MARKER = "_log_setup_owned"
|
||||||
_listener = None
|
_listener = None
|
||||||
|
_atexit_registered = False
|
||||||
|
|
||||||
|
|
||||||
def _level_value(level: Union[int, str]) -> int:
|
def _level_value(level: Union[int, str]) -> int:
|
||||||
"""coerce a level name or int to a logging level int (defaults to INFO)"""
|
"""coerce a level name or int to a logging level int (defaults to INFO)"""
|
||||||
|
if isinstance(level, bool):
|
||||||
|
# bool is an int subclass (True==1, below DEBUG) but is never a real level —
|
||||||
|
# reject it consistently with the per-module path rather than set level 1
|
||||||
|
return logging.INFO
|
||||||
if isinstance(level, int):
|
if isinstance(level, int):
|
||||||
return level
|
return level
|
||||||
return logging.getLevelName(str(level).upper()) if isinstance(level, str) else logging.INFO
|
if not isinstance(level, str):
|
||||||
|
return logging.INFO
|
||||||
|
resolved = logging.getLevelName(level.upper())
|
||||||
|
# getLevelName returns the string "Level XXX" for an unknown name, which
|
||||||
|
# setLevel then rejects — never crash the app over a bad level, fall back to INFO
|
||||||
|
return resolved if isinstance(resolved, int) else logging.INFO
|
||||||
|
|
||||||
|
|
||||||
|
def _strict_level_value(level: Union[int, str]) -> Optional[int]:
|
||||||
|
"""coerce a level name or int to a logging level int, or None if invalid
|
||||||
|
|
||||||
|
unlike `_level_value` (which falls back to INFO for the root `level`), this reports
|
||||||
|
an invalid value as None so the per-module path can skip + warn rather than silently
|
||||||
|
apply INFO to a logger the caller named with a typo'd level
|
||||||
|
"""
|
||||||
|
if isinstance(level, bool):
|
||||||
|
return None
|
||||||
|
if isinstance(level, int):
|
||||||
|
return level
|
||||||
|
if not isinstance(level, str):
|
||||||
|
return None
|
||||||
|
resolved = logging.getLevelName(level.upper())
|
||||||
|
return resolved if isinstance(resolved, int) else None
|
||||||
|
|
||||||
|
|
||||||
|
def _apply_module_levels(module_levels: Optional[Dict[str, Union[int, str]]]) -> None:
|
||||||
|
"""set per-logger level overrides by exact logger name, never crashing
|
||||||
|
|
||||||
|
each name->level entry calls `logging.getLogger(name).setLevel(<level>)`. names are
|
||||||
|
matched exactly (no discovery); stdlib hierarchy still applies, so a parent name
|
||||||
|
quiets its whole subtree. a bad level for one entry is skipped with a warning so the
|
||||||
|
other entries and the rest of setup still proceed.
|
||||||
|
"""
|
||||||
|
if not module_levels:
|
||||||
|
return
|
||||||
|
for mod_name, raw_level in module_levels.items():
|
||||||
|
value = _strict_level_value(raw_level)
|
||||||
|
if value is None:
|
||||||
|
log.warning("log_setup: invalid level %r for logger %r; skipping", raw_level, mod_name)
|
||||||
|
continue
|
||||||
|
logging.getLogger(mod_name).setLevel(value)
|
||||||
|
|
||||||
|
|
||||||
def _clear_owned(root: logging.Logger) -> None:
|
def _clear_owned(root: logging.Logger) -> None:
|
||||||
@@ -36,6 +85,14 @@ def _clear_owned(root: logging.Logger) -> None:
|
|||||||
global _listener
|
global _listener
|
||||||
if _listener is not None:
|
if _listener is not None:
|
||||||
_listener.stop()
|
_listener.stop()
|
||||||
|
# the listener owns the real file/console handlers (only the QueueHandler is
|
||||||
|
# root-attached + marked); stopping it doesn't close them, so close them here
|
||||||
|
# to avoid relying on GC finalizers across a re-setup
|
||||||
|
for wrapped in getattr(_listener, "handlers", ()):
|
||||||
|
try:
|
||||||
|
wrapped.close()
|
||||||
|
except Exception:
|
||||||
|
log.warning("log_setup: failed to close queued handler %r", wrapped, exc_info=True)
|
||||||
_listener = None
|
_listener = None
|
||||||
for handler in list(root.handlers):
|
for handler in list(root.handlers):
|
||||||
if getattr(handler, _MARKER, False):
|
if getattr(handler, _MARKER, False):
|
||||||
@@ -43,7 +100,9 @@ def _clear_owned(root: logging.Logger) -> None:
|
|||||||
try:
|
try:
|
||||||
handler.close()
|
handler.close()
|
||||||
except Exception:
|
except Exception:
|
||||||
pass
|
# a handler failing to close must not abort re-setup, but log it
|
||||||
|
# rather than swallow silently (consistent with the lib's warn pattern)
|
||||||
|
log.warning("log_setup: failed to close handler %r during re-setup", handler, exc_info=True)
|
||||||
|
|
||||||
|
|
||||||
def _tag(handler: logging.Handler) -> logging.Handler:
|
def _tag(handler: logging.Handler) -> logging.Handler:
|
||||||
@@ -52,25 +111,90 @@ def _tag(handler: logging.Handler) -> logging.Handler:
|
|||||||
return handler
|
return handler
|
||||||
|
|
||||||
|
|
||||||
|
def _normalize_name(name: str) -> str:
|
||||||
|
"""strip one trailing '.log' (case-insensitive) so the stem is extension-free
|
||||||
|
|
||||||
|
`name` is allowed to be passed with or without the extension — "latest" and
|
||||||
|
"latest.log" both yield stem "latest" (live file latest.log), never latest.log.log.
|
||||||
|
only one level is stripped: "app.log.log" -> "app.log" so a legit ".log" inside a
|
||||||
|
name survives.
|
||||||
|
"""
|
||||||
|
if name.lower().endswith(".log"):
|
||||||
|
return name[:-4]
|
||||||
|
return name
|
||||||
|
|
||||||
|
|
||||||
|
def _history_stem() -> str:
|
||||||
|
"""the project namespace for historic files: the cwd basename
|
||||||
|
|
||||||
|
a service run from bestbuy/ gives historic files bestbuy.<stamp>.log[.gz]. falls back
|
||||||
|
to an empty string only for a degenerate cwd (e.g. "/"), which the caller resolves to
|
||||||
|
the live stem.
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
return os.path.basename(os.getcwd().rstrip(os.sep))
|
||||||
|
except OSError:
|
||||||
|
return ""
|
||||||
|
|
||||||
|
|
||||||
def _file_handler(
|
def _file_handler(
|
||||||
name: str, live_path: str, log_dir: str, rotate: Optional[str],
|
name: str, history_stem: str, live_path: str, log_dir: str, rotate: Optional[str],
|
||||||
backup_count: int, max_bytes: int, compress: bool,
|
backup_count: int, max_bytes: int, compress: bool,
|
||||||
|
keep_uncompressed: Optional[int], keep_compressed: Optional[int],
|
||||||
) -> logging.Handler:
|
) -> logging.Handler:
|
||||||
"""build the configured file handler with custom rolling into log_dir"""
|
"""build the configured file handler with custom rolling into log_dir
|
||||||
|
|
||||||
|
`name` is the LIVE stem (drives live_path); `history_stem` is the PROJECT stem that
|
||||||
|
rolled/historic files are named off + the retention glob keys on. they are decoupled:
|
||||||
|
the live file keeps its defined name, historic files carry the project namespace.
|
||||||
|
"""
|
||||||
|
tiered = keep_uncompressed is not None or keep_compressed is not None
|
||||||
if rotate == "size":
|
if rotate == "size":
|
||||||
|
# stdlib doRollover is a no-op when backupCount == 0, and its numbered .1/.2 shift
|
||||||
|
# can't manage files redirected into log_dir. force a nonzero backupCount so the
|
||||||
|
# roll always fires regardless of mode — tiered AND legacy — and let
|
||||||
|
# attach_rolling's history namer name + retier/prune bound retention (keyed to
|
||||||
|
# history_stem). the ORIGINAL backup_count (which may be 0) still flows into
|
||||||
|
# attach_rolling below: make_rotator treats backup_count <= 0 there as "keep no
|
||||||
|
# rolled history" and deletes each roll right after it lands, rather than passing
|
||||||
|
# 0 to prune() (whose own backup_count <= 0 means "leave history alone", a no-op —
|
||||||
|
# that mismatch is exactly what silently disabled rotation before).
|
||||||
|
size_backup = max(backup_count, 1)
|
||||||
handler = logging.handlers.RotatingFileHandler(
|
handler = logging.handlers.RotatingFileHandler(
|
||||||
live_path, maxBytes=max_bytes, backupCount=backup_count, encoding="utf-8",
|
live_path, maxBytes=max_bytes, backupCount=size_backup, encoding="utf-8",
|
||||||
|
)
|
||||||
|
attach_rolling(
|
||||||
|
handler, log_dir, compress, prune_stem=history_stem, backup_count=backup_count,
|
||||||
|
keep_uncompressed=keep_uncompressed, keep_compressed=keep_compressed,
|
||||||
|
tiered=tiered,
|
||||||
)
|
)
|
||||||
attach_rolling(handler, log_dir, compress)
|
|
||||||
elif rotate == "daily":
|
elif rotate == "daily":
|
||||||
handler = logging.handlers.TimedRotatingFileHandler(
|
handler = logging.handlers.TimedRotatingFileHandler(
|
||||||
live_path, when="midnight", backupCount=backup_count, encoding="utf-8",
|
live_path, when="midnight", backupCount=backup_count, encoding="utf-8",
|
||||||
)
|
)
|
||||||
attach_rolling(handler, log_dir, compress)
|
attach_rolling(
|
||||||
|
handler, log_dir, compress, prune_stem=history_stem, backup_count=backup_count,
|
||||||
|
keep_uncompressed=keep_uncompressed, keep_compressed=keep_compressed,
|
||||||
|
tiered=tiered,
|
||||||
|
)
|
||||||
else:
|
else:
|
||||||
if rotate == "on_start":
|
if rotate == "on_start":
|
||||||
rotate_on_start(live_path, log_dir, compress)
|
if tiered:
|
||||||
prune(log_dir, name, backup_count)
|
rotate_on_start(
|
||||||
|
live_path, log_dir, compress, history_stem=history_stem,
|
||||||
|
keep_uncompressed=keep_uncompressed, keep_compressed=keep_compressed,
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
rotate_on_start(live_path, log_dir, compress, history_stem=history_stem)
|
||||||
|
prune(log_dir, history_stem, backup_count)
|
||||||
|
elif rotate is not None:
|
||||||
|
# a typo'd rotate value (e.g. "hourly") would otherwise silently fall through
|
||||||
|
# to a non-rotating FileHandler and grow forever — warn, matching the
|
||||||
|
# unknown-`output` convention, rather than degrade silently
|
||||||
|
log.warning(
|
||||||
|
"log_setup: unknown rotate %r; expected 'daily'/'size'/'on_start'/None — "
|
||||||
|
"no rotation applied (single growing file)", rotate,
|
||||||
|
)
|
||||||
handler = logging.FileHandler(live_path, encoding="utf-8")
|
handler = logging.FileHandler(live_path, encoding="utf-8")
|
||||||
return handler
|
return handler
|
||||||
|
|
||||||
@@ -79,33 +203,77 @@ def setup_logging(
|
|||||||
name: str = "run",
|
name: str = "run",
|
||||||
log_dir: str = "logs",
|
log_dir: str = "logs",
|
||||||
level: Union[int, str] = "INFO",
|
level: Union[int, str] = "INFO",
|
||||||
|
module_levels: Optional[Dict[str, Union[int, str]]] = None,
|
||||||
rotate: Optional[str] = "daily",
|
rotate: Optional[str] = "daily",
|
||||||
|
history_name: Optional[str] = None,
|
||||||
backup_count: int = 14,
|
backup_count: int = 14,
|
||||||
|
keep_uncompressed: Optional[int] = None,
|
||||||
|
keep_compressed: Optional[int] = None,
|
||||||
max_bytes: int = 10_000_000,
|
max_bytes: int = 10_000_000,
|
||||||
compress: bool = True,
|
compress: bool = True,
|
||||||
console: bool = False,
|
console: bool = False,
|
||||||
queue: bool = False,
|
queue: bool = False,
|
||||||
|
output: str = "text",
|
||||||
fmt: Optional[str] = None,
|
fmt: Optional[str] = None,
|
||||||
datefmt: Optional[str] = None,
|
datefmt: Optional[str] = None,
|
||||||
) -> logging.Logger:
|
) -> logging.Logger:
|
||||||
"""configure the root logger for the whole process and return it
|
"""configure the root logger for the whole process and return it
|
||||||
|
|
||||||
`name` -> <name>.log live file at cwd; rolled/compressed copies go to `log_dir`.
|
`name` -> <name>.log live file at cwd; rolled/compressed copies go to `log_dir`. a
|
||||||
`rotate` is "daily" (default), "size", "on_start", or None. `console=True` adds a
|
trailing ".log" in `name` is stripped so "latest" and "latest.log" both produce the
|
||||||
|
live file latest.log (never latest.log.log).
|
||||||
|
`history_name` names the rolled/historic files (`<history_name>.<timestamp>.log[.gz]`),
|
||||||
|
independent of the live file: it defaults to the PROJECT namespace = the cwd basename
|
||||||
|
(run from bestbuy/ -> historic files bestbuy.<stamp>...), and can be set explicitly. the
|
||||||
|
live file always keeps `name`; only the historic files carry the project name.
|
||||||
|
`keep_uncompressed`/`keep_compressed` (default None) enable TIERED retention: when
|
||||||
|
either is given, rolled files are kept as the newest `keep_uncompressed` uncompressed
|
||||||
|
+ the next `keep_compressed` gzipped, and the rest are deleted (total retained =
|
||||||
|
sum). this applies to "on_start", "daily", and "size". `backup_count` and the
|
||||||
|
gzip-on-roll behavior of `compress` are IGNORED in tiered mode (the tier counts bound
|
||||||
|
retention). pass NEITHER knob and rotation behaves exactly as before (backup_count +
|
||||||
|
compress) — existing callers are unaffected.
|
||||||
|
`level` is the root default every logger inherits. `module_levels` is an optional
|
||||||
|
map of exact logger name -> level applied after the root is set, the ergonomic way
|
||||||
|
to quiet noisy dependencies (e.g. {"motor": "WARNING", "aiohttp": "WARNING"}) from
|
||||||
|
the one setup call instead of scattering `getLogger(...).setLevel(...)` afterwards —
|
||||||
|
it's stdlib hierarchy under the hood, not new capability. names match EXACTLY (no
|
||||||
|
discovery: a typo'd name silently configures an unused logger), but stdlib hierarchy
|
||||||
|
applies, so naming a parent ("aiohttp") quiets its whole subtree (aiohttp.client,
|
||||||
|
aiohttp.access, ...). each entry accepts a str or int level; a bad value for one
|
||||||
|
entry is skipped with a warning and never aborts the others or the setup.
|
||||||
|
`rotate` is "daily" (default), "size", "on_start", or None. for `rotate="size"`, the
|
||||||
|
live file always rolls at `max_bytes` regardless of `backup_count`: `backup_count=0`
|
||||||
|
means "keep zero rolled files" (each roll lands then is deleted immediately), NOT
|
||||||
|
"disable rotation" — the live file is always bounded. `backup_count>=1` keeps that
|
||||||
|
many rolled files as before. `console=True` adds a
|
||||||
stdout handler (off by default — the file is the output). `queue=True` routes records
|
stdout handler (off by default — the file is the output). `queue=True` routes records
|
||||||
through a background QueueListener so file I/O never blocks the caller (the listener
|
through a background QueueListener so file I/O never blocks the caller (the listener
|
||||||
is stopped at exit). idempotent: a repeat call clears only the handlers this function
|
is stopped at exit). `output` is "text" (default, human `time | module | level |
|
||||||
added. never raises over logging — an unwritable `log_dir` falls back to console-only
|
message`, local time) or "json" (structured one-JSON-object-per-line for the
|
||||||
with a warning even when `console` is off, so output is never silently lost.
|
Grafana/Loki path, UTC timestamps, `extra=` fields surfaced as top-level keys); both
|
||||||
|
file and console use the chosen format and the live-file name is the same regardless.
|
||||||
|
the raw `fmt`/`datefmt` overrides apply to text output only. idempotent: a repeat call
|
||||||
|
clears only the handlers this function added. never raises over logging — an
|
||||||
|
unwritable `log_dir` falls back to console-only with a warning even when `console` is
|
||||||
|
off, so output is never silently lost; an unknown `output` falls back to text.
|
||||||
"""
|
"""
|
||||||
global _listener
|
global _listener, _atexit_registered
|
||||||
|
|
||||||
root = logging.getLogger()
|
root = logging.getLogger()
|
||||||
root.setLevel(_level_value(level))
|
root.setLevel(_level_value(level))
|
||||||
|
_apply_module_levels(module_levels)
|
||||||
_clear_owned(root)
|
_clear_owned(root)
|
||||||
|
|
||||||
formatter = build_formatter(fmt, datefmt)
|
formatter = build_formatter(output, fmt, datefmt)
|
||||||
live_path = f"{name}.log"
|
stem = _normalize_name(name)
|
||||||
|
live_path = f"{stem}.log"
|
||||||
|
# historic/rolled files are named off the project namespace, independent of the live
|
||||||
|
# file: history_name if given, else the cwd basename (e.g. run from bestbuy/ -> historic
|
||||||
|
# files bestbuy.<stamp>.log[.gz]). normalized + basenamed like `name`; falls back to the
|
||||||
|
# live stem for a degenerate cwd so naming/retention never break.
|
||||||
|
history_source = history_name if history_name is not None else _history_stem()
|
||||||
|
history_stem = os.path.basename(_normalize_name(history_source)) or stem
|
||||||
|
|
||||||
handlers = []
|
handlers = []
|
||||||
|
|
||||||
@@ -117,7 +285,10 @@ def setup_logging(
|
|||||||
|
|
||||||
if file_ok:
|
if file_ok:
|
||||||
try:
|
try:
|
||||||
fh = _file_handler(name, live_path, log_dir, rotate, backup_count, max_bytes, compress)
|
fh = _file_handler(
|
||||||
|
stem, history_stem, live_path, log_dir, rotate, backup_count, max_bytes, compress,
|
||||||
|
keep_uncompressed, keep_compressed,
|
||||||
|
)
|
||||||
fh.setFormatter(formatter)
|
fh.setFormatter(formatter)
|
||||||
handlers.append(fh)
|
handlers.append(fh)
|
||||||
except OSError:
|
except OSError:
|
||||||
@@ -129,12 +300,16 @@ def setup_logging(
|
|||||||
handlers.append(sh)
|
handlers.append(sh)
|
||||||
|
|
||||||
if queue:
|
if queue:
|
||||||
record_queue: "queue.Queue" = _make_queue()
|
record_queue: "_queue.Queue" = _make_queue()
|
||||||
qh = _tag(logging.handlers.QueueHandler(record_queue))
|
qh = _tag(logging.handlers.QueueHandler(record_queue))
|
||||||
root.addHandler(qh)
|
root.addHandler(qh)
|
||||||
_listener = logging.handlers.QueueListener(record_queue, *handlers, respect_handler_level=True)
|
_listener = logging.handlers.QueueListener(record_queue, *handlers, respect_handler_level=True)
|
||||||
_listener.start()
|
_listener.start()
|
||||||
|
if not _atexit_registered:
|
||||||
|
# register once — atexit doesn't dedupe, so repeated queue re-setups would
|
||||||
|
# otherwise stack identical callbacks (harmless but unbounded)
|
||||||
atexit.register(_stop_listener)
|
atexit.register(_stop_listener)
|
||||||
|
_atexit_registered = True
|
||||||
else:
|
else:
|
||||||
for handler in handlers:
|
for handler in handlers:
|
||||||
root.addHandler(_tag(handler))
|
root.addHandler(_tag(handler))
|
||||||
@@ -145,9 +320,9 @@ def setup_logging(
|
|||||||
return root
|
return root
|
||||||
|
|
||||||
|
|
||||||
def _make_queue() -> "queue.Queue":
|
def _make_queue() -> "_queue.Queue":
|
||||||
"""unbounded in-memory queue for the QueueHandler -> QueueListener path"""
|
"""unbounded in-memory queue for the QueueHandler -> QueueListener path"""
|
||||||
return queue.Queue(-1)
|
return _queue.Queue(-1)
|
||||||
|
|
||||||
|
|
||||||
def _stop_listener() -> None:
|
def _stop_listener() -> None:
|
||||||
|
|||||||
Reference in New Issue
Block a user