4 Commits
Author SHA1 Message Date
dsql 1207c53742 feat: historic logs named off the project namespace (cwd basename / history_name)
the live file keeps its defined name (latest.log); rolled/historic files are now named off
the PROJECT namespace so you can tell which service a log came from at a glance. default =
os.path.basename(os.getcwd()) (run from bestbuy/ -> historic bestbuy.<stamp>.log[.gz]);
override with the new history_name= param.

- decouple the rolled stem (history) from the live stem (name) in setup.py; thread
  history_stem into BOTH the namers AND prune/retier (same stem, or retention breaks).
- unify size + daily on make_history_namer -> <stem>.<stamp>.log[.gz] (was stdlib .N /
  .log.<date>); rotate_on_start takes history_stem.

behavior-visible: rolled names differ from pre-0.5.0 (were the live stem). pass
history_name=name for the old naming. execute-verified across on_start/daily/size (tiered +
non-tiered), default-cwd + explicit + history==name + normalization edges; v0.4.x fixes
(rotation/tiering/retention) all still hold. bump v0.4.3 -> v0.5.0

Signed-off-by: disqualifier <dev@disqualifier.me>
2026-07-01 01:58:10 -04:00
dsql b52c1d37fa fix: LOW/nit clearance — retier twin dedupe + same-second counter ordering, warn on unknown rotate
- logsetup-2: retier drops a plain file whose .gz twin exists (crash-between-write-and-remove
  residue) up front, so the phantom twin never occupies a retention slot / evicts a real roll.
- logsetup-5: retier breaks mtime ties by the roll counter parsed from the name, so a
  same-second on_start burst tiers newest-first instead of arbitrary listdir order.
- logsetup-3: tiered daily rotator disambiguates an existing dated dest with a counter (no
  clobber on a second same-interval roll).
- logsetup-6: unknown rotate value warns (matching the unknown-output convention) instead of
  silently degrading to a non-rotating file.
- README: size-mode backup naming clarified (numbered flat vs timestamped tiered).
verified stdlib-only; v0.4.0/v0.4.1/size suites still pass. bump v0.4.2 -> v0.4.3

Signed-off-by: disqualifier <dev@disqualifier.me>
2026-07-01 00:28:18 -04:00
dsql 6a10f3acc0 fix: tiered rotate='size' actually rotates and tiers (logsetup-1, logsetup-4)
tiered size mode was broken two ways: (1) stdlib RotatingFileHandler's numbered .1/.2 shift
can't manage files redirected into log_dir, so every roll overwrote slot 1 -> ~99% of
history silently lost; (2) doRollover is gated on backupCount>0, so backup_count=0 (which
the docstring says is ignored in tiered mode) meant NO rotation + unbounded live file.

fix: tiered size now uses a timestamped per-roll namer (make_size_namer) like daily/on_start
so retier manages the pile, and forces a nonzero internal backupCount so the roll always
fires (retier bounds retention, not backupCount). non-tiered size unchanged.

verified: tiered size rotates + bounds to tier total (was stuck at 1); backup_count=0 rotates
+ live file bounded (was unbounded); legacy size back-compat intact; v0.4.0/v0.4.1 suites
still pass. bump v0.4.1 -> v0.4.2

Signed-off-by: disqualifier <dev@disqualifier.me>
2026-06-30 21:04:58 -04:00
dsql ece9a6b9ca fix: retention dead when name contains a directory (basename the stem)
rolled files land in log_dir under their basename (the namer/rotate_on_start basename
them), but prune()/retier() globbed the un-basenamed name. a name like 'sub/run' matched
nothing, so old .log/.gz files piled up forever (slow disk leak; the live file was fine).
basename the stem at the top of both prune() and retier(). regression-verified with
name='sub/run' (10 retained vs 12/dead before). bump v0.4.0 -> v0.4.1

Signed-off-by: disqualifier <dev@disqualifier.me>
2026-06-30 03:49:35 -04:00
5 changed files with 220 additions and 30 deletions
+32 -8
View File
@@ -13,12 +13,12 @@ and emit; their records flow into the handlers `log_setup` wired.
## Install ## Install
``` ```
log_setup @ git+ssh://git@git.rethinkstudios.io/rethink-public/log_setup.git@v0.4.0 log_setup @ git+ssh://git@git.rethinkstudios.io/rethink-public/log_setup.git@v0.5.0
``` ```
No dependencies — stdlib only. No dependencies — stdlib only.
Drop the `@v0.4.0` suffix from the line above to install the latest unpinned. Drop the `@v0.5.0` suffix from the line above to install the latest unpinned.
## Quick start ## Quick start
@@ -43,13 +43,14 @@ emits; the records land in the configured root.
- **Format:** `2026-06-27 19:55:05 | module.name | INFO | message`. `%(name)s` is the - **Format:** `2026-06-27 19:55:05 | module.name | INFO | message`. `%(name)s` is the
`getLogger` name each module used, so you see which lib/module logged. `getLogger` name each module used, so you see which lib/module logged.
- **Rotation** (`rotate=`): - **Rotation** (`rotate=`):
- `"daily"` (default) — rolls at midnight, dated name into `log_dir`, keeps - `"daily"` (default) — rolls at midnight into `log_dir`, keeps `backup_count` days.
`backup_count` days. - `"size"` — rolls at `max_bytes` into `log_dir`, keeps `backup_count`.
- `"size"` — rolls at `max_bytes`, numbered backups in `log_dir`. - `"on_start"` — on startup, moves an existing live file into `log_dir` and starts fresh;
- `"on_start"` — on startup, moves an existing `run.log` into `log_dir` prunes to `backup_count`.
(`run.<timestamp>.log[.gz]`) and starts fresh; prunes to `backup_count`.
- `None` — single file, no rotation. - `None` — single file, no rotation.
- **compress=True** (default) gzips each rolled file (`run.log.2026-06-27.gz`). - **Historic files are named off the project** — see below. Every rolled file is
`<project>.<timestamp>.log[.gz]`; the live file keeps its own name.
- **compress=True** (default) gzips each rolled file.
- **Retention** = `backup_count` (default 14) for every mode — unless tiered retention is - **Retention** = `backup_count` (default 14) for every mode — unless tiered retention is
enabled (below). enabled (below).
- **console=True** (off by default) also logs to stdout in the same format — opt in when - **console=True** (off by default) also logs to stdout in the same format — opt in when
@@ -58,6 +59,28 @@ emits; the records land in the configured root.
The `name` you pass is normalized so it produces exactly one `.log`: `name="latest"` and The `name` you pass is normalized so it produces exactly one `.log`: `name="latest"` and
`name="latest.log"` both yield the live file `latest.log` (never `latest.log.log`). `name="latest.log"` both yield the live file `latest.log` (never `latest.log.log`).
## Historic files are named off the project (`history_name`)
The **live** file keeps its defined `name` (`latest.log`). The **historic** (rolled/gz)
files are named off the **project namespace** — by default the current directory's basename
— so you can tell at a glance which service a log came from:
```python
# app run from bestbuy/run.py , with name="latest":
setup_logging(name="latest", rotate="daily")
# logs/
# latest.log <- live (the tail -f target)
# bestbuy.2026-07-01_02-00-00.log <- historic, named off the project dir
# bestbuy.2026-06-30_02-00-00.log.gz
```
- **Default** = `os.path.basename(os.getcwd())` (the project directory). Zero config.
- Override with **`history_name="foo"`** → historic files become `foo.<timestamp>.log[.gz]`.
- This changed in **v0.5.0**: historic files used to reuse the live `name`. To keep the old
behavior, pass `history_name=name`.
- Retention (tier counts / `backup_count`) is unchanged — it's just keyed to the project
stem now.
## Tiered retention (`keep_uncompressed` / `keep_compressed`) ## Tiered retention (`keep_uncompressed` / `keep_compressed`)
The default is a flat `backup_count`: every rolled file is gzipped on roll and the oldest The default is a flat `backup_count`: every rolled file is gzipped on roll and the oldest
@@ -133,6 +156,7 @@ setup_logging(
level="INFO", # root level everything inherits (str name or logging constant) level="INFO", # root level everything inherits (str name or logging constant)
module_levels=None, # {logger_name: level} per-logger overrides (exact name match) module_levels=None, # {logger_name: level} per-logger overrides (exact name match)
rotate="daily", # "daily" | "size" | "on_start" | None rotate="daily", # "daily" | "size" | "on_start" | None
history_name=None, # stem for rolled/historic files; None -> cwd basename (project)
backup_count=14, # rotated files to keep (flat retention; ignored if tiered) backup_count=14, # rotated files to keep (flat retention; ignored if tiered)
keep_uncompressed=None, # tiered: newest N rolled logs kept PLAIN (opt-in) keep_uncompressed=None, # tiered: newest N rolled logs kept PLAIN (opt-in)
keep_compressed=None, # tiered: next M rolled logs kept GZIPPED (opt-in) keep_compressed=None, # tiered: next M rolled logs kept GZIPPED (opt-in)
+1 -1
View File
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
[project] [project]
name = "log_setup" name = "log_setup"
version = "0.4.0" version = "0.5.0"
description = "stdlib app-entry-point logging setup: live run.log, rotation, gzip, retention, consistent format" description = "stdlib app-entry-point logging setup: live run.log, rotation, gzip, retention, consistent format"
requires-python = ">=3.10" requires-python = ">=3.10"
dependencies = [] dependencies = []
+1 -1
View File
@@ -19,4 +19,4 @@ from .setup import setup_logging
__all__ = ["setup_logging"] __all__ = ["setup_logging"]
__version__ = "0.4.0" __version__ = "0.5.0"
+133 -11
View File
@@ -32,6 +32,24 @@ def _move(source: str, dest: str) -> None:
shutil.move(source, dest) shutil.move(source, dest)
def _free_dest(dest: str) -> str:
"""return `dest`, or a `.N`-suffixed variant if it (or its .gz twin) already exists
used by the tiered rotator so a second roll landing on the same dated/stamped name
(e.g. two daily rolls in one day) doesn't clobber the earlier file. the suffix goes
before nothing here (dest is already the plain path) — checks both the plain and .gz
forms of each candidate.
"""
if not os.path.exists(dest) and not os.path.exists(dest + ".gz"):
return dest
counter = 1
while True:
candidate = f"{dest}.{counter}"
if not os.path.exists(candidate) and not os.path.exists(candidate + ".gz"):
return candidate
counter += 1
def _gzip_file(source: str, dest: str) -> None: def _gzip_file(source: str, dest: str) -> None:
"""gzip source into dest then remove source (the rolled-file compression idiom) """gzip source into dest then remove source (the rolled-file compression idiom)
@@ -62,6 +80,35 @@ def make_namer(log_dir: str, compress: bool) -> Callable[[str], str]:
return namer return namer
def make_history_namer(
stem: str, log_dir: str, compress: bool = False, plain: bool = False,
clock=time.localtime,
) -> Callable[[str], str]:
"""namer minting historic rolled files `<stem>.<Y-m-d_H-M-S>.log[.gz]` in log_dir
used by size and daily (and their tiered variants). `stem` is the HISTORY stem (the
project namespace), independent of the live file's name — the returned rolled files
are keyed off it, and prune/retier glob the same stem.
the stdlib handler's own rolled name (`.N` for size, `.log.<date>` for daily) is
ignored — we mint our own uniform timestamped name so all modes converge on one shape
and retier can rank/tier them. `plain=True` (tiered mode) always lands `.log` and lets
retier decide compression; otherwise `.gz` is appended when `compress`. same-second
collisions are disambiguated with a counter, checking both .log and .log.gz forms.
"""
def namer(default_name: str) -> str:
stamp = time.strftime("%Y-%m-%d_%H-%M-%S", clock())
base = os.path.join(log_dir, f"{stem}.{stamp}")
candidate = base
counter = 1
while os.path.exists(candidate + ".log") or os.path.exists(candidate + ".log.gz"):
candidate = f"{base}.{counter}"
counter += 1
suffix = ".log.gz" if (compress and not plain) else ".log"
return candidate + suffix
return namer
def make_rotator( def make_rotator(
compress: bool, log_dir: Optional[str] = None, compress: bool, log_dir: Optional[str] = None,
prune_stem: Optional[str] = None, backup_count: int = 0, prune_stem: Optional[str] = None, backup_count: int = 0,
@@ -87,8 +134,11 @@ def make_rotator(
return return
if tiered: if tiered:
# dest carries the namer's .gz suffix in compress mode; strip it so the # dest carries the namer's .gz suffix in compress mode; strip it so the
# freshly-rolled file lands plain and retier decides its tier # freshly-rolled file lands plain and retier decides its tier. disambiguate a
plain_dest = dest[:-3] if dest.endswith(".gz") else dest # dest that already exists (a second same-interval daily roll reuses the same
# dated name) with a counter, checking both .log and .log.gz forms, so the
# earlier roll isn't clobbered.
plain_dest = _free_dest(dest[:-3] if dest.endswith(".gz") else dest)
_move(source, plain_dest) _move(source, plain_dest)
if log_dir is not None and prune_stem is not None: if log_dir is not None and prune_stem is not None:
retier(log_dir, prune_stem, keep_uncompressed or 0, keep_compressed or 0) retier(log_dir, prune_stem, keep_uncompressed or 0, keep_compressed or 0)
@@ -105,9 +155,14 @@ def make_rotator(
def rotate_on_start( def rotate_on_start(
live_path: str, log_dir: str, compress: bool, clock=time.localtime, live_path: str, log_dir: str, compress: bool, clock=time.localtime,
keep_uncompressed: Optional[int] = None, keep_compressed: Optional[int] = None, keep_uncompressed: Optional[int] = None, keep_compressed: Optional[int] = None,
history_stem: Optional[str] = None,
) -> None: ) -> None:
"""move an existing live file into log_dir with a timestamp, gzipped if asked """move an existing live file into log_dir with a timestamp, gzipped if asked
the rolled file is named off `history_stem` (the project namespace) when given, so
historic files carry the project name independent of the live file's stem; falls back
to the live file's own stem when history_stem is None/empty.
no-op if the live file doesn't exist. used by rotate="on_start" before the fresh no-op if the live file doesn't exist. used by rotate="on_start" before the fresh
handler opens a new live file. the timestamp form is run.<%Y-%m-%d_%H-%M-%S>.log. handler opens a new live file. the timestamp form is run.<%Y-%m-%d_%H-%M-%S>.log.
@@ -119,7 +174,8 @@ def rotate_on_start(
if not os.path.exists(live_path): if not os.path.exists(live_path):
return return
tiered = keep_uncompressed is not None or keep_compressed is not None tiered = keep_uncompressed is not None or keep_compressed is not None
stem = os.path.splitext(os.path.basename(live_path))[0] live_stem = os.path.splitext(os.path.basename(live_path))[0]
stem = os.path.basename(history_stem) if history_stem else live_stem
stamp = time.strftime("%Y-%m-%d_%H-%M-%S", clock()) stamp = time.strftime("%Y-%m-%d_%H-%M-%S", clock())
suffix = ".log.gz" if (compress and not tiered) else ".log" suffix = ".log.gz" if (compress and not tiered) else ".log"
# the stamp is 1-second resolution; two starts in the same second would collide # the stamp is 1-second resolution; two starts in the same second would collide
@@ -154,7 +210,17 @@ def retier(log_dir: str, stem: str, keep_uncompressed: int, keep_compressed: int
to <name>.gz and the plain source removed), and everything beyond to <name>.gz and the plain source removed), and everything beyond
keep_uncompressed+keep_compressed is deleted. the live <stem>.log is never touched. keep_uncompressed+keep_compressed is deleted. the live <stem>.log is never touched.
fail-soft per file (skip on OSError) so retention never crashes setup. fail-soft per file (skip on OSError) so retention never crashes setup.
`stem` is reduced to its basename: rolled files land in log_dir under the basename
(the namer/rotate_on_start basename them), so a `name` containing a directory (e.g.
"sub/run") must be matched by "run." here or nothing matches and retention silently
never fires (unbounded pileup).
ordering is by mtime, then by the roll counter parsed from the name so a same-second
burst (tied mtimes, counter-disambiguated stamps like run.<t>.log / run.<t>.1.log)
still tiers newest-first correctly rather than falling back to arbitrary listdir order.
""" """
stem = os.path.basename(stem)
try: try:
names = [ names = [
name for name in os.listdir(log_dir) name for name in os.listdir(log_dir)
@@ -163,11 +229,26 @@ def retier(log_dir: str, stem: str, keep_uncompressed: int, keep_compressed: int
except OSError: except OSError:
return return
entries = [os.path.join(log_dir, name) for name in names] entries = [os.path.join(log_dir, name) for name in names]
files = [(p, _safe_mtime(p)) for p in entries if os.path.isfile(p)] # dedupe plain/gz twins FIRST (a crash between _gzip_file's write and its os.remove can
files.sort(key=lambda pair: pair[1], reverse=True) # leave <x>.log beside <x>.log.gz). drop the redundant plain copy up front so the
# phantom twin never occupies a retention slot and evicts a distinct older roll.
present = set(entries)
kept = []
for p in entries:
if not p.endswith(".gz") and (p + ".gz") in present:
try:
os.remove(p)
except OSError:
kept.append(p) # couldn't remove — keep it in the accounting
continue
kept.append(p)
files = [(p, _safe_mtime(p), _roll_counter(p)) for p in kept if os.path.isfile(p)]
# newest-first: higher mtime first, and within a tied second the higher roll counter
# (a later same-second roll) is newer
files.sort(key=lambda t: (t[1], t[2]), reverse=True)
keep = keep_uncompressed + keep_compressed keep = keep_uncompressed + keep_compressed
for index, (path, _) in enumerate(files): for index, (path, _, _) in enumerate(files):
if index >= keep: if index >= keep:
try: try:
os.remove(path) os.remove(path)
@@ -176,6 +257,14 @@ def retier(log_dir: str, stem: str, keep_uncompressed: int, keep_compressed: int
elif index >= keep_uncompressed and not path.endswith(".gz"): elif index >= keep_uncompressed and not path.endswith(".gz"):
dest = path + ".gz" dest = path + ".gz"
if os.path.exists(dest): if os.path.exists(dest):
# a crash/power-loss between _gzip_file's write and its os.remove can leave
# a plain source beside a fresh .gz. don't keep both (they'd double-count
# toward retention and evict a distinct older roll) — drop the redundant
# plain twin, keeping the compressed copy.
try:
os.remove(path)
except OSError:
pass
continue continue
try: try:
_gzip_file(path, dest) _gzip_file(path, dest)
@@ -183,14 +272,38 @@ def retier(log_dir: str, stem: str, keep_uncompressed: int, keep_compressed: int
pass pass
def _roll_counter(path: str) -> int:
"""parse the same-second disambiguation counter out of a rolled filename
only the on_start / size-namer shape carries a counter: `<stem>.<stamp>[.<counter>].log`
(optionally `.gz`), where a colliding same-second roll gets `.1`, `.2`, ... and a higher
counter is the later (newer) roll. the first roll of a second has no counter (0).
daily's dated names (`<stem>.log.<Y-m-d>`) do NOT end in `.log` and are second+-granular
(distinct mtimes), so they never need the counter tie-break — return 0 for them rather
than misparsing the trailing date component as a counter.
"""
base = path[:-3] if path.endswith(".gz") else path
if not base.endswith(".log"):
return 0
base = base[:-4]
tail = base.rsplit(".", 1)[-1]
return int(tail) if tail.isdigit() else 0
def prune(log_dir: str, stem: str, backup_count: int) -> None: def prune(log_dir: str, stem: str, backup_count: int) -> None:
"""keep only the newest `backup_count` rolled files for a given stem in log_dir """keep only the newest `backup_count` rolled files for a given stem in log_dir
matches files beginning with `<stem>.` (e.g. run.*), sorted by mtime, deleting the matches files beginning with `<stem>.` (e.g. run.*), sorted by mtime, deleting the
oldest beyond the count. used for on_start, which the handlers don't auto-prune. oldest beyond the count. used for on_start, which the handlers don't auto-prune.
`stem` is reduced to its basename so a `name` containing a directory (e.g. "sub/run")
still matches the basenamed rolled files in log_dir (else nothing matches and old
files pile up forever).
""" """
if backup_count <= 0: if backup_count <= 0:
return return
stem = os.path.basename(stem)
try: try:
entries = [ entries = [
os.path.join(log_dir, name) os.path.join(log_dir, name)
@@ -220,15 +333,24 @@ def attach_rolling(
handler, log_dir: str, compress: bool, handler, log_dir: str, compress: bool,
prune_stem: Optional[str] = None, backup_count: int = 0, prune_stem: Optional[str] = None, backup_count: int = 0,
keep_uncompressed: Optional[int] = None, keep_compressed: Optional[int] = None, keep_uncompressed: Optional[int] = None, keep_compressed: Optional[int] = None,
tiered: bool = False,
) -> Tuple[Callable, Callable]: ) -> Tuple[Callable, Callable]:
"""wire the custom namer + rotator onto a rotating handler; return them """wire the custom namer + rotator onto a rotating handler; return them
pass `prune_stem`/`backup_count` so the rotator prunes `log_dir` after each roll rolled files are named off `prune_stem` (the HISTORY stem — the project namespace),
(the handler's own retention can't see the redirected rolled files). pass independent of the live file name, via make_history_namer: `<stem>.<stamp>.log[.gz]`
`keep_uncompressed`/`keep_compressed` instead to use tiered retention (newest plain, uniform across size and daily. this replaces the stdlib handler's own rolled-name
next gzipped, rest deleted) — see make_rotator. scheme (`.N` for size, `.log.<date>` for daily), which can't inject a project stem and
(for size) can't be managed once files are redirected into log_dir.
pass `prune_stem`/`backup_count` so the rotator prunes `log_dir` after each roll (the
handler's own retention can't see the redirected rolled files). pass
`keep_uncompressed`/`keep_compressed` for tiered retention (newest plain, next gzipped,
rest deleted) — see make_rotator. `tiered=True` lands rolls plain (retier compresses).
""" """
namer = make_namer(log_dir, compress) namer = make_history_namer(
os.path.basename(prune_stem or ""), log_dir, compress, plain=tiered,
)
rotator = make_rotator( rotator = make_rotator(
compress, log_dir, prune_stem, backup_count, keep_uncompressed, keep_compressed, compress, log_dir, prune_stem, backup_count, keep_uncompressed, keep_compressed,
) )
+53 -9
View File
@@ -120,39 +120,72 @@ def _normalize_name(name: str) -> str:
return name return name
def _history_stem() -> str:
"""the project namespace for historic files: the cwd basename
a service run from bestbuy/ gives historic files bestbuy.<stamp>.log[.gz]. falls back
to an empty string only for a degenerate cwd (e.g. "/"), which the caller resolves to
the live stem.
"""
try:
return os.path.basename(os.getcwd().rstrip(os.sep))
except OSError:
return ""
def _file_handler( def _file_handler(
name: str, live_path: str, log_dir: str, rotate: Optional[str], name: str, history_stem: str, live_path: str, log_dir: str, rotate: Optional[str],
backup_count: int, max_bytes: int, compress: bool, backup_count: int, max_bytes: int, compress: bool,
keep_uncompressed: Optional[int], keep_compressed: Optional[int], keep_uncompressed: Optional[int], keep_compressed: Optional[int],
) -> logging.Handler: ) -> logging.Handler:
"""build the configured file handler with custom rolling into log_dir""" """build the configured file handler with custom rolling into log_dir
`name` is the LIVE stem (drives live_path); `history_stem` is the PROJECT stem that
rolled/historic files are named off + the retention glob keys on. they are decoupled:
the live file keeps its defined name, historic files carry the project namespace.
"""
tiered = keep_uncompressed is not None or keep_compressed is not None tiered = keep_uncompressed is not None or keep_compressed is not None
if rotate == "size": if rotate == "size":
# stdlib doRollover is a no-op when backupCount == 0, and its numbered .1/.2 shift
# can't manage files redirected into log_dir. force a nonzero backupCount so the
# roll always fires, and let attach_rolling's history namer name + retier/prune
# bound retention (keyed to history_stem).
size_backup = backup_count if not tiered else max(backup_count, 1)
handler = logging.handlers.RotatingFileHandler( handler = logging.handlers.RotatingFileHandler(
live_path, maxBytes=max_bytes, backupCount=backup_count, encoding="utf-8", live_path, maxBytes=max_bytes, backupCount=size_backup, encoding="utf-8",
) )
attach_rolling( attach_rolling(
handler, log_dir, compress, prune_stem=name, backup_count=backup_count, handler, log_dir, compress, prune_stem=history_stem, backup_count=backup_count,
keep_uncompressed=keep_uncompressed, keep_compressed=keep_compressed, keep_uncompressed=keep_uncompressed, keep_compressed=keep_compressed,
tiered=tiered,
) )
elif rotate == "daily": elif rotate == "daily":
handler = logging.handlers.TimedRotatingFileHandler( handler = logging.handlers.TimedRotatingFileHandler(
live_path, when="midnight", backupCount=backup_count, encoding="utf-8", live_path, when="midnight", backupCount=backup_count, encoding="utf-8",
) )
attach_rolling( attach_rolling(
handler, log_dir, compress, prune_stem=name, backup_count=backup_count, handler, log_dir, compress, prune_stem=history_stem, backup_count=backup_count,
keep_uncompressed=keep_uncompressed, keep_compressed=keep_compressed, keep_uncompressed=keep_uncompressed, keep_compressed=keep_compressed,
tiered=tiered,
) )
else: else:
if rotate == "on_start": if rotate == "on_start":
if tiered: if tiered:
rotate_on_start( rotate_on_start(
live_path, log_dir, compress, live_path, log_dir, compress, history_stem=history_stem,
keep_uncompressed=keep_uncompressed, keep_compressed=keep_compressed, keep_uncompressed=keep_uncompressed, keep_compressed=keep_compressed,
) )
else: else:
rotate_on_start(live_path, log_dir, compress) rotate_on_start(live_path, log_dir, compress, history_stem=history_stem)
prune(log_dir, name, backup_count) prune(log_dir, history_stem, backup_count)
elif rotate is not None:
# a typo'd rotate value (e.g. "hourly") would otherwise silently fall through
# to a non-rotating FileHandler and grow forever — warn, matching the
# unknown-`output` convention, rather than degrade silently
log.warning(
"log_setup: unknown rotate %r; expected 'daily'/'size'/'on_start'/None — "
"no rotation applied (single growing file)", rotate,
)
handler = logging.FileHandler(live_path, encoding="utf-8") handler = logging.FileHandler(live_path, encoding="utf-8")
return handler return handler
@@ -163,6 +196,7 @@ def setup_logging(
level: Union[int, str] = "INFO", level: Union[int, str] = "INFO",
module_levels: Optional[Dict[str, Union[int, str]]] = None, module_levels: Optional[Dict[str, Union[int, str]]] = None,
rotate: Optional[str] = "daily", rotate: Optional[str] = "daily",
history_name: Optional[str] = None,
backup_count: int = 14, backup_count: int = 14,
keep_uncompressed: Optional[int] = None, keep_uncompressed: Optional[int] = None,
keep_compressed: Optional[int] = None, keep_compressed: Optional[int] = None,
@@ -179,6 +213,10 @@ def setup_logging(
`name` -> <name>.log live file at cwd; rolled/compressed copies go to `log_dir`. a `name` -> <name>.log live file at cwd; rolled/compressed copies go to `log_dir`. a
trailing ".log" in `name` is stripped so "latest" and "latest.log" both produce the trailing ".log" in `name` is stripped so "latest" and "latest.log" both produce the
live file latest.log (never latest.log.log). live file latest.log (never latest.log.log).
`history_name` names the rolled/historic files (`<history_name>.<timestamp>.log[.gz]`),
independent of the live file: it defaults to the PROJECT namespace = the cwd basename
(run from bestbuy/ -> historic files bestbuy.<stamp>...), and can be set explicitly. the
live file always keeps `name`; only the historic files carry the project name.
`keep_uncompressed`/`keep_compressed` (default None) enable TIERED retention: when `keep_uncompressed`/`keep_compressed` (default None) enable TIERED retention: when
either is given, rolled files are kept as the newest `keep_uncompressed` uncompressed either is given, rolled files are kept as the newest `keep_uncompressed` uncompressed
+ the next `keep_compressed` gzipped, and the rest are deleted (total retained = + the next `keep_compressed` gzipped, and the rest are deleted (total retained =
@@ -217,6 +255,12 @@ def setup_logging(
formatter = build_formatter(output, fmt, datefmt) formatter = build_formatter(output, fmt, datefmt)
stem = _normalize_name(name) stem = _normalize_name(name)
live_path = f"{stem}.log" live_path = f"{stem}.log"
# historic/rolled files are named off the project namespace, independent of the live
# file: history_name if given, else the cwd basename (e.g. run from bestbuy/ -> historic
# files bestbuy.<stamp>.log[.gz]). normalized + basenamed like `name`; falls back to the
# live stem for a degenerate cwd so naming/retention never break.
history_source = history_name if history_name is not None else _history_stem()
history_stem = os.path.basename(_normalize_name(history_source)) or stem
handlers = [] handlers = []
@@ -229,7 +273,7 @@ def setup_logging(
if file_ok: if file_ok:
try: try:
fh = _file_handler( fh = _file_handler(
stem, live_path, log_dir, rotate, backup_count, max_bytes, compress, stem, history_stem, live_path, log_dir, rotate, backup_count, max_bytes, compress,
keep_uncompressed, keep_compressed, keep_uncompressed, keep_compressed,
) )
fh.setFormatter(formatter) fh.setFormatter(formatter)