release: 1.0.0

first stable release. pre-1.0.0 verification complete: all surviving MED regressions and
gaps resolved and independently re-fired, tree audited clean across the suite.

Signed-off-by: disqualifier <dev@disqualifier.me>

README.md

@@ -13,12 +13,12 @@ and emit; their records flow into the handlers `log_setup` wired.
 ## Install
 
 ```
-log_setup @ git+ssh://git@git.rethinkstudios.io/rethink-public/log_setup.git@v0.6.0
+log_setup @ git+ssh://git@git.rethinkstudios.io/rethink-public/log_setup.git@v1.0.0
 ```
 
 No dependencies — stdlib only.
 
-Drop the `@v0.6.0` suffix from the line above to install the latest unpinned.
+Drop the `@v1.0.0` suffix from the line above to install the latest unpinned.
 
 ## Quick start
 
@@ -248,13 +248,23 @@ setup_logging(name="run", queue=True)
   documented (keeps that many rolled files). This does not change `"daily"`/`"on_start"`,
   where `backup_count=0` still means "roll, but don't prune the rolled files" (unbounded
   `log_dir` growth) — that is a separate, pre-existing knob, not this fix's scope.
-- **Gzip writes are crash-safe (v0.5.1+).** `_gzip_file` now writes to a `.tmp` sibling and
+- **`console=True` writes to stdout (v0.6.3).** the console handler now binds `sys.stdout`
-  atomically `os.replace`s it onto the final `.gz` path, so a crash/OOM/power-loss mid-write
+  explicitly instead of the stdlib `StreamHandler` default of `sys.stderr`, so a supervisor
-  can never leave a truncated `.gz` at the path retention logic trusts. Tiered retention's
+  capturing stdout sees console log lines. `console` stays off by default — the file is the
-  plain/gz dedupe additionally verifies a `.gz` decompresses cleanly before deleting its
+  primary sink.
-  plain twin — a corrupt `.gz` (from before this fix, or an external cause) is never
+- **`"daily"` regression fixed (v0.6.2).** v0.5.1's `rotate="size"` fix above shared its
-  preferred over an intact plain copy; the plain is kept and the `.gz` gets rewritten
+  rotator with `"daily"`, so a `rotate="daily", backup_count=0` roll was incorrectly
-  cleanly on the next retier pass instead of being deleted.
+  deleted at every midnight rollover instead of just landing unpruned. The rotator is now
+  rotate-mode aware: the zero-retention delete only ever fires for `"size"`, matching the
+  contract in the bullet above — `"daily"`/`"on_start"` with `backup_count=0` were always
+  meant to roll without pruning and now do again.
+- **Gzip is atomic compress-or-skip (v0.5.1+).** `_gzip_file` writes to a `.tmp` sibling,
+  verifies it decompresses cleanly, and only then `os.replace`s it onto the final `.gz` and
+  removes the plain source — a crash/OOM/interrupt at any point leaves the plain `.log`
+  intact with no `.gz` (or the `.tmp` cleaned up), so this code never produces a truncated
+  `.gz` at the path retention trusts. Because a corrupt `.gz` can't arise from this path, the
+  tiered retention dedupe drops a plain twin unconditionally when its `.gz` exists (no
+  runtime `_gz_intact` reconciliation — that was removed once compression became atomic).
 - **Setup-time warnings reach the log file (v0.6.0+).** Previously, a warning raised
   during `setup_logging` itself (an invalid `module_levels` entry, a handler failing to
   close on re-setup, an unknown `rotate` value) was emitted *before* any handler was

pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "log_setup"
-version = "0.6.0"
+version = "1.0.0"
 description = "stdlib app-entry-point logging setup: live run.log, rotation, gzip, retention, consistent format"
 requires-python = ">=3.10"
 dependencies = []

src/log_setup/__init__.py

@@ -1,8 +1,4 @@
-"""log_setup — app-entry-point logging configuration (sync, stdlib only).
+"""log_setup - app-entry-point logging configuration (sync, stdlib only). see README.
-
-call once at an application's entry point to configure the whole process: a live
-run.log, rotation (daily/size/on_start), gzip of rolled files, retention, optional
-console output, and a consistent `time | module | level | message` format.
 
     from log_setup import setup_logging
 
@@ -10,13 +6,16 @@ console output, and a consistent `time | module | level | message` format.
     log = logging.getLogger(__name__)
     log.info("up")                                  # -> run.log (add console=True for stdout too)
 
-reusable libraries do NOT call this — they only `logging.getLogger(__name__)` and
+reusable libraries do NOT call this; the application owns setup, libraries only emit.
-emit; the application owns this setup. shipping logs to a backend is out of scope
-(that's Promtail's job against the produced files).
 """
 
+from importlib.metadata import PackageNotFoundError, version
+
 from .setup import setup_logging
 
 __all__ = ["setup_logging"]
 
-__version__ = "0.6.0"
+try:
+    __version__ = version("log_setup")
+except PackageNotFoundError:
+    __version__ = "0.0.0+unknown"

src/log_setup/formats.py

@@ -1,10 +1,8 @@
 """log formats for the app-wide setup: human-readable text + structured JSON lines.
 
-two output formats, two proven needs. `text` (default) is the human `tail -f` format
+`text` (default) is the human `tail -f` format (`time | module | level | message`,
-(`time | module | level | message`, local time). `json` is the Grafana/Loki path —
+local time). `json` is the Grafana/Loki path - one JSON object per line, UTC
-one JSON object per line (JSON Lines), fields parsed into labels natively, UTC
+timestamps so logs aggregated from many machines sort unambiguously. see README.
-timestamps so logs aggregated from many machines/containers sort unambiguously.
-`%(name)s` is the getLogger name the emitting module used, so each module shows.
 """
 
 import datetime
@@ -15,7 +13,7 @@ DEFAULT_FORMAT = "%(asctime)s | %(name)s | %(levelname)s | %(message)s"
 DEFAULT_DATEFMT = "%Y-%m-%d %H:%M:%S"
 
 _RESERVED = frozenset(vars(logging.makeLogRecord({})).keys()) | {"message", "asctime"}
-# this formatter's own canonical output keys — stdlib's LogRecord rejects `extra` keys
+# this formatter's own canonical output keys - stdlib's LogRecord rejects `extra` keys
 # colliding with real attribute names (e.g. `module`), but `time`/`level`/`ts` are NOT
 # LogRecord attrs, so a caller's extra={"time":...}/{"ts":...} would otherwise overwrite
 # the UTC timestamp / epoch. guard them explicitly
@@ -26,14 +24,10 @@ class JsonLinesFormatter(logging.Formatter):
     """format each record as a single-line JSON object (JSON Lines / .jsonl)
 
     emits at minimum time/ts/level/module/message. `time` is UTC ISO-8601 with a `Z`
-    suffix (e.g. 2026-06-28T14:03:11Z); `ts` (added v0.6.0) is the same instant as a
+    suffix; `ts` is the same instant as a unix epoch int, both sortable across
-    unix epoch int (`int(record.created)`, second resolution) for a consumer that wants
+    machines/containers. any field passed via logging `extra={...}` lands as a
-    a sortable number instead of parsing the ISO string — both sort unambiguously
+    top-level JSON field. a traceback (exc_info) is rendered into an `exc_info`
-    across machines/containers; Grafana converts to local for display. any field
+    string field rather than dropped.
-    passed via logging `extra={...}` lands as a top-level JSON field (how a caller
-    stamps monitor/service/request-id for Loki labels without the lib knowing those
-    domain concepts). a traceback (exc_info) is rendered into an `exc_info` string
-    field rather than dropped.
     """
 
     def format(self, record: logging.LogRecord) -> str:
@@ -49,8 +43,7 @@ class JsonLinesFormatter(logging.Formatter):
             if key not in _RESERVED and key not in _OUTPUT_KEYS and not key.startswith("_"):
                 payload[key] = value
         if record.exc_info:
-            # cache the rendered traceback on the record (as stdlib Formatter does) so a
+            # cache the rendered traceback on the record, like stdlib Formatter does
-            # second handler/format() of the same record doesn't re-render it
             if not record.exc_text:
                 record.exc_text = self.formatException(record.exc_info)
             payload["exc_info"] = record.exc_text
@@ -61,19 +54,27 @@ class JsonLinesFormatter(logging.Formatter):
         return json.dumps(payload, default=str)
 
 
-def build_formatter(output: str = "text", fmt=None, datefmt=None) -> logging.Formatter:
+def build_formatter(output: str = "text", fmt=None, datefmt=None, warnings: list = None) -> logging.Formatter:
     """build the formatter for the chosen output format
 
     `output="text"` (default) returns the human-readable text formatter, honoring
     the raw `fmt`/`datefmt` format-string overrides. `output="json"` returns the
-    structured `JsonLinesFormatter` (which ignores `fmt`/`datefmt` — it builds
+    structured `JsonLinesFormatter` (which ignores `fmt`/`datefmt` - it builds
     fields, not a format string). an unrecognized `output` falls back to text and
-    warns, never raising — a bad format arg must not take the app down.
+    warns, never raising - a bad format arg must not take the app down.
+
+    `warnings` is the setup-time buffer (see `setup_logging`): passing it appends
+    the unknown-output warning there instead of emitting immediately, so it lands
+    in the configured log like the other setup-time warnings rather than only
+    stderr. `None` (default) preserves the old immediate-emit behavior for direct
+    callers outside `setup_logging`.
     """
     if output == "json":
         return JsonLinesFormatter()
     if output != "text":
-        logging.getLogger(__name__).warning(
+        message = "log_setup: unknown output %r; falling back to 'text'"
-            "log_setup: unknown output %r; falling back to 'text'", output
+        if warnings is None:
-        )
+            logging.getLogger(__name__).warning(message, output)
+        else:
+            warnings.append((message, output))
     return logging.Formatter(fmt or DEFAULT_FORMAT, datefmt or DEFAULT_DATEFMT)

src/log_setup/rotation.py

@@ -5,15 +5,16 @@ the namer/rotator so rolled files land in `log_dir` and are gzipped when asked,
 the live file at its stable path, and handle the on-start and prune paths the handlers
 don't manage themselves.
 
-gzip writes are crash-safe: `_gzip_file` writes to a `.tmp` sibling and atomically
+FOOTGUN: compression is atomic - `_gzip_file` writes to a `.tmp` sibling, verifies it
-`os.replace`s it onto the final `.gz` path, so a crash/OOM/power-loss mid-write never
+decompresses, and only then `os.replace`s it onto the final `.gz` and removes the plain.
-leaves a truncated `.gz` where retention would trust it. `retier`'s plain/gz dedupe
+a crash/OOM/power-loss or failed verify at any point leaves the plain `.log` intact and
-additionally verifies a `.gz` decompresses cleanly (`_gz_intact`) before removing its
+no `.gz`, so a corrupt/partial `.gz` is never produced and retention never has to choose
-plain twin, so a corrupt `.gz` is never preferred over an intact plain copy.
+between an intact plain and a corrupt archive.
 """
 
 import gzip
 import os
+import re
 import shutil
 import time
 from typing import Callable, Optional, Tuple
@@ -22,13 +23,10 @@ from typing import Callable, Optional, Tuple
 def _move(source: str, dest: str) -> None:
     """rename source to dest, falling back to copy+unlink across filesystems
 
-    os.replace is atomic but raises OSError(EXDEV) across filesystems — the container
+    FOOTGUN: os.replace is atomic but raises OSError(EXDEV) across filesystems (the
-    bind-mount / separate-logs-volume case this lib targets. falls back to shutil.move
+    container bind-mount / separate-logs-volume case) - falls back to shutil.move so
-    (copy+unlink) so the roll still lands instead of failing rotation silently.
+    the roll still lands instead of failing rotation silently. precondition: `dest` is
-
+    a free, non-directory path.
-    precondition: `dest` is a free, non-directory path (every call site generates a
-    unique timestamped/dated dest) — not safe for arbitrary dests that may already
-    exist as a directory.
     """
     try:
         os.replace(source, dest)
@@ -39,9 +37,8 @@ def _move(source: str, dest: str) -> None:
 def _free_dest(dest: str) -> str:
     """return `dest`, or a `.N`-suffixed variant if it (or its .gz twin) already exists
 
-    used by the tiered rotator so a second roll landing on the same dated/stamped name
+    used by the tiered rotator so a second same-stamp roll doesn't clobber the earlier
-    (two daily rolls in one day) doesn't clobber the earlier file. checks both the plain
+    file; checks both the plain and .gz forms of each candidate.
-    and .gz forms of each candidate.
     """
     if not os.path.exists(dest) and not os.path.exists(dest + ".gz"):
         return dest
@@ -54,22 +51,25 @@ def _free_dest(dest: str) -> str:
 
 
 def _gzip_file(source: str, dest: str) -> None:
-    """gzip source into dest then remove source (the rolled-file compression idiom)
+    """atomically gzip source into dest, then remove source - or leave source untouched
 
-    writes to `dest + ".tmp"` and atomically `os.replace`s it onto `dest` once
+    compression is all-or-nothing: writes to `dest + ".tmp"`, verifies that tmp
-    complete, so a crash/OOM/power-loss mid-write never leaves a truncated `.gz` at
+    decompresses cleanly, and only then `os.replace`s it onto `dest` and removes source.
-    `dest` — the partial write stays quarantined in `.tmp` and source is untouched
+    a crash/OOM/power-loss or a failed verify at ANY point removes the tmp and raises with
-    (safe to retry).
+    source intact and no `.gz` at `dest` - so a corrupt/partial `.gz` is never produced and
+    nothing downstream ever has to choose between an intact plain and a corrupt archive.
 
     the source mtime is carried onto dest so a file keeps its tier position when it
-    crosses the plain->gz boundary — retier ranks by mtime, and a fresh write would
+    crosses the plain->gz boundary - a fresh write would otherwise make a
-    otherwise make a just-compressed file look newest and reshuffle tiers.
+    just-compressed file look newest and reshuffle tiers.
     """
     mtime = _safe_mtime(source)
     tmp_dest = dest + ".tmp"
     try:
         with open(source, "rb") as src, gzip.open(tmp_dest, "wb") as dst:
             shutil.copyfileobj(src, dst)
+        if not _gz_intact(tmp_dest):
+            raise OSError(f"gzip of {source!r} did not verify")
     except BaseException:
         try:
             os.remove(tmp_dest)
@@ -87,10 +87,8 @@ def _gzip_file(source: str, dest: str) -> None:
 def _gz_intact(path: str) -> bool:
     """return True if the gzip file at path decompresses cleanly end to end
 
-    belt-and-suspenders check before a dedupe site removes a plain twin in favor of its
+    used by `_gzip_file` to verify a freshly-written `.gz` before it replaces the plain
-    .gz — a truncated/corrupt .gz must never be trusted over an intact plain copy. reads
+    source; reads the whole stream, any failure means "not intact".
-    the whole stream (gzip.open only validates end-of-stream on a full read); any
-    failure is treated as "not intact" so the caller keeps the plain source.
     """
     try:
         with gzip.open(path, "rb") as handle:
@@ -101,19 +99,6 @@ def _gz_intact(path: str) -> bool:
         return False
 
 
-def make_namer(log_dir: str, compress: bool) -> Callable[[str], str]:
-    """namer: redirect a rolled filename into log_dir, adding .gz when compressing
-
-    keeps the handler's default rolled basename but places it under log_dir, appending
-    .gz so the gzipped name matches.
-    """
-    def namer(default_name: str) -> str:
-        base = os.path.basename(default_name)
-        target = os.path.join(log_dir, base)
-        return target + ".gz" if compress else target
-    return namer
-
-
 def make_history_namer(
     stem: str, log_dir: str, compress: bool = False, plain: bool = False,
     clock=time.localtime,
@@ -121,8 +106,8 @@ def make_history_namer(
     """namer minting historic rolled files `<stem>.<Y-m-d_H-M-S>.log[.gz]` in log_dir
 
     used by size and daily (and their tiered variants). `stem` is the HISTORY stem (the
-    project namespace), independent of the live file's name. FOOTGUN: prune/retier must
+    project namespace), independent of the live file's name. FOOTGUN: prune/retier's
-    glob this same stem or nothing matches and retention silently never fires.
+    `stem` must match this one or nothing matches and retention silently never fires.
 
     ignores the stdlib handler's own rolled name (`.N` for size, `.log.<date>` for
     daily) in favor of a uniform timestamped name so all modes converge on one shape
@@ -147,19 +132,23 @@ def make_rotator(
     compress: bool, log_dir: Optional[str] = None,
     prune_stem: Optional[str] = None, backup_count: int = 0,
     keep_uncompressed: Optional[int] = None, keep_compressed: Optional[int] = None,
+    rotate_mode: Optional[str] = None, live_stem: Optional[str] = None,
 ) -> Callable[[str, str], None]:
     """rotator: move (or gzip) the source live file to the destination rolled path
 
     legacy mode (default): gzip on roll when `compress`, then prune `log_dir` to
-    `backup_count` newest rolled files — the stdlib handler's own retention only scans
+    `backup_count` newest rolled files - the stdlib handler's own retention only scans
     the live file's directory, so it never sees files redirected into `log_dir`; pruning
-    here is what bounds retention for daily/size. FOOTGUN: `backup_count <= 0` means
+    here is what bounds retention for daily/size. FOOTGUN: for `rotate_mode="size"`,
-    "keep no rolled history", but `prune()` itself no-ops at `<= 0` (its own sentinel for
+    `backup_count <= 0` means "keep no rolled history", but `prune()` itself no-ops at
-    "don't touch history") — so a zero-retention roll is deleted by the rotator directly
+    `<= 0` (its own sentinel for "don't touch history") - so a zero-retention roll is
-    right after landing, rather than relying on prune to do it.
+    deleted by the rotator directly right after landing, rather than relying on prune to
+    do it. `rotate_mode` gates this delete-on-land branch to `"size"` only - `"daily"`
+    (and any other non-size mode) with `backup_count <= 0` still rolls without pruning,
+    matching the documented contract that only `size` always bounds the live file.
 
     tiered mode (when `keep_uncompressed`/`keep_compressed` are given): land the rolled
-    file PLAIN and re-tier `log_dir` — newest `keep_uncompressed` stay uncompressed, next
+    file PLAIN and re-tier `log_dir` - newest `keep_uncompressed` stay uncompressed, next
     `keep_compressed` gzipped, rest deleted. `compress`/`backup_count` are ignored.
     """
     tiered = keep_uncompressed is not None or keep_compressed is not None
@@ -168,26 +157,25 @@ def make_rotator(
         if not os.path.exists(source):
             return
         if tiered:
-            # dest carries the namer's .gz suffix in compress mode; strip it so the roll
+            # strip the namer's .gz suffix so the roll lands plain and retier decides
-            # lands plain and retier decides its tier. disambiguate a dest that already
+            # its tier; disambiguate a same-interval collision via _free_dest
-            # exists (a second same-interval daily roll reuses the same dated name) with
-            # a counter, checking both .log and .log.gz forms.
             plain_dest = _free_dest(dest[:-3] if dest.endswith(".gz") else dest)
             _move(source, plain_dest)
             if log_dir is not None and prune_stem is not None:
-                retier(log_dir, prune_stem, keep_uncompressed or 0, keep_compressed or 0)
+                retier(log_dir, prune_stem, keep_uncompressed or 0, keep_compressed or 0, live_stem)
             return
         if compress:
             _gzip_file(source, dest)
         else:
             _move(source, dest)
         if backup_count <= 0:
-            try:
+            if rotate_mode == "size":
-                os.remove(dest)
+                try:
-            except OSError:
+                    os.remove(dest)
-                pass
+                except OSError:
+                    pass
         elif log_dir is not None and prune_stem is not None:
-            prune(log_dir, prune_stem, backup_count)
+            prune(log_dir, prune_stem, backup_count, live_stem)
     return rotator
 
 
@@ -207,7 +195,7 @@ def rotate_on_start(
 
     tiered mode (`keep_uncompressed`/`keep_compressed` given): the rolled file always
     lands PLAIN (so it can occupy the newest uncompressed tier) and `retier` decides
-    compression/deletion across the whole stem — `compress` is ignored here.
+    compression/deletion across the whole stem - `compress` is ignored here.
     """
     if not os.path.exists(live_path):
         return
@@ -217,11 +205,9 @@ def rotate_on_start(
     stamp = time.strftime("%Y-%m-%d_%H-%M-%S", clock())
     suffix = ".log.gz" if (compress and not tiered) else ".log"
 
-    # the 1-second stamp resolution means two starts in the same second collide;
+    # a 1-second stamp collision (rapid crash-restart) is disambiguated with a
-    # disambiguate with a counter so a rapid crash-restart loop doesn't lose the
+    # counter; check BOTH .log and .log.gz since a tiered same-stamp roll may
-    # earlier roll. check BOTH .log and .log.gz forms: in tiered mode an earlier
+    # already be compressed
-    # same-stamp roll may already be compressed, and reusing its bare stem would
-    # create a second file for the same logical roll and break the tier counts
     def _taken(path: str) -> bool:
         base = path[:-3] if path.endswith(".gz") else path
         return os.path.exists(base) or os.path.exists(base + ".gz")
@@ -236,10 +222,11 @@ def rotate_on_start(
     else:
         _move(live_path, dest)
     if tiered:
-        retier(log_dir, stem, keep_uncompressed or 0, keep_compressed or 0)
+        retier(log_dir, stem, keep_uncompressed or 0, keep_compressed or 0, live_stem)
 
 
-def retier(log_dir: str, stem: str, keep_uncompressed: int, keep_compressed: int) -> None:
+def retier(log_dir: str, stem: str, keep_uncompressed: int, keep_compressed: int,
+           live_stem: Optional[str] = None) -> None:
     """re-tier rolled files for stem: newest plain, next gzipped, rest deleted
 
     newest-first by mtime: the first `keep_uncompressed` stay uncompressed, the next
@@ -248,43 +235,45 @@ def retier(log_dir: str, stem: str, keep_uncompressed: int, keep_compressed: int
     fail-soft per file (skip on OSError) so retention never crashes setup.
 
     FOOTGUN: `stem` is reduced to its basename to match how rolled files land in
-    log_dir (namer/rotate_on_start basename them) — a `name` containing a directory
+    log_dir (namer/rotate_on_start basename them) - a `name` containing a directory
     (e.g. "sub/run") must be matched by "run." here or nothing matches and retention
     silently never fires (unbounded pileup).
 
     ordering is by mtime, then by the roll counter parsed from the name, so a
     same-second burst (tied mtimes, counter-disambiguated stamps like run.<t>.log /
     run.<t>.1.log) still tiers newest-first rather than falling back to listdir order.
+
+    candidates are also checked against this lib's own rolled-file shape
+    (`<stem>.<stamp>[.N].log[.gz]`, see `_is_rolled_name`) - a foreign file that only
+    shares the `<stem>.` prefix (e.g. a project's own `<stem>.audit.log` dropped into
+    the same log_dir) is left alone rather than tiered/gzipped/deleted.
     """
     stem = os.path.basename(stem)
+    live_stem = os.path.basename(live_stem) if live_stem else None
     try:
         names = [
             name for name in os.listdir(log_dir)
-            if name.startswith(f"{stem}.") and name != f"{stem}.log"
+            if name != f"{stem}.log" and _is_rolled_name(stem, name, live_stem)
         ]
     except OSError:
         return
     entries = [os.path.join(log_dir, name) for name in names]
-    # dedupe plain/gz twins FIRST (a crash between _gzip_file's write and its os.remove
+    # dedupe plain/gz twins FIRST: a crash between _gzip_file's os.replace and its
-    # can leave <x>.log beside <x>.log.gz) so the phantom twin never occupies a
+    # os.remove(source) can leave <x>.log beside <x>.log.gz. the .gz is guaranteed intact
-    # retention slot and evicts a distinct older roll — but ONLY once the .gz is
+    # (_gzip_file verifies before it replaces, and never produces a partial .gz), so the
-    # verified to decompress cleanly (_gz_intact): a pre-existing corrupt .gz must never
+    # plain twin is redundant and dropped unconditionally.
-    # win over an intact plain copy, which would delete the only good copy.
     present = set(entries)
     kept = []
     for p in entries:
         if not p.endswith(".gz") and (p + ".gz") in present:
-            if _gz_intact(p + ".gz"):
+            try:
-                try:
+                os.remove(p)
-                    os.remove(p)
-                except OSError:
-                    kept.append(p)  # couldn't remove — keep it in the accounting
                 continue
-            # .gz twin is corrupt — keep the intact plain untouched; a later retier
+            except OSError:
-            # retries the compress once it's re-gzipped cleanly
+                pass  # couldn't remove - keep it in the accounting
         kept.append(p)
     files = [(p, _safe_mtime(p), _roll_counter(p)) for p in kept if os.path.isfile(p)]
-    # newest-first: higher mtime first, tied second broken by higher roll counter (later)
+    # newest-first: higher mtime first, ties broken by higher (later) roll counter
     files.sort(key=lambda t: (t[1], t[2]), reverse=True)
 
     keep = keep_uncompressed + keep_compressed
@@ -297,33 +286,52 @@ def retier(log_dir: str, stem: str, keep_uncompressed: int, keep_compressed: int
         elif index >= keep_uncompressed and not path.endswith(".gz"):
             dest = path + ".gz"
             if os.path.exists(dest):
-                # a crash between _gzip_file's write and its os.remove can leave a plain
+                # a verified .gz twin already exists (a prior crashed roll) - drop the
-                # source beside a fresh .gz — drop the redundant plain twin, but ONLY
+                # redundant plain rather than re-gzip over it
-                # once the .gz is verified intact (a corrupt .gz must never win)
+                try:
-                if _gz_intact(dest):
+                    os.remove(path)
-                    try:
+                except OSError:
-                        os.remove(path)
+                    pass
-                    except OSError:
+                continue
-                        pass
-                    continue
-                # .gz is corrupt — fall through and re-gzip the plain over the bad dest
-                # (atomic write replaces it only once a valid archive exists)
             try:
                 _gzip_file(path, dest)
             except OSError:
                 pass
 
 
+def _rolled_name_pattern(stem: str, live_stem: Optional[str] = None) -> "re.Pattern":
+    """compiled regex matching this lib's own rolled-file shapes for stem
+
+    all forms are date-bearing so a foreign same-stem file (e.g. `proj.audit.log`) is
+    never mistaken for a roll and pruned/retiered/deleted:
+    - current uniform namer: `<stem>.<Y-m-d_H-M-S>[.N].log[.gz]`
+      (make_history_namer/rotate_on_start, see attach_rolling)
+    - legacy pre-v0.5.0 daily: `<stem>.log.<Y-m-d>[.gz]` (the stdlib TimedRotatingFileHandler
+      shape) - matched so an upgraded deployment's existing history is still pruned/retired
+      instead of piling up forever. legacy rolls were named off the LIVE file's name, not the
+      history stem, so when `live_stem` is given (and differs) its legacy shape is matched too
+    """
+    stems = [stem] if live_stem is None or live_stem == stem else [stem, live_stem]
+    forms = []
+    for s in stems:
+        esc = re.escape(s)
+        forms.append(rf"{esc}\.\d{{4}}-\d{{2}}-\d{{2}}_\d{{2}}-\d{{2}}-\d{{2}}(?:\.\d+)?\.log(?:\.gz)?")
+        forms.append(rf"{esc}\.log\.\d{{4}}-\d{{2}}-\d{{2}}(?:\.gz)?")
+    return re.compile(rf"^(?:{'|'.join(forms)})$")
+
+
+def _is_rolled_name(stem: str, name: str, live_stem: Optional[str] = None) -> bool:
+    """return whether name has this lib's own rolled-log shape for stem (or the live stem)"""
+    return _rolled_name_pattern(stem, live_stem).match(name) is not None
+
+
 def _roll_counter(path: str) -> int:
     """parse the same-second disambiguation counter out of a rolled filename
 
     only the on_start / size-namer shape carries a counter: `<stem>.<stamp>[.<counter>].log`
-    (optionally `.gz`), where a colliding same-second roll gets `.1`, `.2`, ... and a higher
+    (optionally `.gz`); a higher counter is the later roll, first roll of a second is 0.
-    counter is the later (newer) roll. the first roll of a second has no counter (0).
+    daily's dated names (`<stem>.log.<Y-m-d>`) don't end in `.log`, so they never need
-
+    the tie-break - return 0 rather than misparsing the trailing date as a counter.
-    daily's dated names (`<stem>.log.<Y-m-d>`) do NOT end in `.log` and are second+-granular
-    (distinct mtimes), so they never need the counter tie-break — return 0 for them rather
-    than misparsing the trailing date component as a counter.
     """
     base = path[:-3] if path.endswith(".gz") else path
     if not base.endswith(".log"):
@@ -333,25 +341,31 @@ def _roll_counter(path: str) -> int:
     return int(tail) if tail.isdigit() else 0
 
 
-def prune(log_dir: str, stem: str, backup_count: int) -> None:
+def prune(log_dir: str, stem: str, backup_count: int, live_stem: Optional[str] = None) -> None:
     """keep only the newest `backup_count` rolled files for a given stem in log_dir
 
     matches files beginning with `<stem>.` (e.g. run.*), sorted newest-first by mtime
-    then roll counter (mirrors retier's ordering — see _roll_counter), deleting the
+    then roll counter (mirrors retier's ordering, see _roll_counter), deleting the
     oldest beyond the count. used for on_start, which the handlers don't auto-prune.
 
     FOOTGUN: `stem` is reduced to its basename so a `name` containing a directory (e.g.
-    "sub/run") still matches the basenamed rolled files in log_dir — else nothing
+    "sub/run") still matches the basenamed rolled files in log_dir - else nothing
     matches and old files pile up forever.
+
+    candidates are also checked against this lib's own rolled-file shape
+    (`<stem>.<stamp>[.N].log[.gz]`, see `_is_rolled_name`) - a foreign file that only
+    shares the `<stem>.` prefix (e.g. a project's own `<stem>.audit.log` dropped into
+    the same log_dir) is left alone rather than counted and deleted as a roll.
     """
     if backup_count <= 0:
         return
     stem = os.path.basename(stem)
+    live_stem = os.path.basename(live_stem) if live_stem else None
     try:
         entries = [
             os.path.join(log_dir, name)
             for name in os.listdir(log_dir)
-            if name.startswith(f"{stem}.") and name != f"{stem}.log"
+            if name != f"{stem}.log" and _is_rolled_name(stem, name, live_stem)
         ]
     except OSError:
         return
@@ -376,26 +390,33 @@ def attach_rolling(
     handler, log_dir: str, compress: bool,
     prune_stem: Optional[str] = None, backup_count: int = 0,
     keep_uncompressed: Optional[int] = None, keep_compressed: Optional[int] = None,
-    tiered: bool = False,
+    tiered: bool = False, rotate_mode: Optional[str] = None,
 ) -> Tuple[Callable, Callable]:
     """wire the custom namer + rotator onto a rotating handler; return them
 
-    rolled files are named off `prune_stem` (the HISTORY stem — the project namespace),
+    rolled files are named off `prune_stem` (the HISTORY stem - the project namespace),
     independent of the live file name, via make_history_namer: `<stem>.<stamp>.log[.gz]`
-    uniform across size and daily — replacing the stdlib handler's own rolled-name
+    uniform across size and daily - replacing the stdlib handler's own rolled-name
     scheme (`.N` for size, `.log.<date>` for daily), which can't inject a project stem
     and (for size) can't be managed once files are redirected into log_dir.
 
     pass `prune_stem`/`backup_count` so the rotator prunes `log_dir` after each roll
     (the handler's own retention can't see the redirected files). pass
     `keep_uncompressed`/`keep_compressed` for tiered retention instead (see
-    make_rotator); `tiered=True` lands rolls plain (retier compresses).
+    make_rotator); `tiered=True` lands rolls plain (retier compresses). `rotate_mode`
+    ("size"/"daily") is forwarded to `make_rotator` so the zero-retention delete-on-land
+    branch only ever fires for `"size"`.
     """
     namer = make_history_namer(
         os.path.basename(prune_stem or ""), log_dir, compress, plain=tiered,
     )
+    # the live file name (stem of the handler's baseFilename, minus one .log) - so retention
+    # also recognizes pre-v0.5.0 legacy rolls, which were named off the LIVE name not the stem
+    live_base = os.path.basename(getattr(handler, "baseFilename", "") or "")
+    live_stem = live_base[:-4] if live_base.endswith(".log") else live_base
     rotator = make_rotator(
         compress, log_dir, prune_stem, backup_count, keep_uncompressed, keep_compressed,
+        rotate_mode=rotate_mode, live_stem=live_stem or None,
     )
     handler.namer = namer
     handler.rotator = rotator

src/log_setup/setup.py

@@ -1,15 +1,13 @@
-"""app-entry-point logging setup (sync, stdlib only).
+"""app-entry-point logging setup (sync, stdlib only). see README.
 
-`setup_logging` configures the root logger once for the whole process: a live
+`setup_logging` configures the root logger once for the whole process. called by the
-run.log at a stable path, rotation (daily/size/on_start/none) into a logs/ dir, gzip
+APPLICATION, not by reusable libraries (those stay emit-only). idempotent, never
-of rolled files, retention, console output, and a consistent format. called by the
+crashes the app over logging, and can route through a background queue so an async
-APPLICATION, not by reusable libraries (those stay emit-only). idempotent (no
+event loop doesn't block on file I/O.
-duplicate handlers on repeat calls), never crashes the app over logging, and can
-route through a background queue so an async event loop doesn't block on file I/O.
 
 `rotate="size"` always bounds the live file: the roll fires at `max_bytes` regardless
 of `backup_count`, including `backup_count=0` (means "keep zero rolled files", not
-"never roll" — each roll is deleted right after landing).
+"never roll" - each roll is deleted right after landing).
 """
 
 import atexit
@@ -31,22 +29,29 @@ _listener = None
 _atexit_registered = False
 
 
-def _exc_text() -> str:
+class _PreservingQueueHandler(logging.handlers.QueueHandler):
-    """render sys.exc_info() as text, for capturing a traceback into a buffered warning
+    """QueueHandler that hands the record to the listener untouched
 
-    (log.warning(..., exc_info=True) only works logged live from the except block; setup
+    stdlib's QueueHandler.prepare() calls self.format(record) with the queue
-    warnings are deferred, see _flush_warnings, so render eagerly instead)
+    handler's OWN (default) formatter, then nulls exc_info/exc_text/stack_info -
+    losing structured fields (e.g. JsonLinesFormatter's exc_info key) before the
+    listener's real formatter ever sees the record. that behavior exists to keep a
+    record picklable across a multiprocessing.Queue; this lib only ever uses an
+    in-process queue.Queue, so there is nothing to pickle and nothing to strip -
+    the listener's handler formats the untouched record exactly once.
     """
+
+    def prepare(self, record: logging.LogRecord) -> logging.LogRecord:
+        return record
+
+
+def _exc_text() -> str:
+    """render sys.exc_info() as text, for capturing a traceback into a buffered warning"""
     return "".join(traceback.format_exception(*sys.exc_info())).strip()
 
 
 def _flush_warnings(warnings: list) -> None:
-    """emit buffered setup-time warnings now that handlers are attached
+    """emit buffered setup-time warnings now that handlers are attached"""
-
-    setup-time warnings fire before this call's handlers exist, so logging them
-    immediately would only reach stderr (logging's lastResort) and never the file being
-    configured — buffer, then flush once attached so they land like any other record
-    """
     for message, *args in warnings:
         log.warning(message, *args)
 
@@ -54,25 +59,19 @@ def _flush_warnings(warnings: list) -> None:
 def _level_value(level: Union[int, str]) -> int:
     """coerce a level name or int to a logging level int (defaults to INFO)"""
     if isinstance(level, bool):
-        # bool is an int subclass (True==1, below DEBUG) but is never a real level —
+        # bool is an int subclass (True==1, below DEBUG) but never a real level
-        # reject it consistently with the per-module path rather than set level 1
         return logging.INFO
     if isinstance(level, int):
         return level
     if not isinstance(level, str):
         return logging.INFO
     resolved = logging.getLevelName(level.upper())
-    # getLevelName returns the string "Level XXX" for an unknown name, which
+    # getLevelName returns "Level XXX" for an unknown name; fall back to INFO
-    # setLevel then rejects — never crash the app over a bad level, fall back to INFO
     return resolved if isinstance(resolved, int) else logging.INFO
 
 
 def _strict_level_value(level: Union[int, str]) -> Optional[int]:
-    """coerce a level name or int to a logging level int, or None if invalid
+    """like _level_value but reports invalid as None so the caller can skip + warn"""
-
-    unlike `_level_value` (falls back to INFO for the root `level`), reports invalid as
-    None so the per-module path can skip + warn instead of silently applying INFO
-    """
     if isinstance(level, bool):
         return None
     if isinstance(level, int):
@@ -84,12 +83,7 @@ def _strict_level_value(level: Union[int, str]) -> Optional[int]:
 
 
 def _apply_module_levels(module_levels: Optional[Dict[str, Union[int, str]]], warnings: list) -> None:
-    """set per-logger level overrides by exact logger name, never crashing
+    """set per-logger level overrides by exact logger name, never crashing"""
-
-    names match exactly (no discovery); stdlib hierarchy still applies, so a parent name
-    quiets its whole subtree. a bad level is skipped, its warning appended to `warnings`
-    (no handlers exist yet — see _flush_warnings) rather than emitted directly
-    """
     if not module_levels:
         return
     for mod_name, raw_level in module_levels.items():
@@ -101,17 +95,12 @@ def _apply_module_levels(module_levels: Optional[Dict[str, Union[int, str]]], wa
 
 
 def _clear_owned(root: logging.Logger, warnings: list) -> None:
-    """remove only the handlers this lib previously added; leave app handlers alone
+    """remove only the handlers this lib previously added; leave app handlers alone"""
-
-    close failures are appended to `warnings`, not logged directly — no handlers exist
-    yet at this point in setup (see _flush_warnings)
-    """
     global _listener
     if _listener is not None:
         _listener.stop()
-        # listener owns the real file/console handlers (only QueueHandler is root-
+        # listener owns the real file/console handlers; stopping it doesn't close
-        # attached + marked); stopping it doesn't close them, so close here rather than
+        # them, so close here rather than rely on GC finalizers across a re-setup
-        # rely on GC finalizers across a re-setup
         for wrapped in getattr(_listener, "handlers", ()):
             try:
                 wrapped.close()
@@ -138,10 +127,8 @@ def _tag(handler: logging.Handler) -> logging.Handler:
 def _normalize_name(name: str) -> str:
     """strip one trailing '.log' (case-insensitive) so the stem is extension-free
 
-    `name` is allowed to be passed with or without the extension — "latest" and
+    "latest" and "latest.log" both yield stem "latest" (never latest.log.log); only
-    "latest.log" both yield stem "latest" (live file latest.log), never latest.log.log.
+    one level is stripped, so "app.log.log" -> "app.log".
-    only one level is stripped: "app.log.log" -> "app.log" so a legit ".log" inside a
-    name survives.
     """
     if name.lower().endswith(".log"):
         return name[:-4]
@@ -149,12 +136,7 @@ def _normalize_name(name: str) -> str:
 
 
 def _history_stem() -> str:
-    """the project namespace for historic files: the cwd basename
+    """the project namespace for historic files: the cwd basename, "" for a degenerate cwd"""
-
-    a service run from bestbuy/ gives historic files bestbuy.<stamp>.log[.gz]. falls back
-    to an empty string only for a degenerate cwd (e.g. "/"), which the caller resolves to
-    the live stem.
-    """
     try:
         return os.path.basename(os.getcwd().rstrip(os.sep))
     except OSError:
@@ -162,27 +144,20 @@ def _history_stem() -> str:
 
 
 def _file_handler(
-    name: str, history_stem: str, live_path: str, log_dir: str, rotate: Optional[str],
+    history_stem: str, live_path: str, log_dir: str, rotate: Optional[str],
     backup_count: int, max_bytes: int, compress: bool,
     keep_uncompressed: Optional[int], keep_compressed: Optional[int], warnings: list,
 ) -> logging.Handler:
     """build the configured file handler with custom rolling into log_dir
 
-    `name` is the LIVE stem (drives live_path); `history_stem` is the PROJECT stem that
+    `history_stem` is the PROJECT stem that rolled/historic files are named off,
-    rolled/historic files are named off + the retention glob keys on — decoupled: the
+    decoupled from the live file's own name (`live_path`).
-    live file keeps its defined name, historic files carry the project namespace. an
-    unknown `rotate` is appended to `warnings` rather than logged directly (see
-    _flush_warnings — no handlers exist yet at this point).
     """
     tiered = keep_uncompressed is not None or keep_compressed is not None
     if rotate == "size":
-        # stdlib doRollover no-ops at backupCount==0, and its numbered .1/.2 shift can't
+        # stdlib doRollover no-ops at backupCount==0 - force nonzero so the roll
-        # manage files redirected into log_dir — force nonzero so the roll always fires,
+        # always fires; the REAL backup_count still flows to attach_rolling, whose
-        # and let attach_rolling's namer + retier/prune bound retention instead. the
+        # make_rotator treats <=0 as "keep no rolled history" and deletes each roll
-        # REAL backup_count (maybe 0) still flows to attach_rolling below: make_rotator
-        # treats <=0 there as "keep no rolled history" and deletes each roll right after
-        # landing, rather than passing 0 to prune() (whose own <=0 is a "leave history
-        # alone" no-op — that mismatch is what silently disabled rotation before)
         size_backup = max(backup_count, 1)
         handler = logging.handlers.RotatingFileHandler(
             live_path, maxBytes=max_bytes, backupCount=size_backup, encoding="utf-8",
@@ -190,7 +165,7 @@ def _file_handler(
         attach_rolling(
             handler, log_dir, compress, prune_stem=history_stem, backup_count=backup_count,
             keep_uncompressed=keep_uncompressed, keep_compressed=keep_compressed,
-            tiered=tiered,
+            tiered=tiered, rotate_mode=rotate,
         )
     elif rotate == "daily":
         handler = logging.handlers.TimedRotatingFileHandler(
@@ -199,7 +174,7 @@ def _file_handler(
         attach_rolling(
             handler, log_dir, compress, prune_stem=history_stem, backup_count=backup_count,
             keep_uncompressed=keep_uncompressed, keep_compressed=keep_compressed,
-            tiered=tiered,
+            tiered=tiered, rotate_mode=rotate,
         )
     else:
         if rotate == "on_start":
@@ -210,12 +185,14 @@ def _file_handler(
                 )
             else:
                 rotate_on_start(live_path, log_dir, compress, history_stem=history_stem)
-                prune(log_dir, history_stem, backup_count)
+                live_base = os.path.basename(live_path)
+                live_stem = live_base[:-4] if live_base.endswith(".log") else live_base
+                prune(log_dir, history_stem, backup_count, live_stem or None)
         elif rotate is not None:
-            # a typo'd value (e.g. "hourly") would otherwise silently fall through to a
+            # a typo'd value would otherwise silently fall through to a non-rotating
-            # non-rotating FileHandler and grow forever — warn instead of degrade silently
+            # FileHandler and grow forever - warn instead of degrade silently
             warnings.append((
-                "log_setup: unknown rotate %r; expected 'daily'/'size'/'on_start'/None — "
+                "log_setup: unknown rotate %r; expected 'daily'/'size'/'on_start'/None - "
                 "no rotation applied (single growing file)", rotate,
             ))
         handler = logging.FileHandler(live_path, encoding="utf-8")
@@ -248,8 +225,8 @@ def setup_logging(
 
     `history_name` names the rolled/historic files (`<history_name>.<timestamp>.log[.gz]`),
     independent of the live file: defaults to the PROJECT namespace = the cwd basename
-    (run from bestbuy/ -> historic files bestbuy.<stamp>...), settable explicitly. the
+    (run from bestbuy/ -> historic files bestbuy.<stamp>...). the live file always keeps
-    live file always keeps `name`; only historic files carry the project name.
+    `name`; only historic files carry the project name.
 
     `keep_uncompressed`/`keep_compressed` (default None) enable TIERED retention: when
     either is given, rolled files are kept as the newest `keep_uncompressed` uncompressed
@@ -258,29 +235,28 @@ def setup_logging(
     IGNORED in tiered mode. pass NEITHER knob and rotation behaves exactly as before.
 
     `level` is the root default every logger inherits. `module_levels` is an optional
-    map of exact logger name -> level applied after the root is set — the ergonomic way
+    map of exact logger name -> level applied after the root is set, the ergonomic way
-    to quiet noisy dependencies (e.g. {"motor": "WARNING"}) from the one setup call
+    to quiet noisy dependencies (e.g. {"motor": "WARNING"}) from one call instead of
-    instead of scattering `getLogger(...).setLevel(...)` afterwards (stdlib hierarchy
+    scattering `getLogger(...).setLevel(...)` calls. names match EXACTLY (no discovery:
-    under the hood, not new capability). names match EXACTLY (no discovery: a typo'd
+    a typo'd name silently configures an unused logger), but hierarchy applies, so
-    name silently configures an unused logger), but hierarchy applies, so naming a
+    naming a parent ("aiohttp") quiets its whole subtree. str or int per entry; a bad
-    parent ("aiohttp") quiets its whole subtree. str or int per entry; a bad value is
+    value is skipped with a warning and never aborts the others or the setup.
-    skipped with a warning and never aborts the others or the setup.
 
     `rotate` is "daily" (default), "size", "on_start", or None. for `rotate="size"`, the
     live file always rolls at `max_bytes` regardless of `backup_count`: `backup_count=0`
     means "keep zero rolled files" (each roll lands then is deleted immediately), NOT
     "disable rotation". `backup_count>=1` keeps that many rolled files as before.
 
-    `console=True` adds a stdout handler (off by default — the file is the output).
+    `console=True` adds a stdout handler (off by default - the file is the output).
     `queue=True` routes records through a background QueueListener so file I/O never
     blocks the caller (stopped at exit). `output` is "text" (default, human `time |
-    module | level | message`, local time) or "json" (structured JSON Lines for the
+    module | level | message`, local time) or "json" (structured JSON Lines, UTC
-    Grafana/Loki path, UTC timestamps + a unix-epoch `ts`, `extra=` fields surfaced as
+    timestamps + a unix-epoch `ts`, `extra=` fields surfaced as top-level keys); file
-    top-level keys); file and console use the same format, live-file name unaffected.
+    and console use the same format, live-file name unaffected. `fmt`/`datefmt` apply
-    `fmt`/`datefmt` apply to text output only.
+    to text output only.
 
     idempotent: a repeat call clears only the handlers this function added. never
-    raises over logging — an unwritable `log_dir` falls back to console-only with a
+    raises over logging - an unwritable `log_dir` falls back to console-only with a
     warning even when `console` is off; an unknown `output` falls back to text.
     """
     global _listener, _atexit_registered
@@ -292,12 +268,11 @@ def setup_logging(
     _apply_module_levels(module_levels, warnings)
     _clear_owned(root, warnings)
 
-    formatter = build_formatter(output, fmt, datefmt)
+    formatter = build_formatter(output, fmt, datefmt, warnings)
     stem = _normalize_name(name)
     live_path = f"{stem}.log"
-    # historic/rolled files are named off the project namespace: history_name if given,
+    # history_name if given, else the cwd basename; falls back to the live stem for a
-    # else the cwd basename. normalized + basenamed like `name`; falls back to the live
+    # degenerate cwd so naming/retention never break
-    # stem for a degenerate cwd so naming/retention never break.
     history_source = history_name if history_name is not None else _history_stem()
     history_stem = os.path.basename(_normalize_name(history_source)) or stem
 
@@ -312,7 +287,7 @@ def setup_logging(
     if file_ok:
         try:
             fh = _file_handler(
-                stem, history_stem, live_path, log_dir, rotate, backup_count, max_bytes, compress,
+                history_stem, live_path, log_dir, rotate, backup_count, max_bytes, compress,
                 keep_uncompressed, keep_compressed, warnings,
             )
             fh.setFormatter(formatter)
@@ -321,19 +296,18 @@ def setup_logging(
             file_ok = False
 
     if console or not file_ok:
-        sh = logging.StreamHandler()
+        sh = logging.StreamHandler(sys.stdout)
         sh.setFormatter(formatter)
         handlers.append(sh)
 
     if queue:
         record_queue: "_queue.Queue" = _make_queue()
-        qh = _tag(logging.handlers.QueueHandler(record_queue))
+        qh = _tag(_PreservingQueueHandler(record_queue))
         root.addHandler(qh)
         _listener = logging.handlers.QueueListener(record_queue, *handlers, respect_handler_level=True)
         _listener.start()
         if not _atexit_registered:
-            # register once — atexit doesn't dedupe; repeated re-setups would otherwise
+            # register once - atexit doesn't dedupe, repeated re-setups would stack
-            # stack identical callbacks
             atexit.register(_stop_listener)
             _atexit_registered = True
     else:
@@ -343,8 +317,7 @@ def setup_logging(
     if not file_ok:
         warnings.append(("log_setup: log_dir %r not writable; logging to console only", log_dir))
 
-    # flush now that handlers are attached, so setup-time warnings actually land in the
+    # flush now that handlers are attached, so warnings land in the configured log
-    # configured log rather than being lost to stderr before any handler existed
     _flush_warnings(warnings)
 
     return root