the beyond-keep delete branch had no _gz_intact guard: when an intact plain file and its
CORRUPT .gz twin (same mtime) straddled the keep boundary and listdir yielded the .gz first,
the .gz sorted into the kept range while the intact plain sorted past it and was deleted -
leaving the corrupt archive as the sole copy (permanent data loss), violating the 'corrupt
.gz never wins over intact plain' invariant the two dedupe sites already enforce. the delete
now skips an intact plain whose only surviving twin is a corrupt .gz; a later retier retires
it once a clean .gz exists. reproduced with a forced .gz-first listdir: old deleted the plain,
new keeps it.
Signed-off-by: disqualifier <dev@disqualifier.me>
leftover from the v0.5.0 history-namespace split - every path inside
_file_handler already uses live_path/history_stem/log_dir, never name. the
docstring still explained it as "the LIVE stem" though nothing read it.
Signed-off-by: disqualifier <dev@disqualifier.me>
build_formatter warned immediately via getLogger(__name__).warning, but
setup_logging calls it after _clear_owned strips the lib's handlers and before
any new handler attaches - so the warning hit a handler-less root and landed
on stderr via lastResort, never reaching the configured log. The v0.6.0
warnings buffer already threads this class of setup-time warning through for
an unknown rotate value; build_formatter now takes the same warnings list and
appends to it instead of emitting inline, so setup_logging can flush it once
handlers are attached like the others.
Signed-off-by: disqualifier <dev@disqualifier.me>
prune() and retier() matched any file starting with "<stem>." (minus the live
<stem>.log), with no check that the name actually has this lib's own rolled
shape (<stem>.<stamp>[.N].log[.gz]). Since v0.5.0 the default history_stem is
the cwd basename, so a project dropping its own same-stem file into log_dir
(e.g. proj.audit.log, proj.stats.json) got silently deleted by prune once it
aged past backup_count, or folded into retier's tier accounting (gzipped or
deleted as if it were a real roll). Candidates are now filtered through
_is_rolled_name before being treated as a roll.
Signed-off-by: disqualifier <dev@disqualifier.me>
QueueHandler.prepare() formatted the record with its own default formatter and
nulled exc_info/exc_text/stack_info before the listener's real formatter ran,
so queue=True + output="json" produced a JSON line with no exc_info key and
the traceback folded into message instead. _PreservingQueueHandler skips that
premature format-and-strip (safe here - the queue is in-process only, nothing
needs to stay picklable) so the listener's handler formats the untouched
record exactly once, matching the non-queued path.
Signed-off-by: disqualifier <dev@disqualifier.me>
console=True now binds sys.stdout explicitly (was the stdlib StreamHandler
stderr default), so a supervisor capturing stdout sees console log lines.
console stays off by default (file is the primary sink). docs corrected to
stdout.
Signed-off-by: disqualifier <dev@disqualifier.me>
v0.5.1's rotate="size" fix added an unconditional os.remove(dest) at
backup_count<=0 in make_rotator to bound the live file, but the rotator is
shared with rotate="daily" and had no mode awareness - every midnight roll
under daily with backup_count=0 was deleting the just-rolled log instead of
leaving it unpruned. make_rotator/attach_rolling now take rotate_mode and
gate the delete-on-land branch to "size" only, matching the documented
contract that only size always bounds the live file. Bump 0.6.1 -> 0.6.2.
Signed-off-by: disqualifier <dev@disqualifier.me>
Setup-time warnings (invalid module_levels entry, handler-close failure on
re-setup, unknown rotate) fired before any handler attached, so they hit
stderr via logging's lastResort and never run.log; now buffered and flushed
after handlers attach (logsetup-10).
README's tiered-retention example showed the wrong historic-file naming and
put the live file inside log_dir; corrected to the actual project-stem
naming and cwd location (logsetup-11). Package docstring claimed console
output by default; console=False is the real default (logsetup-12).
JSON output gains an additive ts field (unix epoch int, alongside the
unchanged ISO time) for consumers that want a sortable number.
Compressed essay-length docstrings/comments across setup.py, rotation.py,
and formats.py with zero behavior change (re-verified against baseline
logs/ listings); load-bearing footgun notes kept.
Signed-off-by: disqualifier <dev@disqualifier.me>
_gzip_file now writes to a .tmp sibling and atomically os.replaces it onto
the final .gz path, so a crash/OOM mid-write can never leave a truncated
.gz where retention trusts it; retier's plain/gz dedupe additionally
verifies a .gz decompresses cleanly before removing its plain twin, so a
corrupt .gz is never preferred over the last intact copy.
rotate="size" now always forces the handler's backupCount to fire
regardless of backup_count, so backup_count=0 means "keep zero rolled
files" (each roll deletes itself immediately) instead of silently
disabling rotation and growing the live file unbounded.
Also mirrors retier's mtime+roll-counter sort key into prune() so a
same-second burst with tied mtimes prunes the oldest files instead of
arbitrary listdir order (logsetup-9, adjacent one-line fix).
Bumps to v0.5.1.
Signed-off-by: disqualifier <dev@disqualifier.me>
the live file keeps its defined name (latest.log); rolled/historic files are now named off
the PROJECT namespace so you can tell which service a log came from at a glance. default =
os.path.basename(os.getcwd()) (run from bestbuy/ -> historic bestbuy.<stamp>.log[.gz]);
override with the new history_name= param.
- decouple the rolled stem (history) from the live stem (name) in setup.py; thread
history_stem into BOTH the namers AND prune/retier (same stem, or retention breaks).
- unify size + daily on make_history_namer -> <stem>.<stamp>.log[.gz] (was stdlib .N /
.log.<date>); rotate_on_start takes history_stem.
behavior-visible: rolled names differ from pre-0.5.0 (were the live stem). pass
history_name=name for the old naming. execute-verified across on_start/daily/size (tiered +
non-tiered), default-cwd + explicit + history==name + normalization edges; v0.4.x fixes
(rotation/tiering/retention) all still hold. bump v0.4.3 -> v0.5.0
Signed-off-by: disqualifier <dev@disqualifier.me>
- logsetup-2: retier drops a plain file whose .gz twin exists (crash-between-write-and-remove
residue) up front, so the phantom twin never occupies a retention slot / evicts a real roll.
- logsetup-5: retier breaks mtime ties by the roll counter parsed from the name, so a
same-second on_start burst tiers newest-first instead of arbitrary listdir order.
- logsetup-3: tiered daily rotator disambiguates an existing dated dest with a counter (no
clobber on a second same-interval roll).
- logsetup-6: unknown rotate value warns (matching the unknown-output convention) instead of
silently degrading to a non-rotating file.
- README: size-mode backup naming clarified (numbered flat vs timestamped tiered).
verified stdlib-only; v0.4.0/v0.4.1/size suites still pass. bump v0.4.2 -> v0.4.3
Signed-off-by: disqualifier <dev@disqualifier.me>
tiered size mode was broken two ways: (1) stdlib RotatingFileHandler's numbered .1/.2 shift
can't manage files redirected into log_dir, so every roll overwrote slot 1 -> ~99% of
history silently lost; (2) doRollover is gated on backupCount>0, so backup_count=0 (which
the docstring says is ignored in tiered mode) meant NO rotation + unbounded live file.
fix: tiered size now uses a timestamped per-roll namer (make_size_namer) like daily/on_start
so retier manages the pile, and forces a nonzero internal backupCount so the roll always
fires (retier bounds retention, not backupCount). non-tiered size unchanged.
verified: tiered size rotates + bounds to tier total (was stuck at 1); backup_count=0 rotates
+ live file bounded (was unbounded); legacy size back-compat intact; v0.4.0/v0.4.1 suites
still pass. bump v0.4.1 -> v0.4.2
Signed-off-by: disqualifier <dev@disqualifier.me>
rolled files land in log_dir under their basename (the namer/rotate_on_start basename
them), but prune()/retier() globbed the un-basenamed name. a name like 'sub/run' matched
nothing, so old .log/.gz files piled up forever (slow disk leak; the live file was fine).
basename the stem at the top of both prune() and retier(). regression-verified with
name='sub/run' (10 retained vs 12/dead before). bump v0.4.0 -> v0.4.1
Signed-off-by: disqualifier <dev@disqualifier.me>
- keep_uncompressed/keep_compressed: newest N rolled logs plain, next M gzipped, rest
deleted. applies to on_start/daily/size. opt-in by knob presence; without them the
legacy backup_count + gzip-on-roll path is unchanged (existing consumers unaffected).
- _normalize_name: 'latest' and 'latest.log' both -> live latest.log (no .log.log).
- _gzip_file preserves source mtime (stable tier ordering across re-tiers).
- rotate_on_start collision counter checks both .log and .log.gz (no duplicate logical
roll when a same-stamp file was already compressed).
execute-verified stdlib-only incl. a back-compat control proving the no-knobs path is
unchanged. bump v0.3.2 -> v0.4.0
Signed-off-by: disqualifier <dev@disqualifier.me>
LS-1: re-setup closes the QueueListener's wrapped file/console handlers after stopping it
(was: relied on GC). LS-2: atexit registration guarded by a module flag so repeated
queue=True re-setups don't stack callbacks. JSON formatter caches the rendered traceback on
the record (no per-handler re-render); _move dest precondition documented.
Signed-off-by: disqualifier <dev@disqualifier.me>
- the non-compress rotator and on_start move fall back to shutil.move when os.replace
hits OSError(EXDEV) across filesystems, so rolls land on a separate logs volume /
container bind-mount instead of failing every rotation via the handler's silent
handleError (L18)
- on_start disambiguates a same-second restart with a numeric counter so a rapid
crash-restart loop doesn't clobber the earlier rolled file (L17)
- reject a bool root level (True==1) consistently with the per-module path; alias the
queue module import to drop the queue:bool param shadow; log (not swallow) a
handler.close failure during re-setup (nits).
Signed-off-by: disqualifier <dev@disqualifier.me>
guard the extra-merge loop with the formatter's own output keys (time/level/module/
message). stdlib LogRecord rejects extra keys colliding with real attribute names, but
time/level are NOT LogRecord attrs, so a caller's extra={"time":...}/{"level":...}
previously overwrote the UTC timestamp / levelname — the two fields Loki/Grafana alert
on. now those keys are reserved and a colliding extra is dropped.
Signed-off-by: disqualifier <dev@disqualifier.me>
add an optional module_levels={logger_name: level} param to setup_logging,
the ergonomic way to quiet noisy dependencies (motor/pymongo/aiohttp -> WARNING)
from the one entry-point call instead of scattering setLevel afterwards.
- exact logger-name match, no discovery; stdlib hierarchy applies so naming a
parent quiets its subtree
- str or int level per entry, same normalization as root level
- bad level for one entry is skipped + warned, never raises (never-crash rule)
- module_levels=None/{} (default) is byte-identical to prior behavior
additive, backwards-compatible -> v0.3.0.
Signed-off-by: disqualifier <dev@disqualifier.me>
__init__.py still reported __version__ = 0.1.1 while pyproject, README, and the tag are 0.2.0 — the one version-metadata drift across the libs. bumped to 0.2.0 so __version__/pyproject/tag agree.
Signed-off-by: disqualifier <dev@disqualifier.me>
TimedRotatingFileHandler/RotatingFileHandler retention (getFilesToDelete) only scans the live file's directory, never the log_dir we redirect rolled files into, so daily (the default) and size modes never pruned and .gz files grew unbounded. the rotator now calls the existing prune(log_dir, stem, backup_count) helper (the one on_start already uses) after each roll. verified by execution: daily and size both retain exactly backup_count; a no-prune control retains all.
Signed-off-by: disqualifier <dev@disqualifier.me>
add a selectable output format to setup_logging: text (default, human,
local time) stays unchanged; output="json" emits one-JSON-object-per-line
(JSON Lines) for the Grafana/Loki path. json fields are time (UTC ISO-8601
with Z), level, module, message, plus any extra={...} keys surfaced as
top-level fields and a rendered exc_info traceback on error records. both
file and console use the chosen format; the live-file name is unchanged so
the Promtail glob and tail command don't break across text/json. an unknown
output falls back to text and warns, never crashes. stdlib json only, zero
new deps. minor bump to v0.2.0.
Signed-off-by: disqualifier <dev@disqualifier.me>
_level_value used logging.getLevelName(name), which returns the string 'Level XXX'
for an unknown name; that string then reached setLevel() and raised ValueError,
violating the 'never crashes the app over logging' contract. validate the result is
an int and fall back to INFO otherwise.
verified: level='BOGUS' -> INFO (no crash); 'DEBUG' and int levels still honored.
Signed-off-by: disqualifier <dev@disqualifier.me>
dsql