fix: clear() and stale-tmp sweep resolve the real symlink target and match only <base>.<pid>.tmp

clear() removed the raw path, so on a symlinked store it unlinked only the
symlink while the real data file kept every value - clear() returned True
but the data resurfaced if the symlink was recreated. It now realpaths the
target first, mirroring _save()'s symlink-safe writes.

_sweep_stale_tmp globbed {base}.*.tmp, which crosses dots and can match an
unrelated neighbor file (deleting it outside the lib's own artifacts), and
interpolated the store's basename unescaped, so glob metacharacters in the
filename (e.g. state[prod].json) either missed the store's own orphans or
cross-matched a different store's. The sweep now lists the directory and
matches a re.escape'd regex anchored to the exact <base>.<pid>.tmp shape
_save() creates.

Signed-off-by: disqualifier <dev@disqualifier.me>
This commit is contained in:
2026-07-03 19:05:46 -04:00
parent 5e3eb4e704
commit ddf76d8241
3 changed files with 35 additions and 18 deletions
+14 -8
View File
@@ -12,18 +12,18 @@ you `delete` or `clear` them.
`requirements.txt`:
```
aiokv @ git+ssh://git@git.rethinkstudios.io/rethink-public/aiokv.git@v0.2.1
aiokv @ git+ssh://git@git.rethinkstudios.io/rethink-public/aiokv.git@v0.2.2
```
Direct:
```bash
pip install "aiokv @ git+ssh://git@git.rethinkstudios.io/rethink-public/aiokv.git@v0.2.1"
pip install "aiokv @ git+ssh://git@git.rethinkstudios.io/rethink-public/aiokv.git@v0.2.2"
```
Requires `aiofiles` (pulled transitively).
Drop the `@v0.2.1` suffix from the line above to install the latest unpinned.
Drop the `@v0.2.2` suffix from the line above to install the latest unpinned.
## Usage
@@ -66,11 +66,17 @@ Writes are **atomic**: data is written to a temp file in the same directory and
written through rather than clobbered. A **process** crash mid-write leaves the previous
good file intact, and a reader never observes a partial file. (This is process-crash
safety, not power-loss durability — there's no `fsync`, so an OS/power failure could
still lose the last write; fine for reconstructible single-process state.) Orphaned
`.<pid>.tmp` files left by a hard crash of a *different, dead* process are swept on the
next save. A single `asyncio.Lock` guards every read and write, so concurrent operations
on one instance are consistent and no update is lost. All blocking filesystem calls and
JSON (de)serialization run via `asyncio.to_thread`, so nothing stalls the event loop.
still lose the last write; fine for reconstructible single-process state.) `clear()` is
symlink-safe the same way: it removes the realpath'd target, not the symlink itself, so
a symlinked store is genuinely erased rather than just having its symlink unlinked.
Orphaned `.<pid>.tmp` files left by a hard crash of a *different, dead* process are swept
on the next save; the sweep matches only the exact `<realpath basename>.<pid>.tmp` shape
it creates, so a differently-shaped neighbor file is never touched and a store filename
containing glob-like characters (e.g. `state[prod].json`) doesn't miss its own orphans or
cross-match another store's. A single `asyncio.Lock` guards every read and write, so
concurrent operations on one instance are consistent and no update is lost. All blocking
filesystem calls and JSON (de)serialization run via `asyncio.to_thread`, so nothing stalls
the event loop.
## Scope — read this
+1 -1
View File
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
[project]
name = "aiokv"
version = "0.2.1"
version = "0.2.2"
description = "Async file-backed key-value store for single-process local state — atomic writes, no TTL, config-free, installable."
requires-python = ">=3.10"
dependencies = [
+20 -9
View File
@@ -17,9 +17,9 @@ cross-process storage.
"""
import os
import re
import json
import time
import glob
import asyncio
import logging
from typing import Any, Dict
@@ -71,11 +71,15 @@ class AioKV:
return False
async def clear(self) -> bool:
"""remove the backing file entirely; True on success or if already absent, False on error"""
"""remove the backing file entirely; True on success or if already absent, False on error
symlink-safe: removes the realpath'd target rather than the symlink itself, mirroring
_save()'s symlink-safe writes - a symlinked store's real data file is what gets erased"""
try:
async with self.lock:
target = await asyncio.to_thread(os.path.realpath, self.file)
try:
await asyncio.to_thread(os.remove, self.file)
await asyncio.to_thread(os.remove, target)
except FileNotFoundError:
pass
return True
@@ -134,18 +138,25 @@ class AioKV:
raise
async def _sweep_stale_tmp(self, directory: str) -> None:
"""remove orphaned .<pid>.tmp files left by a hard crash of a different, dead process"""
"""remove orphaned .<pid>.tmp files left by a hard crash of a different, dead process
matches only the exact shape _save() creates (<realpath basename>.<pid>.tmp), via a
re.escape'd regex over a directory listing rather than a raw glob - a glob pattern
both crosses unrelated dots (matching non-aiokv neighbors like foo.backup.999.tmp) and
mistreats glob metacharacters in the store's own filename (e.g. state[prod].json)"""
base = os.path.basename(await asyncio.to_thread(os.path.realpath, self.file))
pattern = os.path.join(directory, f"{base}.*.tmp")
for path in await asyncio.to_thread(glob.glob, pattern):
try:
pid = int(path.rsplit(".", 2)[-2])
except (ValueError, IndexError):
candidate = re.compile(rf"^{re.escape(base)}\.(\d+)\.tmp$")
names = await asyncio.to_thread(os.listdir, directory)
for name in names:
match = candidate.match(name)
if not match:
continue
pid = int(match.group(1))
if pid == os.getpid():
continue
if await asyncio.to_thread(self._pid_alive, pid):
continue
path = os.path.join(directory, name)
try:
await asyncio.to_thread(os.remove, path)
except FileNotFoundError: