6 Commits
Author SHA1 Message Date
dsql a2917cce23 release: 1.0.0
first stable release. pre-1.0.0 verification complete: all surviving MED regressions and
gaps resolved and independently re-fired, tree audited clean across the suite.

Signed-off-by: disqualifier <dev@disqualifier.me>
2026-07-06 21:21:00 -04:00
dsql 68be8f7101 fix: _pid_alive treats an out-of-C-int-range pid as not alive (no OverflowError)
os.kill(pid, 0) raises OverflowError for a pid too large for a C int, which _pid_alive did
not catch; a foreign <base>.<huge-number>.tmp file in the store dir then made every stale-tmp
sweep (so every _save) crash. an over-range pid can't name a live process, so it's treated as
not-alive like ProcessLookupError.

Signed-off-by: disqualifier <dev@disqualifier.me>
2026-07-06 17:17:39 -04:00
dsql 98f7190f18 fix: _save no longer resolves the store's realpath twice per call
_save() already resolves target = realpath(self.file) for the write path;
_sweep_stale_tmp then independently recomputed the same realpath to derive its
match basename. _sweep_stale_tmp now takes that resolved basename as a
parameter instead of re-resolving it, cutting one syscall per save. the sweep
itself still runs on every save (unchanged) - gating it to first-save-per-
instance would change when a crash-orphaned tmp from before this process
started actually gets swept, so that part is left alone.

Signed-off-by: disqualifier <dev@disqualifier.me>
2026-07-06 00:18:16 -04:00
dsql ddf76d8241 fix: clear() and stale-tmp sweep resolve the real symlink target and match only <base>.<pid>.tmp
clear() removed the raw path, so on a symlinked store it unlinked only the
symlink while the real data file kept every value - clear() returned True
but the data resurfaced if the symlink was recreated. It now realpaths the
target first, mirroring _save()'s symlink-safe writes.

_sweep_stale_tmp globbed {base}.*.tmp, which crosses dots and can match an
unrelated neighbor file (deleting it outside the lib's own artifacts), and
interpolated the store's basename unescaped, so glob metacharacters in the
filename (e.g. state[prod].json) either missed the store's own orphans or
cross-matched a different store's. The sweep now lists the directory and
matches a re.escape'd regex anchored to the exact <base>.<pid>.tmp shape
_save() creates.

Signed-off-by: disqualifier <dev@disqualifier.me>
2026-07-03 19:05:46 -04:00
dsql 5e3eb4e704 refactor: derive __version__ from package metadata (single source)
Signed-off-by: disqualifier <dev@disqualifier.me>
2026-07-03 16:58:41 -04:00
dsql 8381b2ecbd docs: compress prose/module docstrings, em-dash->hyphen (de-bloat wave 1)
Signed-off-by: disqualifier <dev@disqualifier.me>
2026-07-03 00:10:19 -04:00
4 changed files with 57 additions and 50 deletions
+14 -8
View File
@@ -12,18 +12,18 @@ you `delete` or `clear` them.
`requirements.txt`:
```
aiokv @ git+ssh://git@git.rethinkstudios.io/rethink-public/aiokv.git@v0.2.0
aiokv @ git+ssh://git@git.rethinkstudios.io/rethink-public/aiokv.git@v0.2.2
```
Direct:
```bash
pip install "aiokv @ git+ssh://git@git.rethinkstudios.io/rethink-public/aiokv.git@v0.2.0"
pip install "aiokv @ git+ssh://git@git.rethinkstudios.io/rethink-public/aiokv.git@v0.2.2"
```
Requires `aiofiles` (pulled transitively).
Drop the `@v0.2.0` suffix from the line above to install the latest unpinned.
Drop the `@v0.2.2` suffix from the line above to install the latest unpinned.
## Usage
@@ -66,11 +66,17 @@ Writes are **atomic**: data is written to a temp file in the same directory and
written through rather than clobbered. A **process** crash mid-write leaves the previous
good file intact, and a reader never observes a partial file. (This is process-crash
safety, not power-loss durability — there's no `fsync`, so an OS/power failure could
still lose the last write; fine for reconstructible single-process state.) Orphaned
`.<pid>.tmp` files left by a hard crash of a *different, dead* process are swept on the
next save. A single `asyncio.Lock` guards every read and write, so concurrent operations
on one instance are consistent and no update is lost. All blocking filesystem calls and
JSON (de)serialization run via `asyncio.to_thread`, so nothing stalls the event loop.
still lose the last write; fine for reconstructible single-process state.) `clear()` is
symlink-safe the same way: it removes the realpath'd target, not the symlink itself, so
a symlinked store is genuinely erased rather than just having its symlink unlinked.
Orphaned `.<pid>.tmp` files left by a hard crash of a *different, dead* process are swept
on the next save; the sweep matches only the exact `<realpath basename>.<pid>.tmp` shape
it creates, so a differently-shaped neighbor file is never touched and a store filename
containing glob-like characters (e.g. `state[prod].json`) doesn't miss its own orphans or
cross-match another store's. A single `asyncio.Lock` guards every read and write, so
concurrent operations on one instance are consistent and no update is lost. All blocking
filesystem calls and JSON (de)serialization run via `asyncio.to_thread`, so nothing stalls
the event loop.
## Scope — read this
+1 -1
View File
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
[project]
name = "aiokv"
version = "0.2.0"
version = "1.0.0"
description = "Async file-backed key-value store for single-process local state — atomic writes, no TTL, config-free, installable."
requires-python = ">=3.10"
dependencies = [
+6 -1
View File
@@ -1,5 +1,10 @@
from importlib.metadata import version, PackageNotFoundError
from .aiokv import AioKV, aiocache
__version__ = "0.2.0"
try:
__version__ = version("aiokv")
except PackageNotFoundError:
__version__ = "0.0.0+unknown"
__all__ = ["AioKV", "aiocache"]
+36 -40
View File
@@ -1,41 +1,25 @@
"""
async file-backed key-value store for single-process local state
a persist-forever KV store (last-used-command, rate-limit timestamps, seen-ids,
simple bot state) backed by a JSON file. NOT a cache: no TTL, no expiry, no
eviction — values live until you delete or clear them.
a persist-forever KV store backed by a JSON file, not a cache (no TTL/expiry/eviction).
see README for usage and the full API.
from aiokv import AioKV
kv = AioKV("state.json")
await kv.set("last_seen", 12345)
await kv.set("ran_cleanup") # value omitted -> stores int(time.time())
when = await kv.get("ran_cleanup")
await kv.delete("last_seen")
when = await kv.get("last_seen")
durability: atomic writes (temp file + os.replace, symlink-safe), stale
`.<pid>.tmp` files from a hard crash are swept on save. this is process-crash
safety, NOT power-loss durability — no fsync, so an OS/power failure can still
lose the last write. a single asyncio.Lock guards every read and write, so
concurrent operations on one instance are consistent. JSON (de)serialization
runs off-loop via asyncio.to_thread so a large store never stalls other tasks.
scope: SINGLE-PROCESS, single-instance local state only. the lock is per-instance —
two AioKV instances (or two processes) pointing at the same file are NOT safe and
will clobber each other. for shared cross-process/cross-bot state, use a database
(e.g. mongo), not this.
config-free: the file path is passed at construction; nothing is read from a global
config. keys must be str (ValueError otherwise — JSON object keys are always
strings, so a non-str key would silently never round-trip). errors in delete/clear
are logged and swallowed (returning False); get/set raise on unexpected i/o so a
real failure is visible.
atomic writes are process-crash safe (temp file + os.replace) but NOT power-loss safe
(no fsync). the lock is per-instance only - two AioKV instances or processes on the
same file will clobber each other; this is single-process local state, not shared
cross-process storage.
"""
import os
import re
import json
import time
import glob
import asyncio
import logging
from typing import Any, Dict
@@ -87,11 +71,15 @@ class AioKV:
return False
async def clear(self) -> bool:
"""remove the backing file entirely; True on success or if already absent, False on error"""
"""remove the backing file entirely; True on success or if already absent, False on error
symlink-safe: removes the realpath'd target rather than the symlink itself, mirroring
_save()'s symlink-safe writes - a symlinked store's real data file is what gets erased"""
try:
async with self.lock:
target = await asyncio.to_thread(os.path.realpath, self.file)
try:
await asyncio.to_thread(os.remove, self.file)
await asyncio.to_thread(os.remove, target)
except FileNotFoundError:
pass
return True
@@ -106,7 +94,7 @@ class AioKV:
@staticmethod
def _check_key(key: str) -> None:
"""raise ValueError if key is not str (JSON object keys are always strings)"""
"""raise ValueError if key is not str"""
if not isinstance(key, str):
raise ValueError(f"aiokv keys must be str, got {type(key).__name__}")
@@ -128,12 +116,12 @@ class AioKV:
async def _save(self, cache: Dict[str, Any]) -> None:
"""write atomically: temp file in the same dir, then os.replace over the realpath'd
target (symlink-safe). process-crash safe (a crash mid-write leaves the prior good
file intact), NOT power-loss safe no fsync, so an OS/power failure can still lose
file intact), NOT power-loss safe - no fsync, so an OS/power failure can still lose
the last write (acceptable: reconstructible single-process state, not a db)"""
target = await asyncio.to_thread(os.path.realpath, self.file)
directory = os.path.dirname(target) or "."
await asyncio.to_thread(os.makedirs, directory, exist_ok=True)
await self._sweep_stale_tmp(directory)
await self._sweep_stale_tmp(directory, os.path.basename(target))
payload = await asyncio.to_thread(json.dumps, cache, allow_nan=False)
tmp = f"{target}.{os.getpid()}.tmp"
@@ -149,19 +137,27 @@ class AioKV:
log.exception("aiokv: failed to clean up temp file %s", tmp)
raise
async def _sweep_stale_tmp(self, directory: str) -> None:
"""remove orphaned .<pid>.tmp files left by a hard crash of a different, dead process"""
base = os.path.basename(await asyncio.to_thread(os.path.realpath, self.file))
pattern = os.path.join(directory, f"{base}.*.tmp")
for path in await asyncio.to_thread(glob.glob, pattern):
try:
pid = int(path.rsplit(".", 2)[-2])
except (ValueError, IndexError):
async def _sweep_stale_tmp(self, directory: str, base: str) -> None:
"""remove orphaned .<pid>.tmp files left by a hard crash of a different, dead process
matches only the exact shape _save() creates (<realpath basename>.<pid>.tmp), via a
re.escape'd regex over a directory listing rather than a raw glob - a glob pattern
both crosses unrelated dots (matching non-aiokv neighbors like foo.backup.999.tmp) and
mistreats glob metacharacters in the store's own filename (e.g. state[prod].json).
`base` is the realpath'd basename _save() already resolved - passed in rather than
re-resolved here so a save does one realpath call, not two."""
candidate = re.compile(rf"^{re.escape(base)}\.(\d+)\.tmp$")
names = await asyncio.to_thread(os.listdir, directory)
for name in names:
match = candidate.match(name)
if not match:
continue
pid = int(match.group(1))
if pid == os.getpid():
continue
if await asyncio.to_thread(self._pid_alive, pid):
continue
path = os.path.join(directory, name)
try:
await asyncio.to_thread(os.remove, path)
except FileNotFoundError:
@@ -174,13 +170,13 @@ class AioKV:
"""check whether pid refers to a live process, without permission to signal counting as alive"""
try:
os.kill(pid, 0)
except ProcessLookupError:
except (ProcessLookupError, OverflowError):
# OverflowError: a pid too large for a C int can't name a live process
return False
except PermissionError:
return True
return True
# back-compat: this lib was originally named aiocache; legacy call sites using
# `aiocache(...)` keep working via this alias. prefer AioKV in new code.
# back-compat: originally named aiocache; prefer AioKV in new code.
aiocache = AioKV