Files
aiokv/README.md
T
dsql 0af38e28dd perf: load the store once and serve reads from memory
T3-17: every get/set/delete re-read and re-parsed the whole file (get 40 ms,
set 63 ms at 100k keys). the file is now loaded once into memory; reads
return deep copies, set stores the value's json round-trip so memory always
equals a reload, and a failed write rolls the in-memory change back. the
stale-temp sweep runs once per instance. an external edit after first load
is no longer seen (single-process by contract). folds into 1.1.0.

Signed-off-by: disqualifier <dev@disqualifier.me>
2026-10-16 04:34:30 -04:00

114 lines
4.8 KiB
Markdown

# aiokv
Async file-backed key-value store for **single-process local state** — last-used
command, rate-limit timestamps, seen-IDs, simple bot state. Persists forever to a JSON
file with atomic writes.
It is **a KV store, not a cache**: no TTL, no expiry, no eviction. Values live until
you `delete` or `clear` them.
## Install
`requirements.txt`:
```
aiokv @ git+ssh://git@git.rethinkstudios.io/rethink-public/aiokv.git@v1.1.0
```
Direct:
```bash
pip install "aiokv @ git+ssh://git@git.rethinkstudios.io/rethink-public/aiokv.git@v1.1.0"
```
Requires `aiofiles` (pulled transitively).
Drop the `@v1.1.0` suffix from the line above to install the latest unpinned.
## Usage
```python
from aiokv import AioKV
kv = AioKV("state.json")
await kv.set("last_command", "ping")
await kv.set("seen_ran_cleanup") # value omitted -> stores int(time.time())
cmd = await kv.get("last_command") # "ping"
when = await kv.get("seen_ran_cleanup") # the unix timestamp
miss = await kv.get("absent", default=0) # 0
await kv.delete("last_command") # True (removed or absent)
everything = await kv.get_all() # {"seen_ran_cleanup": ...}
await kv.clear() # removes the file
```
The timestamp default (`set(key)` with no value) is for "mark that I saw/did X at
time T" — the common rate-limit / seen-ID pattern.
### Back-compat
This lib was originally `aiocache`. Legacy call sites keep working — `aiocache` is a
re-export alias of `AioKV`:
```python
from aiokv import aiocache # alias of AioKV; same API
kv = aiocache("state.json")
```
Prefer `AioKV` in new code.
## Memory model
The file is read once, on first use, and then served from memory: `get` / `get_all` don't
touch the disk, and each `set` / `delete` writes the whole store back atomically. `get` and
`get_all` return deep copies, and `set` stores the value's JSON round-trip (exactly what a
reload would give back), so mutating a value you passed in or got back never changes the
store. Because it is single-process, an edit to the file made by something else after the
first read is not seen (and is overwritten by the next write). Orphaned temp files are swept
once per instance, on its first save.
## Durability
Writes are **atomic**: data is written to a temp file in the same directory and
`os.replace()`d over the target's realpath — symlink-safe, so a symlinked store file is
written through rather than clobbered. A **process** crash mid-write leaves the previous
good file intact, and a reader never observes a partial file. (This is process-crash
safety, not power-loss durability — there's no `fsync`, so an OS/power failure could
still lose the last write; fine for reconstructible single-process state.) `clear()` is
symlink-safe the same way: it removes the realpath'd target, not the symlink itself, so
a symlinked store is genuinely erased rather than just having its symlink unlinked.
Orphaned `.<pid>.tmp` files left by a hard crash of a *different, dead* process are swept
on the next save; the sweep matches only the exact `<realpath basename>.<pid>.tmp` shape
it creates, so a differently-shaped neighbor file is never touched and a store filename
containing glob-like characters (e.g. `state[prod].json`) doesn't miss its own orphans or
cross-match another store's. A single `asyncio.Lock` guards every read and write, so
concurrent operations on one instance are consistent and no update is lost. All blocking
filesystem calls and JSON (de)serialization run via `asyncio.to_thread`, so nothing stalls
the event loop.
## Scope — read this
- **Single-process, single-instance only.** The lock is per-instance. Two `AioKV`
instances — or two processes — pointing at the same file are **not** safe; they will
clobber each other's writes. For shared cross-process / cross-bot state, that is a
database's job (e.g. our `mongo` lib), not aiokv.
- **Not a cache.** No TTL/expiry/eviction. If you need entries that age out, this is
the wrong tool.
## Error contract
- Keys must be `str` — `get` / `set` raise `ValueError` on a non-str key (JSON object
keys are always strings, so a non-str key would silently never round-trip).
- `get` / `set` / `get_all` raise on unexpected I/O. `_load` raises `JSONDecodeError`
on a truncated/corrupt file, and `ValueError` when the file holds valid JSON that
isn't an object (a bare list/number/string/null) — so corruption or a wrong-shaped
file is visible rather than silently masked. Non-finite floats (`NaN`/`Infinity`)
raise `ValueError` on `set` rather than persisting invalid JSON.
- `delete` / `clear` log the exception and return `False` on error, `True` otherwise.
## Versioning
Releases are tagged `vX.Y.Z`. The install line above pins a release; drop the `@vX.Y.Z` suffix to install the latest unpinned. Pin deliberately for reproducible installs.