dsql
ddf76d8241
clear() removed the raw path, so on a symlinked store it unlinked only the
symlink while the real data file kept every value - clear() returned True
but the data resurfaced if the symlink was recreated. It now realpaths the
target first, mirroring _save()'s symlink-safe writes.
_sweep_stale_tmp globbed {base}.*.tmp, which crosses dots and can match an
unrelated neighbor file (deleting it outside the lib's own artifacts), and
interpolated the store's basename unescaped, so glob metacharacters in the
filename (e.g. state[prod].json) either missed the store's own orphans or
cross-matched a different store's. The sweep now lists the directory and
matches a re.escape'd regex anchored to the exact <base>.<pid>.tmp shape
_save() creates.
Signed-off-by: disqualifier <dev@disqualifier.me>
104 lines
4.2 KiB
Markdown
104 lines
4.2 KiB
Markdown
# aiokv
|
|
|
|
Async file-backed key-value store for **single-process local state** — last-used
|
|
command, rate-limit timestamps, seen-IDs, simple bot state. Persists forever to a JSON
|
|
file with atomic writes.
|
|
|
|
It is **a KV store, not a cache**: no TTL, no expiry, no eviction. Values live until
|
|
you `delete` or `clear` them.
|
|
|
|
## Install
|
|
|
|
`requirements.txt`:
|
|
|
|
```
|
|
aiokv @ git+ssh://git@git.rethinkstudios.io/rethink-public/aiokv.git@v0.2.2
|
|
```
|
|
|
|
Direct:
|
|
|
|
```bash
|
|
pip install "aiokv @ git+ssh://git@git.rethinkstudios.io/rethink-public/aiokv.git@v0.2.2"
|
|
```
|
|
|
|
Requires `aiofiles` (pulled transitively).
|
|
|
|
Drop the `@v0.2.2` suffix from the line above to install the latest unpinned.
|
|
|
|
## Usage
|
|
|
|
```python
|
|
from aiokv import AioKV
|
|
|
|
kv = AioKV("state.json")
|
|
|
|
await kv.set("last_command", "ping")
|
|
await kv.set("seen_ran_cleanup") # value omitted -> stores int(time.time())
|
|
|
|
cmd = await kv.get("last_command") # "ping"
|
|
when = await kv.get("seen_ran_cleanup") # the unix timestamp
|
|
miss = await kv.get("absent", default=0) # 0
|
|
|
|
await kv.delete("last_command") # True (removed or absent)
|
|
everything = await kv.get_all() # {"seen_ran_cleanup": ...}
|
|
await kv.clear() # removes the file
|
|
```
|
|
|
|
The timestamp default (`set(key)` with no value) is for "mark that I saw/did X at
|
|
time T" — the common rate-limit / seen-ID pattern.
|
|
|
|
### Back-compat
|
|
|
|
This lib was originally `aiocache`. Legacy call sites keep working — `aiocache` is a
|
|
re-export alias of `AioKV`:
|
|
|
|
```python
|
|
from aiokv import aiocache # alias of AioKV; same API
|
|
kv = aiocache("state.json")
|
|
```
|
|
|
|
Prefer `AioKV` in new code.
|
|
|
|
## Durability
|
|
|
|
Writes are **atomic**: data is written to a temp file in the same directory and
|
|
`os.replace()`d over the target's realpath — symlink-safe, so a symlinked store file is
|
|
written through rather than clobbered. A **process** crash mid-write leaves the previous
|
|
good file intact, and a reader never observes a partial file. (This is process-crash
|
|
safety, not power-loss durability — there's no `fsync`, so an OS/power failure could
|
|
still lose the last write; fine for reconstructible single-process state.) `clear()` is
|
|
symlink-safe the same way: it removes the realpath'd target, not the symlink itself, so
|
|
a symlinked store is genuinely erased rather than just having its symlink unlinked.
|
|
Orphaned `.<pid>.tmp` files left by a hard crash of a *different, dead* process are swept
|
|
on the next save; the sweep matches only the exact `<realpath basename>.<pid>.tmp` shape
|
|
it creates, so a differently-shaped neighbor file is never touched and a store filename
|
|
containing glob-like characters (e.g. `state[prod].json`) doesn't miss its own orphans or
|
|
cross-match another store's. A single `asyncio.Lock` guards every read and write, so
|
|
concurrent operations on one instance are consistent and no update is lost. All blocking
|
|
filesystem calls and JSON (de)serialization run via `asyncio.to_thread`, so nothing stalls
|
|
the event loop.
|
|
|
|
## Scope — read this
|
|
|
|
- **Single-process, single-instance only.** The lock is per-instance. Two `AioKV`
|
|
instances — or two processes — pointing at the same file are **not** safe; they will
|
|
clobber each other's writes. For shared cross-process / cross-bot state, that is a
|
|
database's job (e.g. our `mongo` lib), not aiokv.
|
|
- **Not a cache.** No TTL/expiry/eviction. If you need entries that age out, this is
|
|
the wrong tool.
|
|
|
|
## Error contract
|
|
|
|
- Keys must be `str` — `get` / `set` raise `ValueError` on a non-str key (JSON object
|
|
keys are always strings, so a non-str key would silently never round-trip).
|
|
- `get` / `set` / `get_all` raise on unexpected I/O. `_load` raises `JSONDecodeError`
|
|
on a truncated/corrupt file, and `ValueError` when the file holds valid JSON that
|
|
isn't an object (a bare list/number/string/null) — so corruption or a wrong-shaped
|
|
file is visible rather than silently masked. Non-finite floats (`NaN`/`Infinity`)
|
|
raise `ValueError` on `set` rather than persisting invalid JSON.
|
|
- `delete` / `clear` log the exception and return `False` on error, `True` otherwise.
|
|
|
|
## Versioning
|
|
|
|
Releases are tagged `vX.Y.Z`. The install line above pins a release; drop the `@vX.Y.Z` suffix to install the latest unpinned. Pin deliberately for reproducible installs.
|