dsql
5c2fdd80f9
_load() masked permission/IO errors as an empty store via os.path.exists();
now raises FileNotFoundError only, propagating real failures. Non-str keys
silently never round-tripped through get() since JSON object keys are always
strings; set()/get() now raise ValueError on a non-str key. _save() clobbered
a symlinked store file with os.replace(); now realpaths the target first.
json.dumps() allows NaN/Infinity by default, producing invalid JSON for
strict readers; now allow_nan=False. A whitespace-only file raised while a
zero-byte file returned {}; both now return {}. Orphaned .<pid>.tmp files
from a hard crash of a dead process are swept on save. JSON (de)serialization
now runs via asyncio.to_thread instead of on the event loop.
Signed-off-by: disqualifier <dev@disqualifier.me>
98 lines
3.8 KiB
Markdown
98 lines
3.8 KiB
Markdown
# aiokv
|
|
|
|
Async file-backed key-value store for **single-process local state** — last-used
|
|
command, rate-limit timestamps, seen-IDs, simple bot state. Persists forever to a JSON
|
|
file with atomic writes.
|
|
|
|
It is **a KV store, not a cache**: no TTL, no expiry, no eviction. Values live until
|
|
you `delete` or `clear` them.
|
|
|
|
## Install
|
|
|
|
`requirements.txt`:
|
|
|
|
```
|
|
aiokv @ git+ssh://git@git.rethinkstudios.io/rethink-public/aiokv.git@v0.2.0
|
|
```
|
|
|
|
Direct:
|
|
|
|
```bash
|
|
pip install "aiokv @ git+ssh://git@git.rethinkstudios.io/rethink-public/aiokv.git@v0.2.0"
|
|
```
|
|
|
|
Requires `aiofiles` (pulled transitively).
|
|
|
|
Drop the `@v0.2.0` suffix from the line above to install the latest unpinned.
|
|
|
|
## Usage
|
|
|
|
```python
|
|
from aiokv import AioKV
|
|
|
|
kv = AioKV("state.json")
|
|
|
|
await kv.set("last_command", "ping")
|
|
await kv.set("seen_ran_cleanup") # value omitted -> stores int(time.time())
|
|
|
|
cmd = await kv.get("last_command") # "ping"
|
|
when = await kv.get("seen_ran_cleanup") # the unix timestamp
|
|
miss = await kv.get("absent", default=0) # 0
|
|
|
|
await kv.delete("last_command") # True (removed or absent)
|
|
everything = await kv.get_all() # {"seen_ran_cleanup": ...}
|
|
await kv.clear() # removes the file
|
|
```
|
|
|
|
The timestamp default (`set(key)` with no value) is for "mark that I saw/did X at
|
|
time T" — the common rate-limit / seen-ID pattern.
|
|
|
|
### Back-compat
|
|
|
|
This lib was originally `aiocache`. Legacy call sites keep working — `aiocache` is a
|
|
re-export alias of `AioKV`:
|
|
|
|
```python
|
|
from aiokv import aiocache # alias of AioKV; same API
|
|
kv = aiocache("state.json")
|
|
```
|
|
|
|
Prefer `AioKV` in new code.
|
|
|
|
## Durability
|
|
|
|
Writes are **atomic**: data is written to a temp file in the same directory and
|
|
`os.replace()`d over the target's realpath — symlink-safe, so a symlinked store file is
|
|
written through rather than clobbered. A **process** crash mid-write leaves the previous
|
|
good file intact, and a reader never observes a partial file. (This is process-crash
|
|
safety, not power-loss durability — there's no `fsync`, so an OS/power failure could
|
|
still lose the last write; fine for reconstructible single-process state.) Orphaned
|
|
`.<pid>.tmp` files left by a hard crash of a *different, dead* process are swept on the
|
|
next save. A single `asyncio.Lock` guards every read and write, so concurrent operations
|
|
on one instance are consistent and no update is lost. All blocking filesystem calls and
|
|
JSON (de)serialization run via `asyncio.to_thread`, so nothing stalls the event loop.
|
|
|
|
## Scope — read this
|
|
|
|
- **Single-process, single-instance only.** The lock is per-instance. Two `AioKV`
|
|
instances — or two processes — pointing at the same file are **not** safe; they will
|
|
clobber each other's writes. For shared cross-process / cross-bot state, that is a
|
|
database's job (e.g. our `mongo` lib), not aiokv.
|
|
- **Not a cache.** No TTL/expiry/eviction. If you need entries that age out, this is
|
|
the wrong tool.
|
|
|
|
## Error contract
|
|
|
|
- Keys must be `str` — `get` / `set` raise `ValueError` on a non-str key (JSON object
|
|
keys are always strings, so a non-str key would silently never round-trip).
|
|
- `get` / `set` / `get_all` raise on unexpected I/O. `_load` raises `JSONDecodeError`
|
|
on a truncated/corrupt file, and `ValueError` when the file holds valid JSON that
|
|
isn't an object (a bare list/number/string/null) — so corruption or a wrong-shaped
|
|
file is visible rather than silently masked. Non-finite floats (`NaN`/`Infinity`)
|
|
raise `ValueError` on `set` rather than persisting invalid JSON.
|
|
- `delete` / `clear` log the exception and return `False` on error, `True` otherwise.
|
|
|
|
## Versioning
|
|
|
|
Releases are tagged `vX.Y.Z`. The install line above pins a release; drop the `@vX.Y.Z` suffix to install the latest unpinned. Pin deliberately for reproducible installs.
|