feat: retry/backoff module (commons v0.2.0)
add commons.retry: exponential-backoff retry as sync `retry` and async `aretry`, each usable as a call form or a decorator. backoff is min(backoff*factor**n, max_backoff) with optional full jitter; the schedule is a pure generator so it tests without real sleeps (sleep + rand injectable). `on=` narrows retryable exception types, `give_up(exc)` stops early on a non-retryable error, and after attempts are exhausted the LAST exception is re-raised (fail loud, never swallowed). de-dups retry logic written 3x divergently (aiowebhooks 429/5xx, aioproxies burn/rotate, aiomail reconnect). Signed-off-by: disqualifier <dev@disqualifier.me>
This commit is contained in:
@@ -5,15 +5,16 @@ Small sync helpers shared across projects. Base is stdlib only — **no dependen
|
||||
- `timing` — unix-timestamp deltas + timezone-aware datetime conversions
|
||||
- `paths` — nested dict/list access by dotted path
|
||||
- `masking` — display masking for cards / cvv / tokens
|
||||
- `retry` — exponential-backoff retry, sync (`retry`) and async (`aretry`)
|
||||
- `addr` — ip/address tooling: pure stdlib ip utils in base, async geo lookups
|
||||
behind the `commons[addr]` extra
|
||||
|
||||
## Install
|
||||
|
||||
```
|
||||
commons @ git+ssh://git@git.rethinkstudios.io/rethink-public/commons.git@v0.1.0
|
||||
commons @ git+ssh://git@git.rethinkstudios.io/rethink-public/commons.git@v0.2.0
|
||||
# async address/geo lookups (fetch_ip / ip_location / fetch_location) need the extra:
|
||||
commons[addr] @ git+ssh://git@git.rethinkstudios.io/rethink-public/commons.git@v0.1.0
|
||||
commons[addr] @ git+ssh://git@git.rethinkstudios.io/rethink-public/commons.git@v0.2.0
|
||||
```
|
||||
|
||||
The base install pulls **nothing** (stdlib). Only `commons[addr]` adds `aiohttp`, and
|
||||
@@ -112,6 +113,39 @@ phantom("abcdef1234567890") # "abcdef...7890"
|
||||
provider("4111111111111111") # "VISA" (VISA/MC/AMEX/UPAY/DISC/JCB/DNRS/UNKW)
|
||||
```
|
||||
|
||||
## retry
|
||||
|
||||
Exponential-backoff retry, sync (`retry`) and async (`aretry`). Each works as a **call
|
||||
form** or a **decorator**, with the same kwargs. After the attempts are exhausted the
|
||||
**last exception is re-raised** — it never swallows or returns a default.
|
||||
|
||||
```python
|
||||
from commons import retry, aretry
|
||||
|
||||
# call form
|
||||
rows = retry(lambda: read_db(), attempts=5, on=(IOError,))
|
||||
data = await aretry(lambda: fetch(url), attempts=3, backoff=0.5, on=(TimeoutError,))
|
||||
|
||||
# decorator form (same kwargs)
|
||||
@aretry(attempts=4, backoff=0.5, factor=2.0, on=(ConnectionError,))
|
||||
async def pull():
|
||||
...
|
||||
```
|
||||
|
||||
Knobs: `attempts` (total tries), `backoff` / `factor` / `max_backoff` (delay is
|
||||
`min(backoff * factor**n, max_backoff)`), `jitter` (full jitter, on by default),
|
||||
`on=` (tuple of retryable exception types), and `give_up=lambda exc: ...` to stop early
|
||||
on a non-retryable error (e.g. a 400 vs a 429):
|
||||
|
||||
```python
|
||||
# retry 429/5xx but give up immediately on a 4xx
|
||||
await aretry(send, attempts=4, on=(HTTPError,),
|
||||
give_up=lambda e: 400 <= e.status < 500 and e.status != 429)
|
||||
```
|
||||
|
||||
Each retry is logged (emit-only). `sleep=` and `rand=` are injectable for deterministic
|
||||
tests (no real waits).
|
||||
|
||||
## addr
|
||||
|
||||
IP/address tooling, exposed as a submodule. The pure `ip` utilities ship in the base
|
||||
|
||||
Reference in New Issue
Block a user