10 Commits
Author SHA1 Message Date
dsql 90f67cf9fa fix: bound 429 retry_after and retry connection errors without a proxy (v0.1.5)
A server-controlled 429 retry_after/Retry-After was slept verbatim with no
finiteness check or ceiling, so an inf or huge value (adversarial or a
ms-vs-s unit mismatch) could stall send() for hours outside max_retries
accounting. Non-finite values are now rejected and finite values clamped
to MAX_RETRY_AFTER (300s).

Connection/timeout errors with no proxy provider set failed one-shot,
contradicting the README's documented "normal retry" behavior and skipping
the single most retry-worthy failure class. They now raise the internal
_Retryable signal so commons.aretry retries them under max_retries, same
as a 5xx, while still returning a failed WebhookResult (never raising)
once retries are exhausted.

Signed-off-by: disqualifier <dev@disqualifier.me>
2026-07-02 17:08:14 -04:00
dsql 585a432ae0 chore: ignore .claude/ dir (CLAUDE.md now lives under .claude/)
Signed-off-by: disqualifier <dev@disqualifier.me>
2026-06-29 21:55:13 -04:00
dsql 3d86fc249c docs: clarify 429 honored-retry_after is additive with aretry backoff (F2)
note that an honored retry_after sleeps before aretry's own backoff, so the effective
wait is retry_after + backoff — only ever over-waits, never under-waits the server hint.

Signed-off-by: disqualifier <dev@disqualifier.me>
2026-06-29 21:34:38 -04:00
dsql ef20bc51f0 fix: never-raises net widened to unexpected exceptions; changelog/429 docs (v0.1.4)
M-2: _attempt caught only (aiohttp.ClientError, asyncio.TimeoutError); an unexpected
error escaping the attempt (closed injected session -> RuntimeError, malformed proxy url
-> ValueError) propagated out of send(), breaking the documented 'never raises on a send
failure' contract. add an outer catch-all in _send_loop converting any such exception to a
falsy WebhookResult(ok=False), logged at warning with exc_info.

aiowebhooks-F3: README 429 section + changelog were stale vs the v0.1.3 'retry any 429'
fix; added the no-parseable-wait-still-retries wording and v0.1.3/v0.1.4 changelog entries.

verified by execution: closed-session (RuntimeError) and bad-proxy (ValueError) controls
both fire and now return ok=False instead of raising.

Signed-off-by: disqualifier <dev@disqualifier.me>
2026-06-29 20:47:30 -04:00
dsql 28bad7fc7f docs: pin install line to release, note unpinned-latest option
Signed-off-by: disqualifier <dev@disqualifier.me>
2026-06-29 18:13:35 -04:00
dsql 1d3418a4be docs: show unpinned install line; note tag-pinning for reproducibility
Signed-off-by: disqualifier <dev@disqualifier.me>
2026-06-29 18:07:20 -04:00
dsql f3d2561bf9 fix: retry any 429, not just those with a parseable retry_after (v0.1.3)
treat every status==429 as retryable: sleep only when retry_after parses, but raise
_Retryable either way so aretry's backoff + max_retries cap engages. previously a 429
with no body retry_after and no Retry-After header (edge/Cloudflare/generic webhook)
returned a terminal ok=False with no retry, contradicting the documented retry-on-429.

Signed-off-by: disqualifier <dev@disqualifier.me>
2026-06-29 17:09:30 -04:00
dsql cd93e4f44f fix: per-call attempt counter so concurrent sends don't corrupt attempts
the attempt count used instance state (self._attempt_no), reset in _send_loop and incremented in _attempt; two concurrent send() calls on one Webhook interleaved, corrupting each other's WebhookResult.attempts. it is now a per-call mutable cell created in _send_loop and threaded into _attempt. verified under load: 400 concurrent sends, zero corrupted counts.

Signed-off-by: disqualifier <dev@disqualifier.me>
2026-06-28 17:46:20 -04:00
dsql bcf1f5511c fix: _proxy_string emits no stray colons for a portless proxy
a portless proxy url produced host::user:pass / host: (extra colons), breaking the identity match against the provider pool. the port colon is now omitted when there is no port, mirroring aioproxies' canonical key.

Signed-off-by: disqualifier <dev@disqualifier.me>
2026-06-28 17:18:28 -04:00
dsql aec0a5cc2b fix: remove dead clock param; pin commons v0.2.1 (v0.1.2)
the clock= constructor param was stored (self._clock) but never read — the 429
retry_after wait uses asyncio.sleep directly. it was dead code, and the CLAUDE.md
wrongly claimed it made 429 timing test-controllable. remove the param + the unused
time import, and correct the doc (tests patch commons.retry's sleep + sender.asyncio
.sleep, not a clock seam). bump the commons pin to v0.2.1 (retry attempts floor).

verified: clock param gone, constructs fine, 18/18 fix harness intact.
Signed-off-by: disqualifier <dev@disqualifier.me>
2026-06-28 16:16:13 -04:00
5 changed files with 191 additions and 60 deletions
+1 -1
View File
@@ -1,5 +1,5 @@
# claude
CLAUDE.md
.claude/
# python
__pycache__/
+80 -18
View File
@@ -13,13 +13,16 @@ send to the core — inheriting rotation, proxy, retry, and result for free.
## Install
```
aiowebhooks @ git+ssh://git@git.rethinkstudios.io/rethink-public/aiowebhooks.git@v0.1.0
aiowebhooks @ git+ssh://git@git.rethinkstudios.io/rethink-public/aiowebhooks.git@v0.1.5
# discord embeds / identity helpers need the extra:
aiowebhooks[discord] @ git+ssh://git@git.rethinkstudios.io/rethink-public/aiowebhooks.git@v0.1.0
aiowebhooks[discord] @ git+ssh://git@git.rethinkstudios.io/rethink-public/aiowebhooks.git@v0.1.5
```
The base pulls `aiohttp`. Only `aiowebhooks[discord]` adds `discord.py` (>=2.3,
mainline — not discord.py-self), and only for `DiscordWebhook`.
The base pulls `aiohttp` and `commons` (for the retry/backoff engine). Only
`aiowebhooks[discord]` adds `discord.py` (>=2.3, mainline — not discord.py-self), and
only for `DiscordWebhook`.
Drop the `@v0.1.5` suffix from the line above to install the latest unpinned.
## Core sender
@@ -47,7 +50,9 @@ Webhook(
```
Inject a shared `session` for throughput (one session per process); without one, each
send opens and closes its own.
send opens and closes its own. A single `Webhook` instance is safe to drive from many
concurrent `send()` calls — each call tracks its own attempt count, so concurrent sends
don't corrupt each other's `WebhookResult.attempts`.
## WebhookResult
@@ -65,12 +70,23 @@ result.proxy # canonical proxy string used (host:port:user:pass / host:port)
## Retry & rate limits
- **429** — waits the `retry_after` from the body first (Discord sends seconds), then
the `Retry-After` header, then retries; capped by `max_retries`.
- **5xx** — retried, capped by `max_retries`.
Status retries run through `commons.aretry` (exponential backoff + cap):
- **429** — always retried, capped by `max_retries`. When a wait is parseable (body
`retry_after` first — Discord sends seconds — then the `Retry-After` header) it sleeps
that value before retrying; a 429 with no parseable wait (edge/Cloudflare/generic
webhook) still retries under aretry's backoff rather than failing one-shot. A
non-finite value (`inf`/`nan`) is rejected outright and a finite value is clamped to
`MAX_RETRY_AFTER` (300s) — a server-controlled wait can never stall a `send()` past
that ceiling, however large or malformed the value it sends.
- **5xx** — retried with exponential backoff, capped by `max_retries`.
- **connection/timeout errors** — retried with exponential backoff, capped by
`max_retries`, same as a 5xx (with no proxy provider; see below for the
proxy-rotation path).
- **4xx** (other than 429) — fails immediately (no retry), returned as `ok=False`.
Exceeding a cap returns a failed result rather than looping.
Exceeding a cap returns a failed result rather than looping — and the result carries the
**real** last status/body (not a synthetic placeholder).
## Proxy rotation (optional, duck-typed)
@@ -91,9 +107,11 @@ result = await wh.send(payload) # sends through pm.get(); on a timeout/connect
```
On a timeout/connection error the current proxy is burned and the next is tried, up to
`max_proxy_retries`. A provider `ProxiesExhaustedError` (or hitting the cap) returns a
failed result — never an infinite loop. With no provider, a timeout just fails after
normal retry.
`max_proxy_retries`. Hitting the cap, or **any exception from the provider's
`get()`/`burn()`** (the provider is duck-typed and never imported, so its exception
types can't be caught by class), returns a failed result — never an infinite loop, never
an escape. With no provider, a timeout/connection error is retried directly under
`max_retries` (the same cap and backoff a 5xx gets) instead of burn+rotate.
## Discord (`aiowebhooks[discord]`)
@@ -124,12 +142,56 @@ Without the extra installed, importing `aiowebhooks` still works; constructing o
- Every send returns a `WebhookResult`; the core never raises on a send failure and
never prints. Callers check `result.ok`.
- v0.1.0 is JSON-only: **files/attachments, `tts`, and `allowed_mentions` are out**
(deliberate scope cut, addable later). The Discord surface is content + embeds +
identity.
- v0.1.0 rotation is round-robin only; try-next-on-failure across URLs is a later
feature.
- JSON-only: **files/attachments, `tts`, and `allowed_mentions` are out** (deliberate
scope cut, addable later). The Discord surface is content + embeds + identity.
- Rotation is round-robin only; try-next-on-failure across URLs is a later feature.
## Changelog
### v0.1.5
- **429 `retry_after` bounded:** a non-finite server-controlled wait (`inf`/`nan`, from
the body `retry_after` or the `Retry-After` header) is now rejected outright, and a
finite wait is clamped to `MAX_RETRY_AFTER` (300s). Previously a bare `float()` parse
slept the value verbatim, unbounded and outside `max_retries` accounting — an
adversarial or ms-vs-s-misconfigured server could stall a `send()` for hours.
- **Connection/timeout errors now retry without a proxy provider:** previously
`aiohttp.ClientError`/`asyncio.TimeoutError` with no `proxies=` set failed one-shot,
contradicting both the README and the single most retry-worthy failure class. Now it
retries under `commons.aretry`'s backoff, capped by `max_retries`, same as a 5xx —
and still returns `ok=False` (never raises) once retries are exhausted. Proxy-rotation
behavior (burn + rotate, capped by `max_proxy_retries`) is unchanged when a provider
is set.
### v0.1.4
- **Never-raises net widened:** an unexpected exception that escapes a send attempt (a
closed injected session → `RuntimeError`, a malformed proxy URL → `ValueError`) now
converts to a falsy `WebhookResult(ok=False, ...)` instead of propagating out of
`send()`, restoring the documented contract for those edge triggers.
### v0.1.3
- **429 always retries:** every `429` is now retryable under aretry's backoff + cap, not
only those with a parseable `retry_after`. A 429 with no body `retry_after` and no
`Retry-After` header (edge/Cloudflare/generic webhook) previously failed one-shot.
### v0.1.2
- Removed a dead `clock` constructor param (it was stored but never used). Pinned
`commons` to v0.2.1.
### v0.1.1
- **Never-raises contract hardened:** an error from a duck-typed proxy provider's
`get()`/`burn()` (e.g. `aioproxies.burn()` raising `ValueError` for an unknown proxy)
used to escape `send()`. Now any provider exception is caught and converted to a
failed result.
- **Retry via `commons.aretry`:** 429/5xx retry moved onto the shared backoff engine —
5xx now backs off (was a tight no-backoff loop), and exhausted retries return the
**real** last status/body instead of a synthetic placeholder. Adds a `commons`
dependency.
## Versioning
Tagged `vX.Y.Z`. Pin the tag.
Releases are tagged `vX.Y.Z`. The install line above pins a release; drop the `@vX.Y.Z` suffix to install the latest unpinned. Pin deliberately for reproducible installs.
+2 -2
View File
@@ -4,12 +4,12 @@ build-backend = "hatchling.build"
[project]
name = "aiowebhooks"
version = "0.1.1"
version = "0.1.5"
description = "async webhook sender (aiohttp) with round-robin urls, retry, and proxy rotation; optional discord.py embeds"
requires-python = ">=3.10"
dependencies = [
"aiohttp>=3.9",
"commons @ git+ssh://git@git.rethinkstudios.io/rethink-public/commons.git@v0.2.0",
"commons @ git+ssh://git@git.rethinkstudios.io/rethink-public/commons.git@v0.2.1",
]
[project.optional-dependencies]
+1 -1
View File
@@ -21,4 +21,4 @@ from .sender import Webhook
__all__ = ["Webhook", "WebhookResult", "WebhookError", "NoUrlsError"]
__version__ = "0.1.1"
__version__ = "0.1.4"
+97 -28
View File
@@ -1,14 +1,20 @@
"""core async webhook sender (aiohttp only, no discord knowledge).
`Webhook` posts a JSON dict to a url (or round-robins a pool), handling 429/5xx
retries and optional proxy rotation, and always returns a `WebhookResult` — it
never raises on a send failure. the discord layer builds payloads and delegates
the actual POST here so it inherits rotation / proxy / retry / result.
retries, connection/timeout retries, and optional proxy rotation, and always
returns a `WebhookResult` — it never raises on a send failure. the discord layer
builds payloads and delegates the actual POST here so it inherits rotation / proxy
/ retry / result.
a 429's server-controlled `retry_after` wait is bounded: non-finite values
(inf/nan) are rejected and finite values are clamped to `MAX_RETRY_AFTER`, so an
adversarial or misconfigured server can never stall a send() past a bounded
ceiling.
"""
import asyncio
import logging
import time
import math
from typing import Dict, List, Optional, Union
from urllib.parse import unquote, urlsplit
@@ -20,6 +26,17 @@ from .result import WebhookResult
log = logging.getLogger(__name__)
MAX_RETRY_AFTER = 300.0
"""ceiling (seconds) honored from a server-controlled 429 retry_after/Retry-After
a server can send an arbitrarily large or non-finite wait (float('inf'), a
Cloudflare-scale value, or a ms-vs-s unit mismatch turning 5s into 5000s); sleeping
that verbatim would stall a send() far past its ClientTimeout and outside
max_retries accounting, breaking the 'always returns a WebhookResult' contract.
non-finite values are rejected outright (treated as unparseable); finite values are
clamped to this ceiling.
"""
class _Retryable(Exception):
"""internal signal: a retryable HTTP status (429/5xx); carries the response
@@ -50,12 +67,12 @@ def _proxy_string(proxies_dict: Optional[Dict[str, str]]) -> Optional[str]:
if host is None:
return None
host = host.lower()
port = str(parts.port) if parts.port is not None else ""
hostport = f"{host}:{parts.port}" if parts.port is not None else host
if parts.username:
user = unquote(parts.username)
password = unquote(parts.password) if parts.password is not None else ""
return f"{host}:{port}:{user}:{password}"
return f"{host}:{port}"
return f"{hostport}:{user}:{password}"
return hostport
except ValueError:
return None
@@ -72,7 +89,6 @@ class Webhook:
timeout: float = 15,
max_retries: int = 3,
max_proxy_retries: int = 3,
clock=time.monotonic,
):
self._urls = [urls] if isinstance(urls, str) else list(urls)
if not self._urls:
@@ -82,7 +98,6 @@ class Webhook:
self.timeout = timeout
self.max_retries = max_retries
self.max_proxy_retries = max_proxy_retries
self._clock = clock
self._index = 0
def _next_url(self) -> str:
@@ -93,18 +108,27 @@ class Webhook:
@staticmethod
def _retry_after(status: int, headers, body) -> Optional[float]:
"""seconds to wait on a 429, from body retry_after then Retry-After header"""
"""seconds to wait on a 429, from body retry_after then Retry-After header
non-finite values (inf/nan) are rejected as unparseable; finite values are
clamped to MAX_RETRY_AFTER so a server-controlled wait can never stall a
send() past a bounded ceiling.
"""
if status != 429:
return None
if isinstance(body, dict) and body.get("retry_after") is not None:
try:
return float(body["retry_after"])
value = float(body["retry_after"])
if math.isfinite(value):
return max(0.0, min(value, MAX_RETRY_AFTER))
except (TypeError, ValueError):
pass
header = headers.get("Retry-After") if headers else None
if header is not None:
try:
return float(header)
value = float(header)
if math.isfinite(value):
return max(0.0, min(value, MAX_RETRY_AFTER))
except (TypeError, ValueError):
pass
return None
@@ -113,9 +137,10 @@ class Webhook:
"""post a json payload to the next pool url; always returns a result
handles 429 (wait + retry, capped by max_retries), 5xx (retry, capped), and
timeout/connection errors with optional proxy rotation (burn + next, capped
by max_proxy_retries). 4xx other than 429 fail immediately. never raises on a
send failure.
timeout/connection errors: with a proxy provider, burn + rotate to the next
proxy (capped by max_proxy_retries) before failing; without one, retried
directly under max_retries like a 5xx. 4xx other than 429 fail immediately.
never raises on a send failure.
"""
url = self._next_url()
session = self._session
@@ -133,39 +158,59 @@ class Webhook:
) -> WebhookResult:
"""status-retry (via commons.aretry) wrapping proxy rotation; never raises
commons.aretry owns the 429/5xx backoff schedule + retry cap (max_retries),
retrying on the internal _Retryable signal. on exhaustion it re-raises the
last _Retryable, whose carried result is the REAL last response (not a
synthetic status-0). proxy rotation on connection errors lives inside the
attempt and is capped separately.
commons.aretry owns the 429/5xx/no-provider-connection-error backoff schedule
+ retry cap (max_retries), retrying on the internal _Retryable signal. on
exhaustion it re-raises the last _Retryable, whose carried result is the REAL
last response (not a synthetic status-0). proxy rotation on connection errors
lives inside the attempt and is capped separately (max_proxy_retries); once a
provider's rotation cap is hit that failure is terminal, not retried again
via aretry.
"""
self._attempt_no = 0
counter = [0]
try:
return await aretry(
lambda: self._attempt(session, url, payload),
lambda: self._attempt(session, url, payload, counter),
attempts=self.max_retries + 1,
on=(_Retryable,),
)
except _Retryable as exhausted:
return exhausted.result
except Exception as error:
# never-raises safety net: an unexpected error that escapes the attempt (a
# closed injected session -> RuntimeError, a malformed proxy url -> ValueError,
# anything not aiohttp.ClientError/TimeoutError) must come back as a failed
# result, not propagate out of send()
log.warning("webhook send failed unexpectedly on %s: %s", url, error, exc_info=True)
return WebhookResult(
ok=False, status=None, url=url, attempts=counter[0] or 1,
error=f"{type(error).__name__}: {error}",
)
async def _attempt(
self, session: aiohttp.ClientSession, url: str, payload: Dict
self, session: aiohttp.ClientSession, url: str, payload: Dict, counter: List[int]
) -> WebhookResult:
"""one logical send: proxy rotation + a single POST; may raise _Retryable
raises _Retryable (carrying the real response) on a 429/5xx so the caller's
aretry applies backoff; honors an explicit 429 retry_after by sleeping it
before signalling. returns a final WebhookResult on success or a terminal
(non-retryable) failure — never lets a provider/connection error escape.
before signalling. a connection/timeout error also raises _Retryable when no
proxy provider is set, so it retries under aretry's backoff + max_retries
instead of failing one-shot; with a provider, the same error first drives
burn+rotate up to max_proxy_retries, and only becomes terminal (not retried
via aretry) once that cap is hit. returns a final WebhookResult on success or
a terminal (non-retryable) failure — never lets a provider error escape.
`counter` is a per-call mutable cell ([0]) owned by the calling `_send_loop`,
so the attempt count is local to one `send()` and concurrent sends on the
same instance don't corrupt each other's tally.
"""
timeout = aiohttp.ClientTimeout(total=self.timeout)
last_proxy: Optional[str] = None
proxy_tries = 0
while True:
self._attempt_no += 1
attempts = self._attempt_no
counter[0] += 1
attempts = counter[0]
proxy_url = None
if self._proxies is not None:
try:
@@ -200,17 +245,30 @@ class Webhook:
error=f"http {status}", response=body, proxy=last_proxy,
)
if status == 429:
# every 429 is retryable; honor an explicit retry_after by
# sleeping it, but a 429 with no parseable wait (edge/Cloudflare/
# generic webhook) still retries under aretry's backoff + cap.
# note: aretry ALSO sleeps its backoff between retries, so an
# honored retry_after is additive (retry_after + backoff) — this
# only ever over-waits, never under-waits the server's hint.
# non-finite values (inf/nan) are rejected and finite values are
# clamped to MAX_RETRY_AFTER by _retry_after, so a server-controlled
# wait can never stall this attempt past a bounded ceiling
wait = self._retry_after(status, resp.headers, body)
if wait is not None:
log.warning("webhook 429 on %s; honoring retry_after %.3fs", url, wait)
await asyncio.sleep(wait)
else:
log.warning("webhook 429 on %s; no retry_after, backing off", url)
raise _Retryable(result)
if status >= 500:
raise _Retryable(result)
return result
except (aiohttp.ClientError, asyncio.TimeoutError) as error:
if self._proxies is not None and proxy_tries < self.max_proxy_retries:
if self._proxies is not None:
if proxy_tries < self.max_proxy_retries:
if self._burn(last_proxy):
proxy_tries += 1
continue
@@ -222,6 +280,17 @@ class Webhook:
ok=False, status=None, url=url, attempts=attempts,
error=f"{type(error).__name__}: {error}", proxy=last_proxy,
)
# no proxy provider: a connection/timeout error is the single most
# retry-worthy failure class (the 5xx equivalent already retries under
# aretry) — raise _Retryable so aretry applies backoff + max_retries
# instead of failing one-shot. aretry re-raises this same _Retryable on
# exhaustion, and _send_loop's `except _Retryable` returns its carried
# result, so the never-raises / always-returns-a-WebhookResult contract
# still holds after retries run out.
raise _Retryable(WebhookResult(
ok=False, status=None, url=url, attempts=attempts,
error=f"{type(error).__name__}: {error}", proxy=last_proxy,
))
def _burn(self, proxy: Optional[str]) -> bool:
"""burn the current proxy; return False if it can't be rotated