1 Commits
Author SHA1 Message Date
dsql 90f67cf9fa fix: bound 429 retry_after and retry connection errors without a proxy (v0.1.5)
A server-controlled 429 retry_after/Retry-After was slept verbatim with no
finiteness check or ceiling, so an inf or huge value (adversarial or a
ms-vs-s unit mismatch) could stall send() for hours outside max_retries
accounting. Non-finite values are now rejected and finite values clamped
to MAX_RETRY_AFTER (300s).

Connection/timeout errors with no proxy provider set failed one-shot,
contradicting the README's documented "normal retry" behavior and skipping
the single most retry-worthy failure class. They now raise the internal
_Retryable signal so commons.aretry retries them under max_retries, same
as a 5xx, while still returning a failed WebhookResult (never raising)
once retries are exhausted.

Signed-off-by: disqualifier <dev@disqualifier.me>
2026-07-02 17:08:14 -04:00
3 changed files with 101 additions and 30 deletions
+27 -5
View File
@@ -13,16 +13,16 @@ send to the core — inheriting rotation, proxy, retry, and result for free.
## Install ## Install
``` ```
aiowebhooks @ git+ssh://git@git.rethinkstudios.io/rethink-public/aiowebhooks.git@v0.1.4 aiowebhooks @ git+ssh://git@git.rethinkstudios.io/rethink-public/aiowebhooks.git@v0.1.5
# discord embeds / identity helpers need the extra: # discord embeds / identity helpers need the extra:
aiowebhooks[discord] @ git+ssh://git@git.rethinkstudios.io/rethink-public/aiowebhooks.git@v0.1.4 aiowebhooks[discord] @ git+ssh://git@git.rethinkstudios.io/rethink-public/aiowebhooks.git@v0.1.5
``` ```
The base pulls `aiohttp` and `commons` (for the retry/backoff engine). Only The base pulls `aiohttp` and `commons` (for the retry/backoff engine). Only
`aiowebhooks[discord]` adds `discord.py` (>=2.3, mainline — not discord.py-self), and `aiowebhooks[discord]` adds `discord.py` (>=2.3, mainline — not discord.py-self), and
only for `DiscordWebhook`. only for `DiscordWebhook`.
Drop the `@v0.1.4` suffix from the line above to install the latest unpinned. Drop the `@v0.1.5` suffix from the line above to install the latest unpinned.
## Core sender ## Core sender
@@ -75,8 +75,14 @@ Status retries run through `commons.aretry` (exponential backoff + cap):
- **429** — always retried, capped by `max_retries`. When a wait is parseable (body - **429** — always retried, capped by `max_retries`. When a wait is parseable (body
`retry_after` first — Discord sends seconds — then the `Retry-After` header) it sleeps `retry_after` first — Discord sends seconds — then the `Retry-After` header) it sleeps
that value before retrying; a 429 with no parseable wait (edge/Cloudflare/generic that value before retrying; a 429 with no parseable wait (edge/Cloudflare/generic
webhook) still retries under aretry's backoff rather than failing one-shot. webhook) still retries under aretry's backoff rather than failing one-shot. A
non-finite value (`inf`/`nan`) is rejected outright and a finite value is clamped to
`MAX_RETRY_AFTER` (300s) — a server-controlled wait can never stall a `send()` past
that ceiling, however large or malformed the value it sends.
- **5xx** — retried with exponential backoff, capped by `max_retries`. - **5xx** — retried with exponential backoff, capped by `max_retries`.
- **connection/timeout errors** — retried with exponential backoff, capped by
`max_retries`, same as a 5xx (with no proxy provider; see below for the
proxy-rotation path).
- **4xx** (other than 429) — fails immediately (no retry), returned as `ok=False`. - **4xx** (other than 429) — fails immediately (no retry), returned as `ok=False`.
Exceeding a cap returns a failed result rather than looping — and the result carries the Exceeding a cap returns a failed result rather than looping — and the result carries the
@@ -104,7 +110,8 @@ On a timeout/connection error the current proxy is burned and the next is tried,
`max_proxy_retries`. Hitting the cap, or **any exception from the provider's `max_proxy_retries`. Hitting the cap, or **any exception from the provider's
`get()`/`burn()`** (the provider is duck-typed and never imported, so its exception `get()`/`burn()`** (the provider is duck-typed and never imported, so its exception
types can't be caught by class), returns a failed result — never an infinite loop, never types can't be caught by class), returns a failed result — never an infinite loop, never
an escape. With no provider, a timeout just fails after normal retry. an escape. With no provider, a timeout/connection error is retried directly under
`max_retries` (the same cap and backoff a 5xx gets) instead of burn+rotate.
## Discord (`aiowebhooks[discord]`) ## Discord (`aiowebhooks[discord]`)
@@ -141,6 +148,21 @@ Without the extra installed, importing `aiowebhooks` still works; constructing o
## Changelog ## Changelog
### v0.1.5
- **429 `retry_after` bounded:** a non-finite server-controlled wait (`inf`/`nan`, from
the body `retry_after` or the `Retry-After` header) is now rejected outright, and a
finite wait is clamped to `MAX_RETRY_AFTER` (300s). Previously a bare `float()` parse
slept the value verbatim, unbounded and outside `max_retries` accounting — an
adversarial or ms-vs-s-misconfigured server could stall a `send()` for hours.
- **Connection/timeout errors now retry without a proxy provider:** previously
`aiohttp.ClientError`/`asyncio.TimeoutError` with no `proxies=` set failed one-shot,
contradicting both the README and the single most retry-worthy failure class. Now it
retries under `commons.aretry`'s backoff, capped by `max_retries`, same as a 5xx —
and still returns `ok=False` (never raises) once retries are exhausted. Proxy-rotation
behavior (burn + rotate, capped by `max_proxy_retries`) is unchanged when a provider
is set.
### v0.1.4 ### v0.1.4
- **Never-raises net widened:** an unexpected exception that escapes a send attempt (a - **Never-raises net widened:** an unexpected exception that escapes a send attempt (a
+1 -1
View File
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
[project] [project]
name = "aiowebhooks" name = "aiowebhooks"
version = "0.1.4" version = "0.1.5"
description = "async webhook sender (aiohttp) with round-robin urls, retry, and proxy rotation; optional discord.py embeds" description = "async webhook sender (aiohttp) with round-robin urls, retry, and proxy rotation; optional discord.py embeds"
requires-python = ">=3.10" requires-python = ">=3.10"
dependencies = [ dependencies = [
+73 -24
View File
@@ -1,13 +1,20 @@
"""core async webhook sender (aiohttp only, no discord knowledge). """core async webhook sender (aiohttp only, no discord knowledge).
`Webhook` posts a JSON dict to a url (or round-robins a pool), handling 429/5xx `Webhook` posts a JSON dict to a url (or round-robins a pool), handling 429/5xx
retries and optional proxy rotation, and always returns a `WebhookResult` — it retries, connection/timeout retries, and optional proxy rotation, and always
never raises on a send failure. the discord layer builds payloads and delegates returns a `WebhookResult` — it never raises on a send failure. the discord layer
the actual POST here so it inherits rotation / proxy / retry / result. builds payloads and delegates the actual POST here so it inherits rotation / proxy
/ retry / result.
a 429's server-controlled `retry_after` wait is bounded: non-finite values
(inf/nan) are rejected and finite values are clamped to `MAX_RETRY_AFTER`, so an
adversarial or misconfigured server can never stall a send() past a bounded
ceiling.
""" """
import asyncio import asyncio
import logging import logging
import math
from typing import Dict, List, Optional, Union from typing import Dict, List, Optional, Union
from urllib.parse import unquote, urlsplit from urllib.parse import unquote, urlsplit
@@ -19,6 +26,17 @@ from .result import WebhookResult
log = logging.getLogger(__name__) log = logging.getLogger(__name__)
MAX_RETRY_AFTER = 300.0
"""ceiling (seconds) honored from a server-controlled 429 retry_after/Retry-After
a server can send an arbitrarily large or non-finite wait (float('inf'), a
Cloudflare-scale value, or a ms-vs-s unit mismatch turning 5s into 5000s); sleeping
that verbatim would stall a send() far past its ClientTimeout and outside
max_retries accounting, breaking the 'always returns a WebhookResult' contract.
non-finite values are rejected outright (treated as unparseable); finite values are
clamped to this ceiling.
"""
class _Retryable(Exception): class _Retryable(Exception):
"""internal signal: a retryable HTTP status (429/5xx); carries the response """internal signal: a retryable HTTP status (429/5xx); carries the response
@@ -90,18 +108,27 @@ class Webhook:
@staticmethod @staticmethod
def _retry_after(status: int, headers, body) -> Optional[float]: def _retry_after(status: int, headers, body) -> Optional[float]:
"""seconds to wait on a 429, from body retry_after then Retry-After header""" """seconds to wait on a 429, from body retry_after then Retry-After header
non-finite values (inf/nan) are rejected as unparseable; finite values are
clamped to MAX_RETRY_AFTER so a server-controlled wait can never stall a
send() past a bounded ceiling.
"""
if status != 429: if status != 429:
return None return None
if isinstance(body, dict) and body.get("retry_after") is not None: if isinstance(body, dict) and body.get("retry_after") is not None:
try: try:
return float(body["retry_after"]) value = float(body["retry_after"])
if math.isfinite(value):
return max(0.0, min(value, MAX_RETRY_AFTER))
except (TypeError, ValueError): except (TypeError, ValueError):
pass pass
header = headers.get("Retry-After") if headers else None header = headers.get("Retry-After") if headers else None
if header is not None: if header is not None:
try: try:
return float(header) value = float(header)
if math.isfinite(value):
return max(0.0, min(value, MAX_RETRY_AFTER))
except (TypeError, ValueError): except (TypeError, ValueError):
pass pass
return None return None
@@ -110,9 +137,10 @@ class Webhook:
"""post a json payload to the next pool url; always returns a result """post a json payload to the next pool url; always returns a result
handles 429 (wait + retry, capped by max_retries), 5xx (retry, capped), and handles 429 (wait + retry, capped by max_retries), 5xx (retry, capped), and
timeout/connection errors with optional proxy rotation (burn + next, capped timeout/connection errors: with a proxy provider, burn + rotate to the next
by max_proxy_retries). 4xx other than 429 fail immediately. never raises on a proxy (capped by max_proxy_retries) before failing; without one, retried
send failure. directly under max_retries like a 5xx. 4xx other than 429 fail immediately.
never raises on a send failure.
""" """
url = self._next_url() url = self._next_url()
session = self._session session = self._session
@@ -130,11 +158,13 @@ class Webhook:
) -> WebhookResult: ) -> WebhookResult:
"""status-retry (via commons.aretry) wrapping proxy rotation; never raises """status-retry (via commons.aretry) wrapping proxy rotation; never raises
commons.aretry owns the 429/5xx backoff schedule + retry cap (max_retries), commons.aretry owns the 429/5xx/no-provider-connection-error backoff schedule
retrying on the internal _Retryable signal. on exhaustion it re-raises the + retry cap (max_retries), retrying on the internal _Retryable signal. on
last _Retryable, whose carried result is the REAL last response (not a exhaustion it re-raises the last _Retryable, whose carried result is the REAL
synthetic status-0). proxy rotation on connection errors lives inside the last response (not a synthetic status-0). proxy rotation on connection errors
attempt and is capped separately. lives inside the attempt and is capped separately (max_proxy_retries); once a
provider's rotation cap is hit that failure is terminal, not retried again
via aretry.
""" """
counter = [0] counter = [0]
try: try:
@@ -163,8 +193,12 @@ class Webhook:
raises _Retryable (carrying the real response) on a 429/5xx so the caller's raises _Retryable (carrying the real response) on a 429/5xx so the caller's
aretry applies backoff; honors an explicit 429 retry_after by sleeping it aretry applies backoff; honors an explicit 429 retry_after by sleeping it
before signalling. returns a final WebhookResult on success or a terminal before signalling. a connection/timeout error also raises _Retryable when no
(non-retryable) failure — never lets a provider/connection error escape. proxy provider is set, so it retries under aretry's backoff + max_retries
instead of failing one-shot; with a provider, the same error first drives
burn+rotate up to max_proxy_retries, and only becomes terminal (not retried
via aretry) once that cap is hit. returns a final WebhookResult on success or
a terminal (non-retryable) failure — never lets a provider error escape.
`counter` is a per-call mutable cell ([0]) owned by the calling `_send_loop`, `counter` is a per-call mutable cell ([0]) owned by the calling `_send_loop`,
so the attempt count is local to one `send()` and concurrent sends on the so the attempt count is local to one `send()` and concurrent sends on the
@@ -217,7 +251,10 @@ class Webhook:
# generic webhook) still retries under aretry's backoff + cap. # generic webhook) still retries under aretry's backoff + cap.
# note: aretry ALSO sleeps its backoff between retries, so an # note: aretry ALSO sleeps its backoff between retries, so an
# honored retry_after is additive (retry_after + backoff) — this # honored retry_after is additive (retry_after + backoff) — this
# only ever over-waits, never under-waits the server's hint # only ever over-waits, never under-waits the server's hint.
# non-finite values (inf/nan) are rejected and finite values are
# clamped to MAX_RETRY_AFTER by _retry_after, so a server-controlled
# wait can never stall this attempt past a bounded ceiling
wait = self._retry_after(status, resp.headers, body) wait = self._retry_after(status, resp.headers, body)
if wait is not None: if wait is not None:
log.warning("webhook 429 on %s; honoring retry_after %.3fs", url, wait) log.warning("webhook 429 on %s; honoring retry_after %.3fs", url, wait)
@@ -230,18 +267,30 @@ class Webhook:
return result return result
except (aiohttp.ClientError, asyncio.TimeoutError) as error: except (aiohttp.ClientError, asyncio.TimeoutError) as error:
if self._proxies is not None and proxy_tries < self.max_proxy_retries: if self._proxies is not None:
if self._burn(last_proxy): if proxy_tries < self.max_proxy_retries:
proxy_tries += 1 if self._burn(last_proxy):
continue proxy_tries += 1
continue
return WebhookResult(
ok=False, status=None, url=url, attempts=attempts,
error="proxies exhausted", proxy=last_proxy,
)
return WebhookResult( return WebhookResult(
ok=False, status=None, url=url, attempts=attempts, ok=False, status=None, url=url, attempts=attempts,
error="proxies exhausted", proxy=last_proxy, error=f"{type(error).__name__}: {error}", proxy=last_proxy,
) )
return WebhookResult( # no proxy provider: a connection/timeout error is the single most
# retry-worthy failure class (the 5xx equivalent already retries under
# aretry) — raise _Retryable so aretry applies backoff + max_retries
# instead of failing one-shot. aretry re-raises this same _Retryable on
# exhaustion, and _send_loop's `except _Retryable` returns its carried
# result, so the never-raises / always-returns-a-WebhookResult contract
# still holds after retries run out.
raise _Retryable(WebhookResult(
ok=False, status=None, url=url, attempts=attempts, ok=False, status=None, url=url, attempts=attempts,
error=f"{type(error).__name__}: {error}", proxy=last_proxy, error=f"{type(error).__name__}: {error}", proxy=last_proxy,
) ))
def _burn(self, proxy: Optional[str]) -> bool: def _burn(self, proxy: Optional[str]) -> bool:
"""burn the current proxy; return False if it can't be rotated """burn the current proxy; return False if it can't be rotated