Compare commits

...

10 Commits
v0.1.0 ... main

Author SHA1 Message Date
585a432ae0 chore: ignore .claude/ dir (CLAUDE.md now lives under .claude/)
Signed-off-by: disqualifier <dev@disqualifier.me>
2026-06-29 21:55:13 -04:00
3d86fc249c docs: clarify 429 honored-retry_after is additive with aretry backoff (F2)
note that an honored retry_after sleeps before aretry's own backoff, so the effective
wait is retry_after + backoff — only ever over-waits, never under-waits the server hint.

Signed-off-by: disqualifier <dev@disqualifier.me>
2026-06-29 21:34:38 -04:00
ef20bc51f0 fix: never-raises net widened to unexpected exceptions; changelog/429 docs (v0.1.4)
M-2: _attempt caught only (aiohttp.ClientError, asyncio.TimeoutError); an unexpected
error escaping the attempt (closed injected session -> RuntimeError, malformed proxy url
-> ValueError) propagated out of send(), breaking the documented 'never raises on a send
failure' contract. add an outer catch-all in _send_loop converting any such exception to a
falsy WebhookResult(ok=False), logged at warning with exc_info.

aiowebhooks-F3: README 429 section + changelog were stale vs the v0.1.3 'retry any 429'
fix; added the no-parseable-wait-still-retries wording and v0.1.3/v0.1.4 changelog entries.

verified by execution: closed-session (RuntimeError) and bad-proxy (ValueError) controls
both fire and now return ok=False instead of raising.

Signed-off-by: disqualifier <dev@disqualifier.me>
2026-06-29 20:47:30 -04:00
28bad7fc7f docs: pin install line to release, note unpinned-latest option
Signed-off-by: disqualifier <dev@disqualifier.me>
2026-06-29 18:13:35 -04:00
1d3418a4be docs: show unpinned install line; note tag-pinning for reproducibility
Signed-off-by: disqualifier <dev@disqualifier.me>
2026-06-29 18:07:20 -04:00
f3d2561bf9 fix: retry any 429, not just those with a parseable retry_after (v0.1.3)
treat every status==429 as retryable: sleep only when retry_after parses, but raise
_Retryable either way so aretry's backoff + max_retries cap engages. previously a 429
with no body retry_after and no Retry-After header (edge/Cloudflare/generic webhook)
returned a terminal ok=False with no retry, contradicting the documented retry-on-429.

Signed-off-by: disqualifier <dev@disqualifier.me>
2026-06-29 17:09:30 -04:00
cd93e4f44f fix: per-call attempt counter so concurrent sends don't corrupt attempts
the attempt count used instance state (self._attempt_no), reset in _send_loop and incremented in _attempt; two concurrent send() calls on one Webhook interleaved, corrupting each other's WebhookResult.attempts. it is now a per-call mutable cell created in _send_loop and threaded into _attempt. verified under load: 400 concurrent sends, zero corrupted counts.

Signed-off-by: disqualifier <dev@disqualifier.me>
2026-06-28 17:46:20 -04:00
bcf1f5511c fix: _proxy_string emits no stray colons for a portless proxy
a portless proxy url produced host::user:pass / host: (extra colons), breaking the identity match against the provider pool. the port colon is now omitted when there is no port, mirroring aioproxies' canonical key.

Signed-off-by: disqualifier <dev@disqualifier.me>
2026-06-28 17:18:28 -04:00
aec0a5cc2b fix: remove dead clock param; pin commons v0.2.1 (v0.1.2)
the clock= constructor param was stored (self._clock) but never read — the 429
retry_after wait uses asyncio.sleep directly. it was dead code, and the CLAUDE.md
wrongly claimed it made 429 timing test-controllable. remove the param + the unused
time import, and correct the doc (tests patch commons.retry's sleep + sender.asyncio
.sleep, not a clock seam). bump the commons pin to v0.2.1 (retry attempts floor).

verified: clock param gone, constructs fine, 18/18 fix harness intact.
Signed-off-by: disqualifier <dev@disqualifier.me>
2026-06-28 16:16:13 -04:00
40f8cc5b5f fix: never-raises contract + retry migration to commons.aretry (v0.1.1)
- seam bug: _burn/get() only caught ProxiesExhaustedError, but aioproxies.burn()
  raises ValueError ('proxy not in pool') which escaped send() and broke the
  'never raises on send failure' contract. catch ANY exception across the
  duck-typed provider seam and convert to a failed WebhookResult.
- 5xx hot loop: 5xx retries had no backoff (immediate retry, hammering the
  endpoint). migrate 429/5xx retry onto commons.aretry (>=0.2.0) for correct
  exponential backoff + cap.
- lost response: exhausted retries returned a synthetic status-0 result; now the
  real last 4xx/5xx status + body is returned (aretry re-raises the carried
  _Retryable, the loop unwraps it).

verified by execution: burn/get ValueError no longer escapes, 5xx backs off
(~1.9s over 3 retries vs ~0s hot loop), exhausted 5xx returns real 503 + body,
429 retry_after honored, 4xx/rotation/round-robin intact.

Signed-off-by: disqualifier <dev@disqualifier.me>
2026-06-27 21:43:41 -04:00
5 changed files with 164 additions and 62 deletions

2
.gitignore vendored
View File

@ -1,5 +1,5 @@
# claude # claude
CLAUDE.md .claude/
# python # python
__pycache__/ __pycache__/

View File

@ -13,13 +13,16 @@ send to the core — inheriting rotation, proxy, retry, and result for free.
## Install ## Install
``` ```
aiowebhooks @ git+ssh://git@git.rethinkstudios.io/rethink-public/aiowebhooks.git@v0.1.0 aiowebhooks @ git+ssh://git@git.rethinkstudios.io/rethink-public/aiowebhooks.git@v0.1.4
# discord embeds / identity helpers need the extra: # discord embeds / identity helpers need the extra:
aiowebhooks[discord] @ git+ssh://git@git.rethinkstudios.io/rethink-public/aiowebhooks.git@v0.1.0 aiowebhooks[discord] @ git+ssh://git@git.rethinkstudios.io/rethink-public/aiowebhooks.git@v0.1.4
``` ```
The base pulls `aiohttp`. Only `aiowebhooks[discord]` adds `discord.py` (>=2.3, The base pulls `aiohttp` and `commons` (for the retry/backoff engine). Only
mainline — not discord.py-self), and only for `DiscordWebhook`. `aiowebhooks[discord]` adds `discord.py` (>=2.3, mainline — not discord.py-self), and
only for `DiscordWebhook`.
Drop the `@v0.1.4` suffix from the line above to install the latest unpinned.
## Core sender ## Core sender
@ -47,7 +50,9 @@ Webhook(
``` ```
Inject a shared `session` for throughput (one session per process); without one, each Inject a shared `session` for throughput (one session per process); without one, each
send opens and closes its own. send opens and closes its own. A single `Webhook` instance is safe to drive from many
concurrent `send()` calls — each call tracks its own attempt count, so concurrent sends
don't corrupt each other's `WebhookResult.attempts`.
## WebhookResult ## WebhookResult
@ -65,12 +70,17 @@ result.proxy # canonical proxy string used (host:port:user:pass / host:port)
## Retry & rate limits ## Retry & rate limits
- **429** — waits the `retry_after` from the body first (Discord sends seconds), then Status retries run through `commons.aretry` (exponential backoff + cap):
the `Retry-After` header, then retries; capped by `max_retries`.
- **5xx** — retried, capped by `max_retries`. - **429** — always retried, capped by `max_retries`. When a wait is parseable (body
`retry_after` first — Discord sends seconds — then the `Retry-After` header) it sleeps
that value before retrying; a 429 with no parseable wait (edge/Cloudflare/generic
webhook) still retries under aretry's backoff rather than failing one-shot.
- **5xx** — retried with exponential backoff, capped by `max_retries`.
- **4xx** (other than 429) — fails immediately (no retry), returned as `ok=False`. - **4xx** (other than 429) — fails immediately (no retry), returned as `ok=False`.
Exceeding a cap returns a failed result rather than looping. Exceeding a cap returns a failed result rather than looping — and the result carries the
**real** last status/body (not a synthetic placeholder).
## Proxy rotation (optional, duck-typed) ## Proxy rotation (optional, duck-typed)
@ -91,9 +101,10 @@ result = await wh.send(payload) # sends through pm.get(); on a timeout/connect
``` ```
On a timeout/connection error the current proxy is burned and the next is tried, up to On a timeout/connection error the current proxy is burned and the next is tried, up to
`max_proxy_retries`. A provider `ProxiesExhaustedError` (or hitting the cap) returns a `max_proxy_retries`. Hitting the cap, or **any exception from the provider's
failed result — never an infinite loop. With no provider, a timeout just fails after `get()`/`burn()`** (the provider is duck-typed and never imported, so its exception
normal retry. types can't be caught by class), returns a failed result — never an infinite loop, never
an escape. With no provider, a timeout just fails after normal retry.
## Discord (`aiowebhooks[discord]`) ## Discord (`aiowebhooks[discord]`)
@ -124,12 +135,41 @@ Without the extra installed, importing `aiowebhooks` still works; constructing o
- Every send returns a `WebhookResult`; the core never raises on a send failure and - Every send returns a `WebhookResult`; the core never raises on a send failure and
never prints. Callers check `result.ok`. never prints. Callers check `result.ok`.
- v0.1.0 is JSON-only: **files/attachments, `tts`, and `allowed_mentions` are out** - JSON-only: **files/attachments, `tts`, and `allowed_mentions` are out** (deliberate
(deliberate scope cut, addable later). The Discord surface is content + embeds + scope cut, addable later). The Discord surface is content + embeds + identity.
identity. - Rotation is round-robin only; try-next-on-failure across URLs is a later feature.
- v0.1.0 rotation is round-robin only; try-next-on-failure across URLs is a later
feature. ## Changelog
### v0.1.4
- **Never-raises net widened:** an unexpected exception that escapes a send attempt (a
closed injected session → `RuntimeError`, a malformed proxy URL → `ValueError`) now
converts to a falsy `WebhookResult(ok=False, ...)` instead of propagating out of
`send()`, restoring the documented contract for those edge triggers.
### v0.1.3
- **429 always retries:** every `429` is now retryable under aretry's backoff + cap, not
only those with a parseable `retry_after`. A 429 with no body `retry_after` and no
`Retry-After` header (edge/Cloudflare/generic webhook) previously failed one-shot.
### v0.1.2
- Removed a dead `clock` constructor param (it was stored but never used). Pinned
`commons` to v0.2.1.
### v0.1.1
- **Never-raises contract hardened:** an error from a duck-typed proxy provider's
`get()`/`burn()` (e.g. `aioproxies.burn()` raising `ValueError` for an unknown proxy)
used to escape `send()`. Now any provider exception is caught and converted to a
failed result.
- **Retry via `commons.aretry`:** 429/5xx retry moved onto the shared backoff engine —
5xx now backs off (was a tight no-backoff loop), and exhausted retries return the
**real** last status/body instead of a synthetic placeholder. Adds a `commons`
dependency.
## Versioning ## Versioning
Tagged `vX.Y.Z`. Pin the tag. Releases are tagged `vX.Y.Z`. The install line above pins a release; drop the `@vX.Y.Z` suffix to install the latest unpinned. Pin deliberately for reproducible installs.

View File

@ -4,11 +4,12 @@ build-backend = "hatchling.build"
[project] [project]
name = "aiowebhooks" name = "aiowebhooks"
version = "0.1.0" version = "0.1.4"
description = "async webhook sender (aiohttp) with round-robin urls, retry, and proxy rotation; optional discord.py embeds" description = "async webhook sender (aiohttp) with round-robin urls, retry, and proxy rotation; optional discord.py embeds"
requires-python = ">=3.10" requires-python = ">=3.10"
dependencies = [ dependencies = [
"aiohttp>=3.9", "aiohttp>=3.9",
"commons @ git+ssh://git@git.rethinkstudios.io/rethink-public/commons.git@v0.2.1",
] ]
[project.optional-dependencies] [project.optional-dependencies]
@ -16,5 +17,8 @@ discord = [
"discord.py>=2.3", "discord.py>=2.3",
] ]
[tool.hatch.metadata]
allow-direct-references = true
[tool.hatch.build.targets.wheel] [tool.hatch.build.targets.wheel]
packages = ["src/aiowebhooks"] packages = ["src/aiowebhooks"]

View File

@ -21,4 +21,4 @@ from .sender import Webhook
__all__ = ["Webhook", "WebhookResult", "WebhookError", "NoUrlsError"] __all__ = ["Webhook", "WebhookResult", "WebhookError", "NoUrlsError"]
__version__ = "0.1.0" __version__ = "0.1.4"

View File

@ -8,11 +8,11 @@ the actual POST here so it inherits rotation / proxy / retry / result.
import asyncio import asyncio
import logging import logging
import time
from typing import Dict, List, Optional, Union from typing import Dict, List, Optional, Union
from urllib.parse import unquote, urlsplit from urllib.parse import unquote, urlsplit
import aiohttp import aiohttp
from commons import aretry
from .errors import NoUrlsError from .errors import NoUrlsError
from .result import WebhookResult from .result import WebhookResult
@ -20,6 +20,18 @@ from .result import WebhookResult
log = logging.getLogger(__name__) log = logging.getLogger(__name__)
class _Retryable(Exception):
"""internal signal: a retryable HTTP status (429/5xx); carries the response
raised inside an attempt so commons.aretry drives the backoff + cap; the loop
catches the final one to return the REAL last response, not a synthetic result.
"""
def __init__(self, result: WebhookResult):
super().__init__(f"retryable status {result.status}")
self.result = result
def _proxy_string(proxies_dict: Optional[Dict[str, str]]) -> Optional[str]: def _proxy_string(proxies_dict: Optional[Dict[str, str]]) -> Optional[str]:
"""canonical host:port:user:pass (or host:port) from an aiohttp proxies dict """canonical host:port:user:pass (or host:port) from an aiohttp proxies dict
@ -37,12 +49,12 @@ def _proxy_string(proxies_dict: Optional[Dict[str, str]]) -> Optional[str]:
if host is None: if host is None:
return None return None
host = host.lower() host = host.lower()
port = str(parts.port) if parts.port is not None else "" hostport = f"{host}:{parts.port}" if parts.port is not None else host
if parts.username: if parts.username:
user = unquote(parts.username) user = unquote(parts.username)
password = unquote(parts.password) if parts.password is not None else "" password = unquote(parts.password) if parts.password is not None else ""
return f"{host}:{port}:{user}:{password}" return f"{hostport}:{user}:{password}"
return f"{host}:{port}" return hostport
except ValueError: except ValueError:
return None return None
@ -59,7 +71,6 @@ class Webhook:
timeout: float = 15, timeout: float = 15,
max_retries: int = 3, max_retries: int = 3,
max_proxy_retries: int = 3, max_proxy_retries: int = 3,
clock=time.monotonic,
): ):
self._urls = [urls] if isinstance(urls, str) else list(urls) self._urls = [urls] if isinstance(urls, str) else list(urls)
if not self._urls: if not self._urls:
@ -69,7 +80,6 @@ class Webhook:
self.timeout = timeout self.timeout = timeout
self.max_retries = max_retries self.max_retries = max_retries
self.max_proxy_retries = max_proxy_retries self.max_proxy_retries = max_proxy_retries
self._clock = clock
self._index = 0 self._index = 0
def _next_url(self) -> str: def _next_url(self) -> str:
@ -118,29 +128,71 @@ class Webhook:
async def _send_loop( async def _send_loop(
self, session: aiohttp.ClientSession, url: str, payload: Dict self, session: aiohttp.ClientSession, url: str, payload: Dict
) -> WebhookResult: ) -> WebhookResult:
"""retry/rotation loop for a single send""" """status-retry (via commons.aretry) wrapping proxy rotation; never raises
attempts = 0
retries = 0 commons.aretry owns the 429/5xx backoff schedule + retry cap (max_retries),
proxy_tries = 0 retrying on the internal _Retryable signal. on exhaustion it re-raises the
last_proxy: Optional[str] = None last _Retryable, whose carried result is the REAL last response (not a
synthetic status-0). proxy rotation on connection errors lives inside the
attempt and is capped separately.
"""
counter = [0]
try:
return await aretry(
lambda: self._attempt(session, url, payload, counter),
attempts=self.max_retries + 1,
on=(_Retryable,),
)
except _Retryable as exhausted:
return exhausted.result
except Exception as error:
# never-raises safety net: an unexpected error that escapes the attempt (a
# closed injected session -> RuntimeError, a malformed proxy url -> ValueError,
# anything not aiohttp.ClientError/TimeoutError) must come back as a failed
# result, not propagate out of send()
log.warning("webhook send failed unexpectedly on %s: %s", url, error, exc_info=True)
return WebhookResult(
ok=False, status=None, url=url, attempts=counter[0] or 1,
error=f"{type(error).__name__}: {error}",
)
async def _attempt(
self, session: aiohttp.ClientSession, url: str, payload: Dict, counter: List[int]
) -> WebhookResult:
"""one logical send: proxy rotation + a single POST; may raise _Retryable
raises _Retryable (carrying the real response) on a 429/5xx so the caller's
aretry applies backoff; honors an explicit 429 retry_after by sleeping it
before signalling. returns a final WebhookResult on success or a terminal
(non-retryable) failure never lets a provider/connection error escape.
`counter` is a per-call mutable cell ([0]) owned by the calling `_send_loop`,
so the attempt count is local to one `send()` and concurrent sends on the
same instance don't corrupt each other's tally.
"""
timeout = aiohttp.ClientTimeout(total=self.timeout) timeout = aiohttp.ClientTimeout(total=self.timeout)
last_proxy: Optional[str] = None
proxy_tries = 0
while True: while True:
counter[0] += 1
attempts = counter[0]
proxy_url = None proxy_url = None
if self._proxies is not None: if self._proxies is not None:
try: try:
proxy_dict = self._proxies.get() proxy_dict = self._proxies.get()
except Exception as error: except Exception:
if type(error).__name__ == "ProxiesExhaustedError": # duck-typed provider; any error from get() means no proxy is
# available — fail cleanly rather than escaping send().
log.warning("webhook: proxy get() failed; no proxy available",
exc_info=True)
return WebhookResult( return WebhookResult(
ok=False, status=None, url=url, attempts=attempts, ok=False, status=None, url=url, attempts=attempts,
error="proxies exhausted", proxy=last_proxy, error="proxies unavailable", proxy=last_proxy,
) )
raise
last_proxy = _proxy_string(proxy_dict) last_proxy = _proxy_string(proxy_dict)
proxy_url = (proxy_dict or {}).get("http") or (proxy_dict or {}).get("https") proxy_url = (proxy_dict or {}).get("http") or (proxy_dict or {}).get("https")
attempts += 1
try: try:
async with session.post( async with session.post(
url, json=payload, proxy=proxy_url, timeout=timeout url, json=payload, proxy=proxy_url, timeout=timeout
@ -154,25 +206,29 @@ class Webhook:
response=body, proxy=last_proxy, response=body, proxy=last_proxy,
) )
wait = self._retry_after(status, resp.headers, body) result = WebhookResult(
if wait is not None and retries < self.max_retries:
retries += 1
log.warning("webhook 429 on %s; waiting %.3fs (retry %d/%d)",
url, wait, retries, self.max_retries)
await asyncio.sleep(wait)
continue
if status >= 500 and retries < self.max_retries:
retries += 1
log.warning("webhook %d on %s; retry %d/%d",
status, url, retries, self.max_retries)
continue
return WebhookResult(
ok=False, status=status, url=url, attempts=attempts, ok=False, status=status, url=url, attempts=attempts,
error=f"http {status}", response=body, proxy=last_proxy, error=f"http {status}", response=body, proxy=last_proxy,
) )
if status == 429:
# every 429 is retryable; honor an explicit retry_after by
# sleeping it, but a 429 with no parseable wait (edge/Cloudflare/
# generic webhook) still retries under aretry's backoff + cap.
# note: aretry ALSO sleeps its backoff between retries, so an
# honored retry_after is additive (retry_after + backoff) — this
# only ever over-waits, never under-waits the server's hint
wait = self._retry_after(status, resp.headers, body)
if wait is not None:
log.warning("webhook 429 on %s; honoring retry_after %.3fs", url, wait)
await asyncio.sleep(wait)
else:
log.warning("webhook 429 on %s; no retry_after, backing off", url)
raise _Retryable(result)
if status >= 500:
raise _Retryable(result)
return result
except (aiohttp.ClientError, asyncio.TimeoutError) as error: except (aiohttp.ClientError, asyncio.TimeoutError) as error:
if self._proxies is not None and proxy_tries < self.max_proxy_retries: if self._proxies is not None and proxy_tries < self.max_proxy_retries:
if self._burn(last_proxy): if self._burn(last_proxy):
@ -188,18 +244,20 @@ class Webhook:
) )
def _burn(self, proxy: Optional[str]) -> bool: def _burn(self, proxy: Optional[str]) -> bool:
"""burn the current proxy; return False if the provider is exhausted """burn the current proxy; return False if it can't be rotated
catches a provider ProxiesExhaustedError duck-typed by class name (the the provider is duck-typed and never imported, so we cannot catch its
provider is never imported), so a dying pool ends the loop cleanly. exception types by class. ANY exception from burn (a ProxiesExhaustedError
on a dead pool, a ValueError when the proxy isn't in the pool, etc.) means
we can't rotate — return False so the caller ends the loop with a failed
result rather than letting it escape send() (which must never raise).
""" """
try: try:
self._proxies.burn(proxy) self._proxies.burn(proxy)
return True return True
except Exception as error: except Exception:
if type(error).__name__ == "ProxiesExhaustedError": log.warning("webhook: proxy burn failed; ending rotation", exc_info=True)
return False return False
raise
@staticmethod @staticmethod
async def _read_body(resp) -> Optional[Union[str, Dict]]: async def _read_body(resp) -> Optional[Union[str, Dict]]: