Compare commits

..

No commits in common. "main" and "v0.1.1" have entirely different histories.
main ... v0.1.1

5 changed files with 37 additions and 97 deletions

2
.gitignore vendored
View File

@ -1,5 +1,5 @@
# claude # claude
.claude/ CLAUDE.md
# python # python
__pycache__/ __pycache__/

View File

@ -13,16 +13,13 @@ send to the core — inheriting rotation, proxy, retry, and result for free.
## Install ## Install
``` ```
aiowebhooks @ git+ssh://git@git.rethinkstudios.io/rethink-public/aiowebhooks.git@v0.1.4 aiowebhooks @ git+ssh://git@git.rethinkstudios.io/rethink-public/aiowebhooks.git@v0.1.0
# discord embeds / identity helpers need the extra: # discord embeds / identity helpers need the extra:
aiowebhooks[discord] @ git+ssh://git@git.rethinkstudios.io/rethink-public/aiowebhooks.git@v0.1.4 aiowebhooks[discord] @ git+ssh://git@git.rethinkstudios.io/rethink-public/aiowebhooks.git@v0.1.0
``` ```
The base pulls `aiohttp` and `commons` (for the retry/backoff engine). Only The base pulls `aiohttp`. Only `aiowebhooks[discord]` adds `discord.py` (>=2.3,
`aiowebhooks[discord]` adds `discord.py` (>=2.3, mainline — not discord.py-self), and mainline — not discord.py-self), and only for `DiscordWebhook`.
only for `DiscordWebhook`.
Drop the `@v0.1.4` suffix from the line above to install the latest unpinned.
## Core sender ## Core sender
@ -50,9 +47,7 @@ Webhook(
``` ```
Inject a shared `session` for throughput (one session per process); without one, each Inject a shared `session` for throughput (one session per process); without one, each
send opens and closes its own. A single `Webhook` instance is safe to drive from many send opens and closes its own.
concurrent `send()` calls — each call tracks its own attempt count, so concurrent sends
don't corrupt each other's `WebhookResult.attempts`.
## WebhookResult ## WebhookResult
@ -70,17 +65,12 @@ result.proxy # canonical proxy string used (host:port:user:pass / host:port)
## Retry & rate limits ## Retry & rate limits
Status retries run through `commons.aretry` (exponential backoff + cap): - **429** — waits the `retry_after` from the body first (Discord sends seconds), then
the `Retry-After` header, then retries; capped by `max_retries`.
- **429** — always retried, capped by `max_retries`. When a wait is parseable (body - **5xx** — retried, capped by `max_retries`.
`retry_after` first — Discord sends seconds — then the `Retry-After` header) it sleeps
that value before retrying; a 429 with no parseable wait (edge/Cloudflare/generic
webhook) still retries under aretry's backoff rather than failing one-shot.
- **5xx** — retried with exponential backoff, capped by `max_retries`.
- **4xx** (other than 429) — fails immediately (no retry), returned as `ok=False`. - **4xx** (other than 429) — fails immediately (no retry), returned as `ok=False`.
Exceeding a cap returns a failed result rather than looping — and the result carries the Exceeding a cap returns a failed result rather than looping.
**real** last status/body (not a synthetic placeholder).
## Proxy rotation (optional, duck-typed) ## Proxy rotation (optional, duck-typed)
@ -101,10 +91,9 @@ result = await wh.send(payload) # sends through pm.get(); on a timeout/connect
``` ```
On a timeout/connection error the current proxy is burned and the next is tried, up to On a timeout/connection error the current proxy is burned and the next is tried, up to
`max_proxy_retries`. Hitting the cap, or **any exception from the provider's `max_proxy_retries`. A provider `ProxiesExhaustedError` (or hitting the cap) returns a
`get()`/`burn()`** (the provider is duck-typed and never imported, so its exception failed result — never an infinite loop. With no provider, a timeout just fails after
types can't be caught by class), returns a failed result — never an infinite loop, never normal retry.
an escape. With no provider, a timeout just fails after normal retry.
## Discord (`aiowebhooks[discord]`) ## Discord (`aiowebhooks[discord]`)
@ -135,41 +124,12 @@ Without the extra installed, importing `aiowebhooks` still works; constructing o
- Every send returns a `WebhookResult`; the core never raises on a send failure and - Every send returns a `WebhookResult`; the core never raises on a send failure and
never prints. Callers check `result.ok`. never prints. Callers check `result.ok`.
- JSON-only: **files/attachments, `tts`, and `allowed_mentions` are out** (deliberate - v0.1.0 is JSON-only: **files/attachments, `tts`, and `allowed_mentions` are out**
scope cut, addable later). The Discord surface is content + embeds + identity. (deliberate scope cut, addable later). The Discord surface is content + embeds +
- Rotation is round-robin only; try-next-on-failure across URLs is a later feature. identity.
- v0.1.0 rotation is round-robin only; try-next-on-failure across URLs is a later
## Changelog feature.
### v0.1.4
- **Never-raises net widened:** an unexpected exception that escapes a send attempt (a
closed injected session → `RuntimeError`, a malformed proxy URL → `ValueError`) now
converts to a falsy `WebhookResult(ok=False, ...)` instead of propagating out of
`send()`, restoring the documented contract for those edge triggers.
### v0.1.3
- **429 always retries:** every `429` is now retryable under aretry's backoff + cap, not
only those with a parseable `retry_after`. A 429 with no body `retry_after` and no
`Retry-After` header (edge/Cloudflare/generic webhook) previously failed one-shot.
### v0.1.2
- Removed a dead `clock` constructor param (it was stored but never used). Pinned
`commons` to v0.2.1.
### v0.1.1
- **Never-raises contract hardened:** an error from a duck-typed proxy provider's
`get()`/`burn()` (e.g. `aioproxies.burn()` raising `ValueError` for an unknown proxy)
used to escape `send()`. Now any provider exception is caught and converted to a
failed result.
- **Retry via `commons.aretry`:** 429/5xx retry moved onto the shared backoff engine —
5xx now backs off (was a tight no-backoff loop), and exhausted retries return the
**real** last status/body instead of a synthetic placeholder. Adds a `commons`
dependency.
## Versioning ## Versioning
Releases are tagged `vX.Y.Z`. The install line above pins a release; drop the `@vX.Y.Z` suffix to install the latest unpinned. Pin deliberately for reproducible installs. Tagged `vX.Y.Z`. Pin the tag.

View File

@ -4,12 +4,12 @@ build-backend = "hatchling.build"
[project] [project]
name = "aiowebhooks" name = "aiowebhooks"
version = "0.1.4" version = "0.1.1"
description = "async webhook sender (aiohttp) with round-robin urls, retry, and proxy rotation; optional discord.py embeds" description = "async webhook sender (aiohttp) with round-robin urls, retry, and proxy rotation; optional discord.py embeds"
requires-python = ">=3.10" requires-python = ">=3.10"
dependencies = [ dependencies = [
"aiohttp>=3.9", "aiohttp>=3.9",
"commons @ git+ssh://git@git.rethinkstudios.io/rethink-public/commons.git@v0.2.1", "commons @ git+ssh://git@git.rethinkstudios.io/rethink-public/commons.git@v0.2.0",
] ]
[project.optional-dependencies] [project.optional-dependencies]

View File

@ -21,4 +21,4 @@ from .sender import Webhook
__all__ = ["Webhook", "WebhookResult", "WebhookError", "NoUrlsError"] __all__ = ["Webhook", "WebhookResult", "WebhookError", "NoUrlsError"]
__version__ = "0.1.4" __version__ = "0.1.1"

View File

@ -8,6 +8,7 @@ the actual POST here so it inherits rotation / proxy / retry / result.
import asyncio import asyncio
import logging import logging
import time
from typing import Dict, List, Optional, Union from typing import Dict, List, Optional, Union
from urllib.parse import unquote, urlsplit from urllib.parse import unquote, urlsplit
@ -49,12 +50,12 @@ def _proxy_string(proxies_dict: Optional[Dict[str, str]]) -> Optional[str]:
if host is None: if host is None:
return None return None
host = host.lower() host = host.lower()
hostport = f"{host}:{parts.port}" if parts.port is not None else host port = str(parts.port) if parts.port is not None else ""
if parts.username: if parts.username:
user = unquote(parts.username) user = unquote(parts.username)
password = unquote(parts.password) if parts.password is not None else "" password = unquote(parts.password) if parts.password is not None else ""
return f"{hostport}:{user}:{password}" return f"{host}:{port}:{user}:{password}"
return hostport return f"{host}:{port}"
except ValueError: except ValueError:
return None return None
@ -71,6 +72,7 @@ class Webhook:
timeout: float = 15, timeout: float = 15,
max_retries: int = 3, max_retries: int = 3,
max_proxy_retries: int = 3, max_proxy_retries: int = 3,
clock=time.monotonic,
): ):
self._urls = [urls] if isinstance(urls, str) else list(urls) self._urls = [urls] if isinstance(urls, str) else list(urls)
if not self._urls: if not self._urls:
@ -80,6 +82,7 @@ class Webhook:
self.timeout = timeout self.timeout = timeout
self.max_retries = max_retries self.max_retries = max_retries
self.max_proxy_retries = max_proxy_retries self.max_proxy_retries = max_proxy_retries
self._clock = clock
self._index = 0 self._index = 0
def _next_url(self) -> str: def _next_url(self) -> str:
@ -136,28 +139,18 @@ class Webhook:
synthetic status-0). proxy rotation on connection errors lives inside the synthetic status-0). proxy rotation on connection errors lives inside the
attempt and is capped separately. attempt and is capped separately.
""" """
counter = [0] self._attempt_no = 0
try: try:
return await aretry( return await aretry(
lambda: self._attempt(session, url, payload, counter), lambda: self._attempt(session, url, payload),
attempts=self.max_retries + 1, attempts=self.max_retries + 1,
on=(_Retryable,), on=(_Retryable,),
) )
except _Retryable as exhausted: except _Retryable as exhausted:
return exhausted.result return exhausted.result
except Exception as error:
# never-raises safety net: an unexpected error that escapes the attempt (a
# closed injected session -> RuntimeError, a malformed proxy url -> ValueError,
# anything not aiohttp.ClientError/TimeoutError) must come back as a failed
# result, not propagate out of send()
log.warning("webhook send failed unexpectedly on %s: %s", url, error, exc_info=True)
return WebhookResult(
ok=False, status=None, url=url, attempts=counter[0] or 1,
error=f"{type(error).__name__}: {error}",
)
async def _attempt( async def _attempt(
self, session: aiohttp.ClientSession, url: str, payload: Dict, counter: List[int] self, session: aiohttp.ClientSession, url: str, payload: Dict
) -> WebhookResult: ) -> WebhookResult:
"""one logical send: proxy rotation + a single POST; may raise _Retryable """one logical send: proxy rotation + a single POST; may raise _Retryable
@ -165,18 +158,14 @@ class Webhook:
aretry applies backoff; honors an explicit 429 retry_after by sleeping it aretry applies backoff; honors an explicit 429 retry_after by sleeping it
before signalling. returns a final WebhookResult on success or a terminal before signalling. returns a final WebhookResult on success or a terminal
(non-retryable) failure never lets a provider/connection error escape. (non-retryable) failure never lets a provider/connection error escape.
`counter` is a per-call mutable cell ([0]) owned by the calling `_send_loop`,
so the attempt count is local to one `send()` and concurrent sends on the
same instance don't corrupt each other's tally.
""" """
timeout = aiohttp.ClientTimeout(total=self.timeout) timeout = aiohttp.ClientTimeout(total=self.timeout)
last_proxy: Optional[str] = None last_proxy: Optional[str] = None
proxy_tries = 0 proxy_tries = 0
while True: while True:
counter[0] += 1 self._attempt_no += 1
attempts = counter[0] attempts = self._attempt_no
proxy_url = None proxy_url = None
if self._proxies is not None: if self._proxies is not None:
try: try:
@ -211,19 +200,10 @@ class Webhook:
error=f"http {status}", response=body, proxy=last_proxy, error=f"http {status}", response=body, proxy=last_proxy,
) )
if status == 429: wait = self._retry_after(status, resp.headers, body)
# every 429 is retryable; honor an explicit retry_after by if wait is not None:
# sleeping it, but a 429 with no parseable wait (edge/Cloudflare/ log.warning("webhook 429 on %s; honoring retry_after %.3fs", url, wait)
# generic webhook) still retries under aretry's backoff + cap. await asyncio.sleep(wait)
# note: aretry ALSO sleeps its backoff between retries, so an
# honored retry_after is additive (retry_after + backoff) — this
# only ever over-waits, never under-waits the server's hint
wait = self._retry_after(status, resp.headers, body)
if wait is not None:
log.warning("webhook 429 on %s; honoring retry_after %.3fs", url, wait)
await asyncio.sleep(wait)
else:
log.warning("webhook 429 on %s; no retry_after, backing off", url)
raise _Retryable(result) raise _Retryable(result)
if status >= 500: if status >= 500:
raise _Retryable(result) raise _Retryable(result)