Compare commits
16
Commits
v0.1.2
...
c65326882a
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
c65326882a | ||
|
|
e2f78bd80f | ||
|
|
8a8d99bbb3 | ||
|
|
52fea6e380 | ||
|
|
fb382f9767 | ||
|
|
1662d73d36 | ||
|
|
0d9d93e4c1 | ||
|
|
7a157efc16 | ||
|
|
6d4183948e | ||
|
|
90f67cf9fa | ||
|
|
585a432ae0 | ||
|
|
3d86fc249c | ||
|
|
ef20bc51f0 | ||
|
|
28bad7fc7f | ||
|
|
1d3418a4be | ||
|
|
f3d2561bf9 |
+1
-1
@@ -1,5 +1,5 @@
|
|||||||
# claude
|
# claude
|
||||||
CLAUDE.md
|
.claude/
|
||||||
|
|
||||||
# python
|
# python
|
||||||
__pycache__/
|
__pycache__/
|
||||||
|
|||||||
@@ -13,15 +13,17 @@ send to the core — inheriting rotation, proxy, retry, and result for free.
|
|||||||
## Install
|
## Install
|
||||||
|
|
||||||
```
|
```
|
||||||
aiowebhooks @ git+ssh://git@git.rethinkstudios.io/rethink-public/aiowebhooks.git@v0.1.2
|
aiowebhooks @ git+ssh://git@git.rethinkstudios.io/rethink-public/aiowebhooks.git@v1.0.2
|
||||||
# discord embeds / identity helpers need the extra:
|
# discord embeds / identity helpers need the extra:
|
||||||
aiowebhooks[discord] @ git+ssh://git@git.rethinkstudios.io/rethink-public/aiowebhooks.git@v0.1.2
|
aiowebhooks[discord] @ git+ssh://git@git.rethinkstudios.io/rethink-public/aiowebhooks.git@v1.0.2
|
||||||
```
|
```
|
||||||
|
|
||||||
The base pulls `aiohttp` and `commons` (for the retry/backoff engine). Only
|
The base pulls `aiohttp` and `commons` (for the retry/backoff engine). Only
|
||||||
`aiowebhooks[discord]` adds `discord.py` (>=2.3, mainline — not discord.py-self), and
|
`aiowebhooks[discord]` adds `discord.py` (>=2.3, mainline, not discord.py-self), and
|
||||||
only for `DiscordWebhook`.
|
only for `DiscordWebhook`.
|
||||||
|
|
||||||
|
Drop the `@v1.0.2` suffix from the line above to install the latest unpinned.
|
||||||
|
|
||||||
## Core sender
|
## Core sender
|
||||||
|
|
||||||
```python
|
```python
|
||||||
@@ -70,9 +72,17 @@ result.proxy # canonical proxy string used (host:port:user:pass / host:port)
|
|||||||
|
|
||||||
Status retries run through `commons.aretry` (exponential backoff + cap):
|
Status retries run through `commons.aretry` (exponential backoff + cap):
|
||||||
|
|
||||||
- **429** — honors the `retry_after` from the body first (Discord sends seconds), then
|
- **429** — always retried, capped by `max_retries`. When a wait is parseable (body
|
||||||
the `Retry-After` header, sleeping that exact value; capped by `max_retries`.
|
`retry_after` first — Discord sends seconds — then the `Retry-After` header) it sleeps
|
||||||
|
that value before retrying; a 429 with no parseable wait (edge/Cloudflare/generic
|
||||||
|
webhook) still retries under aretry's backoff rather than failing one-shot. A
|
||||||
|
non-finite value (`inf`/`nan`) is rejected outright and a finite value is clamped to
|
||||||
|
`MAX_RETRY_AFTER` (300s) — a server-controlled wait can never stall a `send()` past
|
||||||
|
that ceiling, however large or malformed the value it sends.
|
||||||
- **5xx** — retried with exponential backoff, capped by `max_retries`.
|
- **5xx** — retried with exponential backoff, capped by `max_retries`.
|
||||||
|
- **connection/timeout errors** — retried with exponential backoff, capped by
|
||||||
|
`max_retries`, same as a 5xx (with no proxy provider; see below for the
|
||||||
|
proxy-rotation path).
|
||||||
- **4xx** (other than 429) — fails immediately (no retry), returned as `ok=False`.
|
- **4xx** (other than 429) — fails immediately (no retry), returned as `ok=False`.
|
||||||
|
|
||||||
Exceeding a cap returns a failed result rather than looping — and the result carries the
|
Exceeding a cap returns a failed result rather than looping — and the result carries the
|
||||||
@@ -100,7 +110,8 @@ On a timeout/connection error the current proxy is burned and the next is tried,
|
|||||||
`max_proxy_retries`. Hitting the cap, or **any exception from the provider's
|
`max_proxy_retries`. Hitting the cap, or **any exception from the provider's
|
||||||
`get()`/`burn()`** (the provider is duck-typed and never imported, so its exception
|
`get()`/`burn()`** (the provider is duck-typed and never imported, so its exception
|
||||||
types can't be caught by class), returns a failed result — never an infinite loop, never
|
types can't be caught by class), returns a failed result — never an infinite loop, never
|
||||||
an escape. With no provider, a timeout just fails after normal retry.
|
an escape. With no provider, a timeout/connection error is retried directly under
|
||||||
|
`max_retries` (the same cap and backoff a 5xx gets) instead of burn+rotate.
|
||||||
|
|
||||||
## Discord (`aiowebhooks[discord]`)
|
## Discord (`aiowebhooks[discord]`)
|
||||||
|
|
||||||
@@ -137,6 +148,58 @@ Without the extra installed, importing `aiowebhooks` still works; constructing o
|
|||||||
|
|
||||||
## Changelog
|
## Changelog
|
||||||
|
|
||||||
|
### v0.1.8
|
||||||
|
|
||||||
|
- Compressed 5 residual internal/trivial docstrings (`MAX_RETRY_AFTER`, `_Retryable`,
|
||||||
|
`_proxy_string`, `_retry_after`, `_attempt`) to one or two lines; no behavior change.
|
||||||
|
|
||||||
|
### v0.1.7
|
||||||
|
|
||||||
|
- Docstrings/comments compressed (module docstrings and internal-method prose); no
|
||||||
|
behavior change. Public method contracts (`Webhook.send`, `WebhookResult` field
|
||||||
|
docs) are unchanged.
|
||||||
|
- Em-dash characters replaced with hyphens across the source.
|
||||||
|
|
||||||
|
### v0.1.6
|
||||||
|
|
||||||
|
- **429 `retry_after` no longer sleeps on an exhausted final attempt:** the wait is
|
||||||
|
now carried to the START of the next attempt instead of slept immediately after
|
||||||
|
seeing the 429. Previously the last (exhausted) attempt slept the full
|
||||||
|
`retry_after` before giving up — a pointless wait since no retry followed. The
|
||||||
|
wait is still honored (additive with aretry's backoff) before every attempt that
|
||||||
|
actually runs.
|
||||||
|
- Docs: `errors.py` no longer implies the missing-`[discord]`-extra raise is one of
|
||||||
|
the exported error types — it's a plain `RuntimeError`.
|
||||||
|
- Docstrings/comments compressed; no behavior change.
|
||||||
|
|
||||||
|
### v0.1.5
|
||||||
|
|
||||||
|
- **429 `retry_after` bounded:** a non-finite server-controlled wait (`inf`/`nan`, from
|
||||||
|
the body `retry_after` or the `Retry-After` header) is now rejected outright, and a
|
||||||
|
finite wait is clamped to `MAX_RETRY_AFTER` (300s). Previously a bare `float()` parse
|
||||||
|
slept the value verbatim, unbounded and outside `max_retries` accounting — an
|
||||||
|
adversarial or ms-vs-s-misconfigured server could stall a `send()` for hours.
|
||||||
|
- **Connection/timeout errors now retry without a proxy provider:** previously
|
||||||
|
`aiohttp.ClientError`/`asyncio.TimeoutError` with no `proxies=` set failed one-shot,
|
||||||
|
contradicting both the README and the single most retry-worthy failure class. Now it
|
||||||
|
retries under `commons.aretry`'s backoff, capped by `max_retries`, same as a 5xx —
|
||||||
|
and still returns `ok=False` (never raises) once retries are exhausted. Proxy-rotation
|
||||||
|
behavior (burn + rotate, capped by `max_proxy_retries`) is unchanged when a provider
|
||||||
|
is set.
|
||||||
|
|
||||||
|
### v0.1.4
|
||||||
|
|
||||||
|
- **Never-raises net widened:** an unexpected exception that escapes a send attempt (a
|
||||||
|
closed injected session → `RuntimeError`, a malformed proxy URL → `ValueError`) now
|
||||||
|
converts to a falsy `WebhookResult(ok=False, ...)` instead of propagating out of
|
||||||
|
`send()`, restoring the documented contract for those edge triggers.
|
||||||
|
|
||||||
|
### v0.1.3
|
||||||
|
|
||||||
|
- **429 always retries:** every `429` is now retryable under aretry's backoff + cap, not
|
||||||
|
only those with a parseable `retry_after`. A 429 with no body `retry_after` and no
|
||||||
|
`Retry-After` header (edge/Cloudflare/generic webhook) previously failed one-shot.
|
||||||
|
|
||||||
### v0.1.2
|
### v0.1.2
|
||||||
|
|
||||||
- Removed a dead `clock` constructor param (it was stored but never used). Pinned
|
- Removed a dead `clock` constructor param (it was stored but never used). Pinned
|
||||||
@@ -155,4 +218,4 @@ Without the extra installed, importing `aiowebhooks` still works; constructing o
|
|||||||
|
|
||||||
## Versioning
|
## Versioning
|
||||||
|
|
||||||
Tagged `vX.Y.Z`. Pin the tag.
|
Releases are tagged `vX.Y.Z`. The install line above pins a release; drop the `@vX.Y.Z` suffix to install the latest unpinned. Pin deliberately for reproducible installs.
|
||||||
|
|||||||
+2
-2
@@ -4,12 +4,12 @@ build-backend = "hatchling.build"
|
|||||||
|
|
||||||
[project]
|
[project]
|
||||||
name = "aiowebhooks"
|
name = "aiowebhooks"
|
||||||
version = "0.1.2"
|
version = "1.1.0"
|
||||||
description = "async webhook sender (aiohttp) with round-robin urls, retry, and proxy rotation; optional discord.py embeds"
|
description = "async webhook sender (aiohttp) with round-robin urls, retry, and proxy rotation; optional discord.py embeds"
|
||||||
requires-python = ">=3.10"
|
requires-python = ">=3.10"
|
||||||
dependencies = [
|
dependencies = [
|
||||||
"aiohttp>=3.9",
|
"aiohttp>=3.9",
|
||||||
"commons @ git+ssh://git@git.rethinkstudios.io/rethink-public/commons.git@v0.2.1",
|
"commons @ git+https://git.rethinkstudios.io/rethink-public/commons.git@v1.0.0",
|
||||||
]
|
]
|
||||||
|
|
||||||
[project.optional-dependencies]
|
[project.optional-dependencies]
|
||||||
|
|||||||
+10
-15
@@ -1,24 +1,19 @@
|
|||||||
"""aiowebhooks — async webhook sender (aiohttp), optional discord.py embeds.
|
"""aiowebhooks - async webhook sender (aiohttp), optional discord.py embeds.
|
||||||
|
|
||||||
post a json payload to a webhook url (or a round-robin pool) with 429/5xx retry and
|
post a json payload to a webhook url (or round-robin pool); every send returns a
|
||||||
optional proxy rotation; every send returns a WebhookResult and never raises on a
|
WebhookResult and never raises on a send failure. see README for usage. the
|
||||||
send failure. the [discord] extra adds DiscordWebhook (username/avatar + Embed
|
[discord] extra adds DiscordWebhook (aiowebhooks.discord).
|
||||||
handling) layered over the same core.
|
|
||||||
|
|
||||||
from aiowebhooks import Webhook
|
|
||||||
|
|
||||||
wh = Webhook("https://example.com/hook")
|
|
||||||
result = await wh.send({"content": "hello"})
|
|
||||||
if not result.ok:
|
|
||||||
...
|
|
||||||
|
|
||||||
DiscordWebhook lives in aiowebhooks.discord and needs the [discord] extra.
|
|
||||||
"""
|
"""
|
||||||
|
|
||||||
|
from importlib.metadata import PackageNotFoundError, version
|
||||||
|
|
||||||
from .errors import NoUrlsError, WebhookError
|
from .errors import NoUrlsError, WebhookError
|
||||||
from .result import WebhookResult
|
from .result import WebhookResult
|
||||||
from .sender import Webhook
|
from .sender import Webhook
|
||||||
|
|
||||||
__all__ = ["Webhook", "WebhookResult", "WebhookError", "NoUrlsError"]
|
__all__ = ["Webhook", "WebhookResult", "WebhookError", "NoUrlsError"]
|
||||||
|
|
||||||
__version__ = "0.1.2"
|
try:
|
||||||
|
__version__ = version("aiowebhooks")
|
||||||
|
except PackageNotFoundError:
|
||||||
|
__version__ = "0.0.0+unknown"
|
||||||
|
|||||||
@@ -2,9 +2,8 @@
|
|||||||
|
|
||||||
`DiscordWebhook` wraps a core `Webhook`, adds discord identity (username/avatar,
|
`DiscordWebhook` wraps a core `Webhook`, adds discord identity (username/avatar,
|
||||||
overridable per send) and `Embed` handling, builds the discord webhook json, and
|
overridable per send) and `Embed` handling, builds the discord webhook json, and
|
||||||
delegates the POST to the core — inheriting rotation / proxy / retry / result.
|
delegates the POST to the core. importing this module without discord.py installed
|
||||||
importing this module without discord.py installed is fine; constructing or sending
|
is fine; constructing or sending raises a clear RuntimeError naming the extra.
|
||||||
raises a clear RuntimeError naming the extra.
|
|
||||||
"""
|
"""
|
||||||
|
|
||||||
import logging
|
import logging
|
||||||
@@ -25,7 +24,7 @@ _MISSING = "discord support requires aiowebhooks[discord]"
|
|||||||
|
|
||||||
|
|
||||||
class DiscordWebhook:
|
class DiscordWebhook:
|
||||||
"""discord webhook sender — builds payloads, delegates sending to a core Webhook"""
|
"""discord webhook sender - builds payloads, delegates sending to a core Webhook"""
|
||||||
|
|
||||||
def __init__(
|
def __init__(
|
||||||
self,
|
self,
|
||||||
|
|||||||
@@ -1,10 +1,8 @@
|
|||||||
"""exception types for aiowebhooks.
|
"""exception types for aiowebhooks.
|
||||||
|
|
||||||
these are surfaced for callers that want to branch on a specific failure cause.
|
`Webhook.send` never raises on a send failure (returns `WebhookResult(ok=False)`
|
||||||
note the core `Webhook.send` does NOT raise on a send failure — it returns a
|
instead); these cover bad construction. the missing-`[discord]`-extra path raises a
|
||||||
`WebhookResult` with `ok=False` and the cause captured in `error`. these types
|
plain `RuntimeError`, not one of these.
|
||||||
exist for the few raise paths (bad construction, missing extra) and as a base for
|
|
||||||
any future raising surface.
|
|
||||||
"""
|
"""
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -1,9 +1,4 @@
|
|||||||
"""the result object every send returns.
|
"""the result object every send returns; `Webhook.send` never raises, callers branch on `ok`."""
|
||||||
|
|
||||||
`Webhook.send` never raises on a send failure; it always returns a `WebhookResult`.
|
|
||||||
callers branch on `result.ok`. success and every failure mode (4xx/5xx, timeout,
|
|
||||||
exhausted proxies) populate the same shape so call sites stay uniform.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from dataclasses import dataclass
|
from dataclasses import dataclass
|
||||||
from typing import Dict, Optional, Union
|
from typing import Dict, Optional, Union
|
||||||
|
|||||||
+64
-52
@@ -1,13 +1,14 @@
|
|||||||
"""core async webhook sender (aiohttp only, no discord knowledge).
|
"""core async webhook sender (aiohttp only, no discord knowledge).
|
||||||
|
|
||||||
`Webhook` posts a JSON dict to a url (or round-robins a pool), handling 429/5xx
|
`Webhook` posts a JSON dict to a url (or round-robins a pool), handling 429/5xx
|
||||||
retries and optional proxy rotation, and always returns a `WebhookResult` — it
|
retries, connection/timeout retries, and optional proxy rotation; always returns a
|
||||||
never raises on a send failure. the discord layer builds payloads and delegates
|
`WebhookResult`, never raises on a send failure. the discord layer delegates its
|
||||||
the actual POST here so it inherits rotation / proxy / retry / result.
|
POST here so it inherits rotation/proxy/retry/result.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
import asyncio
|
import asyncio
|
||||||
import logging
|
import logging
|
||||||
|
import math
|
||||||
from typing import Dict, List, Optional, Union
|
from typing import Dict, List, Optional, Union
|
||||||
from urllib.parse import unquote, urlsplit
|
from urllib.parse import unquote, urlsplit
|
||||||
|
|
||||||
@@ -19,13 +20,12 @@ from .result import WebhookResult
|
|||||||
|
|
||||||
log = logging.getLogger(__name__)
|
log = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
MAX_RETRY_AFTER = 300.0
|
||||||
|
"""ceiling (seconds) honored from a 429 retry_after/Retry-After; non-finite values (inf/nan) rejected outright"""
|
||||||
|
|
||||||
|
|
||||||
class _Retryable(Exception):
|
class _Retryable(Exception):
|
||||||
"""internal signal: a retryable HTTP status (429/5xx); carries the response
|
"""internal signal for commons.aretry on a retryable 429/5xx, carrying the real response"""
|
||||||
|
|
||||||
raised inside an attempt so commons.aretry drives the backoff + cap; the loop
|
|
||||||
catches the final one to return the REAL last response, not a synthetic result.
|
|
||||||
"""
|
|
||||||
|
|
||||||
def __init__(self, result: WebhookResult):
|
def __init__(self, result: WebhookResult):
|
||||||
super().__init__(f"retryable status {result.status}")
|
super().__init__(f"retryable status {result.status}")
|
||||||
@@ -33,11 +33,7 @@ class _Retryable(Exception):
|
|||||||
|
|
||||||
|
|
||||||
def _proxy_string(proxies_dict: Optional[Dict[str, str]]) -> Optional[str]:
|
def _proxy_string(proxies_dict: Optional[Dict[str, str]]) -> Optional[str]:
|
||||||
"""canonical host:port:user:pass (or host:port) from an aiohttp proxies dict
|
"""canonical host:port:user:pass (or host:port) from an aiohttp proxies dict, or None if unparseable"""
|
||||||
|
|
||||||
duck-typed: reads whatever the provider's get() returned without importing it.
|
|
||||||
returns None if the dict is empty or unparseable.
|
|
||||||
"""
|
|
||||||
if not proxies_dict:
|
if not proxies_dict:
|
||||||
return None
|
return None
|
||||||
url = proxies_dict.get("http") or proxies_dict.get("https")
|
url = proxies_dict.get("http") or proxies_dict.get("https")
|
||||||
@@ -90,18 +86,22 @@ class Webhook:
|
|||||||
|
|
||||||
@staticmethod
|
@staticmethod
|
||||||
def _retry_after(status: int, headers, body) -> Optional[float]:
|
def _retry_after(status: int, headers, body) -> Optional[float]:
|
||||||
"""seconds to wait on a 429, from body retry_after then Retry-After header"""
|
"""seconds to wait on a 429 from body retry_after then Retry-After header, clamped to MAX_RETRY_AFTER"""
|
||||||
if status != 429:
|
if status != 429:
|
||||||
return None
|
return None
|
||||||
if isinstance(body, dict) and body.get("retry_after") is not None:
|
if isinstance(body, dict) and body.get("retry_after") is not None:
|
||||||
try:
|
try:
|
||||||
return float(body["retry_after"])
|
value = float(body["retry_after"])
|
||||||
|
if math.isfinite(value):
|
||||||
|
return max(0.0, min(value, MAX_RETRY_AFTER))
|
||||||
except (TypeError, ValueError):
|
except (TypeError, ValueError):
|
||||||
pass
|
pass
|
||||||
header = headers.get("Retry-After") if headers else None
|
header = headers.get("Retry-After") if headers else None
|
||||||
if header is not None:
|
if header is not None:
|
||||||
try:
|
try:
|
||||||
return float(header)
|
value = float(header)
|
||||||
|
if math.isfinite(value):
|
||||||
|
return max(0.0, min(value, MAX_RETRY_AFTER))
|
||||||
except (TypeError, ValueError):
|
except (TypeError, ValueError):
|
||||||
pass
|
pass
|
||||||
return None
|
return None
|
||||||
@@ -110,9 +110,10 @@ class Webhook:
|
|||||||
"""post a json payload to the next pool url; always returns a result
|
"""post a json payload to the next pool url; always returns a result
|
||||||
|
|
||||||
handles 429 (wait + retry, capped by max_retries), 5xx (retry, capped), and
|
handles 429 (wait + retry, capped by max_retries), 5xx (retry, capped), and
|
||||||
timeout/connection errors with optional proxy rotation (burn + next, capped
|
timeout/connection errors: with a proxy provider, burn + rotate to the next
|
||||||
by max_proxy_retries). 4xx other than 429 fail immediately. never raises on a
|
proxy (capped by max_proxy_retries) before failing; without one, retried
|
||||||
send failure.
|
directly under max_retries like a 5xx. 4xx other than 429 fail immediately.
|
||||||
|
never raises on a send failure.
|
||||||
"""
|
"""
|
||||||
url = self._next_url()
|
url = self._next_url()
|
||||||
session = self._session
|
session = self._session
|
||||||
@@ -128,43 +129,47 @@ class Webhook:
|
|||||||
async def _send_loop(
|
async def _send_loop(
|
||||||
self, session: aiohttp.ClientSession, url: str, payload: Dict
|
self, session: aiohttp.ClientSession, url: str, payload: Dict
|
||||||
) -> WebhookResult:
|
) -> WebhookResult:
|
||||||
"""status-retry (via commons.aretry) wrapping proxy rotation; never raises
|
"""status-retry (commons.aretry, on _Retryable) wrapping proxy rotation; never raises"""
|
||||||
|
|
||||||
commons.aretry owns the 429/5xx backoff schedule + retry cap (max_retries),
|
|
||||||
retrying on the internal _Retryable signal. on exhaustion it re-raises the
|
|
||||||
last _Retryable, whose carried result is the REAL last response (not a
|
|
||||||
synthetic status-0). proxy rotation on connection errors lives inside the
|
|
||||||
attempt and is capped separately.
|
|
||||||
"""
|
|
||||||
counter = [0]
|
counter = [0]
|
||||||
|
pending_wait: List[Optional[float]] = [None]
|
||||||
try:
|
try:
|
||||||
return await aretry(
|
return await aretry(
|
||||||
lambda: self._attempt(session, url, payload, counter),
|
lambda: self._attempt(session, url, payload, counter, pending_wait),
|
||||||
attempts=self.max_retries + 1,
|
attempts=self.max_retries + 1,
|
||||||
on=(_Retryable,),
|
on=(_Retryable,),
|
||||||
)
|
)
|
||||||
except _Retryable as exhausted:
|
except _Retryable as exhausted:
|
||||||
return exhausted.result
|
return exhausted.result
|
||||||
|
except Exception as error:
|
||||||
|
# never-raises net: anything else (closed session, bad proxy url, ...) -> failed result.
|
||||||
|
# ERROR: an UNEXPECTED failure is swallowed here into a failed result (op failed, result
|
||||||
|
# lost, a human may need to act) - the lib is the last code that sees this exception.
|
||||||
|
log.error("webhook send failed unexpectedly on %s: %s", url, error, exc_info=True)
|
||||||
|
return WebhookResult(
|
||||||
|
ok=False, status=None, url=url, attempts=counter[0] or 1,
|
||||||
|
error=f"{type(error).__name__}: {error}",
|
||||||
|
)
|
||||||
|
|
||||||
async def _attempt(
|
async def _attempt(
|
||||||
self, session: aiohttp.ClientSession, url: str, payload: Dict, counter: List[int]
|
self, session: aiohttp.ClientSession, url: str, payload: Dict, counter: List[int],
|
||||||
|
pending_wait: List[Optional[float]],
|
||||||
) -> WebhookResult:
|
) -> WebhookResult:
|
||||||
"""one logical send: proxy rotation + a single POST; may raise _Retryable
|
"""one logical send: proxy rotation + a single POST; may raise _Retryable
|
||||||
|
|
||||||
raises _Retryable (carrying the real response) on a 429/5xx so the caller's
|
`counter`/`pending_wait` are per-call mutable cells threaded from `_send_loop`, not
|
||||||
aretry applies backoff; honors an explicit 429 retry_after by sleeping it
|
instance state - a plain local wouldn't survive aretry calling this fresh each retry.
|
||||||
before signalling. returns a final WebhookResult on success or a terminal
|
|
||||||
(non-retryable) failure — never lets a provider/connection error escape.
|
|
||||||
|
|
||||||
`counter` is a per-call mutable cell ([0]) owned by the calling `_send_loop`,
|
|
||||||
so the attempt count is local to one `send()` and concurrent sends on the
|
|
||||||
same instance don't corrupt each other's tally.
|
|
||||||
"""
|
"""
|
||||||
timeout = aiohttp.ClientTimeout(total=self.timeout)
|
timeout = aiohttp.ClientTimeout(total=self.timeout)
|
||||||
last_proxy: Optional[str] = None
|
last_proxy: Optional[str] = None
|
||||||
proxy_tries = 0
|
proxy_tries = 0
|
||||||
|
|
||||||
while True:
|
while True:
|
||||||
|
if pending_wait[0] is not None:
|
||||||
|
wait = pending_wait[0]
|
||||||
|
pending_wait[0] = None
|
||||||
|
log.warning("webhook 429 on %s; honoring retry_after %.3fs (attempt %d/%d)",
|
||||||
|
url, wait, counter[0] + 1, self.max_retries + 1)
|
||||||
|
await asyncio.sleep(wait)
|
||||||
counter[0] += 1
|
counter[0] += 1
|
||||||
attempts = counter[0]
|
attempts = counter[0]
|
||||||
proxy_url = None
|
proxy_url = None
|
||||||
@@ -172,9 +177,10 @@ class Webhook:
|
|||||||
try:
|
try:
|
||||||
proxy_dict = self._proxies.get()
|
proxy_dict = self._proxies.get()
|
||||||
except Exception:
|
except Exception:
|
||||||
# duck-typed provider; any error from get() means no proxy is
|
# duck-typed provider: any get() error means no proxy available.
|
||||||
# available — fail cleanly rather than escaping send().
|
# ERROR: swallowed into a failed result (op failed, result lost) - the
|
||||||
log.warning("webhook: proxy get() failed; no proxy available",
|
# provider fault vanishes here unless it is logged loudly.
|
||||||
|
log.error("webhook: proxy get() failed; no proxy available",
|
||||||
exc_info=True)
|
exc_info=True)
|
||||||
return WebhookResult(
|
return WebhookResult(
|
||||||
ok=False, status=None, url=url, attempts=attempts,
|
ok=False, status=None, url=url, attempts=attempts,
|
||||||
@@ -201,17 +207,23 @@ class Webhook:
|
|||||||
error=f"http {status}", response=body, proxy=last_proxy,
|
error=f"http {status}", response=body, proxy=last_proxy,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
if status == 429:
|
||||||
|
# every 429 retries; retry_after (if any) is carried to the NEXT
|
||||||
|
# attempt's sleep, additive with aretry's backoff
|
||||||
wait = self._retry_after(status, resp.headers, body)
|
wait = self._retry_after(status, resp.headers, body)
|
||||||
if wait is not None:
|
if wait is not None:
|
||||||
log.warning("webhook 429 on %s; honoring retry_after %.3fs", url, wait)
|
pending_wait[0] = wait
|
||||||
await asyncio.sleep(wait)
|
else:
|
||||||
|
log.warning("webhook 429 on %s; no retry_after, backing off (attempt %d/%d)",
|
||||||
|
url, attempts, self.max_retries + 1)
|
||||||
raise _Retryable(result)
|
raise _Retryable(result)
|
||||||
if status >= 500:
|
if status >= 500:
|
||||||
raise _Retryable(result)
|
raise _Retryable(result)
|
||||||
return result
|
return result
|
||||||
|
|
||||||
except (aiohttp.ClientError, asyncio.TimeoutError) as error:
|
except (aiohttp.ClientError, asyncio.TimeoutError) as error:
|
||||||
if self._proxies is not None and proxy_tries < self.max_proxy_retries:
|
if self._proxies is not None:
|
||||||
|
if proxy_tries < self.max_proxy_retries:
|
||||||
if self._burn(last_proxy):
|
if self._burn(last_proxy):
|
||||||
proxy_tries += 1
|
proxy_tries += 1
|
||||||
continue
|
continue
|
||||||
@@ -223,21 +235,21 @@ class Webhook:
|
|||||||
ok=False, status=None, url=url, attempts=attempts,
|
ok=False, status=None, url=url, attempts=attempts,
|
||||||
error=f"{type(error).__name__}: {error}", proxy=last_proxy,
|
error=f"{type(error).__name__}: {error}", proxy=last_proxy,
|
||||||
)
|
)
|
||||||
|
# no proxy provider: retry like a 5xx, under aretry's backoff + max_retries
|
||||||
|
raise _Retryable(WebhookResult(
|
||||||
|
ok=False, status=None, url=url, attempts=attempts,
|
||||||
|
error=f"{type(error).__name__}: {error}", proxy=last_proxy,
|
||||||
|
))
|
||||||
|
|
||||||
def _burn(self, proxy: Optional[str]) -> bool:
|
def _burn(self, proxy: Optional[str]) -> bool:
|
||||||
"""burn the current proxy; return False if it can't be rotated
|
"""burn the current proxy; return False (never raise) if it can't be rotated"""
|
||||||
|
|
||||||
the provider is duck-typed and never imported, so we cannot catch its
|
|
||||||
exception types by class. ANY exception from burn (a ProxiesExhaustedError
|
|
||||||
on a dead pool, a ValueError when the proxy isn't in the pool, etc.) means
|
|
||||||
we can't rotate — return False so the caller ends the loop with a failed
|
|
||||||
result rather than letting it escape send() (which must never raise).
|
|
||||||
"""
|
|
||||||
try:
|
try:
|
||||||
self._proxies.burn(proxy)
|
self._proxies.burn(proxy)
|
||||||
return True
|
return True
|
||||||
except Exception:
|
except Exception:
|
||||||
log.warning("webhook: proxy burn failed; ending rotation", exc_info=True)
|
# ERROR: an unexpected provider fault (burn() raising) is swallowed here and ends
|
||||||
|
# rotation, failing the send - it must be loud, not a quiet WARNING.
|
||||||
|
log.error("webhook: proxy burn failed; ending rotation", exc_info=True)
|
||||||
return False
|
return False
|
||||||
|
|
||||||
@staticmethod
|
@staticmethod
|
||||||
|
|||||||
Reference in New Issue
Block a user