fix: preserve exception subtype in request(), render params/timeout in as_curl()

request() re-raised every aiohttp.ClientError subclass as the bare base class,
losing ClientConnectorError/ClientProxyConnectionError/ClientResponseError
(.status/.headers)/TooManyRedirects and their attributes; direct callers
branching by type never matched. Now the original exception is bare-raised,
preserving subtype, attributes, and __cause__. request_with_retries (which
catches the base ClientError) is unaffected.

as_curl() silently dropped params and timeout, so a debug=True replay of a
params-driven request hit a different URL than the one actually sent. params
are now merged into the URL via yarl before quoting, and timeout renders as
--max-time.

Bumps 0.1.7 -> 0.1.8.

Signed-off-by: disqualifier <dev@disqualifier.me>
This commit is contained in:
2026-07-02 16:57:04 -04:00
parent e1ab5d38a0
commit ebfd7bf35b
4 changed files with 61 additions and 9 deletions
+33 -4
View File
@@ -11,18 +11,18 @@ and swap the HTTP client while inheriting everything else.
`requirements.txt`: `requirements.txt`:
``` ```
aioweb @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb.git@v0.1.7 aioweb @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb.git@v0.1.8
``` ```
Direct: Direct:
```bash ```bash
pip install "aioweb @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb.git@v0.1.7" pip install "aioweb @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb.git@v0.1.8"
``` ```
Requires `aiohttp` and `yarl` (pulled transitively). Requires `aiohttp` and `yarl` (pulled transitively).
Drop the `@v0.1.7` suffix from the line above to install the latest unpinned. Drop the `@v0.1.8` suffix from the line above to install the latest unpinned.
## Usage ## Usage
@@ -65,6 +65,15 @@ resp = await s.request_with_retries(
Returns a `FailureResponse` (falsy) if every attempt fails. Returns a `FailureResponse` (falsy) if every attempt fails.
`request()` (the non-retrying call) raises on failure: a total timeout raises
`aiohttp.ServerTimeoutError`, and any other network/protocol failure raises its real
`aiohttp.ClientError` subtype as-is (`ClientConnectorError`, `ClientProxyConnectionError`,
`ClientResponseError` with `.status`/`.headers`, `TooManyRedirects`, ...) — it is not
flattened into the base `ClientError`, so direct callers can branch by type or read
subtype attributes. `request_with_retries` catches the base `aiohttp.ClientError` (and
`asyncio.TimeoutError`) across all attempts and returns a falsy `FailureResponse` instead
of raising.
## Header overwrites & ephemeral headers ## Header overwrites & ephemeral headers
```python ```python
@@ -88,7 +97,10 @@ s.overwrite_domain("internal.local", "127.0.0.1") # host-substring rewrite
print(s.preview("POST", url, json={"a": 1}).as_curl()) # equivalent cURL command print(s.preview("POST", url, json={"a": 1}).as_curl()) # equivalent cURL command
``` ```
Pass `debug=True` to `request_with_retries` to log the cURL preview and request flow. `as_curl()` renders `params` (merged into the url's query string) and `timeout` (as
`--max-time`) as well as headers/body/proxy, so the emitted command is faithful to what
`request()` actually sends. Pass `debug=True` to `request_with_retries` to log the cURL
preview and request flow.
## Custom backends ## Custom backends
@@ -132,6 +144,23 @@ Two changes can't be shimmed without re-introducing the bugs they fix:
## Changelog ## Changelog
### v0.1.8
- **`request()` no longer flattens `aiohttp.ClientError` subtypes.** Every failure
(connect errors, proxy errors, `raise_for_status()`-style response errors,
redirect limits, ...) was re-raised as a bare `aiohttp.ClientError`, losing the
real subtype and its attributes (`.os_error`, `.status`, `.headers`, ...) — a
direct caller doing `except ClientProxyConnectionError:` or `if e.status == 401`
would silently never match. Now the original exception is re-raised as-is (its
subtype, attributes, and `__cause__` all preserved). `request_with_retries`
still catches the base `aiohttp.ClientError` across attempts, so its behavior
(and its `FailureResponse` return on exhaustion) is unchanged.
- **`as_curl()` now renders `params` and `timeout`.** Previously a preview built
with `params=` silently omitted the query string (and a `timeout=` omitted
`--max-time`), so a `debug=True` cURL replay of a params-driven request hit a
different URL than the one actually sent. `params` are now merged into the url's
query string (via `yarl`) and `timeout` is emitted as `--max-time`.
### v0.1.7 ### v0.1.7
- **`get_cookies()` now returns real cookies.** Previously called `filter_cookies()` - **`get_cookies()` now returns real cookies.** Previously called `filter_cookies()`
+1 -1
View File
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
[project] [project]
name = "aioweb" name = "aioweb"
version = "0.1.7" version = "0.1.8"
description = "Async HTTP session wrapper over aiohttp — proxies, header overwrites, retries, previews. Config-free, installable." description = "Async HTTP session wrapper over aiohttp — proxies, header overwrites, retries, previews. Config-free, installable."
requires-python = ">=3.10" requires-python = ">=3.10"
dependencies = [ dependencies = [
+12 -2
View File
@@ -5,6 +5,8 @@ request preview for aioweb — format or export a request without sending it
import json as _json import json as _json
import shlex import shlex
from yarl import URL
class RequestPreview: class RequestPreview:
"""a formatted preview of an HTTP request (does not send)""" """a formatted preview of an HTTP request (does not send)"""
@@ -30,7 +32,10 @@ class RequestPreview:
every interpolated value is shell-quoted with shlex.quote, so headers, every interpolated value is shell-quoted with shlex.quote, so headers,
body, url, or proxy containing quotes/spaces/metacharacters produce a body, url, or proxy containing quotes/spaces/metacharacters produce a
valid, non-injectable command rather than a broken or unsafe one. valid, non-injectable command rather than a broken or unsafe one. `params`
is merged into the url's query string (via yarl) and `timeout` is rendered
as `--max-time`, so the emitted command is faithful to what request()
actually sends rather than dropping either silently.
""" """
parts = [f"curl -X {shlex.quote(self.details['method'])}"] parts = [f"curl -X {shlex.quote(self.details['method'])}"]
for header, value in (self.details["headers"] or {}).items(): for header, value in (self.details["headers"] or {}).items():
@@ -41,7 +46,12 @@ class RequestPreview:
# is-not-None, not truthiness: an empty-but-valid body ({} / []) must still # is-not-None, not truthiness: an empty-but-valid body ({} / []) must still
# render rather than being dropped as falsy # render rather than being dropped as falsy
parts.append(f"--data {shlex.quote(_json.dumps(self.details['json']))}") parts.append(f"--data {shlex.quote(_json.dumps(self.details['json']))}")
parts.append(shlex.quote(str(self.details["url"]))) url = self.details["url"]
if self.details["params"]:
url = str(URL(url).update_query(self.details["params"]))
parts.append(shlex.quote(str(url)))
if self.details["proxy"]: if self.details["proxy"]:
parts.append(f"--proxy {shlex.quote(str(self.details['proxy']))}") parts.append(f"--proxy {shlex.quote(str(self.details['proxy']))}")
if self.details["timeout"] is not None:
parts.append(f"--max-time {shlex.quote(str(self.details['timeout']))}")
return " \\\n ".join(parts) return " \\\n ".join(parts)
+15 -2
View File
@@ -342,7 +342,14 @@ class ExtendedSession:
) )
async def request(self, method, url, **kwargs) -> Response: async def request(self, method, url, **kwargs) -> Response:
"""make a request, applying overwrites/domain rewrites/proxy resolution""" """make a request, applying overwrites/domain rewrites/proxy resolution
raises the real exception subtype on failure: a total timeout raises
aiohttp.ServerTimeoutError, and any other aiohttp.ClientError (e.g.
ClientConnectorError, ClientProxyConnectionError, ClientResponseError with
.status/.headers, TooManyRedirects) is re-raised as-is, not flattened into
the base ClientError — callers can branch by type or read subtype attributes.
"""
kwargs["proxy"] = self._get_proxy(url, kwargs.pop("proxies", None)) kwargs["proxy"] = self._get_proxy(url, kwargs.pop("proxies", None))
debug = kwargs.pop("debug", False) debug = kwargs.pop("debug", False)
@@ -376,7 +383,13 @@ class ExtendedSession:
# request_with_retries can still label it a timeout # request_with_retries can still label it a timeout
raise aiohttp.ServerTimeoutError(f"timeout for {url}: {error}") from error raise aiohttp.ServerTimeoutError(f"timeout for {url}: {error}") from error
except aiohttp.ClientError as error: except aiohttp.ClientError as error:
raise aiohttp.ClientError(f"client error for {url}: {error}") from error # re-raise the ORIGINAL subtype (ClientConnectorError, ClientProxyConnectionError,
# ClientResponseError with .status/.headers, TooManyRedirects, ...) instead of
# flattening into the base class — direct callers branching by type or reading
# subtype attributes need the real exception; request_with_retries still catches
# the base aiohttp.ClientError below and is unaffected
log.error("client error for %s: %s", url, error)
raise
async def request_with_retries( async def request_with_retries(
self, method, url, *, data=None, proxies=None, timeout=None, attempts=None, self, method, url, *, data=None, proxies=None, timeout=None, attempts=None,