8 Commits
Author SHA1 Message Date
dsql d3f2bed7fe docs: pin install line to release, note unpinned-latest option
Signed-off-by: disqualifier <dev@disqualifier.me>
2026-06-29 18:13:34 -04:00
dsql 849200985c docs: show unpinned install line; note tag-pinning for reproducibility
Signed-off-by: disqualifier <dev@disqualifier.me>
2026-06-29 18:07:19 -04:00
dsql 7da06443c8 fix: label timeouts, render empty-but-valid preview bodies, snapshot override dicts (v0.1.4)
- request_with_retries labels an exhausted total-timeout as 'timeout' instead of the
  generic 'unexpected error' catch-all (nit)
- as_curl() renders an empty-but-valid json body ({} / []) via is-not-None instead of
  dropping it as falsy (nit)
- _apply_overwrites snapshots the shared override dicts before iterating, so a
  concurrent mutation can't raise 'dict changed size during iteration' (nit).

Signed-off-by: disqualifier <dev@disqualifier.me>
2026-06-29 17:57:54 -04:00
dsql 382b8aa632 fix: request() wraps bare asyncio.TimeoutError on total-timeout (v0.1.3)
broaden the except to (aiohttp.ClientError, asyncio.TimeoutError) and re-wrap into
the same typed aiohttp.ClientError path. a total ClientTimeout raises a bare
asyncio.TimeoutError, which is NOT an aiohttp.ClientError subclass, so it previously
leaked raw out of request()/test_proxies(). add the missing asyncio import.

Signed-off-by: disqualifier <dev@disqualifier.me>
2026-06-29 17:09:20 -04:00
dsql d527174a2b fix: make session-default headers fully mutable; preview matches request in all cases
headers passed at construction were baked into aiohttp's ClientSession(headers=) (an immutable per-session map) AND merged in request(), a double path that made clear_headers()/update_headers() unable to remove or change what reached the wire. dropped headers= from the aiohttp session so _default_headers is our sole, mutable layer that request() and preview() both merge (defaults -> per-request -> overwrites). preview() now merges identically to request() even when explicit per-request headers are passed (it previously dropped session defaults in that case), so preview == wire in every case. clear_headers clears our defaults (not zero headers — per-request + overwrites still flow).

Signed-off-by: disqualifier <dev@disqualifier.me>
2026-06-29 01:10:25 -04:00
dsql bad3ea2677 fix: merge session-default headers into request() so the header API works
request() built outgoing headers from per-request kwargs only and never read self._default_headers, so update_headers()/clear_headers() mutated a field that never reached the wire — while preview() DID read it, so preview diverged from the real send. request() now merges _default_headers (defaults -> per-request -> overwrites), making the session-default header API functional and preview consistent with request.

Signed-off-by: disqualifier <dev@disqualifier.me>
2026-06-28 18:45:25 -04:00
dsql dc3fb70a1e fix: shlex.quote values in as_curl() so the command is valid and not injectable
header/body/url/proxy values were wrapped in raw single quotes, so a value containing a quote or shell metacharacter produced a broken or injectable command. every interpolated value is now shell-quoted.

Signed-off-by: disqualifier <dev@disqualifier.me>
2026-06-28 17:18:28 -04:00
dsql 7a2f24be9e chore: pin commons v0.2.1 (v0.1.2)
bump the commons dependency pin to v0.2.1 (retry attempts-floor fix). no code change;
the aretry migration is unaffected.

verified: 18/18 migration harness passes against commons 0.2.1.
Signed-off-by: disqualifier <dev@disqualifier.me>
2026-06-28 16:17:38 -04:00
4 changed files with 67 additions and 23 deletions
+22 -3
View File
@@ -11,17 +11,19 @@ and swap the HTTP client while inheriting everything else.
`requirements.txt`: `requirements.txt`:
``` ```
aioweb @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb.git@v0.1.0 aioweb @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb.git@v0.1.4
``` ```
Direct: Direct:
```bash ```bash
pip install "aioweb @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb.git@v0.1.0" pip install "aioweb @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb.git@v0.1.4"
``` ```
Requires `aiohttp` and `yarl` (pulled transitively). Requires `aiohttp` and `yarl` (pulled transitively).
Drop the `@v0.1.4` suffix from the line above to install the latest unpinned.
## Usage ## Usage
```python ```python
@@ -128,6 +130,23 @@ Two changes can't be shimmed without re-introducing the bugs they fix:
`await s.close()`; a leaked session emits a `ResourceWarning`. The old finalizer-based `await s.close()`; a leaked session emits a `ResourceWarning`. The old finalizer-based
auto-close was unsafe and was removed. auto-close was unsafe and was removed.
## Changelog
### v0.1.2
- Pinned `commons` to v0.2.1 (retry `attempts` floor fix).
### v0.1.1
- **JSON list bodies** now route to `json=` (were wrongly form-encoded via `data=`
only dicts went to `json=` before).
- **Exhausted retries return the real last response.** When every attempt hit a
retryable status (429/5xx), the loop discarded it and returned a synthetic
`FailureResponse` (status 0); now the real last 4xx/5xx `Response` is returned (only a
pure-exception failure yields `FailureResponse`).
- Retry/backoff moved onto `commons.aretry` (shared engine); backoff schedule unchanged.
Adds a `commons` dependency.
## Versioning ## Versioning
Tagged `vX.Y.Z`. Pin the tag in `requirements.txt`. Releases are tagged `vX.Y.Z`. The install line above pins a release; drop the `@vX.Y.Z` suffix to install the latest unpinned. Pin deliberately for reproducible installs.
+2 -2
View File
@@ -4,13 +4,13 @@ build-backend = "hatchling.build"
[project] [project]
name = "aioweb" name = "aioweb"
version = "0.1.1" version = "0.1.4"
description = "Async HTTP session wrapper over aiohttp — proxies, header overwrites, retries, previews. Config-free, installable." description = "Async HTTP session wrapper over aiohttp — proxies, header overwrites, retries, previews. Config-free, installable."
requires-python = ">=3.10" requires-python = ">=3.10"
dependencies = [ dependencies = [
"aiohttp>=3.9", "aiohttp>=3.9",
"yarl>=1.9", "yarl>=1.9",
"commons @ git+ssh://git@git.rethinkstudios.io/rethink-public/commons.git@v0.2.0", "commons @ git+ssh://git@git.rethinkstudios.io/rethink-public/commons.git@v0.2.1",
] ]
[tool.hatch.metadata] [tool.hatch.metadata]
+17 -9
View File
@@ -3,6 +3,7 @@ request preview for aioweb — format or export a request without sending it
""" """
import json as _json import json as _json
import shlex
class RequestPreview: class RequestPreview:
@@ -25,15 +26,22 @@ class RequestPreview:
return "\n".join(f"{key}: {value}" for key, value in self.details.items()) return "\n".join(f"{key}: {value}" for key, value in self.details.items())
def as_curl(self): def as_curl(self):
"""equivalent cURL command for the request""" """equivalent cURL command for the request
parts = [f"curl -X {self.details['method']}"]
every interpolated value is shell-quoted with shlex.quote, so headers,
body, url, or proxy containing quotes/spaces/metacharacters produce a
valid, non-injectable command rather than a broken or unsafe one.
"""
parts = [f"curl -X {shlex.quote(self.details['method'])}"]
for header, value in (self.details["headers"] or {}).items(): for header, value in (self.details["headers"] or {}).items():
parts.append(f"-H '{header}: {value}'") parts.append(f"-H {shlex.quote(f'{header}: {value}')}")
if self.details["data"]: if self.details["data"] is not None:
parts.append(f"--data '{self.details['data']}'") parts.append(f"--data {shlex.quote(str(self.details['data']))}")
elif self.details["json"]: elif self.details["json"] is not None:
parts.append(f"--data '{_json.dumps(self.details['json'])}'") # is-not-None, not truthiness: an empty-but-valid body ({} / []) must still
parts.append(f"'{self.details['url']}'") # render rather than being dropped as falsy
parts.append(f"--data {shlex.quote(_json.dumps(self.details['json']))}")
parts.append(shlex.quote(str(self.details["url"])))
if self.details["proxy"]: if self.details["proxy"]:
parts.append(f"--proxy '{self.details['proxy']}'") parts.append(f"--proxy {shlex.quote(str(self.details['proxy']))}")
return " \\\n ".join(parts) return " \\\n ".join(parts)
+26 -9
View File
@@ -17,6 +17,7 @@ sessions must be closed explicitly (async with, or await s.close()); there is no
__del__ auto-close (that pattern is unsafe for async resources). __del__ auto-close (that pattern is unsafe for async resources).
""" """
import asyncio
import logging import logging
import warnings import warnings
@@ -95,9 +96,15 @@ class ExtendedSession:
proxy/retry/preview logic in this class never touches the session object proxy/retry/preview logic in this class never touches the session object
directly (only _raw_request, the cookie methods, and close do), so those directly (only _raw_request, the cookie methods, and close do), so those
features work unchanged on any backend. features work unchanged on any backend.
`headers` is the session-default header set; the default aiohttp backend
does NOT bake it into the ClientSession (which would copy it into an
immutable per-session map that update_headers/clear_headers can't touch).
instead `_default_headers` is our own mutable layer that request() and
preview() merge per call, so the mutable session-header API actually works.
a backend that needs the defaults baked at construction may use `headers`.
""" """
return aiohttp.ClientSession( return aiohttp.ClientSession(
headers=headers,
timeout=aiohttp.ClientTimeout( timeout=aiohttp.ClientTimeout(
total=timeout, total=timeout,
connect=timeout / 2, connect=timeout / 2,
@@ -117,10 +124,13 @@ class ExtendedSession:
def _apply_overwrites(self, request_headers): def _apply_overwrites(self, request_headers):
"""apply static overwrites and ephemeral headers to a request's headers""" """apply static overwrites and ephemeral headers to a request's headers"""
request_headers = dict(request_headers or {}) request_headers = dict(request_headers or {})
for header, value in self.header_overwrites.items(): # snapshot the shared override dicts so a concurrent mutation (e.g. a command
# editing ephemerals on a shared session) can't raise "dict changed size during
# iteration" — the loop body is sync, but the snapshot is cheap insurance
for header, value in list(self.header_overwrites.items()):
if self.inject or header in request_headers: if self.inject or header in request_headers:
request_headers[header] = value request_headers[header] = value
for header, value_callable in self.ephemeral_headers.items(): for header, value_callable in list(self.ephemeral_headers.items()):
if self.inject or header in request_headers: if self.inject or header in request_headers:
value = value_callable() value = value_callable()
if isinstance(value, dict): if isinstance(value, dict):
@@ -233,10 +243,8 @@ class ExtendedSession:
def preview(self, method, url, **kwargs): def preview(self, method, url, **kwargs):
"""build a RequestPreview for a request without sending it""" """build a RequestPreview for a request without sending it"""
proxy = self._get_proxy(url, kwargs.pop("proxies", None)) proxy = self._get_proxy(url, kwargs.pop("proxies", None))
if kwargs.get("headers"): merged = {**self._default_headers, **(kwargs.pop("headers", None) or {})}
headers = self._apply_overwrites(kwargs.pop("headers")) headers = self._apply_overwrites(merged)
else:
headers = dict(self.get_headers())
timeout = kwargs.get("timeout") timeout = kwargs.get("timeout")
timeout_total = timeout if isinstance(timeout, (int, float)) else None timeout_total = timeout if isinstance(timeout, (int, float)) else None
@@ -291,7 +299,8 @@ class ExtendedSession:
kwargs["proxy"] = self._get_proxy(url, kwargs.pop("proxies", None)) kwargs["proxy"] = self._get_proxy(url, kwargs.pop("proxies", None))
debug = kwargs.pop("debug", False) debug = kwargs.pop("debug", False)
kwargs["headers"] = self._apply_overwrites(kwargs.get("headers")) merged = {**self._default_headers, **(kwargs.get("headers") or {})}
kwargs["headers"] = self._apply_overwrites(merged)
kwargs["headers"] = {str(k): str(v) for k, v in kwargs["headers"].items()} kwargs["headers"] = {str(k): str(v) for k, v in kwargs["headers"].items()}
timeout = kwargs.get("timeout") timeout = kwargs.get("timeout")
@@ -307,7 +316,10 @@ class ExtendedSession:
if debug and result.redirect_chain: if debug and result.redirect_chain:
log.info("redirect chain: %s", result.redirect_chain) log.info("redirect chain: %s", result.redirect_chain)
return result return result
except aiohttp.ClientError as error: except (aiohttp.ClientError, asyncio.TimeoutError) as error:
# a total ClientTimeout raises a bare asyncio.TimeoutError, which is NOT an
# aiohttp.ClientError subclass — wrap it into the same typed path so direct
# callers get a consistent failure instead of a raw timeout
raise aiohttp.ClientError(f"client error for {url}: {error}") from error raise aiohttp.ClientError(f"client error for {url}: {error}") from error
async def request_with_retries( async def request_with_retries(
@@ -355,6 +367,11 @@ class ExtendedSession:
except aiohttp.ClientError as error: except aiohttp.ClientError as error:
log.error("all %d attempts failed for %s (client error: %s)", attempts, url, error) log.error("all %d attempts failed for %s (client error: %s)", attempts, url, error)
return FailureResponse(reason=f"client error: {error}", url=url) return FailureResponse(reason=f"client error: {error}", url=url)
except asyncio.TimeoutError:
# a total ClientTimeout surfaces as a bare asyncio.TimeoutError; label it as
# a timeout rather than letting it fall to the generic "unexpected" branch
log.error("all %d attempts timed out for %s", attempts, url)
return FailureResponse(reason="timeout", url=url)
except Exception as error: except Exception as error:
log.error("all %d attempts failed for %s (unexpected: %s)", attempts, url, error) log.error("all %d attempts failed for %s (unexpected: %s)", attempts, url, error)
return FailureResponse(reason=f"unexpected error: {error}", url=url) return FailureResponse(reason=f"unexpected error: {error}", url=url)