5 Commits
Author SHA1 Message Date
dsql e1ab5d38a0 fix: cookie methods actually work; session builds lazily (v0.1.7)
get_cookies() called filter_cookies() with no URL and always returned {}
for domain-bound cookies; now iterates the jar directly. set_cookie()
ignored path and leaked a shared cookie to every host when given a bare
domain string; scheme is now normalized and path honored, with
domain=None made an intentionally shared cookie instead of a silent
localhost-only no-op.

ExtendedSession also built its aiohttp.ClientSession synchronously in
__init__, which requires a running event loop under aiohttp>=3.14 and
crashed the common construct-before-the-loop-starts host pattern. The
session now builds lazily on first access, preserving the
_create_session subclass override seam used by aioweb_tls.

Signed-off-by: disqualifier <dev@disqualifier.me>
2026-07-02 16:42:30 -04:00
dsql b8cd184c64 fix: request_with_retries honors session timeout (timeout=None no longer disables it)
request() forwarded an explicit timeout=None to aiohttp as ClientTimeout(total=None),
which disables the timeout and overrides the session default. request_with_retries
defaults timeout=None, so its flagship path had zero timeout protection: a hung server
stalled the coroutine forever, pinning connector-pool slots. Pop a None timeout in
request() so the session-level timeout applies; numeric per-call timeouts still wrap.

aioweb-1 (v0.1.6).

Signed-off-by: disqualifier <dev@disqualifier.me>
2026-07-02 16:33:28 -04:00
dsql 74ed83cf73 chore: ignore .claude/ dir (CLAUDE.md now lives under .claude/)
Signed-off-by: disqualifier <dev@disqualifier.me>
2026-06-29 21:55:13 -04:00
dsql 14a3ee1456 fix: AW-2 json() returns None on a non-UTF-8 body instead of raising
responses.json() catches UnicodeDecodeError alongside JSONDecodeError — text() can raise
it on a non-UTF-8 payload, which is a 'not valid JSON' outcome per the docstring, not an
error to propagate.

Signed-off-by: disqualifier <dev@disqualifier.me>
2026-06-29 21:34:37 -04:00
dsql 3737af0cf5 fix: total-timeout labeled 'timeout' in request_with_retries (dead branch live) (v0.1.5)
AW-1: request() wraps a total ClientTimeout's bare asyncio.TimeoutError before
request_with_retries sees it, so the dedicated 'timeout' branch was dead and its comment
lied. wrap it as aiohttp.ServerTimeoutError (which IS both a ClientError AND a
TimeoutError) so direct request() callers still get a typed failure (M1 preserved) while
request_with_retries catches the timeout case first and labels it 'timeout'.

verified by execution: request() raises ServerTimeoutError (typed, M1 intact);
request_with_retries returns reason='timeout'; control confirms a real client error still
labels 'client error'. sibling-grep: aioweb_tls/aiowebhooks catch ClientError/TimeoutError,
both of which ServerTimeoutError satisfies — no consumer break.

Signed-off-by: disqualifier <dev@disqualifier.me>
2026-06-29 20:47:55 -04:00
5 changed files with 122 additions and 25 deletions
+1 -1
View File
@@ -1,5 +1,5 @@
# claude
CLAUDE.md
.claude/
# python
__pycache__/
+26 -3
View File
@@ -11,18 +11,18 @@ and swap the HTTP client while inheriting everything else.
`requirements.txt`:
```
aioweb @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb.git@v0.1.4
aioweb @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb.git@v0.1.7
```
Direct:
```bash
pip install "aioweb @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb.git@v0.1.4"
pip install "aioweb @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb.git@v0.1.7"
```
Requires `aiohttp` and `yarl` (pulled transitively).
Drop the `@v0.1.4` suffix from the line above to install the latest unpinned.
Drop the `@v0.1.7` suffix from the line above to install the latest unpinned.
## Usage
@@ -132,6 +132,29 @@ Two changes can't be shimmed without re-introducing the bugs they fix:
## Changelog
### v0.1.7
- **`get_cookies()` now returns real cookies.** Previously called `filter_cookies()`
with no URL, which only ever returns domain-less shared cookies — every normal
domain-bound cookie (including ones set by a real `Set-Cookie` response) was
silently omitted. Now iterates the jar directly.
- **`set_cookie()` no longer leaks a shared cookie to every host.** A bare hostname
(`domain="example.com"`) built a schemeless URL, which aiohttp's jar treats as a
domain-less "shared" cookie sent with every request the session makes, including
unrelated hosts. A scheme is now added when missing so the cookie is scoped to
that host.
- **`set_cookie()` now honors `path`** (previously ignored — the cookie always
landed at `path="/"`). `domain=None` is unchanged in meaning but now stores a
truly shared cookie (sent to every host) instead of one silently bound to
`localhost` only, which made `set_cookie(name, value)` (no domain) a silent
no-op for any real request.
- **The backend session is built lazily**, not in `__init__`. Under aiohttp 3.14,
constructing `aiohttp.ClientSession` requires a running event loop; eager
construction crashed the common host pattern of attaching a session before the
loop starts (e.g. `bot.http = ExtendedSession(...)` in `Bot.__init__`). The
session (and any subclass's `_create_session` override) now builds on first
access instead.
### v0.1.2
- Pinned `commons` to v0.2.1 (retry `attempts` floor fix).
+1 -1
View File
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
[project]
name = "aioweb"
version = "0.1.4"
version = "0.1.7"
description = "Async HTTP session wrapper over aiohttp — proxies, header overwrites, retries, previews. Config-free, installable."
requires-python = ">=3.10"
dependencies = [
+3 -1
View File
@@ -92,7 +92,9 @@ class Response:
"""parsed JSON content, or None if not valid JSON"""
try:
return _json.loads(self.text())
except _json.JSONDecodeError:
except (_json.JSONDecodeError, UnicodeDecodeError):
# text() decodes the body and can raise UnicodeDecodeError on a non-UTF-8
# payload — that's a "not valid JSON" outcome, not an error to propagate
return None
def raise_for_status(self):
+91 -19
View File
@@ -12,7 +12,10 @@ subclass and override just that one method, inheriting everything else.
if resp: # FailureResponse is falsy
data = resp.json()
config-free: proxies/headers/timeouts are passed at construction or per call.
config-free: proxies/headers/timeouts are passed at construction or per call. the
backend HTTP session is built lazily on first use (request/cookie access/close), not
in __init__, so construction is safe before an event loop is running (e.g. attaching
to a host object at process startup).
sessions must be closed explicitly (async with, or await s.close()); there is no
__del__ auto-close (that pattern is unsafe for async resources).
"""
@@ -20,6 +23,7 @@ __del__ auto-close (that pattern is unsafe for async resources).
import asyncio
import logging
import warnings
from http.cookies import SimpleCookie
import aiohttp
from yarl import URL
@@ -86,7 +90,27 @@ class ExtendedSession:
self.proxies = proxies or {}
# track our own default headers instead of touching aiohttp privates
self._default_headers = dict(headers or {})
self.session = self._create_session(self._default_headers, timeout, **kwargs)
self._session_timeout = timeout
self._session_kwargs = kwargs
# aiohttp.ClientSession (via _create_session) requires a running event loop
# (aiohttp >= 3.14 raises RuntimeError otherwise); building it here would
# break the common host pattern of constructing before the loop starts
# (e.g. bot.http = ExtendedSession(...) in Bot.__init__). build lazily on
# first access instead, via the `session` property / _ensure_session().
self._session = None
@property
def session(self):
"""the backend session, built lazily on first access (needs a running loop)"""
self._ensure_session()
return self._session
def _ensure_session(self):
"""build the backend session on first use — idempotent"""
if self._session is None:
self._session = self._create_session(
self._default_headers, self._session_timeout, **self._session_kwargs,
)
def _create_session(self, headers, timeout, **kwargs):
"""create the backend HTTP session — override to use a different client
@@ -217,13 +241,36 @@ class ExtendedSession:
# cookies
def get_cookies(self):
"""cookies stored in the session jar"""
return self.session.cookie_jar.filter_cookies()
"""all cookies stored in the session jar, regardless of domain binding
iterates the jar directly rather than filter_cookies() (which needs a url
and, given none, returns only domain-less shared cookies — i.e. {} for any
normal domain-bound cookie).
"""
return {c.key: c.value for c in self.session.cookie_jar}
def set_cookie(self, name, value, domain=None, path="/"):
"""set a cookie in the session jar"""
response_url = URL(domain or "http://localhost")
self.session.cookie_jar.update_cookies({name: value}, response_url=response_url)
"""set a cookie in the session jar
domain=None (the default) stores a truly shared cookie sent with every
request regardless of host — the jar's own "no response_url" behavior.
pass domain='example.com' (a scheme is optional and defaulted to http://)
to scope the cookie to one host; a bare hostname like 'example.com' is
normalized into a URL so the jar binds it by host instead of silently
storing another domain-less shared cookie (a schemeless domain has no
raw_host, so the jar can't tell it apart from the shared case).
`path` is honored via the morsel itself, since the jar only derives a path
from response_url when the morsel doesn't already carry one.
"""
cookie = SimpleCookie()
cookie[name] = value
if domain is None:
self.session.cookie_jar.update_cookies(cookie)
return
if "://" not in domain:
domain = "http://" + domain
cookie[name]["path"] = path
self.session.cookie_jar.update_cookies(cookie, response_url=URL(domain))
def clear_cookies(self):
"""clear the session cookie jar"""
@@ -306,6 +353,12 @@ class ExtendedSession:
timeout = kwargs.get("timeout")
if isinstance(timeout, (int, float)):
kwargs["timeout"] = aiohttp.ClientTimeout(total=timeout)
elif timeout is None and "timeout" in kwargs:
# an explicit timeout=None reaches aiohttp as ClientTimeout(total=None),
# which DISABLES the timeout and overrides the session default; drop it so
# the session-level timeout applies (matters for request_with_retries, whose
# timeout kwarg defaults to None)
del kwargs["timeout"]
url = self._apply_domain_overwrites(url)
if debug:
@@ -316,10 +369,13 @@ class ExtendedSession:
if debug and result.redirect_chain:
log.info("redirect chain: %s", result.redirect_chain)
return result
except (aiohttp.ClientError, asyncio.TimeoutError) as error:
except asyncio.TimeoutError as error:
# a total ClientTimeout raises a bare asyncio.TimeoutError, which is NOT an
# aiohttp.ClientError subclass — wrap it into the same typed path so direct
# callers get a consistent failure instead of a raw timeout
# aiohttp.ClientError subclass — wrap it as ServerTimeoutError (which IS both
# a ClientError AND a TimeoutError) so direct callers get a typed failure and
# request_with_retries can still label it a timeout
raise aiohttp.ServerTimeoutError(f"timeout for {url}: {error}") from error
except aiohttp.ClientError as error:
raise aiohttp.ClientError(f"client error for {url}: {error}") from error
async def request_with_retries(
@@ -332,6 +388,10 @@ class ExtendedSession:
returns a Response on success (or non-retryable status), or a falsy
FailureResponse if every attempt fails. backoff is exponential
(backoff_base ** attempt).
timeout defaults to None, which falls back to the session-level timeout set
at construction (aioweb pops a None timeout so it does not reach aiohttp as an
infinite ClientTimeout); pass a number to override per call.
"""
attempts = attempts or DEFAULT_ATTEMPTS
body_data, body_json = _route_body(data)
@@ -364,14 +424,15 @@ class ExtendedSession:
log.error("all %d attempts failed for %s (last status %s)",
attempts, url, exhausted.response.status_code)
return exhausted.response
except asyncio.TimeoutError:
# request() wraps a total timeout as ServerTimeoutError (a ClientError AND a
# TimeoutError); catch the timeout case first so it's labeled a timeout rather
# than falling into the generic client-error branch below
log.error("all %d attempts timed out for %s", attempts, url)
return FailureResponse(reason="timeout", url=url)
except aiohttp.ClientError as error:
log.error("all %d attempts failed for %s (client error: %s)", attempts, url, error)
return FailureResponse(reason=f"client error: {error}", url=url)
except asyncio.TimeoutError:
# a total ClientTimeout surfaces as a bare asyncio.TimeoutError; label it as
# a timeout rather than letting it fall to the generic "unexpected" branch
log.error("all %d attempts timed out for %s", attempts, url)
return FailureResponse(reason="timeout", url=url)
except Exception as error:
log.error("all %d attempts failed for %s (unexpected: %s)", attempts, url, error)
return FailureResponse(reason=f"unexpected error: {error}", url=url)
@@ -380,12 +441,23 @@ class ExtendedSession:
# lifecycle
async def close(self):
"""close the backend session — override if the backend's close differs"""
await self.session.close()
"""close the backend session — override if the backend's close differs
a no-op if the session was never built (lazy construction means a session
that made no request and was never otherwise touched has nothing to close).
"""
if self._session is not None:
await self._session.close()
def _is_closed(self) -> bool:
"""whether the backend session is closed — override for non-aiohttp backends"""
return self.session.closed
"""whether the backend session is closed — override for non-aiohttp backends
an unbuilt (never-lazily-created) session counts as closed: nothing was
opened, so there is nothing to leak and __del__ should not warn.
"""
if self._session is None:
return True
return self._session.closed
async def __aenter__(self):
return self