fix: proxy/attempts truthiness bugs, stale text() cache, preview cookie order (v0.1.9)

_get_proxy() and request() checked proxies/proxy with truthiness instead of
is None, so a per-call proxies={} silently fell back to session proxies, and a
native proxy= kwarg was clobbered by resolved session proxies (unmasking the
caller's real IP) instead of raising on conflicting sources. Response.text()
cached the first decode regardless of a later explicit encoding= argument.
preview() computed the Cookie header from the pre-rewrite url instead of the
rewritten one actually used by request(). request_with_retries(attempts=0)
became DEFAULT_ATTEMPTS via truthiness instead of flooring to 1.

Also compresses docstrings/comments across the module (no behavior change).

Signed-off-by: disqualifier <dev@disqualifier.me>

README.md

@@ -11,18 +11,18 @@ and swap the HTTP client while inheriting everything else.
 `requirements.txt`:
 
 ```
-aioweb @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb.git@v0.1.5
+aioweb @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb.git@v0.1.9
 ```
 
 Direct:
 
 ```bash
-pip install "aioweb @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb.git@v0.1.5"
+pip install "aioweb @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb.git@v0.1.9"
 ```
 
 Requires `aiohttp` and `yarl` (pulled transitively).
 
-Drop the `@v0.1.5` suffix from the line above to install the latest unpinned.
+Drop the `@v0.1.9` suffix from the line above to install the latest unpinned.
 
 ## Usage
 
@@ -65,6 +65,15 @@ resp = await s.request_with_retries(
 
 Returns a `FailureResponse` (falsy) if every attempt fails.
 
+`request()` (the non-retrying call) raises on failure: a total timeout raises
+`aiohttp.ServerTimeoutError`, and any other network/protocol failure raises its real
+`aiohttp.ClientError` subtype as-is (`ClientConnectorError`, `ClientProxyConnectionError`,
+`ClientResponseError` with `.status`/`.headers`, `TooManyRedirects`, ...) — it is not
+flattened into the base `ClientError`, so direct callers can branch by type or read
+subtype attributes. `request_with_retries` catches the base `aiohttp.ClientError` (and
+`asyncio.TimeoutError`) across all attempts and returns a falsy `FailureResponse` instead
+of raising.
+
 ## Header overwrites & ephemeral headers
 
 ```python
@@ -88,7 +97,10 @@ s.overwrite_domain("internal.local", "127.0.0.1")   # host-substring rewrite
 print(s.preview("POST", url, json={"a": 1}).as_curl())   # equivalent cURL command
 ```
 
-Pass `debug=True` to `request_with_retries` to log the cURL preview and request flow.
+`as_curl()` renders `params` (merged into the url's query string) and `timeout` (as
+`--max-time`) as well as headers/body/proxy, so the emitted command is faithful to what
+`request()` actually sends. Pass `debug=True` to `request_with_retries` to log the cURL
+preview and request flow.
 
 ## Custom backends
 
@@ -132,6 +144,64 @@ Two changes can't be shimmed without re-introducing the bugs they fix:
 
 ## Changelog
 
+### v0.1.9
+
+- **`_get_proxy()`/`request()` proxy resolution now checks `is None`, not
+  truthiness.** A per-call `proxies={}` previously fell back to the session's
+  configured proxies instead of disabling them for that call. `request()` also no
+  longer clobbers a native `proxy=` kwarg with the resolved session/`proxies=`
+  value (a real IP-unmasking leak) — passing both now raises `ValueError` instead
+  of silently picking one.
+- **`Response.text(encoding=...)` no longer returns a stale cached decode.** A
+  second call with an explicit `encoding=` previously still returned the first
+  (possibly differently-encoded) cached decode; an explicit encoding now bypasses
+  the cache.
+- **`preview()` now rewrites the url before resolving cookies**, matching what
+  `request()` actually sends — previously cookies were resolved against the
+  pre-rewrite host, which could miss or misattribute host-bound cookies.
+- **`request_with_retries(attempts=0)` now floors to 1 attempt**, not silently
+  `DEFAULT_ATTEMPTS` (3). `attempts` is checked with `is None`, not truthiness.
+
+### v0.1.8
+
+- **`request()` no longer flattens `aiohttp.ClientError` subtypes.** Every failure
+  (connect errors, proxy errors, `raise_for_status()`-style response errors,
+  redirect limits, ...) was re-raised as a bare `aiohttp.ClientError`, losing the
+  real subtype and its attributes (`.os_error`, `.status`, `.headers`, ...) — a
+  direct caller doing `except ClientProxyConnectionError:` or `if e.status == 401`
+  would silently never match. Now the original exception is re-raised as-is (its
+  subtype, attributes, and `__cause__` all preserved). `request_with_retries`
+  still catches the base `aiohttp.ClientError` across attempts, so its behavior
+  (and its `FailureResponse` return on exhaustion) is unchanged.
+- **`as_curl()` now renders `params` and `timeout`.** Previously a preview built
+  with `params=` silently omitted the query string (and a `timeout=` omitted
+  `--max-time`), so a `debug=True` cURL replay of a params-driven request hit a
+  different URL than the one actually sent. `params` are now merged into the url's
+  query string (via `yarl`) and `timeout` is emitted as `--max-time`.
+
+### v0.1.7
+
+- **`get_cookies()` now returns real cookies.** Previously called `filter_cookies()`
+  with no URL, which only ever returns domain-less shared cookies — every normal
+  domain-bound cookie (including ones set by a real `Set-Cookie` response) was
+  silently omitted. Now iterates the jar directly.
+- **`set_cookie()` no longer leaks a shared cookie to every host.** A bare hostname
+  (`domain="example.com"`) built a schemeless URL, which aiohttp's jar treats as a
+  domain-less "shared" cookie sent with every request the session makes, including
+  unrelated hosts. A scheme is now added when missing so the cookie is scoped to
+  that host.
+- **`set_cookie()` now honors `path`** (previously ignored — the cookie always
+  landed at `path="/"`). `domain=None` is unchanged in meaning but now stores a
+  truly shared cookie (sent to every host) instead of one silently bound to
+  `localhost` only, which made `set_cookie(name, value)` (no domain) a silent
+  no-op for any real request.
+- **The backend session is built lazily**, not in `__init__`. Under aiohttp 3.14,
+  constructing `aiohttp.ClientSession` requires a running event loop; eager
+  construction crashed the common host pattern of attaching a session before the
+  loop starts (e.g. `bot.http = ExtendedSession(...)` in `Bot.__init__`). The
+  session (and any subclass's `_create_session` override) now builds on first
+  access instead.
+
 ### v0.1.2
 
 - Pinned `commons` to v0.2.1 (retry `attempts` floor fix).

pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "aioweb"
-version = "0.1.5"
+version = "0.1.9"
 description = "Async HTTP session wrapper over aiohttp — proxies, header overwrites, retries, previews. Config-free, installable."
 requires-python = ">=3.10"
 dependencies = [

src/aioweb/preview.py

@@ -5,6 +5,8 @@ request preview for aioweb — format or export a request without sending it
 import json as _json
 import shlex
 
+from yarl import URL
+
 
 class RequestPreview:
     """a formatted preview of an HTTP request (does not send)"""
@@ -28,9 +30,10 @@ class RequestPreview:
     def as_curl(self):
         """equivalent cURL command for the request
 
-        every interpolated value is shell-quoted with shlex.quote, so headers,
-        body, url, or proxy containing quotes/spaces/metacharacters produce a
-        valid, non-injectable command rather than a broken or unsafe one.
+        every interpolated value is shlex.quote'd (non-injectable even with quotes/
+        spaces/metacharacters). `params` is merged into the url's query string and
+        `timeout` rendered as `--max-time`, so the command matches what request()
+        actually sends.
         """
         parts = [f"curl -X {shlex.quote(self.details['method'])}"]
         for header, value in (self.details["headers"] or {}).items():
@@ -38,10 +41,14 @@ class RequestPreview:
         if self.details["data"] is not None:
             parts.append(f"--data {shlex.quote(str(self.details['data']))}")
         elif self.details["json"] is not None:
-            # is-not-None, not truthiness: an empty-but-valid body ({} / []) must still
-            # render rather than being dropped as falsy
+            # is-not-None: an empty-but-valid body ({} / []) must still render
             parts.append(f"--data {shlex.quote(_json.dumps(self.details['json']))}")
-        parts.append(shlex.quote(str(self.details["url"])))
+        url = self.details["url"]
+        if self.details["params"]:
+            url = str(URL(url).update_query(self.details["params"]))
+        parts.append(shlex.quote(str(url)))
         if self.details["proxy"]:
             parts.append(f"--proxy {shlex.quote(str(self.details['proxy']))}")
+        if self.details["timeout"] is not None:
+            parts.append(f"--max-time {shlex.quote(str(self.details['timeout']))}")
         return " \\\n  ".join(parts)

src/aioweb/responses.py

@@ -1,10 +1,8 @@
 """
-backend-agnostic response objects for aioweb
-
-Response is built from primitives (status, headers, content, url, history) rather
-than holding a raw aiohttp object, so any backend can produce one. FailureResponse
-mirrors the same surface so callers can branch uniformly — every status/predicate is
-a property on both.
+backend-agnostic response objects for aioweb — Response is built from primitives
+(status, headers, content, url, history), not a raw aiohttp object, so any backend can
+produce one; FailureResponse mirrors the same surface (every status/predicate a
+property on both) so callers can branch uniformly.
 """
 
 import json as _json
@@ -83,9 +81,11 @@ class Response:
         return self._content
 
     def text(self, encoding: Optional[str] = None) -> str:
-        """decoded text content (cached)"""
+        """decoded text content, cached only for the default encoding"""
+        if encoding is not None:
+            return self._content.decode(encoding)
         if self._text is None:
-            self._text = self._content.decode(encoding or self._encoding)
+            self._text = self._content.decode(self._encoding)
         return self._text
 
     def json(self):

src/aioweb/session.py

@@ -1,25 +1,24 @@
 """
-async HTTP session wrapper over aiohttp
-
-ExtendedSession adds session-level proxies, header overwrites, ephemeral
-(per-request generated) headers, domain rewriting, request previews, and
-retry/backoff on top of aiohttp. The actual byte-sending is isolated in
-_raw_request() so a different backend (e.g. a TLS-fingerprinting client) can
-subclass and override just that one method, inheriting everything else.
+async HTTP session wrapper over aiohttp — proxies, header overwrites, ephemeral
+headers, domain rewriting, previews, retry/backoff; byte-sending is isolated in
+_raw_request() so a subclass can swap backends and inherit everything else.
 
     async with ExtendedSession(proxies={"https": "http://..."}) as s:
         resp = await s.request_with_retries("GET", url)
         if resp:                       # FailureResponse is falsy
             data = resp.json()
 
-config-free: proxies/headers/timeouts are passed at construction or per call.
-sessions must be closed explicitly (async with, or await s.close()); there is no
-__del__ auto-close (that pattern is unsafe for async resources).
+config-free (proxies/headers/timeouts passed at construction or per call). the
+backend session is built lazily on first use, not in __init__, so construction is
+safe before an event loop is running (e.g. bot.http = ExtendedSession(...)).
+sessions must be closed explicitly (async with, or await s.close()) — no __del__
+auto-close (unsafe for async resources).
 """
 
 import asyncio
 import logging
 import warnings
+from http.cookies import SimpleCookie
 
 import aiohttp
 from yarl import URL
@@ -30,12 +29,7 @@ from .responses import Response, FailureResponse
 
 
 def _route_body(data):
-    """split a body into (data=, json=) kwargs
-
-    dict OR list bodies are valid JSON and route to json=; everything else
-    (str/bytes/form) routes to data=. previously only dicts went to json=, so a
-    JSON list was wrongly form-encoded.
-    """
+    """split a body into (data=, json=) kwargs; dict OR list routes to json=, else data="""
     if isinstance(data, (dict, list)):
         return None, data
     return data, None
@@ -84,25 +78,36 @@ class ExtendedSession:
         self.domain_overwrites = domain_overwrites or {}
         self.ephemeral_headers = {}
         self.proxies = proxies or {}
-        # track our own default headers instead of touching aiohttp privates
+        # own header layer, not aiohttp privates, so update_headers/clear_headers work
         self._default_headers = dict(headers or {})
-        self.session = self._create_session(self._default_headers, timeout, **kwargs)
+        self._session_timeout = timeout
+        self._session_kwargs = kwargs
+        # aiohttp>=3.14 requires a running loop to build ClientSession; build lazily
+        # (via `session` / _ensure_session()) so construction before the loop starts
+        # (e.g. bot.http = ExtendedSession(...) in Bot.__init__) doesn't crash
+        self._session = None
+
+    @property
+    def session(self):
+        """the backend session, built lazily on first access (needs a running loop)"""
+        self._ensure_session()
+        return self._session
+
+    def _ensure_session(self):
+        """build the backend session on first use — idempotent"""
+        if self._session is None:
+            self._session = self._create_session(
+                self._default_headers, self._session_timeout, **self._session_kwargs,
+            )
 
     def _create_session(self, headers, timeout, **kwargs):
-        """create the backend HTTP session — override to use a different client
+        """create the backend HTTP session — override to swap HTTP clients
 
-        a subclass swapping the HTTP backend (e.g. a TLS-fingerprinting client)
-        overrides this to return its own session object. the overwrite/domain/
-        proxy/retry/preview logic in this class never touches the session object
-        directly (only _raw_request, the cookie methods, and close do), so those
-        features work unchanged on any backend.
-
-        `headers` is the session-default header set; the default aiohttp backend
-        does NOT bake it into the ClientSession (which would copy it into an
-        immutable per-session map that update_headers/clear_headers can't touch).
-        instead `_default_headers` is our own mutable layer that request() and
-        preview() merge per call, so the mutable session-header API actually works.
-        a backend that needs the defaults baked at construction may use `headers`.
+        overwrite/domain/proxy/retry/preview logic never touches the session object
+        directly (only _raw_request, cookies, close do), so those work unchanged on
+        any backend. `headers` isn't baked in here — aiohttp would copy it into an
+        immutable map update_headers/clear_headers can't touch; `_default_headers`
+        is the mutable layer request()/preview() merge per call instead.
         """
         return aiohttp.ClientSession(
             timeout=aiohttp.ClientTimeout(
@@ -124,9 +129,8 @@ class ExtendedSession:
     def _apply_overwrites(self, request_headers):
         """apply static overwrites and ephemeral headers to a request's headers"""
         request_headers = dict(request_headers or {})
-        # snapshot the shared override dicts so a concurrent mutation (e.g. a command
-        # editing ephemerals on a shared session) can't raise "dict changed size during
-        # iteration" — the loop body is sync, but the snapshot is cheap insurance
+        # list()-snapshot so a concurrent mutation of the shared dicts can't raise
+        # "dict changed size during iteration"
         for header, value in list(self.header_overwrites.items()):
             if self.inject or header in request_headers:
                 request_headers[header] = value
@@ -208,8 +212,13 @@ class ExtendedSession:
         self.proxies.clear()
 
     def _get_proxy(self, url, proxies=None):
-        """resolve the proxy for a url's scheme"""
-        proxies = proxies or self.proxies
+        """resolve the proxy for a url's scheme
+
+        proxies is checked with `is None`, not truthiness, so an explicit
+        proxies={} disables session proxies for that one call instead of
+        falling back to them.
+        """
+        proxies = self.proxies if proxies is None else proxies
         scheme = url.split("://")[0]
         return proxies.get(scheme)
 
@@ -217,31 +226,50 @@ class ExtendedSession:
     #   cookies
 
     def get_cookies(self):
-        """cookies stored in the session jar"""
-        return self.session.cookie_jar.filter_cookies()
+        """all cookies in the session jar, regardless of domain binding
+
+        iterates the jar directly rather than filter_cookies() (which needs a url and,
+        given none, returns only domain-less shared cookies — {} for any normal one).
+        """
+        return {c.key: c.value for c in self.session.cookie_jar}
 
     def set_cookie(self, name, value, domain=None, path="/"):
-        """set a cookie in the session jar"""
-        response_url = URL(domain or "http://localhost")
-        self.session.cookie_jar.update_cookies({name: value}, response_url=response_url)
+        """set a cookie in the session jar
+
+        domain=None stores a truly shared cookie sent to every host. domain='example.com'
+        (scheme optional, defaulted to http://) scopes it to that host — a bare hostname
+        is normalized into a URL so the jar binds by host instead of silently storing
+        another domain-less shared cookie. `path` is honored via the morsel itself.
+        """
+        cookie = SimpleCookie()
+        cookie[name] = value
+        if domain is None:
+            self.session.cookie_jar.update_cookies(cookie)
+            return
+        if "://" not in domain:
+            domain = "http://" + domain
+        cookie[name]["path"] = path
+        self.session.cookie_jar.update_cookies(cookie, response_url=URL(domain))
 
     def clear_cookies(self):
         """clear the session cookie jar"""
         self.session.cookie_jar.clear()
 
     def _cookies_for_url(self, url):
-        """dict of cookies the backend would send for url — override per backend
-
-        used by preview(). non-aiohttp backends override this (or return {}); the
-        rest of preview (domain rewrites, header overwrites) is backend-agnostic.
-        """
+        """dict of cookies the backend would send for url — override per backend (used by preview())"""
         return {k: v.value for k, v in self.session.cookie_jar.filter_cookies(URL(url)).items()}
 
     # -------------------------------------------------------------------------
     #   preview
 
     def preview(self, method, url, **kwargs):
-        """build a RequestPreview for a request without sending it"""
+        """build a RequestPreview for a request without sending it
+
+        url is domain-rewritten before cookies are resolved, matching the real
+        request — cookie binding is host-based, so resolving against the pre-rewrite
+        host could miss or misattribute cookies.
+        """
+        url = self._apply_domain_overwrites(url)
         proxy = self._get_proxy(url, kwargs.pop("proxies", None))
         merged = {**self._default_headers, **(kwargs.pop("headers", None) or {})}
         headers = self._apply_overwrites(merged)
@@ -260,7 +288,7 @@ class ExtendedSession:
 
         return RequestPreview(
             method=method,
-            url=self._apply_domain_overwrites(url),
+            url=url,
             headers=headers,
             proxy=proxy,
             data=kwargs.get("data"),
@@ -275,10 +303,9 @@ class ExtendedSession:
     async def _raw_request(self, method, url, **kwargs) -> Response:
         """send one request with aiohttp and adapt it into a Response
 
-        this is the backend seam: a subclass can override only this method (using a
-        different HTTP client, e.g. a TLS-fingerprinting one), building a Response
-        from that backend's primitives. everything else (overwrites, retries,
-        preview) is inherited.
+        the backend seam: a subclass overrides only this to swap HTTP clients,
+        building a Response from that backend's primitives; everything else
+        (overwrites, retries, preview) is inherited.
         """
         response = await self.session.request(method, url, **kwargs)
         async with response:
@@ -295,8 +322,18 @@ class ExtendedSession:
             )
 
     async def request(self, method, url, **kwargs) -> Response:
-        """make a request, applying overwrites/domain rewrites/proxy resolution"""
-        kwargs["proxy"] = self._get_proxy(url, kwargs.pop("proxies", None))
+        """make a request, applying overwrites/domain rewrites/proxy resolution
+
+        raises the real exception subtype on failure (ServerTimeoutError on a total
+        timeout; any other aiohttp.ClientError re-raised as-is, not flattened, so
+        callers can branch by type or read subtype attributes like .status/.headers).
+        a native proxy= is never silently clobbered by resolved proxies= (that would
+        unmask the caller's real IP) — pass only one of them.
+        """
+        resolved_proxy = self._get_proxy(url, kwargs.pop("proxies", None))
+        if "proxy" in kwargs and kwargs["proxy"] is not None and resolved_proxy is not None:
+            raise ValueError("pass only one of proxy= or proxies= (session/per-call), not both")
+        kwargs.setdefault("proxy", resolved_proxy)
         debug = kwargs.pop("debug", False)
 
         merged = {**self._default_headers, **(kwargs.get("headers") or {})}
@@ -306,6 +343,11 @@ class ExtendedSession:
         timeout = kwargs.get("timeout")
         if isinstance(timeout, (int, float)):
             kwargs["timeout"] = aiohttp.ClientTimeout(total=timeout)
+        elif timeout is None and "timeout" in kwargs:
+            # explicit timeout=None would reach aiohttp as ClientTimeout(total=None) —
+            # infinite, disabling the session default — so drop it instead (matters for
+            # request_with_retries, whose timeout kwarg defaults to None)
+            del kwargs["timeout"]
 
         url = self._apply_domain_overwrites(url)
         if debug:
@@ -317,13 +359,15 @@ class ExtendedSession:
                 log.info("redirect chain: %s", result.redirect_chain)
             return result
         except asyncio.TimeoutError as error:
-            # a total ClientTimeout raises a bare asyncio.TimeoutError, which is NOT an
-            # aiohttp.ClientError subclass — wrap it as ServerTimeoutError (which IS both
-            # a ClientError AND a TimeoutError) so direct callers get a typed failure and
-            # request_with_retries can still label it a timeout
+            # a bare asyncio.TimeoutError isn't an aiohttp.ClientError subclass — wrap it
+            # as ServerTimeoutError (both a ClientError AND a TimeoutError) so callers get
+            # a typed failure and request_with_retries can still label it a timeout
             raise aiohttp.ServerTimeoutError(f"timeout for {url}: {error}") from error
         except aiohttp.ClientError as error:
-            raise aiohttp.ClientError(f"client error for {url}: {error}") from error
+            # re-raise the original subtype (not flattened) — request_with_retries still
+            # catches the base aiohttp.ClientError below and is unaffected
+            log.error("client error for %s: %s", url, error)
+            raise
 
     async def request_with_retries(
         self, method, url, *, data=None, proxies=None, timeout=None, attempts=None,
@@ -334,9 +378,11 @@ class ExtendedSession:
 
         returns a Response on success (or non-retryable status), or a falsy
         FailureResponse if every attempt fails. backoff is exponential
-        (backoff_base ** attempt).
+        (backoff_base ** attempt). timeout=None falls back to the session-level
+        timeout (see request()'s note on why explicit None is dropped, not passed
+        through). attempts=0 floors to 1 (via commons.aretry), not DEFAULT_ATTEMPTS.
         """
-        attempts = attempts or DEFAULT_ATTEMPTS
+        attempts = DEFAULT_ATTEMPTS if attempts is None else attempts
         body_data, body_json = _route_body(data)
 
         if debug:
@@ -368,9 +414,8 @@ class ExtendedSession:
                       attempts, url, exhausted.response.status_code)
             return exhausted.response
         except asyncio.TimeoutError:
-            # request() wraps a total timeout as ServerTimeoutError (a ClientError AND a
-            # TimeoutError); catch the timeout case first so it's labeled a timeout rather
-            # than falling into the generic client-error branch below
+            # request() wraps timeouts as ServerTimeoutError (ClientError + TimeoutError);
+            # catch it first so it's labeled a timeout, not a generic client error below
             log.error("all %d attempts timed out for %s", attempts, url)
             return FailureResponse(reason="timeout", url=url)
         except aiohttp.ClientError as error:
@@ -384,12 +429,19 @@ class ExtendedSession:
     #   lifecycle
 
     async def close(self):
-        """close the backend session — override if the backend's close differs"""
-        await self.session.close()
+        """close the backend session — override if the backend's close differs; no-op if never built"""
+        if self._session is not None:
+            await self._session.close()
 
     def _is_closed(self) -> bool:
-        """whether the backend session is closed — override for non-aiohttp backends"""
-        return self.session.closed
+        """whether the backend session is closed — override for non-aiohttp backends
+
+        an unbuilt session counts as closed: nothing was opened, nothing to leak,
+        __del__ should not warn.
+        """
+        if self._session is None:
+            return True
+        return self._session.closed
 
     async def __aenter__(self):
         return self
@@ -398,8 +450,8 @@ class ExtendedSession:
         await self.close()
 
     def __del__(self):
-        # do NOT attempt async cleanup here — spinning event loops in a finalizer
-        # is unsafe. just warn so the leak is visible; callers must close explicitly.
+        # no async cleanup here — spinning event loops in a finalizer is unsafe;
+        # just warn so the leak is visible, callers must close explicitly
         try:
             closed = self._is_closed()
         except Exception: