add package: pyproject + src

TLSSession over aioweb's backend seam by composition: one session class
delegates the four seams to an injected backend. ships CurlCffi (curl_cffi
impersonate) and Noble (noble_tls Client) backends plus the TLSBackend
protocol for custom clients. tls clients are optional extras
([curl]/[noble]/[all]) with guarded imports; all aioweb features (domain/
header/ephemeral/proxy/retry/preview) inherited unchanged. src/ multi-module
layout, hatchling.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: disqualifier <dev@disqualifier.me>

.gitignore

@@ -1,5 +1,5 @@
 # claude
-.claude/
+CLAUDE.md
 
 # python
 __pycache__/

README.md

@@ -22,17 +22,17 @@ you want; importing the package never fails because an extra is missing.
 `requirements.txt` (pick the extra you need):
 
 ```
-aioweb_tls[curl]  @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb_tls.git@v0.1.4
-aioweb_tls[noble] @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb_tls.git@v0.1.4
-aioweb_tls[all]   @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb_tls.git@v0.1.4
+aioweb_tls[curl]  @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb_tls.git@v0.1.0
+aioweb_tls[noble] @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb_tls.git@v0.1.0
+aioweb_tls[all]   @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb_tls.git@v0.1.0
 ```
 
 Direct:
 
 ```bash
-pip install "aioweb_tls[curl] @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb_tls.git@v0.1.4"
-pip install "aioweb_tls[noble] @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb_tls.git@v0.1.4"
-pip install "aioweb_tls[all] @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb_tls.git@v0.1.4"
+pip install "aioweb_tls[curl] @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb_tls.git@v0.1.0"
+pip install "aioweb_tls[noble] @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb_tls.git@v0.1.0"
+pip install "aioweb_tls[all] @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb_tls.git@v0.1.0"
 ```
 
 - `[curl]` → curl_cffi backend · `[noble]` → noble_tls backend · `[all]` → both.
@@ -44,8 +44,6 @@ pip install "aioweb_tls[all] @ git+ssh://git@git.rethinkstudios.io/rethink-publi
 Constructing a backend whose client isn't installed raises that `RuntimeError` at
 construction, never at import.
 
-Drop the `@v0.1.4` suffix from the line above to install the latest unpinned.
-
 ## curl_cffi backend
 
 ```python
@@ -57,11 +55,8 @@ async with TLSSession(backend=CurlCffi(impersonate="chrome"), proxies={"https":
         print(resp.json()["tls"]["ja3"])
 ```
 
-- `CurlCffi(impersonate="chrome")` sets the forged profile; override it per call by
-  passing `impersonate=` to the low-level `request()` (which forwards `**kwargs` to the
-  backend). `request_with_retries` has a fixed signature and does **not** accept extra
-  backend kwargs — passing `impersonate=` there raises `TypeError`; set the profile on
-  the `CurlCffi` instance for the retrying path.
+- `CurlCffi(impersonate="chrome")` sets the forged profile; override per call by
+  passing `impersonate=` to any request method.
 - curl_cffi forges JA3/JA4 + HTTP/2 fingerprints via the bundled curl-impersonate binary.
 
 ## noble backend
@@ -78,13 +73,8 @@ async with TLSSession(backend=Noble(client="chrome_133")) as s:
 
 - `Noble(client="chrome_133")` — accepts a `noble_tls.Client` enum or a string name.
 - noble_tls downloads a Go shared library on first use. `await s.setup()` fetches it
-  once at startup; if you skip it, the first request fetches it lazily. The fetch is
-  guarded by a lock, so even concurrent first requests download it exactly once.
-- Binary bodies (images, zips, PDFs, protobuf) round-trip as true bytes: noble_tls
-  returns response bodies as a UTF-8 JSON string by default, which mangles non-UTF-8
-  bytes (`U+FFFD` replacement, wrong length) even on a 200 response — the Noble
-  backend always requests `is_byte_response=True` and decodes the resulting
-  base64 data-URI back into raw bytes, so `resp.content` is never lossy.
+  once at startup; if you skip it, the first request fetches it lazily (guarded to run
+  once).
 
 ## Writing your own backend (the `TLSBackend` protocol)
 
@@ -104,9 +94,6 @@ for the authoritative contract):
 | `raw_request` | **required** | `async (session, method, url, **kwargs) -> aioweb.Response` | send one request; adapt the client's response into an `aioweb.Response` |
 | `is_closed` | **required** | `(session) -> bool` | whether the session is closed |
 | `cookies_for_url` | optional | `(session, url) -> dict` | cookies for `preview()`; defaults to `{}` |
-| `set_cookie` | optional | `(session, name, value, domain=None, path="/") -> None` | backs `TLSSession.set_cookie()`; raises `NotImplementedError` if absent |
-| `get_cookies` | optional | `(session) -> dict` | backs `TLSSession.get_cookies()`; raises `NotImplementedError` if absent |
-| `clear_cookies` | optional | `(session) -> None` | backs `TLSSession.clear_cookies()`; raises `NotImplementedError` if absent |
 | `setup` | optional | `async () -> None` | one-time prep (e.g. fetch a native lib); idempotent |
 | `close` | optional | `async (session) -> None` | close the session; defaults to `await session.close()` |
 
@@ -164,9 +151,8 @@ async with TLSSession(backend=GoTLSBackend("http://localhost:8080")) as s:
 ## Inherited features work unchanged
 
 aioweb's overwrite/domain/ephemeral/proxy/retry/preview logic operates on plain dicts
-and never touches the HTTP backend — only the seams do. Header overwrites, domain
-rewriting, ephemeral headers, proxies, retries, and previews behave identically on
-any backend:
+and never touches the HTTP backend — only the seams do. Every aioweb feature behaves
+identically on any backend:
 
 ```python
 async with TLSSession(backend=CurlCffi(impersonate="chrome")) as s:
@@ -176,19 +162,6 @@ async with TLSSession(backend=CurlCffi(impersonate="chrome")) as s:
     print(s.preview("GET", "https://internal.local/x").as_curl())   # reflects all of the above
 ```
 
-Session-default headers are never baked into the underlying client (neither
-`CurlCffi` nor `Noble` passes `headers=` to their client's constructor) — they flow
-through aioweb's own per-request `_default_headers` merge instead. That keeps
-`update_headers()` / `clear_headers()` accurate for both backends: what
-`get_headers()` and `preview()` report is what actually goes out on the wire, with
-no stale, already-baked value resurfacing after a clear.
-
-The mutable cookie API — `set_cookie()` / `get_cookies()` / `clear_cookies()` — is
-also backend-aware: `CurlCffi` and `Noble` each route it through their own client's
-cookie store (both expose a `requests`-style `session.cookies` with `set()` /
-`items()` / `clear()`), so these calls work the same way they do on the base
-`aioweb.ExtendedSession`, not just `_cookies_for_url()` (used by `preview()`).
-
 ## Honesty note
 
 TLS fingerprinting changes one layer — the TLS/HTTP fingerprint. It does **not** by
@@ -197,4 +170,4 @@ are separate signals. Use this as one component, not a complete anti-bot solutio
 
 ## Versioning
 
-Releases are tagged `vX.Y.Z`. The install line above pins a release; drop the `@vX.Y.Z` suffix to install the latest unpinned. Pin deliberately for reproducible installs.
+Tagged `vX.Y.Z`. Pin the tag in `requirements.txt`.

pyproject.toml

@@ -4,11 +4,11 @@ build-backend = "hatchling.build"
 
 [project]
 name = "aioweb_tls"
-version = "0.1.4"
-description = "TLS-fingerprinting backends (curl_cffi / noble_tls) for aioweb via one injectable TLSSession, config-free, installable."
+version = "0.1.0"
+description = "TLS-fingerprinting backends for aioweb — curl_cffi / noble_tls ExtendedSession subclasses, config-free, installable."
 requires-python = ">=3.10"
 dependencies = [
-    "aioweb @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb.git@v0.1.5",
+    "aioweb @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb.git@v0.1.0",
 ]
 
 [project.optional-dependencies]

src/aioweb_tls/backends.py

@@ -11,28 +11,12 @@ is not installed raises a clear RuntimeError naming the extra to install. import
 this module never fails because an extra is missing.
 """
 
-import asyncio
-import base64
 import logging
-import math
 
-import aiohttp
 from aioweb import Response
 
 log = logging.getLogger(__name__)
 
-
-def _as_client_error(error: Exception, backend: str) -> aiohttp.ClientError:
-    """wrap a backend-native network exception as an aiohttp.ClientError
-
-    aioweb's request() only re-wraps aiohttp.ClientError; curl_cffi raises
-    RequestException(OSError) and noble_tls raises TLSClientException(IOError), neither
-    of which is an aiohttp.ClientError. translating here gives TLS backends the same
-    typed failure contract as the aiohttp path on the bare request() route.
-    """
-    return aiohttp.ClientError(f"{backend} request failed: {error}")
-
-
 try:
     from curl_cffi import AsyncSession as _CurlAsyncSession
     _CURL_ERROR = None
@@ -60,46 +44,8 @@ def _coerce_timeout(value):
     return total if isinstance(total, (int, float)) else None
 
 
-def _noble_timeout_seconds(value):
-    """coerce a raw or wrapped timeout into whole seconds for noble's Go int field
-
-    single source of truth for both of Noble's timeout paths (create_session's
-    session-default and raw_request's per-call override): noble's Go field
-    timeoutSeconds is an int, so a sub-second float (e.g. 0.5) truncates to 0 — which
-    Go reads as no/instant timeout — and any non-integer value fails Go-side JSON
-    unmarshal outright. rounds UP so a sub-second timeout still waits at least 1s
-    instead of truncating away. returns None when there is no timeout to coerce.
-    """
-    timeout = _coerce_timeout(value)
-    if timeout is None:
-        return None
-    return max(1, math.ceil(timeout))
-
-
-def _noble_content(response) -> bytes:
-    """extract true response bytes from a noble_tls response fetched with is_byte_response
-
-    with is_byte_response=True, noble_tls's Go side returns the body as a
-    `data:<mime>;base64,<payload>` URI string in response.text (the whole Go response is
-    UTF-8-decoded before JSON parsing, so raw bytes have to travel as base64 to survive
-    that round trip) — response.content is useless here since it is just
-    `self.text.encode()`, re-encoding the data-URI string itself rather than decoding it.
-    falls back to a plain utf-8 encode if the body isn't a data-URI (e.g. an error body).
-    """
-    text = getattr(response, "text", "") or ""
-    if text.startswith("data:") and ";base64," in text:
-        _, _, payload = text.partition(";base64,")
-        return base64.b64decode(payload)
-    return text.encode()
-
-
 def _jar_to_dict(session):
-    """best-effort map of a requests-style cookie jar on session to a plain dict
-
-    intentionally broad: this feeds preview() only, the two backends expose differently-
-    shaped jars, and a cookie read must never crash a request — so any jar that doesn't
-    iterate cleanly degrades to {} rather than raising.
-    """
+    """best-effort map of a requests-style cookie jar on session to a plain dict"""
     jar = getattr(session, "cookies", None)
     if not jar:
         return {}
@@ -114,10 +60,7 @@ class CurlCffi:
 
     config:
         impersonate: browser profile to forge (default "chrome"); override per call
-            by passing impersonate= to the low-level request()/_raw_request path,
-            which forwards **kwargs to the backend. NOT request_with_retries — its
-            signature is fixed (no **kwargs) and would raise TypeError. for a
-            per-call profile under retries, set it on the CurlCffi instance instead.
+            by passing impersonate= to any request method.
 
     requires the [curl] extra (pip install "aioweb_tls[curl]").
     """
@@ -131,17 +74,8 @@ class CurlCffi:
         self.impersonate = impersonate
 
     def create_session(self, headers, timeout, **kwargs):
-        """build the curl_cffi AsyncSession
-
-        deliberately does NOT pass `headers` to AsyncSession: curl_cffi bakes a
-        constructor `headers=` into the client and re-merges it under whatever
-        per-request headers() sends, so update_headers()/clear_headers() would stop
-        matching what's actually on the wire (aioweb's session-default headers are
-        already applied per request by the base's _default_headers merge — see
-        aioweb.ExtendedSession._create_session's docstring for why baking breaks the
-        mutable header api).
-        """
-        return _CurlAsyncSession(timeout=timeout, **kwargs)
+        """build the curl_cffi AsyncSession"""
+        return _CurlAsyncSession(headers=headers, timeout=timeout, **kwargs)
 
     async def raw_request(self, session, method, url, **kwargs) -> Response:
         """send via curl_cffi and adapt the result into an aioweb.Response"""
@@ -155,18 +89,10 @@ class CurlCffi:
         if proxy:
             kwargs["proxy"] = proxy
 
-        try:
         response = await session.request(method, url, impersonate=impersonate, **kwargs)
-        except aiohttp.ClientError:
-            raise
-        except asyncio.TimeoutError:
-            raise
-        except OSError as error:
-            # curl_cffi's RequestException subclasses OSError; translate the native
-            # network error into aiohttp.ClientError. narrowed from a bare Exception so a
-            # real bug (AttributeError/TypeError) isn't laundered into 'client error'
-            raise _as_client_error(error, "curl_cffi") from error
-        content = response.content if response.content is not None else b""
+        content = response.content
+        if content is None:
+            content = response.text.encode() if response.text else b""
         return Response(
             status_code=response.status_code,
             headers=dict(response.headers),
@@ -177,34 +103,13 @@ class CurlCffi:
         )
 
     def is_closed(self, session) -> bool:
-        """whether the curl_cffi session is closed
-
-        curl_cffi tracks closed state in the private `_closed` (no public `closed`
-        property), so read that; fall back to a public `closed` if a future version
-        adds one. TLSSession's own `_closed` flag is the primary signal — this is a
-        best-effort backend check for out-of-band closes.
-        """
-        closed = getattr(session, "_closed", None)
-        if closed is None:
-            closed = getattr(session, "closed", False)
-        return bool(closed)
+        """whether the curl_cffi session is closed"""
+        return bool(getattr(session, "closed", False))
 
     def cookies_for_url(self, session, url) -> dict:
         """cookies curl_cffi would send for url (best-effort)"""
         return _jar_to_dict(session)
 
-    def set_cookie(self, session, name, value, domain=None, path="/") -> None:
-        """set a cookie in curl_cffi's own cookie store"""
-        session.cookies.set(name, value, domain=domain or "", path=path)
-
-    def get_cookies(self, session) -> dict:
-        """all cookies stored in curl_cffi's cookie store"""
-        return dict(session.cookies.items())
-
-    def clear_cookies(self, session) -> None:
-        """clear curl_cffi's cookie store"""
-        session.cookies.clear()
-
     async def close(self, session) -> None:
         """close the curl_cffi session"""
         await session.close()
@@ -230,7 +135,6 @@ class Noble:
             ) from _NOBLE_ERROR
         self.client = self._resolve_client(client)
         self._updated = False
-        self._setup_lock = asyncio.Lock()
 
     @staticmethod
     def _resolve_client(client):
@@ -240,80 +144,42 @@ class Noble:
         return client
 
     async def setup(self) -> None:
-        """fetch the noble_tls Go shared library once; idempotent and concurrency-safe
+        """fetch the noble_tls Go shared library once; idempotent
 
-        uses noble_tls.download_if_necessary (the current API: it fetches the asset on
-        first use and no-ops when it already exists). older noble_tls without that name
-        is handled via update_if_necessary as a fallback.
-
-        guarded by an asyncio.Lock with a check-lock-recheck so concurrent first
-        requests don't both run the fetch: the fast path returns once _updated is
-        set, and only the first caller through the lock does the work.
+        download_if_necessary handles the first-time fetch (no lib present);
+        update_if_necessary refreshes an existing one. try download first so a
+        clean environment works, falling back to update.
         """
-        if self._updated:
-            return
-        async with self._setup_lock:
         if self._updated:
             return
         download = getattr(noble_tls, "download_if_necessary", None)
         if download is not None:
             await download()
-            elif hasattr(noble_tls, "update_if_necessary"):
+        else:
             await noble_tls.update_if_necessary()
         self._updated = True
 
     def create_session(self, headers, timeout, **kwargs):
-        """build the noble_tls Session, honoring the session-default timeout
-
-        noble_tls.Session takes neither headers nor timeout in its constructor.
-        deliberately does NOT bake `headers` into session.headers: aioweb's
-        session-default headers are already applied per request by the base's
-        _default_headers merge, and baking them here would make
-        update_headers()/clear_headers() stop matching what's actually on the wire
-        (see aioweb.ExtendedSession._create_session's docstring, and CurlCffi.create_session
-        above for the same rationale) — the per-request merge means headers are never
-        silently dropped, contrary to what this docstring used to claim.
-
-        the coerced timeout IS applied here (matching raw_request's per-call path): noble's
-        Go field timeoutSeconds is an int, so a raw sub-second/float session-default (e.g.
-        timeout=7.5) would fail Go-side JSON unmarshal on every request that doesn't
-        override it per call. max(1, ceil()) mirrors the guard raw_request already has.
-        """
-        session = noble_tls.Session(client=self.client, **kwargs)
-        timeout_seconds = _noble_timeout_seconds(timeout)
-        if timeout_seconds is not None:
-            session.timeout_seconds = timeout_seconds
-        return session
+        """build the noble_tls Session"""
+        return noble_tls.Session(client=self.client, **kwargs)
 
     async def raw_request(self, session, method, url, **kwargs) -> Response:
         """send via noble_tls and adapt the result into an aioweb.Response"""
         await self.setup()
 
-        timeout_seconds = _noble_timeout_seconds(kwargs.pop("timeout", None))
-        if timeout_seconds is not None:
-            kwargs["timeout_seconds"] = timeout_seconds
+        timeout = _coerce_timeout(kwargs.pop("timeout", None))
+        if timeout is not None:
+            kwargs["timeout_seconds"] = int(timeout)
 
         proxy = kwargs.pop("proxy", None)
         if proxy:
             kwargs["proxy"] = proxy
 
-        # force byte-safe transport: without this, noble_tls's Go side returns the body
-        # as a plain UTF-8 JSON string, replacing invalid bytes with U+FFFD — silently
-        # corrupting any binary payload (image/zip/pdf) even though the request succeeds
-        # with status 200. see _noble_content() for how the byte-safe body is decoded back.
-        kwargs.setdefault("is_byte_response", True)
-
-        try:
         response = await session.execute_request(method=method.upper(), url=url, **kwargs)
-        except aiohttp.ClientError:
-            raise
-        except asyncio.TimeoutError:
-            raise
-        except OSError as error:
-            # noble_tls's TLSClientException subclasses IOError (== OSError); translate
-            # the native network error, narrowed from bare Exception so a real bug surfaces
-            raise _as_client_error(error, "noble_tls") from error
-        content = _noble_content(response)
+        content = getattr(response, "content", None)
+        if content is None:
+            text = getattr(response, "text", "") or ""
+            content = text.encode()
         return Response(
             status_code=response.status_code,
             headers=dict(getattr(response, "headers", {}) or {}),
@@ -331,18 +197,6 @@ class Noble:
         """cookies noble_tls would send for url (best-effort)"""
         return _jar_to_dict(session)
 
-    def set_cookie(self, session, name, value, domain=None, path="/") -> None:
-        """set a cookie in noble_tls's own cookie jar"""
-        session.cookies.set(name, value, domain=domain or "", path=path)
-
-    def get_cookies(self, session) -> dict:
-        """all cookies stored in noble_tls's cookie jar"""
-        return dict(session.cookies.items())
-
-    def clear_cookies(self, session) -> None:
-        """clear noble_tls's cookie jar"""
-        session.cookies.clear()
-
     async def close(self, session) -> None:
         """close the noble_tls session if it exposes a close"""
         close = getattr(session, "close", None)

src/aioweb_tls/protocol.py

@@ -30,15 +30,6 @@ optional:
         cookies the client would send for url, for preview(). default {} (used when
         the backend has no introspectable jar).
 
-    set_cookie(session, name, value, domain=None, path="/") -> None
-    get_cookies(session) -> dict
-    clear_cookies(session) -> None
-        the mutable cookie api TLSSession.set_cookie/get_cookies/clear_cookies
-        delegate to. the aiohttp-only base implementations reach into
-        session.cookie_jar, which TLS backends don't have, so a backend without
-        these raises NotImplementedError from TLSSession rather than crashing with
-        an AttributeError on a private aiohttp attribute.
-
     async setup() -> None
         one-time async preparation (e.g. fetch a native lib). called once via
         TLSSession.setup() and lazily before the first request; make it idempotent.

src/aioweb_tls/session.py

@@ -81,38 +81,6 @@ class TLSSession(ExtendedSession):
             return True
         return self.backend.is_closed(self.session)
 
-    # -------------------------------------------------------------------------
-    #   mutable cookie api — the base's set_cookie/get_cookies/clear_cookies reach
-    #   into self.session.cookie_jar (aiohttp-only), so TLS backends override them
-    #   to route through the backend instead of crashing with AttributeError
-
-    def set_cookie(self, name, value, domain=None, path="/"):
-        """set a cookie via the backend's own cookie store"""
-        set_cookie = getattr(self.backend, "set_cookie", None)
-        if set_cookie is None:
-            raise NotImplementedError(
-                f"{type(self.backend).__name__} does not support the mutable cookie api"
-            )
-        set_cookie(self.session, name, value, domain=domain, path=path)
-
-    def get_cookies(self) -> dict:
-        """all cookies stored in the backend's cookie store"""
-        get_cookies = getattr(self.backend, "get_cookies", None)
-        if get_cookies is None:
-            raise NotImplementedError(
-                f"{type(self.backend).__name__} does not support the mutable cookie api"
-            )
-        return get_cookies(self.session)
-
-    def clear_cookies(self) -> None:
-        """clear the backend's cookie store"""
-        clear_cookies = getattr(self.backend, "clear_cookies", None)
-        if clear_cookies is None:
-            raise NotImplementedError(
-                f"{type(self.backend).__name__} does not support the mutable cookie api"
-            )
-        clear_cookies(self.session)
-
     # -------------------------------------------------------------------------
     #   lifecycle — backends close differently, so route through the backend