rethink-public/aioweb_tls
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix: seam mutable cookie api, byte-safe noble bodies, drop header baking, coerce noble session timeout (v0.1.4)

Browse Source 
TLSSession.set_cookie/get_cookies/clear_cookies crashed with AttributeError on
both backends (they reached into self.session.cookie_jar, which curl_cffi and
noble_tls sessions don't have); both backends now route the mutable cookie api
through their own requests-style session.cookies store.

Noble.raw_request now requests is_byte_response=True and decodes the resulting
base64 data-URI body, since noble_tls's default text response silently corrupts
any binary payload (image/zip/pdf) via lossy UTF-8 decoding on the Go side.

CurlCffi/Noble.create_session no longer bake session-default headers into the
underlying client; baking caused clear_headers()/get_headers() to lie about
what's actually still on the wire (a credential-leak divergence from the
aiohttp base, which never bakes). Headers flow through aioweb's per-request
merge only, matching the base's documented contract.

Noble.create_session now applies the same max(1, ceil()) timeout coercion
raw_request already had (extracted into a shared _noble_timeout_seconds
helper) — without it, a sub-second/float session-default timeout made every
request fail Go-side JSON unmarshal.

README corrected: dropped the 'every aioweb feature behaves identically'
overclaim re: cookies, documented the binary-body handling and the
no-header-baking rationale, bumped install pins to v0.1.4.

Signed-off-by: disqualifier <dev@disqualifier.me>
This commit is contained in:
dsql
2026-07-02 17:09:11 -04:00
parent ce0762a37c
commit eb7745ae9b
5 changed files with 169 additions and 30 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+31 -9
View File
@@ -22,17 +22,17 @@ you want; importing the package never fails because an extra is missing.
`requirements.txt` (pick the extra you need):
```
aioweb_tls[curl] @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb_tls.git@v0.1.3
aioweb_tls[noble] @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb_tls.git@v0.1.3
aioweb_tls[all] @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb_tls.git@v0.1.3
aioweb_tls[curl] @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb_tls.git@v0.1.4
aioweb_tls[noble] @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb_tls.git@v0.1.4
aioweb_tls[all] @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb_tls.git@v0.1.4
```
Direct:
```bash
pip install "aioweb_tls[curl] @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb_tls.git@v0.1.3"
pip install "aioweb_tls[noble] @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb_tls.git@v0.1.3"
pip install "aioweb_tls[all] @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb_tls.git@v0.1.3"
pip install "aioweb_tls[curl] @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb_tls.git@v0.1.4"
pip install "aioweb_tls[noble] @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb_tls.git@v0.1.4"
pip install "aioweb_tls[all] @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb_tls.git@v0.1.4"
```
- `[curl]` → curl_cffi backend · `[noble]` → noble_tls backend · `[all]` → both.
@@ -44,7 +44,7 @@ pip install "aioweb_tls[all] @ git+ssh://git@git.rethinkstudios.io/rethink-publi
Constructing a backend whose client isn't installed raises that `RuntimeError` at
construction, never at import.
Drop the `@v0.1.3` suffix from the line above to install the latest unpinned.
Drop the `@v0.1.4` suffix from the line above to install the latest unpinned.
## curl_cffi backend
@@ -80,6 +80,11 @@ async with TLSSession(backend=Noble(client="chrome_133")) as s:
- noble_tls downloads a Go shared library on first use. `await s.setup()` fetches it
once at startup; if you skip it, the first request fetches it lazily. The fetch is
guarded by a lock, so even concurrent first requests download it exactly once.
- Binary bodies (images, zips, PDFs, protobuf) round-trip as true bytes: noble_tls
returns response bodies as a UTF-8 JSON string by default, which mangles non-UTF-8
bytes (`U+FFFD` replacement, wrong length) even on a 200 response — the Noble
backend always requests `is_byte_response=True` and decodes the resulting
base64 data-URI back into raw bytes, so `resp.content` is never lossy.
## Writing your own backend (the `TLSBackend` protocol)
@@ -99,6 +104,9 @@ for the authoritative contract):
| `raw_request` | **required** | `async (session, method, url, **kwargs) -> aioweb.Response` | send one request; adapt the client's response into an `aioweb.Response` |
| `is_closed` | **required** | `(session) -> bool` | whether the session is closed |
| `cookies_for_url` | optional | `(session, url) -> dict` | cookies for `preview()`; defaults to `{}` |
| `set_cookie` | optional | `(session, name, value, domain=None, path="/") -> None` | backs `TLSSession.set_cookie()`; raises `NotImplementedError` if absent |
| `get_cookies` | optional | `(session) -> dict` | backs `TLSSession.get_cookies()`; raises `NotImplementedError` if absent |
| `clear_cookies` | optional | `(session) -> None` | backs `TLSSession.clear_cookies()`; raises `NotImplementedError` if absent |
| `setup` | optional | `async () -> None` | one-time prep (e.g. fetch a native lib); idempotent |
| `close` | optional | `async (session) -> None` | close the session; defaults to `await session.close()` |
@@ -156,8 +164,9 @@ async with TLSSession(backend=GoTLSBackend("http://localhost:8080")) as s:
## Inherited features work unchanged
aioweb's overwrite/domain/ephemeral/proxy/retry/preview logic operates on plain dicts
and never touches the HTTP backend — only the seams do. Every aioweb feature behaves
identically on any backend:
and never touches the HTTP backend — only the seams do. Header overwrites, domain
rewriting, ephemeral headers, proxies, retries, and previews behave identically on
any backend:
```python
async with TLSSession(backend=CurlCffi(impersonate="chrome")) as s:
@@ -167,6 +176,19 @@ async with TLSSession(backend=CurlCffi(impersonate="chrome")) as s:
print(s.preview("GET", "https://internal.local/x").as_curl()) # reflects all of the above
```
Session-default headers are never baked into the underlying client (neither
`CurlCffi` nor `Noble` passes `headers=` to their client's constructor) — they flow
through aioweb's own per-request `_default_headers` merge instead. That keeps
`update_headers()` / `clear_headers()` accurate for both backends: what
`get_headers()` and `preview()` report is what actually goes out on the wire, with
no stale, already-baked value resurfacing after a clear.
The mutable cookie API — `set_cookie()` / `get_cookies()` / `clear_cookies()` — is
also backend-aware: `CurlCffi` and `Noble` each route it through their own client's
cookie store (both expose a `requests`-style `session.cookies` with `set()` /
`items()` / `clear()`), so these calls work the same way they do on the base
`aioweb.ExtendedSession`, not just `_cookies_for_url()` (used by `preview()`).
## Honesty note
TLS fingerprinting changes one layer — the TLS/HTTP fingerprint. It does **not** by