fix: thread redirect history through both TLS backends

CurlCffi.raw_request and Noble.raw_request built their Response without
history=, so resp.history/redirect_chain were always empty after a real
redirect despite inheriting aioweb's feature set unchanged. A shared
_history_entries() now maps each client's native history shape (curl_cffi
list[dict], noble_tls list[Response]) into aioweb's (status, url) tuples.

Signed-off-by: disqualifier <dev@disqualifier.me>
This commit is contained in:
2026-07-03 16:19:46 -04:00
parent 226f273695
commit 92ddd5dc39
3 changed files with 53 additions and 8 deletions
+27 -7
View File
@@ -22,17 +22,17 @@ you want; importing the package never fails because an extra is missing.
`requirements.txt` (pick the extra you need): `requirements.txt` (pick the extra you need):
``` ```
aioweb_tls[curl] @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb_tls.git@v0.1.6 aioweb_tls[curl] @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb_tls.git@v0.1.7
aioweb_tls[noble] @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb_tls.git@v0.1.6 aioweb_tls[noble] @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb_tls.git@v0.1.7
aioweb_tls[all] @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb_tls.git@v0.1.6 aioweb_tls[all] @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb_tls.git@v0.1.7
``` ```
Direct: Direct:
```bash ```bash
pip install "aioweb_tls[curl] @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb_tls.git@v0.1.6" pip install "aioweb_tls[curl] @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb_tls.git@v0.1.7"
pip install "aioweb_tls[noble] @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb_tls.git@v0.1.6" pip install "aioweb_tls[noble] @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb_tls.git@v0.1.7"
pip install "aioweb_tls[all] @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb_tls.git@v0.1.6" pip install "aioweb_tls[all] @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb_tls.git@v0.1.7"
``` ```
- `[curl]` → curl_cffi backend · `[noble]` → noble_tls backend · `[all]` → both. - `[curl]` → curl_cffi backend · `[noble]` → noble_tls backend · `[all]` → both.
@@ -44,7 +44,7 @@ pip install "aioweb_tls[all] @ git+ssh://git@git.rethinkstudios.io/rethink-publi
Constructing a backend whose client isn't installed raises that `RuntimeError` at Constructing a backend whose client isn't installed raises that `RuntimeError` at
construction, never at import. construction, never at import.
Drop the `@v0.1.6` suffix from the line above to install the latest unpinned. Drop the `@v0.1.7` suffix from the line above to install the latest unpinned.
## curl_cffi backend ## curl_cffi backend
@@ -198,12 +198,32 @@ cookie store (both expose a `requests`-style `session.cookies` with `set()` /
`items()` / `clear()`), so these calls work the same way they do on the base `items()` / `clear()`), so these calls work the same way they do on the base
`aioweb.ExtendedSession`, not just `_cookies_for_url()` (used by `preview()`). `aioweb.ExtendedSession`, not just `_cookies_for_url()` (used by `preview()`).
`resp.history` / `resp.redirect_chain` (`[(status, url), ...]`) and `resp.is_redirect`
are also threaded through on both backends: `CurlCffi.raw_request` maps curl_cffi's
`Response.history` (`list[dict]`) and `Noble.raw_request` maps noble_tls's
`list[Response]` into aioweb's `(status, url)` tuple shape, so `debug=True`'s "redirect
chain:" log line and any caller reading `resp.history` after a redirecting request see
the real hops on either backend, matching the base `aioweb.ExtendedSession` behavior.
## Honesty note ## Honesty note
TLS fingerprinting changes one layer — the TLS/HTTP fingerprint. It does **not** by TLS fingerprinting changes one layer — the TLS/HTTP fingerprint. It does **not** by
itself defeat modern bot protection: behavioral analysis, captchas, and JS challenges itself defeat modern bot protection: behavioral analysis, captchas, and JS challenges
are separate signals. Use this as one component, not a complete anti-bot solution. are separate signals. Use this as one component, not a complete anti-bot solution.
## Changelog
### v0.1.7
- **Both backends now thread redirect history.** `CurlCffi.raw_request` and
`Noble.raw_request` built their `Response` without `history=`, so
`resp.history`/`resp.redirect_chain`/`resp.is_redirect`-after-follow were
always empty and aioweb's own `debug=True` "redirect chain:" log line was
permanently dead on both TLS backends, despite the "inherits every aioweb
feature unchanged" claim. Each backend's native history (curl_cffi
`list[dict]`, noble_tls `list[Response]`) is now mapped into aioweb's
`(status, url)` tuple shape and passed through.
## Versioning ## Versioning
Releases are tagged `vX.Y.Z`. The install line above pins a release; drop the `@vX.Y.Z` suffix to install the latest unpinned. Pin deliberately for reproducible installs. Releases are tagged `vX.Y.Z`. The install line above pins a release; drop the `@vX.Y.Z` suffix to install the latest unpinned. Pin deliberately for reproducible installs.
+1 -1
View File
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
[project] [project]
name = "aioweb_tls" name = "aioweb_tls"
version = "0.1.6" version = "0.1.7"
description = "TLS-fingerprinting backends (curl_cffi / noble_tls) for aioweb via one injectable TLSSession, config-free, installable." description = "TLS-fingerprinting backends (curl_cffi / noble_tls) for aioweb via one injectable TLSSession, config-free, installable."
requires-python = ">=3.10" requires-python = ">=3.10"
dependencies = [ dependencies = [
+25
View File
@@ -98,6 +98,29 @@ def _flatten_headers(headers) -> dict:
} }
def _history_entries(history) -> list:
"""map a backend-native redirect history into aioweb's `Response(history=...)` shape
aioweb's own `_raw_request` threads `[(status, url), ...]`; curl_cffi's
`Response.history` is `list[dict]` (dict-shaped hop records) and noble_tls's is
`list[Response]` (object-shaped, same accessors as the top-level response) - this
reads a hop's status/url either way. an unparseable hop is skipped rather than
raising, so a redirect-history quirk never breaks the response it's attached to.
"""
entries = []
for hop in history or []:
if isinstance(hop, dict):
status = hop.get("status_code", hop.get("status"))
url = hop.get("url")
else:
status = getattr(hop, "status_code", getattr(hop, "status", None))
url = getattr(hop, "url", None)
if status is None or url is None:
continue
entries.append((status, str(url)))
return entries
def _jar_to_dict(session): def _jar_to_dict(session):
"""best-effort map of a requests-style cookie jar on session to a plain dict """best-effort map of a requests-style cookie jar on session to a plain dict
@@ -172,6 +195,7 @@ class CurlCffi:
content=content, content=content,
url=str(response.url), url=str(response.url),
reason=getattr(response, "reason", None), reason=getattr(response, "reason", None),
history=_history_entries(getattr(response, "history", None)),
cookies=getattr(response, "cookies", None), cookies=getattr(response, "cookies", None),
) )
@@ -310,6 +334,7 @@ class Noble:
content=content, content=content,
url=str(getattr(response, "url", url)), url=str(getattr(response, "url", url)),
reason=getattr(response, "reason", None), reason=getattr(response, "reason", None),
history=_history_entries(getattr(response, "history", None)),
cookies=getattr(response, "cookies", None), cookies=getattr(response, "cookies", None),
) )