fix: thread redirect history through both TLS backends
CurlCffi.raw_request and Noble.raw_request built their Response without history=, so resp.history/redirect_chain were always empty after a real redirect despite inheriting aioweb's feature set unchanged. A shared _history_entries() now maps each client's native history shape (curl_cffi list[dict], noble_tls list[Response]) into aioweb's (status, url) tuples. Signed-off-by: disqualifier <dev@disqualifier.me>
This commit is contained in:
@@ -22,17 +22,17 @@ you want; importing the package never fails because an extra is missing.
|
||||
`requirements.txt` (pick the extra you need):
|
||||
|
||||
```
|
||||
aioweb_tls[curl] @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb_tls.git@v0.1.6
|
||||
aioweb_tls[noble] @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb_tls.git@v0.1.6
|
||||
aioweb_tls[all] @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb_tls.git@v0.1.6
|
||||
aioweb_tls[curl] @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb_tls.git@v0.1.7
|
||||
aioweb_tls[noble] @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb_tls.git@v0.1.7
|
||||
aioweb_tls[all] @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb_tls.git@v0.1.7
|
||||
```
|
||||
|
||||
Direct:
|
||||
|
||||
```bash
|
||||
pip install "aioweb_tls[curl] @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb_tls.git@v0.1.6"
|
||||
pip install "aioweb_tls[noble] @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb_tls.git@v0.1.6"
|
||||
pip install "aioweb_tls[all] @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb_tls.git@v0.1.6"
|
||||
pip install "aioweb_tls[curl] @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb_tls.git@v0.1.7"
|
||||
pip install "aioweb_tls[noble] @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb_tls.git@v0.1.7"
|
||||
pip install "aioweb_tls[all] @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb_tls.git@v0.1.7"
|
||||
```
|
||||
|
||||
- `[curl]` → curl_cffi backend · `[noble]` → noble_tls backend · `[all]` → both.
|
||||
@@ -44,7 +44,7 @@ pip install "aioweb_tls[all] @ git+ssh://git@git.rethinkstudios.io/rethink-publi
|
||||
Constructing a backend whose client isn't installed raises that `RuntimeError` at
|
||||
construction, never at import.
|
||||
|
||||
Drop the `@v0.1.6` suffix from the line above to install the latest unpinned.
|
||||
Drop the `@v0.1.7` suffix from the line above to install the latest unpinned.
|
||||
|
||||
## curl_cffi backend
|
||||
|
||||
@@ -198,12 +198,32 @@ cookie store (both expose a `requests`-style `session.cookies` with `set()` /
|
||||
`items()` / `clear()`), so these calls work the same way they do on the base
|
||||
`aioweb.ExtendedSession`, not just `_cookies_for_url()` (used by `preview()`).
|
||||
|
||||
`resp.history` / `resp.redirect_chain` (`[(status, url), ...]`) and `resp.is_redirect`
|
||||
are also threaded through on both backends: `CurlCffi.raw_request` maps curl_cffi's
|
||||
`Response.history` (`list[dict]`) and `Noble.raw_request` maps noble_tls's
|
||||
`list[Response]` into aioweb's `(status, url)` tuple shape, so `debug=True`'s "redirect
|
||||
chain:" log line and any caller reading `resp.history` after a redirecting request see
|
||||
the real hops on either backend, matching the base `aioweb.ExtendedSession` behavior.
|
||||
|
||||
## Honesty note
|
||||
|
||||
TLS fingerprinting changes one layer — the TLS/HTTP fingerprint. It does **not** by
|
||||
itself defeat modern bot protection: behavioral analysis, captchas, and JS challenges
|
||||
are separate signals. Use this as one component, not a complete anti-bot solution.
|
||||
|
||||
## Changelog
|
||||
|
||||
### v0.1.7
|
||||
|
||||
- **Both backends now thread redirect history.** `CurlCffi.raw_request` and
|
||||
`Noble.raw_request` built their `Response` without `history=`, so
|
||||
`resp.history`/`resp.redirect_chain`/`resp.is_redirect`-after-follow were
|
||||
always empty and aioweb's own `debug=True` "redirect chain:" log line was
|
||||
permanently dead on both TLS backends, despite the "inherits every aioweb
|
||||
feature unchanged" claim. Each backend's native history (curl_cffi
|
||||
`list[dict]`, noble_tls `list[Response]`) is now mapped into aioweb's
|
||||
`(status, url)` tuple shape and passed through.
|
||||
|
||||
## Versioning
|
||||
|
||||
Releases are tagged `vX.Y.Z`. The install line above pins a release; drop the `@vX.Y.Z` suffix to install the latest unpinned. Pin deliberately for reproducible installs.
|
||||
|
||||
+1
-1
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
|
||||
|
||||
[project]
|
||||
name = "aioweb_tls"
|
||||
version = "0.1.6"
|
||||
version = "0.1.7"
|
||||
description = "TLS-fingerprinting backends (curl_cffi / noble_tls) for aioweb via one injectable TLSSession, config-free, installable."
|
||||
requires-python = ">=3.10"
|
||||
dependencies = [
|
||||
|
||||
@@ -98,6 +98,29 @@ def _flatten_headers(headers) -> dict:
|
||||
}
|
||||
|
||||
|
||||
def _history_entries(history) -> list:
|
||||
"""map a backend-native redirect history into aioweb's `Response(history=...)` shape
|
||||
|
||||
aioweb's own `_raw_request` threads `[(status, url), ...]`; curl_cffi's
|
||||
`Response.history` is `list[dict]` (dict-shaped hop records) and noble_tls's is
|
||||
`list[Response]` (object-shaped, same accessors as the top-level response) - this
|
||||
reads a hop's status/url either way. an unparseable hop is skipped rather than
|
||||
raising, so a redirect-history quirk never breaks the response it's attached to.
|
||||
"""
|
||||
entries = []
|
||||
for hop in history or []:
|
||||
if isinstance(hop, dict):
|
||||
status = hop.get("status_code", hop.get("status"))
|
||||
url = hop.get("url")
|
||||
else:
|
||||
status = getattr(hop, "status_code", getattr(hop, "status", None))
|
||||
url = getattr(hop, "url", None)
|
||||
if status is None or url is None:
|
||||
continue
|
||||
entries.append((status, str(url)))
|
||||
return entries
|
||||
|
||||
|
||||
def _jar_to_dict(session):
|
||||
"""best-effort map of a requests-style cookie jar on session to a plain dict
|
||||
|
||||
@@ -172,6 +195,7 @@ class CurlCffi:
|
||||
content=content,
|
||||
url=str(response.url),
|
||||
reason=getattr(response, "reason", None),
|
||||
history=_history_entries(getattr(response, "history", None)),
|
||||
cookies=getattr(response, "cookies", None),
|
||||
)
|
||||
|
||||
@@ -310,6 +334,7 @@ class Noble:
|
||||
content=content,
|
||||
url=str(getattr(response, "url", url)),
|
||||
reason=getattr(response, "reason", None),
|
||||
history=_history_entries(getattr(response, "history", None)),
|
||||
cookies=getattr(response, "cookies", None),
|
||||
)
|
||||
|
||||
|
||||
Reference in New Issue
Block a user