5 Commits
Author SHA1 Message Date
dsql 5eb689ba73 docs: narrow the per-call impersonate= claim (v0.1.2)
per-call impersonate= is honored only via the low-level request()/_raw_request path
(which forwards **kwargs to the backend), NOT request_with_retries — its inherited
aioweb signature is fixed with no **kwargs and raises TypeError on an extra kwarg.
docs-only across README + backend docstring + CLAUDE.md; for the retrying path, set
the profile on the CurlCffi/Noble instance. no code change.

Signed-off-by: disqualifier <dev@disqualifier.me>
2026-06-29 17:11:57 -04:00
dsql 87debe8465 fix: is_closed reads curl_cffi's private _closed (no public closed attr)
CurlCffi.is_closed read getattr(session, 'closed', False), but curl_cffi tracks closed state only in the private _closed and exposes no public 'closed' property, so it always returned False. it now reads _closed, falling back to a public 'closed' if a future version adds one. TLSSession's own flag remains the primary signal; this is the best-effort backend check for out-of-band closes.

Signed-off-by: disqualifier <dev@disqualifier.me>
2026-06-29 01:10:25 -04:00
dsql b2876d005e fix: Noble honors session-default headers and timeout like CurlCffi
noble_tls.Session takes neither headers nor timeout in its constructor, and Noble.create_session forwarded only client+kwargs, so TLSSession(backend=Noble(...), headers=...) silently dropped the headers while CurlCffi passed them through. create_session now applies headers via session.headers.update and sets timeout_seconds after construction. verified against the contract with a stubbed noble_tls; the real Go-lib + a live request remain an untested gap (noble_tls not installable in this env).

Signed-off-by: disqualifier <dev@disqualifier.me>
2026-06-28 18:45:25 -04:00
dsql ae4c653ecc fix: lock Noble.setup so concurrent first requests fetch the Go-lib once
Noble.setup guarded the one-time Go shared-library fetch with a bare 'if self._updated' flag — a TOCTOU race where concurrent first requests both passed the check before either set the flag, running the download multiple times. now guarded by a per-instance asyncio.Lock with a check-lock-recheck. verified under load: 2/10/100/500 concurrent setups run the fetch exactly once each (a no-lock control runs it N times).

Signed-off-by: disqualifier <dev@disqualifier.me>
2026-06-28 17:46:20 -04:00
dsql 7ea8ecf888 fix: don't truncate sub-second timeout to 0 in noble backend (v0.1.1)
int(timeout) truncated a fractional timeout (e.g. 0.5s) to 0, which noble treats as
no/instant timeout. round up with math.ceil and floor at 1 so a sub-second timeout
stays a real (>=1s) timeout.

verified: 0.5/0.1/0.001 -> 1 (was 0); whole seconds unchanged.
Signed-off-by: disqualifier <dev@disqualifier.me>
2026-06-28 15:49:30 -04:00
3 changed files with 62 additions and 24 deletions
+13 -10
View File
@@ -22,17 +22,17 @@ you want; importing the package never fails because an extra is missing.
`requirements.txt` (pick the extra you need):
```
aioweb_tls[curl] @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb_tls.git@v0.1.0
aioweb_tls[noble] @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb_tls.git@v0.1.0
aioweb_tls[all] @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb_tls.git@v0.1.0
aioweb_tls[curl] @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb_tls.git@v0.1.2
aioweb_tls[noble] @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb_tls.git@v0.1.2
aioweb_tls[all] @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb_tls.git@v0.1.2
```
Direct:
```bash
pip install "aioweb_tls[curl] @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb_tls.git@v0.1.0"
pip install "aioweb_tls[noble] @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb_tls.git@v0.1.0"
pip install "aioweb_tls[all] @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb_tls.git@v0.1.0"
pip install "aioweb_tls[curl] @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb_tls.git@v0.1.2"
pip install "aioweb_tls[noble] @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb_tls.git@v0.1.2"
pip install "aioweb_tls[all] @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb_tls.git@v0.1.2"
```
- `[curl]` → curl_cffi backend · `[noble]` → noble_tls backend · `[all]` → both.
@@ -55,8 +55,11 @@ async with TLSSession(backend=CurlCffi(impersonate="chrome"), proxies={"https":
print(resp.json()["tls"]["ja3"])
```
- `CurlCffi(impersonate="chrome")` sets the forged profile; override per call by
passing `impersonate=` to any request method.
- `CurlCffi(impersonate="chrome")` sets the forged profile; override it per call by
passing `impersonate=` to the low-level `request()` (which forwards `**kwargs` to the
backend). `request_with_retries` has a fixed signature and does **not** accept extra
backend kwargs — passing `impersonate=` there raises `TypeError`; set the profile on
the `CurlCffi` instance for the retrying path.
- curl_cffi forges JA3/JA4 + HTTP/2 fingerprints via the bundled curl-impersonate binary.
## noble backend
@@ -73,8 +76,8 @@ async with TLSSession(backend=Noble(client="chrome_133")) as s:
- `Noble(client="chrome_133")` — accepts a `noble_tls.Client` enum or a string name.
- noble_tls downloads a Go shared library on first use. `await s.setup()` fetches it
once at startup; if you skip it, the first request fetches it lazily (guarded to run
once).
once at startup; if you skip it, the first request fetches it lazily. The fetch is
guarded by a lock, so even concurrent first requests download it exactly once.
## Writing your own backend (the `TLSBackend` protocol)
+1 -1
View File
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
[project]
name = "aioweb_tls"
version = "0.1.0"
version = "0.1.2"
description = "TLS-fingerprinting backends for aioweb — curl_cffi / noble_tls ExtendedSession subclasses, config-free, installable."
requires-python = ">=3.10"
dependencies = [
+48 -13
View File
@@ -11,7 +11,9 @@ is not installed raises a clear RuntimeError naming the extra to install. import
this module never fails because an extra is missing.
"""
import asyncio
import logging
import math
from aioweb import Response
@@ -60,7 +62,10 @@ class CurlCffi:
config:
impersonate: browser profile to forge (default "chrome"); override per call
by passing impersonate= to any request method.
by passing impersonate= to the low-level request()/_raw_request path,
which forwards **kwargs to the backend. NOT request_with_retries — its
signature is fixed (no **kwargs) and would raise TypeError. for a
per-call profile under retries, set it on the CurlCffi instance instead.
requires the [curl] extra (pip install "aioweb_tls[curl]").
"""
@@ -103,8 +108,17 @@ class CurlCffi:
)
def is_closed(self, session) -> bool:
"""whether the curl_cffi session is closed"""
return bool(getattr(session, "closed", False))
"""whether the curl_cffi session is closed
curl_cffi tracks closed state in the private `_closed` (no public `closed`
property), so read that; fall back to a public `closed` if a future version
adds one. TLSSession's own `_closed` flag is the primary signal — this is a
best-effort backend check for out-of-band closes.
"""
closed = getattr(session, "_closed", None)
if closed is None:
closed = getattr(session, "closed", False)
return bool(closed)
def cookies_for_url(self, session, url) -> dict:
"""cookies curl_cffi would send for url (best-effort)"""
@@ -135,6 +149,7 @@ class Noble:
) from _NOBLE_ERROR
self.client = self._resolve_client(client)
self._updated = False
self._setup_lock = asyncio.Lock()
@staticmethod
def _resolve_client(client):
@@ -144,24 +159,42 @@ class Noble:
return client
async def setup(self) -> None:
"""fetch the noble_tls Go shared library once; idempotent
"""fetch the noble_tls Go shared library once; idempotent and concurrency-safe
download_if_necessary handles the first-time fetch (no lib present);
update_if_necessary refreshes an existing one. try download first so a
clean environment works, falling back to update.
guarded by an asyncio.Lock with a check-lock-recheck so concurrent first
requests don't both run the fetch: the fast path returns once _updated is
set, and only the first caller through the lock does the work.
"""
if self._updated:
return
download = getattr(noble_tls, "download_if_necessary", None)
if download is not None:
await download()
else:
await noble_tls.update_if_necessary()
self._updated = True
async with self._setup_lock:
if self._updated:
return
download = getattr(noble_tls, "download_if_necessary", None)
if download is not None:
await download()
else:
await noble_tls.update_if_necessary()
self._updated = True
def create_session(self, headers, timeout, **kwargs):
"""build the noble_tls Session"""
return noble_tls.Session(client=self.client, **kwargs)
"""build the noble_tls Session, honoring session-default headers + timeout
noble_tls.Session takes neither headers nor timeout in its constructor, so
aioweb's session-default headers (and the coerced timeout) are applied after
construction — matching CurlCffi, which passes both through. without this a
TLSSession(backend=Noble(...), headers=...) would silently drop the headers.
"""
session = noble_tls.Session(client=self.client, **kwargs)
if headers:
session.headers.update(headers)
if timeout is not None:
session.timeout_seconds = timeout
return session
async def raw_request(self, session, method, url, **kwargs) -> Response:
"""send via noble_tls and adapt the result into an aioweb.Response"""
@@ -169,7 +202,9 @@ class Noble:
timeout = _coerce_timeout(kwargs.pop("timeout", None))
if timeout is not None:
kwargs["timeout_seconds"] = int(timeout)
# noble takes whole seconds; round UP so a sub-second timeout (e.g. 0.5)
# doesn't truncate to 0 (which would mean no/instant timeout)
kwargs["timeout_seconds"] = max(1, math.ceil(timeout))
proxy = kwargs.pop("proxy", None)
if proxy: