aioweb/README.md

# aioweb

Async HTTP session wrapper over `aiohttp`. Adds session-level proxies, header
overwrites, ephemeral (per-request generated) headers, domain rewriting, request
previews / cURL export, and retry-with-backoff. The byte-sending is isolated behind
one overridable method (`_raw_request`), so a TLS-fingerprinting backend can subclass
and swap the HTTP client while inheriting everything else.

## Install

`requirements.txt`:

```
aioweb @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb.git@v0.1.7
```

Direct:

```bash
pip install "aioweb @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioweb.git@v0.1.7"
```

Requires `aiohttp` and `yarl` (pulled transitively).

Drop the `@v0.1.7` suffix from the line above to install the latest unpinned.

## Usage

```python
from aioweb import ExtendedSession

async with ExtendedSession(proxies={"https": "http://user:pass@host:port"}, timeout=15) as s:
    resp = await s.request_with_retries("GET", "https://example.com")
    if resp:                          # FailureResponse is falsy
        data = resp.json()            # or resp.text(), resp.content
```

Sessions must be closed explicitly — use `async with` or `await s.close()`. There is
no `__del__` auto-close (unsafe for async resources); leaking a session emits a
`ResourceWarning`.

## Responses

`request`/`request_with_retries` return a `Response` (success or non-retryable status)
or a falsy `FailureResponse` (all retries failed). Both expose the same surface as
**properties**, so callers branch uniformly:

- `status_code`, `headers`, `url`, `reason`, `cookies`, `history`, `redirect_chain`
- `is_success` (2xx), `is_redirect`
- `content` (bytes), `text(encoding=None)`, `json()` (None if not JSON)
- `raise_for_status()` raises `AiowebError` on non-2xx
- `bool(resp)` / `if resp:` is `is_success`

## Retries

`request_with_retries` retries on exceptions **and** retryable statuses (429, 500,
502, 503, 504 by default), with exponential backoff (`backoff_base ** attempt`).

```python
resp = await s.request_with_retries(
    "GET", url, attempts=5, backoff_base=2.0,
    retry_statuses={429, 503},        # override which statuses retry
)
```

Returns a `FailureResponse` (falsy) if every attempt fails.

## Header overwrites & ephemeral headers

```python
s.overwrite_header("User-Agent", "custom")     # replace per request
s.overwrite_inject(True)                        # also add when absent
s.set_ephemeral("X-Time", lambda: str(time.time()))  # generated fresh each request
```

With `inject=False` (default) overwrites only replace headers already present in a
request; with `inject=True` they're added regardless.

## Domain rewriting

```python
s.overwrite_domain("internal.local", "127.0.0.1")   # host-substring rewrite
```

## Preview / debug

```python
print(s.preview("POST", url, json={"a": 1}).as_curl())   # equivalent cURL command
```

Pass `debug=True` to `request_with_retries` to log the cURL preview and request flow.

## Custom backends

The `_raw_request(method, url, **kwargs) -> Response` method is the only place that
touches the HTTP client. To use a different backend (e.g. a TLS-fingerprinting client
like curl_cffi), subclass `ExtendedSession` and override just `_raw_request`, building
a `Response` from that backend's primitives:

```python
class MySession(ExtendedSession):
    async def _raw_request(self, method, url, **kwargs):
        r = await my_client.request(method, url, **kwargs)
        return Response(
            status_code=r.status, headers=r.headers, content=await r.read(),
            url=str(r.url), reason=r.reason,
        )
```

Everything else — header overwrites, ephemeral headers, domain rewriting, proxy
resolution, retries, previews — is inherited. `Response` is built from primitives
(status, headers, content, url, history) precisely so any backend can produce one.

## Migrating from the original

Back-compat shims are in place for the common path:

- `aiowebResponse` is aliased to `Response` (the class was renamed) — old imports work.
- `request_retries(session, ...)` and `test_proxies(session)` remain as module functions.
- `raise_for_status` now raises `AiowebError` (a subclass of `Exception`), so
  `except Exception` still catches it.

Two changes can't be shimmed without re-introducing the bugs they fix:

- **`is_success` is now a property on `FailureResponse`, not a method.** Code that
  *called* `failure.is_success()` must drop the parens to `failure.is_success`. (Code
  that wrote `if failure.is_success` was previously always-truthy — a bug — and now
  behaves correctly.)
- **No `__del__` auto-close.** Sessions must be closed via `async with` or
  `await s.close()`; a leaked session emits a `ResourceWarning`. The old finalizer-based
  auto-close was unsafe and was removed.

## Changelog

### v0.1.7

- **`get_cookies()` now returns real cookies.** Previously called `filter_cookies()`
  with no URL, which only ever returns domain-less shared cookies — every normal
  domain-bound cookie (including ones set by a real `Set-Cookie` response) was
  silently omitted. Now iterates the jar directly.
- **`set_cookie()` no longer leaks a shared cookie to every host.** A bare hostname
  (`domain="example.com"`) built a schemeless URL, which aiohttp's jar treats as a
  domain-less "shared" cookie sent with every request the session makes, including
  unrelated hosts. A scheme is now added when missing so the cookie is scoped to
  that host.
- **`set_cookie()` now honors `path`** (previously ignored — the cookie always
  landed at `path="/"`). `domain=None` is unchanged in meaning but now stores a
  truly shared cookie (sent to every host) instead of one silently bound to
  `localhost` only, which made `set_cookie(name, value)` (no domain) a silent
  no-op for any real request.
- **The backend session is built lazily**, not in `__init__`. Under aiohttp 3.14,
  constructing `aiohttp.ClientSession` requires a running event loop; eager
  construction crashed the common host pattern of attaching a session before the
  loop starts (e.g. `bot.http = ExtendedSession(...)` in `Bot.__init__`). The
  session (and any subclass's `_create_session` override) now builds on first
  access instead.

### v0.1.2

- Pinned `commons` to v0.2.1 (retry `attempts` floor fix).

### v0.1.1

- **JSON list bodies** now route to `json=` (were wrongly form-encoded via `data=` —
  only dicts went to `json=` before).
- **Exhausted retries return the real last response.** When every attempt hit a
  retryable status (429/5xx), the loop discarded it and returned a synthetic
  `FailureResponse` (status 0); now the real last 4xx/5xx `Response` is returned (only a
  pure-exception failure yields `FailureResponse`).
- Retry/backoff moved onto `commons.aretry` (shared engine); backoff schedule unchanged.
  Adds a `commons` dependency.

## Versioning

Releases are tagged `vX.Y.Z`. The install line above pins a release; drop the `@vX.Y.Z` suffix to install the latest unpinned. Pin deliberately for reproducible installs.