aioproxies/README.md

# aioproxies

Proxy parsing, formatting, health, and pool management. Renders proxies for
aiohttp/aioweb, camoufox, and socks5; manages session templates (with
caller-supplied fields like country/ttl), rotating lists, or a static proxy; and
(for rotating lists) tracks burn/timeout, usage, reuse cooldown, and live pool
edits. **Credentials are always injected — never hardcoded.**

## Install

```
aioproxies @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioproxies.git@v0.2.1
# network helpers (current_ip / reset) need the extra:
aioproxies[net] @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioproxies.git@v0.2.1
```

The core has no dependencies. The `net` extra adds `aiohttp` for `current_ip` /
`reset`.

## Formatting

```python
from aioproxies import parse

p = parse("1.2.3.4:8080:user:pass")        # or "host:port" for IP-authenticated proxies
p.aiohttp()    # {"http": "...", "https": "..."}  -> aioweb ExtendedSession(proxies=)
p.camoufox()   # {"server": "...", "username": ..., "password": ...}
p.socks5()     # {"server": "socks5://...", ...}
p.url()        # "http://user:pass@host:port"
p.key()        # "1.2.3.4:8080:user:pass"  (canonical identity; "host:port" if auth-less)
```

Auth-less (IP-authenticated) proxies are first-class: `"host:port"` parses and
every render shape omits the credentials. The 4-part form splits on the first three
colons, so a password may itself contain colons (`host:port:user:pa:ss:word`).

## Sources

Construct with exactly one source:

```python
from aioproxies import AioProxies

# rotating list (round-robin, shuffled by default)
m = AioProxies(proxies=["h1:1:u:p", "h2:2:u:p"])
m.get()        # next proxy as an aiohttp dict

# session template — {session} is filled with a fresh id each call
m = AioProxies(template="gw.example.io:9000:user_X,sess_{session}:pw")
# a bare {} also works and is treated as the session slot

# static
m = AioProxies(static="1.2.3.4:8080:u:p")

# from a file (raises FileNotFoundError if missing — never exits the process)
m = AioProxies.from_file("proxies.txt")
```

The class is also exported as `ProxyManager` and lowercase `aioproxies` (aliases of
`AioProxies`) — use whichever reads best at your call site.

## Location / per-call fields

Templates can carry extra named placeholders the caller fills per call; the lib
always fills `{session}`. This replaces the old `location_proxy(country, ttl)` /
`dynamic_proxy(user)` helpers — one template, fields supplied at call time:

```python
m = AioProxies(template="portal.io:1080:user_X,country_{country},ttl_{ttl},sess_{session}:pw")

m.next(country="ca", ttl=30)   # lib fills {session}; caller fills {country}/{ttl}
m.get(country="us", ttl=60)    # same, returned as an aiohttp dict
```

Provider-specific values (account, password, the host, country codes, ASN tables,
which providers support geo) are **your config** — bake them into the template or
pass them as fields. The lib only fills placeholders; it never holds credentials.

### Provider session strings (e.g. mobile rotation)

```python
# creds come from your config — placeholders shown here
template = (
    "portal.anyip.io:1080:"
    "user_{ACCOUNT},type_mobile,country_{{country}},asn_{{asn}},session_{{session}}:{PASSWORD}"
).format(ACCOUNT=acct, PASSWORD=pw)   # double-braced fields survive this .format()
m = AioProxies(template=template)     # and stay as {country}/{asn}/{session} for the lib
m.next(country="us", asn="7922")
```

The credentials are baked in once with `.format()`; the per-call fields and `{session}`
are double-braced (`{{country}}`) so they pass through that `.format()` untouched and
remain for `next(**fields)` / the lib to fill.

## Proxy health & pool management (rotating list source)

For `proxies=` / `from_file` sources, the manager tracks each proxy's health and
usage and lets you edit the pool live. (On `template=` / `static=` these methods are
**no-ops that log a warning** and return cleanly — generic caller code can call them
regardless of source.)

```python
from aioproxies import AioProxies, ProxiesExhaustedError

pm = AioProxies(proxies=[...], cooldown=5)   # 5s reuse spacing; cooldown defaults to 0 (off)

try:
    proxy = pm.get()                          # next usable proxy, aiohttp dict
    resp = await session.get(url, proxies=proxy)
    if response_looks_blocked(resp):
        pm.burn(proxy, 600)                   # time out 10 min ... or pm.burn(proxy) for dead
except ProxiesExhaustedError:
    ...                                       # whole pool permanently dead — back off / refetch

pm.replace(fresh_batch)                       # swap in a new provider batch
pm.stats()                                    # monitor uses + timeout state
```

### Selection

Rotation is **sequential round-robin over usable proxies**:

1. proxies that are fine (never burned, or a timed burn already expired) cycle in
   order — same as v0.1.0.
2. if none are fine but some are merely timed, the manager **warns** and hands out
   the one recovering soonest (still counts a use).
3. if every proxy is permanently dead (`-1`), `next()`/`get()` raise
   `ProxiesExhaustedError`.

`next()` still returns a `Proxy`; `get()` still returns an aiohttp dict.

### Burn / restore

```python
pm.burn(proxy)            # dead/permanent (-1) — only manual restore() brings it back
pm.burn(proxy, 600)       # timed — usable again automatically after 600s (lazy, no timers)
pm.restore(proxy)         # clear any burn/timeout, back to fine
pm.is_burned(proxy)       # current state (expired timed burns read False)
```

`burn`/`restore`/`is_burned`/`remove` accept **any proxy shape** — a spec string, a
`Proxy`, an aiohttp/camoufox/socks5 dict, or a url — all resolve to the same canonical
key (`host:port:user:pass`, or `host:port` auth-less). The password is part of the key,
so two proxies differing only by password are distinct slots. `burn` on a proxy not in
the pool raises `ValueError`.

### Cooldown

`AioProxies(proxies=[...], cooldown=5)` spaces reuse: each handout times the proxy out
for `cooldown` seconds so it isn't reused if avoidable. It is **soft** — under load
(everything cooling) it falls through to the soonest-to-recover and never raises on
cooldown alone. Default `0` = off (exact v0.1.0 behavior).

### Stats

```python
pm.stats()        # [{"proxy": "h:p:u:pw", "uses": int, "state": "active"|"timed"|"dead",
                  #   "timeout": <ts | -1 | None>}, ...]
pm.reset_stats()  # zero all use counters; leave timeouts untouched
```

`uses` is a pure counter (every handout, including forced ones); it never drives
selection and survives burns — a proxy can read "used 500× and dead". The `proxy`
field is the full canonical spec (passwords included).

### Live pool edits

```python
pm.replace(new_batch)                    # swap the whole list; wipes per-proxy state
pm.replace(new_batch, keep_state=True)   # survivors keep uses/timeout; new ones start clean
pm.add("h:p:u:pw")                       # append (single or list); skip exact-duplicate keys
pm.remove(proxy)                         # drop a slot entirely (any shape) — distinct from burn
```

`replace` resets the rotation index and honors the manager's `shuffle` setting on the
incoming list. `remove` differs from `burn`: burn = unusable but still tracked; remove =
gone from the pool. Like the burn family, `add`/`replace` accept **any proxy shape**
(spec/`Proxy`/url/aiohttp dict/camoufox/socks5 dict). `canonical_key(shape)` and
`to_proxy(shape)` are exported if you need the key or a normalized `Proxy` yourself.

## Network helpers (optional)

```python
from aioproxies.net import current_ip, reset

ip = await current_ip("1.2.3.4:8080:u:p")   # egress ip through the proxy (ipify by default)
await reset("https://provider/reset-url")    # rotate upstream ip
```

`current_ip` defaults to ipify and is opt-in; pass `test_url=` to point elsewhere.

## Notes

- No module-level globals; rotation state is per-instance.
- A missing proxy file raises, it does not exit the process.
- Country/ASN tables, provider accounts, and reset URLs are project config —
  inject them; do not hardcode credentials in shared code.
- aioweb integration is the manual loop shown above (get → use → burn on block).
  A provider-protocol auto-rotation is a possible later enhancement, not in this lib.

## Changelog

### v0.2.1

- **Legible missing-template-field error:** a `template=` placeholder not supplied to
  `next(**fields)` now raises a clear `ValueError` naming the field, instead of leaking
  a bare `KeyError` from `str.format`.

### v0.2.0

- **Proxy health for rotating lists:** `burn`/`restore`/`is_burned` (dead `-1` vs
  timed), `stats`/`reset_stats`, and the new `ProxiesExhaustedError` (all-dead pool).
- **Cooldown:** new `cooldown=` constructor arg spaces reuse; default `0` = off.
- **Live pool edits:** `replace` (with `keep_state=`), `add`, `remove`, keyed by a
  canonical proxy key that accepts every input shape (incl. auth-less / IP-auth).
- **`{session}` default is now 8-char alphanumeric** (was 10-digit numeric);
  `session_len` default is `8`. Templates that set `session_len` explicitly are
  unaffected by the length change; the charset is now alphanumeric regardless.
- **Backward-compatible:** a v0.1.0-style manager (no burns, `cooldown=0`) behaves
  byte-for-byte identically — sequential round-robin, `next()`→`Proxy`,
  `get()`→aiohttp dict, never raises.