aioproxies/README.md

# aioproxies

Proxy parsing, formatting, health, and pool management. Renders proxies for
aiohttp/aioweb, camoufox, and socks5; manages session templates (with
caller-supplied fields like country/ttl), rotating lists, or a static proxy; and
(for rotating lists) tracks burn/timeout, usage, reuse cooldown, and live pool
edits. **Credentials are always injected — never hardcoded.**

## Install

```
aioproxies @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioproxies.git@v0.3.2
# network helpers (current_ip / reset) need the extra:
aioproxies[net] @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioproxies.git@v0.3.2
```

The core has no dependencies. The `net` extra adds `aiohttp` for `current_ip` /
`reset`.

Drop the `@v0.3.2` suffix from the line above to install the latest unpinned.

## Formatting

```python
from aioproxies import parse

p = parse("1.2.3.4:8080:user:pass")        # or "host:port" for IP-authenticated proxies
p.aiohttp()    # {"http": "...", "https": "..."}  -> aioweb ExtendedSession(proxies=)
p.camoufox()   # {"server": "...", "username": ..., "password": ...}
p.socks5()     # {"server": "socks5://...", ...}
p.url()        # "http://user:pass@host:port"
p.key()        # "1.2.3.4:8080:user:pass"  (canonical identity; "host:port" if auth-less)
```

Auth-less (IP-authenticated) proxies are first-class: `"host:port"` parses and
every render shape omits the credentials. The 4-part form splits on the first three
colons, so a password may itself contain colons (`host:port:user:pa:ss:word`).
`url()` / `aiohttp()` percent-encode the credentials, so reserved characters
(`/ # ? @`) in a user or password still produce a valid URL.

## Sources

Construct with exactly one source:

```python
from aioproxies import AioProxies

# rotating list (round-robin, shuffled by default)
m = AioProxies(proxies=["h1:1:u:p", "h2:2:u:p"])
m.get()        # next proxy as an aiohttp dict

# session template — {session} is filled with a fresh id each call
m = AioProxies(template="gw.example.io:9000:user_X,sess_{session}:pw")
# a bare {} also works and is treated as the session slot

# static
m = AioProxies(static="1.2.3.4:8080:u:p")

# from a file (raises FileNotFoundError if missing — never exits the process)
m = AioProxies.from_file("proxies.txt")
```

The class is also exported as `ProxyManager` and lowercase `aioproxies` (aliases of
`AioProxies`) — use whichever reads best at your call site.

## Location / per-call fields

Templates can carry extra named placeholders the caller fills per call; the lib
always fills `{session}`. This replaces the old `location_proxy(country, ttl)` /
`dynamic_proxy(user)` helpers — one template, fields supplied at call time:

```python
m = AioProxies(template="portal.io:1080:user_X,country_{country},ttl_{ttl},sess_{session}:pw")

m.next(country="ca", ttl=30)   # lib fills {session}; caller fills {country}/{ttl}
m.get(country="us", ttl=60)    # same, returned as an aiohttp dict
```

Provider-specific values (account, password, the host, country codes, ASN tables,
which providers support geo) are **your config** — bake them into the template or
pass them as fields. The lib only fills placeholders; it never holds credentials.

### Provider session strings (e.g. mobile rotation)

```python
# creds come from your config — placeholders shown here
template = (
    "portal.anyip.io:1080:"
    "user_{ACCOUNT},type_mobile,country_{{country}},asn_{{asn}},session_{{session}}:{PASSWORD}"
).format(ACCOUNT=acct, PASSWORD=pw)   # double-braced fields survive this .format()
m = AioProxies(template=template)     # and stay as {country}/{asn}/{session} for the lib
m.next(country="us", asn="7922")
```

The credentials are baked in once with `.format()`; the per-call fields and `{session}`
are double-braced (`{{country}}`) so they pass through that `.format()` untouched and
remain for `next(**fields)` / the lib to fill.

## Proxy health & pool management (rotating list source)

For `proxies=` / `from_file` sources, the manager tracks each proxy's health and
usage and lets you edit the pool live. (On `template=` / `static=` these methods are
**no-ops that log a warning** and return cleanly — generic caller code can call them
regardless of source.)

```python
from aioproxies import AioProxies, ProxiesExhaustedError

pm = AioProxies(proxies=[...], cooldown=5)   # 5s reuse spacing; cooldown defaults to 0 (off)

try:
    proxy = pm.get()                          # next usable proxy, aiohttp dict
    resp = await session.get(url, proxies=proxy)
    if response_looks_blocked(resp):
        pm.burn(proxy, 600)                   # time out 10 min ... or pm.burn(proxy) for dead
except ProxiesExhaustedError:
    ...                                       # whole pool permanently dead — back off / refetch

pm.replace(fresh_batch)                       # swap in a new provider batch
pm.stats()                                    # monitor uses + timeout state
```

### Selection

Rotation is **sequential round-robin over usable proxies**:

1. proxies that are fine (never burned, or a timed burn already expired) cycle in
   order — same as v0.1.0.
2. if none are fine but some are merely timed, the manager **warns** and hands out
   the one recovering soonest (still counts a use).
3. if every proxy is permanently dead (`-1`), `next()`/`get()` raise
   `ProxiesExhaustedError`.

`next()` still returns a `Proxy`; `get()` still returns an aiohttp dict.

### Burn / restore

```python
pm.burn(proxy)            # dead/permanent (-1) — only manual restore() brings it back
pm.burn(proxy, 600)       # timed — usable again automatically after 600s (lazy, no timers)
pm.restore(proxy)         # clear any burn/timeout, back to fine
pm.is_burned(proxy)       # current state (expired timed burns read False)
```

`burn`/`restore`/`is_burned`/`remove` accept **any proxy shape** — a spec string, a
`Proxy`, an aiohttp/camoufox/socks5 dict, or a url — all resolve to the same canonical
key (`host:port:user:pass`, or `host:port` auth-less; the port is normalized so
`host:080` and `host:80` are one slot). The password is part of the key, so two proxies
differing only by password are distinct slots. `burn` on a proxy not in the pool raises
`ValueError`; `restore` on a proxy not in the pool logs a warning and no-ops (matching
`remove`'s contract, as of v0.3.0 — previously it silently did nothing with no signal).
A **portless** url (`http://user:pass@host`, no `:port`) parses fine via `to_proxy()`,
but keying it (`canonical_key`/`.key()`/`burn`/`add`/`remove`) raises `ValueError`
naming the missing port — a proxy needs a port to have a canonical identity.

### Cooldown

`AioProxies(proxies=[...], cooldown=5)` spaces reuse: each handout times the proxy out
for `cooldown` seconds so it isn't reused if avoidable. It is **soft** — under load
(everything cooling) it falls through to the soonest-to-recover and never raises on
cooldown alone. Default `0` = off (exact v0.1.0 behavior).

### Stats

```python
pm.stats()        # [{"proxy": "h:p:u:pw", "uses": int, "state": "active"|"timed"|"dead",
                  #   "timeout": <ts | -1 | None>}, ...]
pm.reset_stats()  # zero all use counters; leave timeouts untouched
```

`uses` is a pure counter (every handout, including forced ones); it never drives
selection and survives burns — a proxy can read "used 500× and dead". The `proxy`
field is the full canonical spec (passwords included).

### Live pool edits

```python
pm.replace(new_batch)                    # swap the whole list; wipes per-proxy state
pm.replace(new_batch, keep_state=True)   # survivors keep uses/timeout; new ones start clean
pm.add("h:p:u:pw")                       # append (single or list); skip exact-duplicate keys
pm.remove(proxy)                         # drop a slot entirely (any shape) — distinct from burn
```

`replace` resets the rotation index and honors the manager's `shuffle` setting on the
incoming list. `remove` differs from `burn`: burn = unusable but still tracked; remove =
gone from the pool — removing a slot that precedes the rotation cursor adjusts the
cursor so `next()` doesn't skip a proxy. Like the burn family, `add`/`replace` accept
**any proxy shape** (spec/`Proxy`/url/aiohttp dict/camoufox/socks5 dict). The
constructor and `from_file` also dedupe by canonical key, same as `add`.
`canonical_key(shape)` and `to_proxy(shape)` are exported if you need the key or a
normalized `Proxy` yourself.

## Network helpers (optional)

```python
from aioproxies.net import current_ip, reset

ip = await current_ip("1.2.3.4:8080:u:p")   # egress ip through the proxy (ipify by default)
await reset("https://provider/reset-url")    # rotate upstream ip
```

`current_ip` defaults to ipify and is opt-in; pass `test_url=` to point elsewhere.

## Notes

- No module-level globals; rotation state is per-instance.
- A missing proxy file raises, it does not exit the process.
- Country/ASN tables, provider accounts, and reset URLs are project config —
  inject them; do not hardcode credentials in shared code.
- aioweb integration is the manual loop shown above (get → use → burn on block).
  A provider-protocol auto-rotation is a possible later enhancement, not in this lib.

## Changelog

### v0.3.2

- **Portless proxy url now fails loud and legible.** `canonical_key`/`.key()`
  (and therefore `burn`/`add`/`remove`) on a proxy url with no `:port`
  (`http://user:pass@host`) used to raise a bare `ValueError("invalid literal
  for int() with base 10: ''")` from `normalize_port('')` — an unrelated `int()`
  error that broke `burn()`'s documented "raises ValueError naming the key if
  not in pool" contract. `normalize_port` now raises `ValueError("missing port:
  ...")` naming the problem instead. `to_proxy()` on a portless url still
  succeeds (construction tolerates a missing port); only keying it raises.

### v0.3.1

- **Docs-only de-bloat pass.** Compressed module/internal docstrings and comments,
  replaced mojibake em-dashes with plain hyphens in source. No behavior change.

### v0.3.0

- **Stored-shape change: int unix deadlines.** `burn(proxy, seconds)` and the
  cooldown timeout write `int(time.time()) + n` instead of a `float`. Comparisons
  (`is_burned`/`stats`/selection) are unaffected — this only tightens what gets
  persisted into per-proxy state.
- **Port key normalization:** `key()` strips leading zeros from the port, so
  `host:080` and `host:80` are the same canonical slot. Previously a zero-padded
  port could break a `get()` → `burn()` pairing (the burn would raise "not in
  pool" against the proxy that was just handed out).
- **Constructor / `from_file` dedupe by canonical key**, matching `add()`. A
  proxy list with repeated entries (e.g. same host:port:user:pass twice) no
  longer inflates the pool or double-weights rotation.
- **`remove()` no longer skews rotation.** Removing a proxy that precedes the
  rotation cursor now decrements the cursor, so the next `next()` call doesn't
  skip or double-serve a proxy.
- **`restore()` on an unknown proxy now logs a warning and no-ops**, matching
  `remove()`'s contract (previously it silently did nothing, with `burn()`
  raising for the same precondition and `remove()` warning — `restore()` was the
  odd one out).
- **Genuine timed burns under `cooldown>0` now log at WARNING**, not DEBUG. A
  real `burn(proxy, seconds)` is now tracked separately from the manager's own
  cooldown resting, so it no longer gets buried as routine cooldown noise when
  every proxy in the pool happens to also be cooling down.
- **Source truthiness → presence.** The constructor's exactly-one-source check
  now uses `is not None` instead of truthiness, so `template=""` (or another
  falsy-but-explicitly-provided source) is accepted rather than silently
  rejected as "no source given."
- **Escape-aware bare-`{}` template normalization.** A template's bare `{}` is
  still filled as the session slot, but an escaped `{{}}` (str.format's own
  convention for a literal `{}` in the output) now survives untouched instead of
  being corrupted into `{{session}}`.

### v0.2.1

- **Legible missing-template-field error:** a `template=` placeholder not supplied to
  `next(**fields)` now raises a clear `ValueError` naming the field, instead of leaking
  a bare `KeyError` from `str.format`.

### v0.2.0

- **Proxy health for rotating lists:** `burn`/`restore`/`is_burned` (dead `-1` vs
  timed), `stats`/`reset_stats`, and the new `ProxiesExhaustedError` (all-dead pool).
- **Cooldown:** new `cooldown=` constructor arg spaces reuse; default `0` = off.
- **Live pool edits:** `replace` (with `keep_state=`), `add`, `remove`, keyed by a
  canonical proxy key that accepts every input shape (incl. auth-less / IP-auth).
- **`{session}` default is now 8-char alphanumeric** (was 10-digit numeric);
  `session_len` default is `8`. Templates that set `session_len` explicitly are
  unaffected by the length change; the charset is now alphanumeric regardless.
- **Backward-compatible:** a v0.1.0-style manager (no burns, `cooldown=0`) behaves
  byte-for-byte identically — sequential round-robin, `next()`→`Proxy`,
  `get()`→aiohttp dict, never raises.