aioproxies

Proxy parsing, formatting, health, and pool management. Renders proxies for aiohttp/aioweb, camoufox, and socks5; manages session templates (with caller-supplied fields like country/ttl), rotating lists, or a static proxy; and (for rotating lists) tracks burn/timeout, usage, reuse cooldown, and live pool edits. Credentials are always injected — never hardcoded.

Install

aioproxies @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioproxies.git@v0.2.2
# network helpers (current_ip / reset) need the extra:
aioproxies[net] @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioproxies.git@v0.2.2

The core has no dependencies. The net extra adds aiohttp for current_ip / reset.

Drop the @v0.2.2 suffix from the line above to install the latest unpinned.

Formatting

from aioproxies import parse

p = parse("1.2.3.4:8080:user:pass")        # or "host:port" for IP-authenticated proxies
p.aiohttp()    # {"http": "...", "https": "..."}  -> aioweb ExtendedSession(proxies=)
p.camoufox()   # {"server": "...", "username": ..., "password": ...}
p.socks5()     # {"server": "socks5://...", ...}
p.url()        # "http://user:pass@host:port"
p.key()        # "1.2.3.4:8080:user:pass"  (canonical identity; "host:port" if auth-less)

Auth-less (IP-authenticated) proxies are first-class: "host:port" parses and every render shape omits the credentials. The 4-part form splits on the first three colons, so a password may itself contain colons (host:port:user:pa:ss:word).

Sources

Construct with exactly one source:

from aioproxies import AioProxies

# rotating list (round-robin, shuffled by default)
m = AioProxies(proxies=["h1:1:u:p", "h2:2:u:p"])
m.get()        # next proxy as an aiohttp dict

# session template — {session} is filled with a fresh id each call
m = AioProxies(template="gw.example.io:9000:user_X,sess_{session}:pw")
# a bare {} also works and is treated as the session slot

# static
m = AioProxies(static="1.2.3.4:8080:u:p")

# from a file (raises FileNotFoundError if missing — never exits the process)
m = AioProxies.from_file("proxies.txt")

The class is also exported as ProxyManager and lowercase aioproxies (aliases of AioProxies) — use whichever reads best at your call site.

Location / per-call fields

Templates can carry extra named placeholders the caller fills per call; the lib always fills {session}. This replaces the old location_proxy(country, ttl) / dynamic_proxy(user) helpers — one template, fields supplied at call time:

m = AioProxies(template="portal.io:1080:user_X,country_{country},ttl_{ttl},sess_{session}:pw")

m.next(country="ca", ttl=30)   # lib fills {session}; caller fills {country}/{ttl}
m.get(country="us", ttl=60)    # same, returned as an aiohttp dict

Provider-specific values (account, password, the host, country codes, ASN tables, which providers support geo) are your config — bake them into the template or pass them as fields. The lib only fills placeholders; it never holds credentials.

Provider session strings (e.g. mobile rotation)

# creds come from your config — placeholders shown here
template = (
    "portal.anyip.io:1080:"
    "user_{ACCOUNT},type_mobile,country_{{country}},asn_{{asn}},session_{{session}}:{PASSWORD}"
).format(ACCOUNT=acct, PASSWORD=pw)   # double-braced fields survive this .format()
m = AioProxies(template=template)     # and stay as {country}/{asn}/{session} for the lib
m.next(country="us", asn="7922")

The credentials are baked in once with .format(); the per-call fields and {session} are double-braced ({{country}}) so they pass through that .format() untouched and remain for next(**fields) / the lib to fill.

Proxy health & pool management (rotating list source)

For proxies= / from_file sources, the manager tracks each proxy's health and usage and lets you edit the pool live. (On template= / static= these methods are no-ops that log a warning and return cleanly — generic caller code can call them regardless of source.)

from aioproxies import AioProxies, ProxiesExhaustedError

pm = AioProxies(proxies=[...], cooldown=5)   # 5s reuse spacing; cooldown defaults to 0 (off)

try:
    proxy = pm.get()                          # next usable proxy, aiohttp dict
    resp = await session.get(url, proxies=proxy)
    if response_looks_blocked(resp):
        pm.burn(proxy, 600)                   # time out 10 min ... or pm.burn(proxy) for dead
except ProxiesExhaustedError:
    ...                                       # whole pool permanently dead — back off / refetch

pm.replace(fresh_batch)                       # swap in a new provider batch
pm.stats()                                    # monitor uses + timeout state

Selection

Rotation is sequential round-robin over usable proxies:

1. proxies that are fine (never burned, or a timed burn already expired) cycle in order — same as v0.1.0.
2. if none are fine but some are merely timed, the manager warns and hands out the one recovering soonest (still counts a use).
3. if every proxy is permanently dead (-1), next()/get() raise ProxiesExhaustedError.

next() still returns a Proxy; get() still returns an aiohttp dict.

Burn / restore

pm.burn(proxy)            # dead/permanent (-1) — only manual restore() brings it back
pm.burn(proxy, 600)       # timed — usable again automatically after 600s (lazy, no timers)
pm.restore(proxy)         # clear any burn/timeout, back to fine
pm.is_burned(proxy)       # current state (expired timed burns read False)

burn/restore/is_burned/remove accept any proxy shape — a spec string, a Proxy, an aiohttp/camoufox/socks5 dict, or a url — all resolve to the same canonical key (host:port:user:pass, or host:port auth-less). The password is part of the key, so two proxies differing only by password are distinct slots. burn on a proxy not in the pool raises ValueError.

Cooldown

AioProxies(proxies=[...], cooldown=5) spaces reuse: each handout times the proxy out for cooldown seconds so it isn't reused if avoidable. It is soft — under load (everything cooling) it falls through to the soonest-to-recover and never raises on cooldown alone. Default 0 = off (exact v0.1.0 behavior).

Stats

pm.stats()        # [{"proxy": "h:p:u:pw", "uses": int, "state": "active"|"timed"|"dead",
                  #   "timeout": <ts | -1 | None>}, ...]
pm.reset_stats()  # zero all use counters; leave timeouts untouched

uses is a pure counter (every handout, including forced ones); it never drives selection and survives burns — a proxy can read "used 500× and dead". The proxy field is the full canonical spec (passwords included).

Live pool edits

pm.replace(new_batch)                    # swap the whole list; wipes per-proxy state
pm.replace(new_batch, keep_state=True)   # survivors keep uses/timeout; new ones start clean
pm.add("h:p:u:pw")                       # append (single or list); skip exact-duplicate keys
pm.remove(proxy)                         # drop a slot entirely (any shape) — distinct from burn

replace resets the rotation index and honors the manager's shuffle setting on the incoming list. remove differs from burn: burn = unusable but still tracked; remove = gone from the pool. Like the burn family, add/replace accept any proxy shape (spec/Proxy/url/aiohttp dict/camoufox/socks5 dict). canonical_key(shape) and to_proxy(shape) are exported if you need the key or a normalized Proxy yourself.

Network helpers (optional)

from aioproxies.net import current_ip, reset

ip = await current_ip("1.2.3.4:8080:u:p")   # egress ip through the proxy (ipify by default)
await reset("https://provider/reset-url")    # rotate upstream ip

current_ip defaults to ipify and is opt-in; pass test_url= to point elsewhere.

Notes

• No module-level globals; rotation state is per-instance.
• A missing proxy file raises, it does not exit the process.
• Country/ASN tables, provider accounts, and reset URLs are project config — inject them; do not hardcode credentials in shared code.
• aioweb integration is the manual loop shown above (get → use → burn on block). A provider-protocol auto-rotation is a possible later enhancement, not in this lib.

Changelog

v0.2.1

• Legible missing-template-field error: a template= placeholder not supplied to next(**fields) now raises a clear ValueError naming the field, instead of leaking a bare KeyError from str.format.

v0.2.0

• Proxy health for rotating lists: burn/restore/is_burned (dead -1 vs timed), stats/reset_stats, and the new ProxiesExhaustedError (all-dead pool).
• Cooldown: new cooldown= constructor arg spaces reuse; default 0 = off.
• Live pool edits: replace (with keep_state=), add, remove, keyed by a canonical proxy key that accepts every input shape (incl. auth-less / IP-auth).
• {session} default is now 8-char alphanumeric (was 10-digit numeric); session_len default is 8. Templates that set session_len explicitly are unaffected by the length change; the charset is now alphanumeric regardless.
• Backward-compatible: a v0.1.0-style manager (no burns, cooldown=0) behaves byte-for-byte identically — sequential round-robin, next()→Proxy, get()→aiohttp dict, never raises.