fix: canonical port key, remove()/restore() drift, escape-aware templates

Port keys no longer drift on zero-padding (host:080 vs host:80 collapsed
one canonical key), so a get() -> burn() pairing on a padded port no
longer raises a false "not in pool". Constructor/from_file now dedupe by
canonical key like add(). remove() decrements the rotation cursor when
the removed slot precedes it, fixing a skip/double-serve. restore() on
an unknown key now warns instead of silently no-op, matching remove()'s
contract. A genuine timed burn() under cooldown>0 is now tracked
separately from routine cooldown resting, so it logs WARNING instead of
being buried as DEBUG. Source validation uses is-not-None instead of
truthiness, so a falsy-but-provided source (e.g. template="") is
accepted. Bare "{}" template normalization is now escape-aware, so an
intentional "{{}}" literal survives instead of being corrupted.

Stored-shape change: burn()/cooldown deadlines now write int(time.time())
instead of float; comparisons are unaffected.

Docstrings compressed (module + next()); no behavior change.

Signed-off-by: disqualifier <dev@disqualifier.me>

.gitignore

@@ -1,5 +1,5 @@
 # claude
-CLAUDE.md
+.claude/
 
 # python
 __pycache__/

README.md

@@ -9,14 +9,16 @@ edits. **Credentials are always injected — never hardcoded.**
 ## Install
 
 ```
-aioproxies @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioproxies.git@v0.2.0
+aioproxies @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioproxies.git@v0.3.0
 # network helpers (current_ip / reset) need the extra:
-aioproxies[net] @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioproxies.git@v0.2.0
+aioproxies[net] @ git+ssh://git@git.rethinkstudios.io/rethink-public/aioproxies.git@v0.3.0
 ```
 
 The core has no dependencies. The `net` extra adds `aiohttp` for `current_ip` /
 `reset`.
 
+Drop the `@v0.3.0` suffix from the line above to install the latest unpinned.
+
 ## Formatting
 
 ```python
@@ -33,6 +35,8 @@ p.key()        # "1.2.3.4:8080:user:pass"  (canonical identity; "host:port" if a
 Auth-less (IP-authenticated) proxies are first-class: `"host:port"` parses and
 every render shape omits the credentials. The 4-part form splits on the first three
 colons, so a password may itself contain colons (`host:port:user:pa:ss:word`).
+`url()` / `aiohttp()` percent-encode the credentials, so reserved characters
+(`/ # ? @`) in a user or password still produce a valid URL.
 
 ## Sources
 
@@ -140,9 +144,11 @@ pm.is_burned(proxy)       # current state (expired timed burns read False)
 
 `burn`/`restore`/`is_burned`/`remove` accept **any proxy shape** — a spec string, a
 `Proxy`, an aiohttp/camoufox/socks5 dict, or a url — all resolve to the same canonical
-key (`host:port:user:pass`, or `host:port` auth-less). The password is part of the key,
-so two proxies differing only by password are distinct slots. `burn` on a proxy not in
-the pool raises `ValueError`.
+key (`host:port:user:pass`, or `host:port` auth-less; the port is normalized so
+`host:080` and `host:80` are one slot). The password is part of the key, so two proxies
+differing only by password are distinct slots. `burn` on a proxy not in the pool raises
+`ValueError`; `restore` on a proxy not in the pool logs a warning and no-ops (matching
+`remove`'s contract, as of v0.3.0 — previously it silently did nothing with no signal).
 
 ### Cooldown
 
@@ -174,9 +180,12 @@ pm.remove(proxy)                         # drop a slot entirely (any shape) —
 
 `replace` resets the rotation index and honors the manager's `shuffle` setting on the
 incoming list. `remove` differs from `burn`: burn = unusable but still tracked; remove =
-gone from the pool. Like the burn family, `add`/`replace` accept **any proxy shape**
-(spec/`Proxy`/url/aiohttp dict/camoufox/socks5 dict). `canonical_key(shape)` and
-`to_proxy(shape)` are exported if you need the key or a normalized `Proxy` yourself.
+gone from the pool — removing a slot that precedes the rotation cursor adjusts the
+cursor so `next()` doesn't skip a proxy. Like the burn family, `add`/`replace` accept
+**any proxy shape** (spec/`Proxy`/url/aiohttp dict/camoufox/socks5 dict). The
+constructor and `from_file` also dedupe by canonical key, same as `add`.
+`canonical_key(shape)` and `to_proxy(shape)` are exported if you need the key or a
+normalized `Proxy` yourself.
 
 ## Network helpers (optional)
 
@@ -200,6 +209,45 @@ await reset("https://provider/reset-url")    # rotate upstream ip
 
 ## Changelog
 
+### v0.3.0
+
+- **Stored-shape change: int unix deadlines.** `burn(proxy, seconds)` and the
+  cooldown timeout write `int(time.time()) + n` instead of a `float`. Comparisons
+  (`is_burned`/`stats`/selection) are unaffected — this only tightens what gets
+  persisted into per-proxy state.
+- **Port key normalization:** `key()` strips leading zeros from the port, so
+  `host:080` and `host:80` are the same canonical slot. Previously a zero-padded
+  port could break a `get()` → `burn()` pairing (the burn would raise "not in
+  pool" against the proxy that was just handed out).
+- **Constructor / `from_file` dedupe by canonical key**, matching `add()`. A
+  proxy list with repeated entries (e.g. same host:port:user:pass twice) no
+  longer inflates the pool or double-weights rotation.
+- **`remove()` no longer skews rotation.** Removing a proxy that precedes the
+  rotation cursor now decrements the cursor, so the next `next()` call doesn't
+  skip or double-serve a proxy.
+- **`restore()` on an unknown proxy now logs a warning and no-ops**, matching
+  `remove()`'s contract (previously it silently did nothing, with `burn()`
+  raising for the same precondition and `remove()` warning — `restore()` was the
+  odd one out).
+- **Genuine timed burns under `cooldown>0` now log at WARNING**, not DEBUG. A
+  real `burn(proxy, seconds)` is now tracked separately from the manager's own
+  cooldown resting, so it no longer gets buried as routine cooldown noise when
+  every proxy in the pool happens to also be cooling down.
+- **Source truthiness → presence.** The constructor's exactly-one-source check
+  now uses `is not None` instead of truthiness, so `template=""` (or another
+  falsy-but-explicitly-provided source) is accepted rather than silently
+  rejected as "no source given."
+- **Escape-aware bare-`{}` template normalization.** A template's bare `{}` is
+  still filled as the session slot, but an escaped `{{}}` (str.format's own
+  convention for a literal `{}` in the output) now survives untouched instead of
+  being corrupted into `{{session}}`.
+
+### v0.2.1
+
+- **Legible missing-template-field error:** a `template=` placeholder not supplied to
+  `next(**fields)` now raises a clear `ValueError` naming the field, instead of leaking
+  a bare `KeyError` from `str.format`.
+
 ### v0.2.0
 
 - **Proxy health for rotating lists:** `burn`/`restore`/`is_burned` (dead `-1` vs

pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "aioproxies"
-version = "0.2.0"
+version = "0.3.0"
 description = "proxy parsing, formatting, health, and pool management for aiohttp/aioweb, camoufox, and socks5"
 requires-python = ">=3.10"
 dependencies = []

src/aioproxies/__init__.py

@@ -18,4 +18,4 @@ __all__ = [
     "to_proxy",
 ]
 
-__version__ = "0.2.0"
+__version__ = "0.3.0"

src/aioproxies/manager.py

@@ -1,35 +1,35 @@
 """proxy source management: session templates, rotation, static.
 
-`AioProxies` is constructed with exactly one source and hands out `Proxy`
-objects. it carries no module-level globals (rotation state is per-instance) and
-never exits the process — a missing proxy file raises, it does not `sys.exit`.
+`AioProxies` is constructed with exactly one source and hands out `Proxy` objects.
+no module-level globals (rotation state is per-instance); a missing proxy file
+raises, never `sys.exit`.
 
 sources:
-- template: a format string with named placeholders. `{session}` is filled with
-  a fresh session id on each `next()`; any other placeholder (e.g. `{country}`,
-  `{ttl}`) is filled from keyword args passed to `next(**fields)`. a bare `{}` is
-  also accepted and treated as the session slot (back-compat with simple templates).
-- proxies:  a list of specs cycled round-robin
+- template: a format string. `{session}` is filled with a fresh id on each
+  `next()`; other placeholders (`{country}`, `{ttl}`, ...) come from
+  `next(**fields)`. a bare `{}` is treated as the session slot unless escaped as
+  `{{}}` (back-compat with simple templates).
+- proxies:  a list of specs cycled round-robin (deduped by canonical key)
 - static:   one fixed proxy
 
-v0.2.0 adds proxy health to the rotating list source only — burn/timeout, usage
-counters, reuse cooldown, and pool management (replace/add/remove). these are
-keyed by each proxy's canonical key (`host:port:user:pass`, or `host:port` for
-auth-less / IP-authenticated proxies). on template/static sources they are no-ops
-that log a warning and return cleanly, so generic caller code can call them
-regardless of source.
+proxy health (rotating list source only) — burn/timeout, usage counters, reuse
+cooldown, pool edits (replace/add/remove) — keyed by each proxy's canonical key
+(`host:port:user:pass`, or `host:port` auth-less; port is normalized so `:080`
+and `:80` collapse to one key). on template/static sources these are no-ops that
+log a warning and return, so generic caller code can call them regardless of
+source.
 
-per-proxy state (keyed by canonical key):
-- `uses`: pure counter, incremented on every handout. never drives selection;
-  survives burns (a proxy can read "used 500x and dead").
-- `timeout`: availability state. `None`/`0` = fine; `-1` = dead/permanent (manual
-  restore only); a future unix ts = timed out until then (lazy, checked against
-  `time.time()`, no timers). cooldown and timed burns share this field. durations
-  (seconds) only ever exist as arguments — converted to `now + seconds` and
-  discarded; the lib never stores a raw duration.
+per-proxy state (keyed by canonical key): `uses` is a pure counter, incremented on
+every handout, never drives selection, survives burns. `timeout` is the
+availability state — `None`/`0` fine, `-1` dead/permanent (manual restore only), a
+future unix ts = timed out until then. this core is sync: no timers, lazy expiry
+checked only against `time.time()` at call time. cooldown and timed burns share
+this field; durations only ever exist as call arguments, converted to `now +
+seconds` and discarded — a raw duration is never stored.
 """
 import logging
 import random
+import re
 import string
 import time
 from typing import Dict, List, Optional, Union
@@ -39,6 +39,7 @@ from .proxy import Proxy, canonical_key, parse, to_proxy
 log = logging.getLogger(__name__)
 
 _DEAD = -1
+_BARE_SESSION_SLOT = re.compile(r"(?<!\{)\{\}(?!\})")
 
 
 class ProxiesExhaustedError(Exception):
@@ -58,22 +59,48 @@ class AioProxies:
         shuffle: bool = True,
         cooldown: int = 0,
     ):
-        sources = [s for s in (template, proxies, static) if s]
+        if proxies is not None and len(proxies) == 0:
+            raise ValueError("proxies list is empty; provide at least one proxy")
+        sources = [s for s in (template, proxies, static) if s is not None]
         if len(sources) != 1:
             raise ValueError("provide exactly one of: template, proxies, static")
-        self.template = template.replace("{}", "{session}") if template else None
+        self.template = self._normalize_template(template) if template is not None else None
         self.session_len = session_len
         self.cooldown = cooldown
         self._shuffle = shuffle
-        self._static = parse(static) if static else None
-        self._proxies = [parse(p) for p in proxies] if proxies else []
+        self._static = parse(static) if static is not None else None
+        self._proxies = self._dedupe([parse(p) for p in proxies]) if proxies is not None else []
         if self._proxies and shuffle:
             random.shuffle(self._proxies)
         self._state: Dict[str, Dict[str, object]] = {}
         for proxy in self._proxies:
-            self._state.setdefault(proxy.key(), {"uses": 0, "timeout": None})
+            self._state.setdefault(proxy.key(), self._fresh_state())
         self._index = 0
 
+    @staticmethod
+    def _dedupe(proxies: List[Proxy]) -> List[Proxy]:
+        """drop later entries sharing a canonical key, preserving first-seen order"""
+        seen: Dict[str, Proxy] = {}
+        for proxy in proxies:
+            seen.setdefault(proxy.key(), proxy)
+        return list(seen.values())
+
+    @staticmethod
+    def _fresh_state() -> Dict[str, object]:
+        """a clean per-proxy state entry: no uses, no timeout, not burned"""
+        return {"uses": 0, "timeout": None, "burned": False}
+
+    @staticmethod
+    def _normalize_template(template: str) -> str:
+        """fill a bare `{}` session slot, escape-aware
+
+        a lone `{}` is treated as `{session}` (back-compat with simple templates).
+        an escaped literal `{{}}` — str.format's own convention for a literal `{}`
+        in the output — is left untouched so it survives to render as `{}`, not
+        `{session}`.
+        """
+        return _BARE_SESSION_SLOT.sub("{session}", template)
+
     @classmethod
     def from_file(cls, path: str, **kwargs) -> "AioProxies":
         """build a rotating manager from a newline-delimited proxy file
@@ -110,20 +137,33 @@ class AioProxies:
     def next(self, **fields: object) -> Proxy:
         """return the next proxy from the configured source
 
-        for template sources, `{session}` is always filled with a fresh id and any
-        other named placeholder is filled from `fields` (e.g. next(country="ca",
-        ttl=30)). fields are ignored by list/static sources.
+        template sources: `{session}` is always filled with a fresh id, other
+        placeholders come from `fields` (e.g. next(country="ca", ttl=30)). fields
+        are ignored by list/static sources.
 
-        for list sources, skips proxies whose timeout is active (-1 dead, or a
-        future ts not yet passed), increments `uses` on handout, and applies the
-        manager's cooldown. if none are fine but some are merely timed, hands out
-        the one recovering soonest (with a warning); raises ProxiesExhaustedError
-        if every proxy is permanently dead.
+        list sources: skips proxies whose timeout is active (-1 dead, or a future
+        ts not yet passed), increments `uses` on handout, applies the manager's
+        cooldown. if none are fine but some are merely timed, hands out the one
+        recovering soonest (warns); raises ProxiesExhaustedError if all are dead.
         """
         if self._static is not None:
             return self._static
         if self.template is not None:
-            return parse(self.template.format(session=self.session_id(), **fields))
+            if "session" in fields:
+                # `session` is auto-filled with a fresh id; a caller-supplied one would
+                # collide in str.format with an opaque TypeError — reject it clearly
+                raise ValueError("'session' is filled automatically; do not pass it to next()")
+            try:
+                filled = self.template.format(session=self.session_id(), **fields)
+            except (KeyError, IndexError) as exc:
+                raise ValueError(
+                    f"template placeholder {exc} not provided; pass it to next(**fields)"
+                ) from exc
+            except ValueError as exc:
+                # a malformed template (e.g. an unmatched '{') makes str.format raise a
+                # bare ValueError; re-raise naming the cause so it isn't cryptic
+                raise ValueError(f"malformed proxy template {self.template!r}: {exc}") from exc
+            return parse(filled)
         return self._next_from_list()
 
     def _next_from_list(self) -> Proxy:
@@ -156,24 +196,19 @@ class AioProxies:
         state = self._state[proxy.key()]
         state["uses"] = int(state["uses"]) + 1
         if self.cooldown > 0:
-            state["timeout"] = now + self.cooldown
+            state["timeout"] = int(now) + self.cooldown
         return proxy
 
     def _has_burned(self) -> bool:
-        """whether the empty 'fine' tier reflects a genuine burn, not just cooldown
+        """whether the empty 'fine' tier reflects a genuine burn(), not just cooldown
 
-        a dead (-1) proxy is always a real burn. a future-ts timeout is a real timed
-        burn only when cooldown is off; with cooldown on, future-ts entries are the
-        manager's own resting and not a pool-health signal. used to pick warning
-        (real trouble) vs debug (normal cooldown) on the soonest-recovering path.
+        tracked directly via each state's `burned` flag (set by burn(), cleared by
+        restore()) so a real burn is never masked by the manager's own cooldown
+        resting on the same `timeout` field, regardless of whether cooldown is on.
+        used to pick warning (real trouble) vs debug (normal cooldown) on the
+        soonest-recovering path.
         """
-        for state in self._state.values():
-            timeout = state["timeout"]
-            if timeout == _DEAD:
-                return True
-            if self.cooldown == 0 and timeout not in (None, 0):
-                return True
-        return False
+        return any(state["burned"] for state in self._state.values())
 
     def _soonest_recovering(self) -> Optional[int]:
         """index of the timed (non-dead) proxy recovering soonest, or None"""
@@ -208,16 +243,24 @@ class AioProxies:
         if seconds is None:
             self._state[key]["timeout"] = _DEAD
         else:
-            self._state[key]["timeout"] = time.time() + seconds
+            self._state[key]["timeout"] = int(time.time()) + seconds
+        self._state[key]["burned"] = True
 
     def restore(self, proxy: Union[str, Proxy, Dict[str, str]]) -> None:
-        """clear any burn/timeout on a proxy (back to fine). no-op if already fine"""
+        """clear any burn/timeout on a proxy (back to fine). no-op if already fine
+
+        matches remove()'s contract for an unknown proxy: logs a warning and returns
+        rather than silently doing nothing.
+        """
         if not self._is_list_source():
             self._warn_non_list("restore()")
             return
         key = canonical_key(proxy)
-        if key in self._state:
-            self._state[key]["timeout"] = None
+        if key not in self._state:
+            log.warning("restore(): proxy not in pool: %s", key)
+            return
+        self._state[key]["timeout"] = None
+        self._state[key]["burned"] = False
 
     def is_burned(self, proxy: Union[str, Proxy, Dict[str, str]]) -> bool:
         """whether a proxy is currently unavailable (lazy expiry of timed burns)
@@ -294,7 +337,7 @@ class AioProxies:
             if keep_state and key in old_state:
                 new_state[key] = old_state[key]
             else:
-                new_state[key] = {"uses": 0, "timeout": None}
+                new_state[key] = self._fresh_state()
         self._proxies = new_proxies
         self._state = new_state
         self._index = 0
@@ -311,13 +354,15 @@ class AioProxies:
             if key in self._state:
                 continue
             self._proxies.append(proxy)
-            self._state[key] = {"uses": 0, "timeout": None}
+            self._state[key] = self._fresh_state()
 
     def remove(self, proxy: Union[str, Proxy, Dict[str, str]]) -> None:
         """drop a proxy from the pool entirely (by canonical key, any shape)
 
-        distinct from burn (burn = unusable but tracked; remove = gone). clamps the
-        rotation index if needed. no-op + warning if the proxy is not present.
+        distinct from burn (burn = unusable but tracked; remove = gone). decrements
+        the rotation index when the removed slot precedes it, so the next() cursor
+        still lands on the same upcoming proxy instead of skipping one. no-op +
+        warning if the proxy is not present.
         """
         if not self._is_list_source():
             self._warn_non_list("remove()")
@@ -326,12 +371,15 @@ class AioProxies:
         if key not in self._state:
             log.warning("remove(): proxy not in pool: %s", key)
             return
+        removed_idx = next(i for i, p in enumerate(self._proxies) if p.key() == key)
         self._proxies = [p for p in self._proxies if p.key() != key]
         del self._state[key]
-        if self._proxies:
-            self._index %= len(self._proxies)
-        else:
+        if not self._proxies:
             self._index = 0
+        else:
+            if removed_idx < self._index:
+                self._index -= 1
+            self._index %= len(self._proxies)
 
 
 # name aliases — same class, call it whichever reads best at your call site

src/aioproxies/net.py

@@ -37,8 +37,10 @@ async def current_ip(
     try:
         async with aiohttp.ClientSession(timeout=t) as session:
             async with session.get(test_url, proxy=p.url()) as resp:
-                data = await resp.json()
-                return data.get("ip")
+                # content_type=None: an echo endpoint may return text/plain json; the
+                # default would raise ContentTypeError and silently return None
+                data = await resp.json(content_type=None)
+                return data.get("ip") if isinstance(data, dict) else None
     except Exception as exc:
         log.warning("ip check failed: %s", exc)
         return None

src/aioproxies/proxy.py

@@ -13,12 +13,21 @@ input shape. `canonical_key()` extends that to dict/url forms.
 """
 from dataclasses import dataclass
 from typing import Dict, Optional, Union
-from urllib.parse import unquote, urlsplit
+from urllib.parse import quote, unquote, urlsplit
 
 SCHEME_HTTP = "http"
 SCHEME_SOCKS5 = "socks5"
 
 
+def normalize_port(port: Union[str, int]) -> str:
+    """canonical port string: parsed to int, rendered without zero-padding
+
+    keeps host:080 and host:80 as one canonical key. raises ValueError on a
+    non-integer port rather than silently keying it as-is.
+    """
+    return str(int(port))
+
+
 @dataclass
 class Proxy:
     """a single proxy endpoint with optional auth"""
@@ -38,19 +47,27 @@ class Proxy:
 
         the host is lowercased (hostnames are case-insensitive per DNS, so
         PROXY.example.com and proxy.example.com are the same host and collapse to one
-        key); port and credentials are kept verbatim. the password is included in
-        full (two proxies differing only by password are distinct slots). auth-less
-        proxies collapse to host:port with no trailing colons.
+        key); credentials are kept verbatim. the port is normalized (leading zeros
+        stripped) so host:080 and host:80 collapse to one key. the password is
+        included in full (two proxies differing only by password are distinct
+        slots). auth-less proxies collapse to host:port with no trailing colons.
         """
         host = self.host.lower()
+        port = normalize_port(self.port)
         if self.has_auth:
-            return f"{host}:{self.port}:{self.user}:{self.password}"
-        return f"{host}:{self.port}"
+            return f"{host}:{port}:{self.user}:{self.password}"
+        return f"{host}:{port}"
 
     def url(self, scheme: str = SCHEME_HTTP) -> str:
-        """render as a url, embedding auth when present"""
+        """render as a url, embedding auth when present
+
+        credentials are percent-encoded so reserved chars (/ # ? @ :) in a user or
+        password produce a valid url; this mirrors the unquote() on the parse side.
+        """
         if self.has_auth:
-            return f"{scheme}://{self.user}:{self.password}@{self.host}:{self.port}"
+            user = quote(str(self.user), safe="")
+            password = quote(str(self.password), safe="")
+            return f"{scheme}://{user}:{password}@{self.host}:{self.port}"
         return f"{scheme}://{self.host}:{self.port}"
 
     def aiohttp(self) -> Dict[str, str]:
@@ -133,8 +150,11 @@ def _proxy_from_dict(spec: Dict[str, str]) -> Proxy:
     """normalize an aiohttp / camoufox / socks5 dict into a Proxy"""
     if "server" in spec:
         proxy = _proxy_from_url(spec["server"])
-        user = spec.get("username")
-        password = spec.get("password")
+        # prefer explicit dict auth; otherwise fall back to auth embedded in the server
+        # URL (http://user:pass@host:port) so it isn't silently dropped — which would
+        # key the proxy auth-less and collide with a genuinely auth-less one
+        user = spec.get("username") or proxy.user
+        password = spec.get("password") or proxy.password
         if user and password:
             return Proxy(proxy.host, proxy.port, user, password)
         return Proxy(proxy.host, proxy.port)