fix: provider() unicode-digit crash + phantom() short-value false-mask (v0.3.2)

provider() used str.isdigit(), which accepts unicode digit lookalikes (e.g. superscript
²) that int() can't parse, crashing on digit-only-looking input; _digits() now filters
on isdecimal()+isascii() so only real ASCII digits pass through. phantom() revealed the
entire value for len <= 10 while still looking masked (first6+last4 fully covers or
double-covers short strings); those now fully mask instead. Also compresses several
essay-length docstrings (retry, masking, paths) to a line plus the real nuance, with
zero behavior change, and bumps the README install pin.

Signed-off-by: disqualifier <dev@disqualifier.me>

.gitignore

@@ -1,5 +1,5 @@
 # claude
-CLAUDE.md
+.claude/
 
 # python
 __pycache__/

README.md

@@ -5,20 +5,23 @@ Small sync helpers shared across projects. Base is stdlib only — **no dependen
 - `timing` — unix-timestamp deltas + timezone-aware datetime conversions
 - `paths` — nested dict/list access by dotted path
 - `masking` — display masking for cards / cvv / tokens
+- `retry` — exponential-backoff retry, sync (`retry`) and async (`aretry`)
 - `addr` — ip/address tooling: pure stdlib ip utils in base, async geo lookups
   behind the `commons[addr]` extra
 
 ## Install
 
 ```
-commons @ git+ssh://git@git.rethinkstudios.io/rethink-public/commons.git@v0.1.0
+commons @ git+ssh://git@git.rethinkstudios.io/rethink-public/commons.git@v0.3.2
 # async address/geo lookups (fetch_ip / ip_location / fetch_location) need the extra:
-commons[addr] @ git+ssh://git@git.rethinkstudios.io/rethink-public/commons.git@v0.1.0
+commons[addr] @ git+ssh://git@git.rethinkstudios.io/rethink-public/commons.git@v0.3.2
 ```
 
 The base install pulls **nothing** (stdlib). Only `commons[addr]` adds `aiohttp`, and
 only for the geo lookups — the pure `commons.addr.ip` utilities ship in base.
 
+Drop the `@v0.3.2` suffix from the line above to install the latest unpinned.
+
 ## timing
 
 Unix ints stay the storable value; datetimes are produced on demand in whatever
@@ -96,22 +99,78 @@ deep_get(data, "items.1.id")             # "b"  (numeric segment indexes a list)
 deep_get(data, "in.nope.here", "DEF")    # "DEF" (missing -> default, no raise)
 
 deep_set({}, "a.b.c", 9)                 # {"a": {"b": {"c": 9}}}
+deep_set(data, "items.1.id", "B")        # updates the list element in place
+deep_get(data, "items.1.id")             # "B"  (get/set share the same grammar)
+deep_set(data, "items.9.id", "X")        # raises IndexError (out of range, no
+                                          # silent mis-store)
 ```
 
+`deep_set` mirrors `deep_get`'s list indexing: a numeric segment over an existing
+list/tuple updates that element in place rather than replacing the list with a dict.
+An out-of-range numeric segment raises `IndexError` instead of silently corrupting
+the structure.
+
 ## masking
 
 Display helpers only — they format a value for showing; they are **not a security
 control** (the underlying value is unchanged and still needs proper handling).
 
 ```python
-from commons import credit, cvv, phantom, provider
+from commons import credit, cvv, phantom, provider, mask_url, mask_proxy
 
 credit("4111 1111 1111 1234")   # "•••• •••• •••• 1234"
 cvv("123")                       # "•••"
 phantom("abcdef1234567890")      # "abcdef...7890"
+phantom("1234567890")            # "••••••••••"  (len <= 10 fully masks, never a false reveal)
 provider("4111111111111111")     # "VISA"  (VISA/MC/AMEX/UPAY/DISC/JCB/DNRS/UNKW)
+provider("4111²111111111234")    # "VISA"  (unicode digit lookalikes ignored, never raises)
+
+# redact credentials in a connection string before logging it
+mask_url("https://u:pw@api.x/v2?apiKey=SECRET&ip=8.8.8.8")
+#   -> "https://api.x/v2?apiKey=***&ip=8.8.8.8"   (userinfo dropped, secret query masked)
+mask_url("redis://:pw@127.0.0.1:6379/0")     # -> "redis://127.0.0.1:6379/0"
+mask_proxy("1.2.3.4:8080:user:supersecret")  # -> "1.2.3.4:8080:user:****"
+mask_proxy("1.2.3.4:8080")                    # -> "1.2.3.4:8080"  (no auth, untouched)
 ```
 
+`mask_url` strips `user:pass@` userinfo and replaces the values of sensitive query
+params (`apiKey`, `token`, `password`, `secret`, …; override via `keys=`) with `***`.
+`mask_proxy` bullets the password of a `host:port:user:password` spec. Non-URL /
+non-conforming input is returned unchanged.
+
+## retry
+
+Exponential-backoff retry, sync (`retry`) and async (`aretry`). Each works as a **call
+form** or a **decorator**, with the same kwargs. After the attempts are exhausted the
+**last exception is re-raised** — it never swallows or returns a default.
+
+```python
+from commons import retry, aretry
+
+# call form
+rows = retry(lambda: read_db(), attempts=5, on=(IOError,))
+data = await aretry(lambda: fetch(url), attempts=3, backoff=0.5, on=(TimeoutError,))
+
+# decorator form (same kwargs)
+@aretry(attempts=4, backoff=0.5, factor=2.0, on=(ConnectionError,))
+async def pull():
+    ...
+```
+
+Knobs: `attempts` (total tries), `backoff` / `factor` / `max_backoff` (delay is
+`min(backoff * factor**n, max_backoff)`), `jitter` (full jitter, on by default),
+`on=` (tuple of retryable exception types), and `give_up=lambda exc: ...` to stop early
+on a non-retryable error (e.g. a 400 vs a 429):
+
+```python
+# retry 429/5xx but give up immediately on a 4xx
+await aretry(send, attempts=4, on=(HTTPError,),
+             give_up=lambda e: 400 <= e.status < 500 and e.status != 429)
+```
+
+Each retry is logged (emit-only). `sleep=` and `rand=` are injectable for deterministic
+tests (no real waits).
+
 ## addr
 
 IP/address tooling, exposed as a submodule. The pure `ip` utilities ship in the base

pyproject.toml

@@ -4,8 +4,8 @@ build-backend = "hatchling.build"
 
 [project]
 name = "commons"
-version = "0.1.0"
-description = "small stdlib-only sync helpers: time/timezone deltas, dotted-path dict access, display masking, and ip/address tooling"
+version = "0.3.2"
+description = "small stdlib-based sync helpers: time/timezone deltas, dotted-path dict access, display masking, ip/address tooling, and retry/backoff"
 requires-python = ">=3.10"
 dependencies = []
 

src/commons/__init__.py

@@ -2,10 +2,15 @@
 
 timing:  unix-timestamp deltas + timezone-aware datetime conversions, bare
          functions and a configurable Clock.
-paths:   nested dict/list access by dotted path (deep_get / deep_set).
-masking: display masking for cards / cvv / tokens (cosmetic, not a security control).
+paths:   nested dict/list access by dotted path (deep_get / deep_set); deep_set
+         mirrors deep_get's list indexing, so a get/set round-trip on the same
+         path never destroys a list.
+masking: display masking for cards / cvv / tokens, plus url/proxy credential
+         redaction for logging (cosmetic, not a security control).
 addr:    ip/address tooling — pure stdlib ip utils (`commons.addr.ip`) in base,
          async geo lookups (`commons.addr.geo`) behind the `commons[addr]` extra.
+retry:   exponential-backoff retry, sync (`retry`) and async (`aretry`), call or
+         decorator form; re-raises the last exception (fail loud), never swallows.
 
 base is stdlib only, no dependencies (the addr geo lookups add aiohttp via the
 extra). to toggle the timing test mode for bare calls, set it on the module —
@@ -14,8 +19,9 @@ for instance-scoped control. addr is exposed as a submodule (`from commons impor
 addr`); its ip helpers live under `commons.addr.ip` to keep top-level uncluttered.
 """
 from . import addr, masking, paths, timing
-from .masking import credit, cvv, phantom, provider
+from .masking import credit, cvv, mask_proxy, mask_url, phantom, provider
 from .paths import deep_get, deep_set
+from .retry import aretry, retry
 from .timing import (
     UTC,
     Clock,
@@ -56,6 +62,10 @@ __all__ = [
     "cvv",
     "phantom",
     "provider",
+    "mask_url",
+    "mask_proxy",
+    "retry",
+    "aretry",
 ]
 
-__version__ = "0.1.0"
+__version__ = "0.3.2"

src/commons/addr/geo.py

@@ -12,6 +12,7 @@ security: the only secret is geo.ipify's `api_key`, which is a REQUIRED keyword
 `ip_location` — the caller injects it. nothing is hardcoded here.
 """
 import logging
+import unicodedata
 from typing import Optional
 from urllib.parse import urlencode
 
@@ -51,12 +52,26 @@ def _reverse_url(lat: float, lon: float) -> str:
 
 def _parse_ipify(data: dict) -> Optional[str]:
     """pull the ip string out of an ipify response"""
+    if not isinstance(data, dict):
+        return None
     ip = data.get("ip")
     return ip or None
 
 
+def _state_slug(state) -> str:
+    """lowercase ascii-folded state slug with underscores (e.g. 'New York' -> 'new_york')
+
+    coerces to str first so a malformed non-string `state` doesn't raise TypeError out
+    of unicodedata.normalize and break the 'None on any parse failure' contract.
+    """
+    folded = unicodedata.normalize("NFKD", str(state)).encode("ascii", "ignore").decode("ascii")
+    return folded.lower().replace(" ", "_")
+
+
 def _parse_reverse(data: dict) -> Optional[dict]:
     """parse a nominatim reverse response into {country: iso2 lower, state: slug|None}"""
+    if not isinstance(data, dict):
+        return None
     address = data.get("address")
     if not isinstance(address, dict):
         return None
@@ -66,7 +81,7 @@ def _parse_reverse(data: dict) -> Optional[dict]:
     state = address.get("state")
     return {
         "country": str(country).lower(),
-        "state": state.lower().replace(" ", "-") if state else None,
+        "state": _state_slug(state) if state else None,
     }
 
 
@@ -77,10 +92,11 @@ async def _get_json(
     if not _HAVE_AIOHTTP:
         raise RuntimeError(_MISSING)
     owns = session is None
+    request_timeout = aiohttp.ClientTimeout(total=timeout)
     if owns:
-        session = aiohttp.ClientSession(timeout=aiohttp.ClientTimeout(total=timeout))
+        session = aiohttp.ClientSession(timeout=request_timeout)
     try:
-        async with session.get(url, headers=headers) as resp:
+        async with session.get(url, headers=headers, timeout=request_timeout) as resp:
             if resp.status != 200:
                 log.warning("address lookup %s -> %s", url, resp.status)
                 return None

src/commons/masking.py

@@ -3,12 +3,22 @@
 these are DISPLAY helpers only — they format a value for showing in a UI or log
 (e.g. "•••• •••• •••• 1234"). they are not a security control: the underlying
 value is unchanged and still needs proper handling (encryption at rest, etc.).
+
+card helpers: `credit`, `cvv`, `provider`. token: `phantom`. url/proxy redaction
+for logging connection strings: `mask_url`, `mask_proxy`.
 """
+from urllib.parse import parse_qsl, urlencode, urlsplit, urlunsplit
+
+SENSITIVE_QUERY_KEYS = frozenset(
+    {"apikey", "api_key", "key", "token", "access_token", "refresh_token",
+     "auth", "password", "passwd", "pwd", "pass", "secret", "client_secret",
+     "sig", "signature", "session"}
+)
 
 
 def _digits(value: str) -> str:
-    """keep only the digit characters of a string"""
-    return "".join(c for c in value if c.isdigit())
+    """keep only ascii-decimal characters of a string (excludes unicode digit lookalikes)"""
+    return "".join(c for c in value if c.isdecimal() and c.isascii())
 
 
 def credit(card_number: str) -> str:
@@ -23,7 +33,13 @@ def cvv(value: str) -> str:
 
 
 def phantom(value: str) -> str:
-    """show a long token as first-six...last-four (e.g. a hash or id)"""
+    """show a long token as first-six...last-four (e.g. a hash or id)
+
+    values of length <= 10 fully mask instead — first6+last4 would otherwise
+    reveal (or double-reveal) every character while still looking masked.
+    """
+    if len(value) <= 10:
+        return "•" * len(value)
     return f"{value[:6]}...{value[-4:]}"
 
 
@@ -51,3 +67,43 @@ def provider(card_number: str) -> str:
     if n.startswith(("30", "36", "38", "39")):
         return "DNRS"
     return "UNKW"
+
+
+def mask_url(url: str, *, keys: "frozenset[str] | None" = None) -> str:
+    """redact credentials in a url for logging: drop userinfo, ``***`` sensitive query values
+
+    param-name matching is case-insensitive; ``keys`` overrides the default sensitive-name
+    set. non-url or unparseable input is returned unchanged.
+    """
+    sensitive = SENSITIVE_QUERY_KEYS if keys is None else frozenset(k.lower() for k in keys)
+    try:
+        parts = urlsplit(url)
+    except ValueError:
+        return url
+    if not parts.scheme and not parts.netloc:
+        return url
+    netloc = parts.netloc
+    if "@" in netloc:
+        netloc = netloc.rsplit("@", 1)[1]
+    query = parts.query
+    if query:
+        pairs = parse_qsl(query, keep_blank_values=True)
+        query = urlencode(
+            [(k, "***" if k.lower() in sensitive else v) for k, v in pairs],
+            quote_via=lambda s, *_: s,
+        )
+    return urlunsplit((parts.scheme, netloc, parts.path, query, parts.fragment))
+
+
+def mask_proxy(spec: str) -> str:
+    """redact the password of a ``host:port:user:password`` proxy string to ``:****``
+
+    a plain ``host:port`` (no auth) and any other non-conforming input pass through unchanged.
+    """
+    parts = spec.split(":")
+    if len(parts) == 2:
+        return spec
+    if len(parts) == 4:
+        host, port, user, _ = parts
+        return f"{host}:{port}:{user}:****"
+    return spec

src/commons/paths.py

@@ -2,7 +2,10 @@
 
 `deep_get(data, "in.this.old.notation")` walks dicts (and lists, when a segment is
 a number) and returns a default instead of raising on a missing/!wrong path.
-`deep_set` writes a nested value, creating intermediate dicts.
+`deep_set` writes a nested value, creating intermediate dicts and indexing into
+existing lists on numeric segments (mirroring deep_get), so a get/set round-trip
+on the same path never corrupts a list. an out-of-range numeric segment raises
+IndexError rather than silently mis-storing.
 """
 from typing import Any
 
@@ -33,18 +36,36 @@ def deep_get(data: Any, path: str, default: Any = None, *, sep: str = ".") -> An
 
 
 def deep_set(data: dict, path: str, value: Any, *, sep: str = ".") -> dict:
-    """set a nested value by dotted path, creating intermediate dicts
+    """set a nested value by dotted path, creating intermediate dicts; returns data for chaining
 
-    returns the same top-level dict for chaining. overwrites any non-dict value
-    sitting where an intermediate dict is needed.
+    mirrors deep_get's list indexing (a numeric segment updates an existing list/tuple
+    element rather than corrupting it); unlike deep_get, an out-of-range index raises
+    IndexError instead of silently mis-storing, since there's no safe default to fall back to.
     """
     segments = path.split(sep)
     cur = data
     for seg in segments[:-1]:
+        if isinstance(cur, list):
+            idx = int(seg)
+            if not -len(cur) <= idx < len(cur):
+                raise IndexError(f"deep_set: index {seg!r} out of range for list of length {len(cur)}")
+            nxt = cur[idx]
+            if not isinstance(nxt, (dict, list)):
+                nxt = {}
+                cur[idx] = nxt
+            cur = nxt
+        else:
             nxt = cur.get(seg)
-        if not isinstance(nxt, dict):
+            if not isinstance(nxt, (dict, list)):
                 nxt = {}
                 cur[seg] = nxt
             cur = nxt
-    cur[segments[-1]] = value
+    last = segments[-1]
+    if isinstance(cur, list):
+        idx = int(last)
+        if not -len(cur) <= idx < len(cur):
+            raise IndexError(f"deep_set: index {last!r} out of range for list of length {len(cur)}")
+        cur[idx] = value
+    else:
+        cur[last] = value
     return data

src/commons/retry.py

@@ -0,0 +1,159 @@
+"""retry with exponential backoff — sync and async, one backoff engine.
+
+de-duplicates retry logic that was written divergently across several libs (HTTP
+429/5xx caps, proxy burn/rotate caps, IMAP reconnects). both a call form and a
+decorator form share one implementation per flavor; the backoff schedule is a pure
+generator so it tests without real sleeps.
+
+    from commons import retry, aretry
+
+    # call form
+    rows = retry(lambda: db_read(), attempts=5, on=(IOError,))
+    data = await aretry(lambda: fetch(url), attempts=3, on=(TimeoutError,))
+
+    # decorator form (same kwargs)
+    @aretry(attempts=4, backoff=0.5, on=(ConnectionError,))
+    async def pull():
+        ...
+
+after `attempts` are exhausted the LAST exception is re-raised (fail loud, never
+swallowed). `on` narrows which exceptions retry; `give_up(exc) -> bool` stops early
+on a non-retryable error (e.g. a 400 vs a 429). each retry is logged (emit-only),
+never printed.
+"""
+
+import asyncio
+import functools
+import logging
+import random
+import time
+from typing import Callable, Iterable, Optional, Tuple, Type
+
+log = logging.getLogger(__name__)
+
+ExcTypes = Tuple[Type[BaseException], ...]
+
+
+def _delays(attempts: int, backoff: float, factor: float, max_backoff: float):
+    """yield the wait before each retry: min(backoff * factor**n, max_backoff)
+
+    yields `attempts - 1` delays (one before each retry after the first try). the
+    raw, un-jittered schedule — jitter is applied at call time so the schedule stays
+    pure and testable.
+    """
+    for n in range(max(0, attempts - 1)):
+        yield min(backoff * (factor ** n), max_backoff)
+
+
+def _jittered(delay: float, jitter: bool, rand: Callable[[], float]) -> float:
+    """apply full jitter to a delay when enabled: uniform(0, delay)"""
+    if not jitter or delay <= 0:
+        return delay
+    return rand() * delay
+
+
+def _as_types(on: Iterable[Type[BaseException]]) -> ExcTypes:
+    """coerce the `on` argument into a tuple of exception types"""
+    if isinstance(on, type):
+        return (on,)
+    return tuple(on)
+
+
+def retry(
+    fn: Optional[Callable] = None,
+    *,
+    attempts: int = 3,
+    backoff: float = 0.5,
+    factor: float = 2.0,
+    max_backoff: float = 30.0,
+    jitter: bool = True,
+    on: Iterable[Type[BaseException]] = (Exception,),
+    give_up: Optional[Callable[[BaseException], bool]] = None,
+    sleep: Callable[[float], None] = time.sleep,
+    rand: Callable[[], float] = random.random,
+):
+    """retry a sync callable with exponential backoff; call form (`retry(fn, ...)`) or decorator
+
+    `attempts` is floored at 1 so the callable always runs at least once.
+    """
+    types = _as_types(on)
+    attempts = max(1, attempts)
+
+    def run(target: Callable, args, kwargs):
+        delays = list(_delays(attempts, backoff, factor, max_backoff))
+        last_index = attempts - 1
+        for index in range(attempts):
+            try:
+                return target(*args, **kwargs)
+            except types as exc:
+                if give_up is not None and give_up(exc):
+                    raise
+                if index == last_index:
+                    raise
+                wait = _jittered(delays[index], jitter, rand)
+                log.warning(
+                    "retry %d/%d after %s: %s",
+                    index + 1, attempts, type(exc).__name__, exc,
+                )
+                if wait > 0:
+                    sleep(wait)
+
+    def decorator(target: Callable) -> Callable:
+        @functools.wraps(target)
+        def wrapper(*args, **kwargs):
+            return run(target, args, kwargs)
+        return wrapper
+
+    if fn is not None:
+        return run(fn, (), {})
+    return decorator
+
+
+def aretry(
+    fn: Optional[Callable] = None,
+    *,
+    attempts: int = 3,
+    backoff: float = 0.5,
+    factor: float = 2.0,
+    max_backoff: float = 30.0,
+    jitter: bool = True,
+    on: Iterable[Type[BaseException]] = (Exception,),
+    give_up: Optional[Callable[[BaseException], bool]] = None,
+    sleep: Callable[[float], "asyncio.Future"] = asyncio.sleep,
+    rand: Callable[[], float] = random.random,
+):
+    """async twin of `retry`; call form (`await aretry(coro_fn, ...)`) or decorator, same semantics
+
+    `attempts` is floored at 1 so the callable always runs at least once.
+    """
+    types = _as_types(on)
+    attempts = max(1, attempts)
+
+    async def run(target: Callable, args, kwargs):
+        delays = list(_delays(attempts, backoff, factor, max_backoff))
+        last_index = attempts - 1
+        for index in range(attempts):
+            try:
+                return await target(*args, **kwargs)
+            except types as exc:
+                if give_up is not None and give_up(exc):
+                    raise
+                if index == last_index:
+                    raise
+                wait = _jittered(delays[index], jitter, rand)
+                log.warning(
+                    "retry %d/%d after %s: %s",
+                    index + 1, attempts, type(exc).__name__, exc,
+                )
+                if wait > 0:
+                    await sleep(wait)
+
+    def decorator(target: Callable) -> Callable:
+        @functools.wraps(target)
+        async def wrapper(*args, **kwargs):
+            return await run(target, args, kwargs)
+        return wrapper
+
+    if fn is not None:
+        return run(fn, (), {})
+    return decorator