Compare commits

...
4 Commits
Author SHA1 Message Date
dsql 1120083e45 Merge pull request 'build_product_details: skip non-finite prices instead of raising OverflowError' (#26) from development into main 2026-10-10 03:24:57 +00:00
dsqlandClaude Code Opus 5.5 6ea3187b33 build_product_details: skip non-finite prices instead of raising OverflowError
PR #25 guarded calculate_priority but missed the same calculation here.
On 2026-10-09 06:25:50Z the consumer hit a product with an infinite price;
runner.py:201

    percentage = int(((original_price - current_price) / original_price) * 100)

raised `OverflowError: cannot convert float infinity to integer`. The
Discord leg dropped that one alert, but the API leg's caller does not
catch OverflowError, so the exception escaped and the consumer logged
CRITICAL "api deal loop stopped" -- no deals reached the API from then on
(nothing sent at all after 22:00Z), while the bot itself stayed "alive".

build_product_details now returns None (with a warning, like the
existing missing-price skip) when either price is a non-finite float, on
BOTH paths: the MSRP-less path never computes a percentage and would
otherwise carry inf into the Product and the API payload. A finite pair
whose ratio still overflows (a subnormal original price) is skipped too.
Ints and other types keep their existing behaviour.

Tests (tests/test_build_product_details.py): five new cases fail on the
old code -- OverflowError, ValueError for nan, and the MSRP-less case
building a Product with inf -- and all 25 pass now. Ruff findings on the
two files are unchanged from main (5, all pre-existing).

Rebuild every service that installs monitor-utils with --no-cache to pick
this up (it is pulled unpinned from main).

Co-Authored-By: Claude Code Opus 5.5 (ansible-ops, auto-repair) <noreply@anthropic.com>
Signed-off-by: disqualifier <dev@disqualifier.me>
2026-10-09 23:22:36 -04:00
dsql 7d514a5401 Merge pull request 'calculate_priority: resolve non-finite prices to 0 instead of crashing the monitor' (#25) from development into main 2026-10-08 22:48:57 +00:00
dsqlandClaude Code Opus 5.5 f51aed2c9f calculate_priority: resolve non-finite prices to 0 instead of crashing the monitor
tpv-academysports died on 2026-10-08 06:27 UTC and stayed down: six
identical crashes in 8 minutes (each ~2 min after start), then docker's
on-failure:5 gave up. Every one was

    database.py, in calculate_priority
        percentage = int(((original_price - current_price) / original_price) * 100)
    OverflowError: cannot convert float infinity to integer

A monitor builds prices with float() on the retailer's raw field, and
float() accepts "Infinity" / "1e999", so a non-finite price reaches here.
The `not original_price` guard catches None and 0, not inf or nan, and the
exception escapes add_products_to_database -> run_monitor_cycle and kills
the whole process -- one bad product takes the monitor down, and it
crashes again on every restart while that product is listed. Every
monitor that uses this lib has the same exposure.

calculate_priority now returns 0 (the existing "broken prices" bucket)
for a non-finite current/original price, and for a finite pair whose
ratio still overflows (a subnormal original price). Tests cover both;
they fail on the old code (OverflowError, plus ValueError for nan) and
pass now (tests/test_alert_priority.py: 10 passed).

Not changed: the monitor-side float() parsing, which could reject these
prices before they are stored -- worth a look by the monitor owners.

Co-Authored-By: Claude Code Opus 5.5 (ansible-ops, auto-repair) <noreply@anthropic.com>
Signed-off-by: disqualifier <dev@disqualifier.me>
2026-10-08 18:46:59 -04:00
4 changed files with 99 additions and 8 deletions

No files matched your search

+16 -5
View File
@@ -7,6 +7,7 @@ alert references for new products and price changes.
"""
import logging
import math
from dataclasses import asdict
from datetime import datetime, timezone
from functools import cache
@@ -318,10 +319,13 @@ def calculate_priority(document: dict) -> int:
Always recalculated from current and original price so the priority
reflects the prices at alert time rather than any stale stored
percentage. Missing prices, zero original prices, and price increases
resolve to priority 0. A product with no MSRP has no percentage to rank
on and takes NO_MSRP_PRIORITY instead, since a real 0 would leave it
permanently behind every other alert in the claim query's sort. A new
percentage. Missing prices, zero original prices, non-finite prices, a
discount too large to represent, and price increases resolve to priority
0, so one product's bad pricing can never raise out of the whole cycle (a
retailer field like "Infinity" survives float() and used to crash the
monitor with an OverflowError). A product with no MSRP has no percentage
to rank on and takes NO_MSRP_PRIORITY instead, since a real 0 would leave
it permanently behind every other alert in the claim query's sort. A new
listing is floored at NEW_LISTING_PRIORITY for the same reason: it may
carry any discount at all, and its percentage rank still wins when higher.
@@ -337,10 +341,17 @@ def calculate_priority(document: dict) -> int:
if current_price is None or not original_price:
return 0
if not (math.isfinite(current_price) and math.isfinite(original_price)):
return 0
if not document.get("hasMsrp", True):
return NO_MSRP_PRIORITY
percentage = int(((original_price - current_price) / original_price) * 100)
discount = ((original_price - current_price) / original_price) * 100
if not math.isfinite(discount):
return 0
percentage = int(discount)
priority = min(max(percentage // 10, 0), 10)
if document.get("newListing"):
return max(priority, NEW_LISTING_PRIORITY)
+28 -3
View File
@@ -1,5 +1,6 @@
import asyncio
import logging
import math
from collections.abc import Awaitable, Callable, Iterable
from contextlib import asynccontextmanager, nullcontext
@@ -160,7 +161,10 @@ def build_product_details(
) -> Product | None:
"""Build a Product from a Mongo document if it passes discount thresholds.
Skips documents with a missing or zero original price. Passing requires
Skips documents with a missing or zero original price, a non-finite
price, or a discount too large to represent: a retailer field like
"Infinity" survives float(), and the percentage math used to raise
OverflowError out of every caller. Passing requires
meeting min_percentage, or min_price_drop when provided. The caller may
bypass both thresholds (new listings send at any discount for monitors
that opt in); which alerts qualify is the caller's decision, not read
@@ -173,7 +177,7 @@ def build_product_details(
min_percentage: Minimum discount percentage to pass.
min_price_drop: Optional minimum absolute price drop to pass.
bypass_thresholds: Build the product even when it fails both
thresholds. The missing-price skip still applies.
thresholds. The missing- and non-finite-price skips still apply.
Returns:
The built Product, or None when skipped or below thresholds.
@@ -191,6 +195,19 @@ def build_product_details(
)
return
# Skipped on BOTH paths, not just the percentage one: an MSRP-less product
# would otherwise carry the infinite price into its payload. Only floats can
# be non-finite, so ints and other types keep their existing behaviour.
if any(
isinstance(price, float) and not math.isfinite(price)
for price in (current_price, original_price)
):
LOGGER.warning(
"non-finite price, skipping",
extra={"product_id": product_id},
)
return
percentage = 0
# Defaulted, not indexed: documents written before the field existed have no
# hasMsrp at all, and stale ones that drop out of a monitor's feed are never
@@ -198,7 +215,15 @@ def build_product_details(
# pre-existing threshold path rather than silently routing them as MSRP-less.
has_msrp = product.get("hasMsrp", True)
if has_msrp:
percentage = int(((original_price - current_price) / original_price) * 100)
discount = ((original_price - current_price) / original_price) * 100
if not math.isfinite(discount):
LOGGER.warning(
"discount not representable, skipping",
extra={"product_id": product_id},
)
return
percentage = int(discount)
price_drop = original_price - current_price
# A percentage code compounds onto the sale price, so an item can clear the
# threshold on the combined discount even when the base discount would not.
+17
View File
@@ -50,6 +50,23 @@ def test_broken_prices_still_outrank_nothing():
assert calculate_priority(document(None, 100.0, hasMsrp=False)) == 0
def test_non_finite_prices_resolve_to_zero_instead_of_raising():
"""float("Infinity") from a retailer field once crashed the whole monitor."""
inf, nan = float("inf"), float("nan")
assert calculate_priority(document(50.0, inf)) == 0
assert calculate_priority(document(inf, 100.0)) == 0
assert calculate_priority(document(-inf, 100.0)) == 0
assert calculate_priority(document(nan, 100.0)) == 0
assert calculate_priority(document(50.0, nan)) == 0
assert calculate_priority(document(50.0, inf, newListing=True)) == 0
def test_a_discount_too_large_to_represent_resolves_to_zero():
"""Finite prices can still overflow the ratio (a subnormal original)."""
assert calculate_priority(document(-1e308, 5e-324)) == 0
assert calculate_priority(document(1e308, 5e-324)) == 0
def test_new_listing_is_floored_not_buried():
"""A low-discount new listing sends regardless, so it must not sort last."""
assert (
+38
View File
@@ -203,3 +203,41 @@ def test_msrpless_doc_with_prices_builds_unthresholded():
product = build_product_details(doc, min_percentage=40)
assert product is not None
assert product.pricing.has_msrp is False
# ------------------------------------------------------------ non-finite prices
def test_infinite_price_skips_instead_of_raising():
"""Regression (2026-10-09 06:25Z): float("Infinity") from a retailer field
raised OverflowError here and stopped the consumer's API deal loop."""
inf = float("inf")
assert build_product_details(_doc(originalPrice=inf), min_percentage=40) is None
assert build_product_details(_doc(currentPrice=inf), min_percentage=40) is None
assert build_product_details(_doc(currentPrice=-inf), min_percentage=40) is None
def test_nan_price_skips_instead_of_raising():
nan = float("nan")
assert build_product_details(_doc(currentPrice=nan), min_percentage=40) is None
assert build_product_details(_doc(originalPrice=nan), min_percentage=40) is None
def test_msrpless_infinite_price_skips_rather_than_reaching_the_payload():
"""Without the skip the MSRP-less path would carry inf into the Product."""
doc = _doc(hasMsrp=False, currentPrice=float("inf"))
assert build_product_details(doc, min_percentage=40) is None
def test_bypass_does_not_rescue_a_non_finite_price():
doc = _doc(originalPrice=float("inf"))
assert build_product_details(doc, min_percentage=40, bypass_thresholds=True) is None
def test_unrepresentable_discount_skips_instead_of_raising():
"""Finite prices can still overflow the ratio (a subnormal original)."""
doc = _doc(currentPrice=-1e308, originalPrice=5e-324)
assert build_product_details(doc, min_percentage=40) is None