Commit Graph
125 Commits
Author SHA1 Message Date
dsqlandClaude Code Opus 5.5 6ea3187b33 build_product_details: skip non-finite prices instead of raising OverflowError
PR #25 guarded calculate_priority but missed the same calculation here.
On 2026-10-09 06:25:50Z the consumer hit a product with an infinite price;
runner.py:201

    percentage = int(((original_price - current_price) / original_price) * 100)

raised `OverflowError: cannot convert float infinity to integer`. The
Discord leg dropped that one alert, but the API leg's caller does not
catch OverflowError, so the exception escaped and the consumer logged
CRITICAL "api deal loop stopped" -- no deals reached the API from then on
(nothing sent at all after 22:00Z), while the bot itself stayed "alive".

build_product_details now returns None (with a warning, like the
existing missing-price skip) when either price is a non-finite float, on
BOTH paths: the MSRP-less path never computes a percentage and would
otherwise carry inf into the Product and the API payload. A finite pair
whose ratio still overflows (a subnormal original price) is skipped too.
Ints and other types keep their existing behaviour.

Tests (tests/test_build_product_details.py): five new cases fail on the
old code -- OverflowError, ValueError for nan, and the MSRP-less case
building a Product with inf -- and all 25 pass now. Ruff findings on the
two files are unchanged from main (5, all pre-existing).

Rebuild every service that installs monitor-utils with --no-cache to pick
this up (it is pulled unpinned from main).

Co-Authored-By: Claude Code Opus 5.5 (ansible-ops, auto-repair) <noreply@anthropic.com>
Signed-off-by: disqualifier <dev@disqualifier.me>
2026-10-09 23:22:36 -04:00
dsqlandClaude Code Opus 5.5 f51aed2c9f calculate_priority: resolve non-finite prices to 0 instead of crashing the monitor
tpv-academysports died on 2026-10-08 06:27 UTC and stayed down: six
identical crashes in 8 minutes (each ~2 min after start), then docker's
on-failure:5 gave up. Every one was

    database.py, in calculate_priority
        percentage = int(((original_price - current_price) / original_price) * 100)
    OverflowError: cannot convert float infinity to integer

A monitor builds prices with float() on the retailer's raw field, and
float() accepts "Infinity" / "1e999", so a non-finite price reaches here.
The `not original_price` guard catches None and 0, not inf or nan, and the
exception escapes add_products_to_database -> run_monitor_cycle and kills
the whole process -- one bad product takes the monitor down, and it
crashes again on every restart while that product is listed. Every
monitor that uses this lib has the same exposure.

calculate_priority now returns 0 (the existing "broken prices" bucket)
for a non-finite current/original price, and for a finite pair whose
ratio still overflows (a subnormal original price). Tests cover both;
they fail on the old code (OverflowError, plus ValueError for nan) and
pass now (tests/test_alert_priority.py: 10 passed).

Not changed: the monitor-side float() parsing, which could reject these
prices before they are stored -- worth a look by the monitor owners.

Co-Authored-By: Claude Code Opus 5.5 (ansible-ops, auto-repair) <noreply@anthropic.com>
Signed-off-by: disqualifier <dev@disqualifier.me>
2026-10-08 18:46:59 -04:00
TPV 6720592685 small cleanup of comments/added optional impersonate into run_bounded_cycle for monitors like bhphoto that need different clients than what we support/dont need to rotate 2026-09-29 18:55:52 -07:00
TPV 4563cdf920 updated tls profiles 2026-09-26 21:11:37 -07:00
TPV b9fbb64364 added get_response_json function to return json from aioweb 2026-09-25 20:00:19 -07:00
TPV ae637a134f rename monitor_name parameters to monitor
Applies to setup_monitor_logging, init_monitor, setup_config, get_config,
add_products_to_database and build_alert. Breaking for callers passing
monitor_name by keyword; every monitor repo is updated alongside.
2026-09-25 19:02:09 -07:00
TPV 401258f52a renamed monitor_name to monitor 2026-09-25 17:29:08 -07:00
TPV 9a34ae5ca6 added asins to build_product_details 2026-09-10 06:38:44 -07:00
TPV 0275f94500 added price drops to alerts/fixed race condition where late claimed docs got their tags wiped and sent to the api without tags 2026-09-09 09:52:15 -07:00
TPV 03cc3a35cb added asins for woot listings 2026-09-08 20:14:49 -07:00
TPV 93013c5d55 updated the db name for dsql mongo 2026-08-30 17:59:05 -07:00
TPV 39a6c40154 updated the mongo_url to dsqls new url/small type hint fixes 2026-08-30 17:54:40 -07:00
TPV 1218ba0b3c added optional mongo client for already configured monitors 2026-08-21 08:21:57 -07:00
TPV b599d34e1e added optional mongo url 2026-08-21 07:52:21 -07:00
TPV b071ac35bf bug fix to avoid swallowing exceptions/failed pages inside fetch_all_pages/small refactor 2026-08-12 17:45:59 -07:00
TPV 321546f9b9 removed retry_failed_pages from docstrings 2026-08-08 18:59:52 -07:00
TPV 0b462aef06 removed failed page retrying 2026-08-08 13:49:35 -07:00
TPV dee090edaf added no msrp prirority so they don't starve in queue 2026-08-04 13:53:32 -07:00
TPV 6c309b1e62 use .get for has_msrp 2026-08-04 11:49:19 -07:00
TPV 1f5224a92c small typo fix 2026-08-04 10:07:31 -07:00
TPV 61c4887c38 added has_msrp for no msrp based products(new lists) 2026-08-04 10:04:32 -07:00
TPV 2e6e7ebc69 added passing headers 2026-07-30 14:29:58 -07:00
TPV 8903be2685 abstracted out bounded cycles 2026-07-30 14:13:26 -07:00
TPV aa6ed4cb06 added pagination fan out pagination abstraction 2026-07-30 11:52:36 -07:00
TPV 3f823b3a6e added price history to build_product_details/models 2026-07-28 11:29:15 -07:00
TPV 3573551957 removed past_prices 2026-07-28 10:38:33 -07:00
TPV 8a1c1893b7 added bounded close session for curl_cffi/past_prices to pricing model 2026-07-28 09:15:38 -07:00
TPV c29032a765 cleanup 2026-07-27 12:02:56 -07:00
TPV 1e53904cc6 added max_clients arg to get_default_session/cycle timeout seconds 2026-07-27 12:02:41 -07:00
TPV 99e68b1fa9 added first_error for asyncio task groups 2026-07-27 10:27:57 -07:00
TPV b6cd692258 added first_error for asyncio task groups 2026-07-27 10:23:07 -07:00
TPV 763f40988f added cycle timeout seconds 2026-07-27 08:49:59 -07:00
TPV d4f4830ff2 added motor/pinned the curl version 2026-07-27 08:48:04 -07:00
TPV 4c1033057a added combined discount for consumer so codes can be calculated on the fly for any monitor 2026-07-26 17:55:13 -07:00
TPV e9632470fd small refactor 2026-07-26 14:17:42 -07:00
TPV 5effb835a7 Merge pull request 'removed timeout from send_request and added promo options to product metadata' (#6) from aioweb-migration into main
Reviewed-on: #6
2026-07-26 21:09:51 +00:00
TPV b876c3d7b3 removed timeout from send_request and added promo options to product metadata 2026-07-26 14:09:05 -07:00
TPV 9af7e7adcf Merge pull request 'added optional semphore' (#5) from aioweb-migration into main
Reviewed-on: #5
2026-07-21 03:05:50 +00:00
TPV 4b63a2d39c added optional semphore 2026-07-20 19:27:47 -07:00
TPV 3cbe5beb43 Merge pull request 'removed user agent setting since curl_impersonate does it' (#4) from aioweb-migration into main
Reviewed-on: #4
2026-07-19 16:36:29 +00:00
TPV 0cfcca01bb removed user agent setting since curl_impersonate does it 2026-07-19 09:36:03 -07:00
TPV 65ddb40bc3 Merge pull request 'Aioweb migration' (#3) from aioweb-migration into main
Reviewed-on: #3
2026-07-19 16:29:00 +00:00
TPV 17219507bf added requiring call sites to set own content-type for anything other than json(aioweb doesn't set content-type properly) 2026-07-19 09:28:22 -07:00
TPV db947a6354 raise default session timeout to 30s 2026-07-19 09:04:40 -07:00
TPV 638d8fb359 raise default session timeout to 30s 2026-07-19 08:55:13 -07:00
TPV 666046bdd1 Merge pull request 'added ability to set default user agent' (#2) from aioweb-migration into main
Reviewed-on: #2
2026-07-19 04:33:03 +00:00
TPV 20ad4c6535 added ability to set default user agent 2026-07-18 19:31:02 -07:00
TPV e9559f3248 Merge pull request 'Aioweb migration' (#1) from aioweb-migration into main
Reviewed-on: #1
2026-07-19 01:26:35 +00:00
TPV d307be60b0 added default headers for get_default_session 2026-07-18 11:45:46 -07:00
TPV bb24d0c13a named collection as a required arg in run_monitor_cycle 2026-07-18 11:40:35 -07:00