rethink-public/dpy_cache
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix: reject duplicate-name uploads and rebuild fresh Files per send

Browse Source 
cache() silently lost a FileRef when a list input had two sources
resolving to the same name (two paths with the same basename, or two
discord.File objects with the same .filename): both files were
uploaded (real cost) but the merge step kept only the last, orphaning
the first. _normalize now rejects a duplicate name with DPYCacheError
before any upload happens.

_to_file returned the same discord.File object for a discord.File
source, only mutating .filename. Reusing one File under two keys
swapped filename/URL between refs; reusing a BytesIO-backed File
re-sent 0 bytes; reusing a path-backed File after discord.py's own
close() raised ValueError. _to_file now builds a genuinely distinct
discord.File per send, re-opening a path-backed source or seeking and
copying a caller-supplied stream; an unrebuildable opaque stream now
raises DPYCacheError instead of corrupting or crashing a later send.

Also: lookup() accepts a digit-only message-id string (previously only
bare int fetched directly, a numeric string routed into the jump-url
parser and raised); dropped a redundant discord.Forbidden member from
an except tuple (Forbidden already subclasses HTTPException); tightened
_parse_jump_url's return type off a dead Optional and added `from err`
to its raise.

Signed-off-by: disqualifier <dev@disqualifier.me>
This commit is contained in:
dsql
2026-07-03 15:32:28 -04:00
parent 2e93a0bcee
commit 248d6ebb6b
3 changed files with 105 additions and 29 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+32 -6
View File
@@ -9,7 +9,7 @@ bites you.
## Install
```
dpy_cache @ git+ssh://git@git.rethinkstudios.io/rethink-public/dpy_cache.git@v0.1.0
dpy_cache @ git+ssh://git@git.rethinkstudios.io/rethink-public/dpy_cache.git@v0.1.1
```
## Usage
@@ -46,6 +46,9 @@ refs = await cache.lookup(result.message_url) # jump url, message id, or disco
refs["avatar.png"].url
```
`lookup()` accepts a message id as an `int` or as a digit-only `str` (the snowflake shape
JSON/DBs store), e.g. `cache.lookup("123456789012345678")`.
## What you inject
A resolved Discord channel object (`discord.abc.Messageable` that supports `.send` and
@@ -58,7 +61,16 @@ first use.
`cache_one` / `cache` accept, per file: `bytes`, a filesystem path (`str` or
`os.PathLike`), or a `discord.File`. In a **list** input, raw `bytes` have no filename and
raise `ValueError` — pass a dict `{name: bytes}` or a `discord.File` to name bytes.
raise `ValueError` — pass a dict `{name: bytes}` or a `discord.File` to name bytes. A
**list** input whose sources resolve to the same name (two paths with the same basename,
two `discord.File`s with the same `.filename`, ...) raises `DPYCacheError` before any
upload — a dict input can't collide since dict keys are already unique.
A `discord.File` source is safe to reuse across multiple names/batches: `cache()` always
builds a genuinely fresh `discord.File` per send (re-opening a path-backed source, or
seeking + copying a caller-supplied stream) rather than reusing the object you passed in.
An opaque stream that can't be safely re-read (already closed, not seekable) raises
`DPYCacheError` instead of silently sending stale or empty data.
## Why `resolve()` exists
@@ -80,12 +92,26 @@ API. In short:
- `lookup(message) -> dict[str, FileRef]`
**Fail-loud.** Nothing is swallowed. Lib-specific invariants raise `DPYCacheError`
(attachment-count mismatch after a send, `cache_url` non-200, a `lookup` jump-URL pointing
at a different channel than the injected one). Unnamed `bytes` in a list raise `ValueError`.
Raw Discord errors (`Forbidden` / `HTTPException` / `NotFound`) propagate **unwrapped** so
you can still branch on Discord's own types.
(attachment-count mismatch after a send, a duplicate filename in a list input to `cache()`,
`cache_url` non-200, a `lookup` jump-URL pointing at a different channel than the injected
one, a `discord.File` source that can't be safely rebuilt for a repeat send). Unnamed
`bytes` in a list raise `ValueError`. Raw Discord errors (`Forbidden` / `HTTPException` /
`NotFound`) propagate **unwrapped** so you can still branch on Discord's own types.
## Versioning
Tagged `vX.Y.Z`; pin a tag in your install line. Targets `discord.py>=2.0` (not
`discord.py-self`).
### v0.1.1
- `cache()` now rejects a duplicate filename in a **list** input with `DPYCacheError`
before any upload, instead of silently uploading both files and losing the first
`FileRef` at merge time.
- `_to_file` now builds a genuinely fresh `discord.File` per send for `discord.File`
sources (previously returned the same object, which could swap filename/URL when one
`File` was reused under two names, or re-send 0 bytes / raise `ValueError` on a spent
stream).
- `lookup()` now accepts a digit-only message-id string in addition to `int`.
- Dropped the redundant `discord.Forbidden` member from an `except` tuple (it already
subclasses `discord.HTTPException`); no behavior change.
pyproject.toml
+1 -1
View File
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
[project]
name = "dpy_cache"
version = "0.1.0"
version = "0.1.1"
description = "Persist images/files to a Discord storage channel and get durable, re-resolvable CDN references — config-free, injectable, installable."
requires-python = ">=3.10"
dependencies = [
src/dpy_cache/dpy_cache.py
+72 -22
View File
@@ -40,10 +40,11 @@ error on first use.
error contract (fail-loud)
--------------------------
nothing is swallowed. lib-specific invariants raise :class:`DPYCacheError` (attachment-count
mismatch after a send, ``cache_url`` non-200, a ``lookup`` jump-url pointing at a different
channel than the injected one). unnamed ``bytes`` in a list input raises ``ValueError``. raw
discord errors (``Forbidden`` / ``HTTPException`` / ``NotFound``) propagate UNWRAPPED so
callers can still branch on discord's types.
mismatch after a send, a duplicate filename in a list input to ``cache()``, ``cache_url``
non-200, a ``lookup`` jump-url pointing at a different channel than the injected one, a
``discord.File`` source that can't be safely rebuilt for a repeat send). unnamed ``bytes`` in
a list input raises ``ValueError``. raw discord errors (``Forbidden`` / ``HTTPException`` /
``NotFound``) propagate UNWRAPPED so callers can still branch on discord's types.
"""
from __future__ import annotations
@@ -64,8 +65,9 @@ FileContent = Union[bytes, str, "os.PathLike[str]", discord.File]
class DPYCacheError(Exception):
"""raised for dpy_cache's own invariants (count mismatch, remote fetch failure,
channel mismatch); raw discord errors propagate unwrapped instead"""
"""raised for dpy_cache's own invariants (count mismatch, duplicate filename, remote
fetch failure, channel mismatch, an unrebuildable discord.File source); raw discord
errors propagate unwrapped instead"""
@dataclass
@@ -119,8 +121,10 @@ class DPYCache:
content: "str | None" = None,
) -> CacheResult:
"""upload files to the storage channel (batching over 10/message) and return refs
keyed by original filename; raises on unnamed bytes, count mismatch, or a discord
send error"""
keyed by original filename; raises ValueError on unnamed bytes in a list input,
DPYCacheError on a duplicate filename in a list input (checked before any upload, so
a collision costs zero requests) or an attachment-count mismatch after a send, and
propagates discord's own Forbidden/HTTPException unwrapped on a send failure"""
items = self._normalize(files)
merged: dict[str, FileRef] = {}
jump_urls: list[str] = []
@@ -130,7 +134,7 @@ class DPYCache:
discord_files = [self._to_file(name, source) for name, source in group]
try:
message = await self._channel.send(content=content, files=discord_files)
except (discord.Forbidden, discord.HTTPException):
except discord.HTTPException:
log.error("dpy_cache: send of %d file(s) to the storage channel failed", len(group))
raise
@@ -194,7 +198,8 @@ class DPYCache:
)
async def lookup(self, message: "str | int | discord.Message") -> dict[str, FileRef]:
"""reverse lookup: a jump-url, message id, or Message -> {filename: FileRef}; raises
"""reverse lookup: a jump-url, message id (``int`` or a digit-only ``str``, matching
the snowflake shape stored in JSON/DBs), or Message -> {filename: FileRef}; raises
(NotFound) if the message is gone, never an empty map for a missing message"""
resolved = await self._resolve_message(message)
return {
@@ -204,16 +209,18 @@ class DPYCache:
def _normalize(self, files: "dict[str, FileContent] | list[FileContent]") -> "list[tuple[str, FileContent]]":
"""normalize dict/list input to an ordered list of (name, source); raises ValueError
on unnamed bytes in a list"""
on unnamed bytes in a list, or DPYCacheError on a duplicate name in a list (dict keys
can't collide by construction)"""
if isinstance(files, dict):
return list(files.items())
items: "list[tuple[str, FileContent]]" = []
seen: "set[str]" = set()
for source in files:
if isinstance(source, discord.File):
items.append((source.filename, source))
name = source.filename
elif isinstance(source, (str, os.PathLike)):
items.append((_basename(source), source))
name = _basename(source)
elif isinstance(source, (bytes, bytearray)):
raise ValueError(
"raw bytes in a list input have no filename; pass a dict {name: bytes} "
@@ -221,29 +228,72 @@ class DPYCache:
)
else:
raise ValueError(f"unsupported file content type in list: {type(source).__name__}")
if name in seen:
raise DPYCacheError(
f"duplicate filename {name!r} in list input; every list source must resolve "
f"to a unique name, or the later upload silently orphans the earlier one"
)
seen.add(name)
items.append((name, source))
return items
def _to_file(self, name: str, source: FileContent) -> discord.File:
"""build a fresh discord.File for ``source`` named ``name``; a fresh File is built
per send so the source survives multi-batch and retries"""
"""build a genuinely fresh discord.File for ``source`` named ``name``, distinct from
any File previously handed to a send; a fresh File is built per send so the source
survives multi-batch, retries, and reuse of the same source under multiple names"""
if isinstance(source, discord.File):
source.filename = name
return source
return self._fresh_file_from_source(source, name)
if isinstance(source, (bytes, bytearray)):
return discord.File(io.BytesIO(bytes(source)), filename=name)
if isinstance(source, (str, os.PathLike)):
return discord.File(source, filename=name)
raise ValueError(f"unsupported file content type: {type(source).__name__}")
def _fresh_file_from_source(self, source: discord.File, name: str) -> discord.File:
"""rebuild a distinct discord.File from an existing one without mutating or
re-sending ``source`` itself; re-opens the path for an owner-opened File, or
seeks to the original position and copies the bytes for a caller-supplied stream;
raises DPYCacheError if the underlying stream can't be safely re-read"""
if source._owner and hasattr(source.fp, "name"):
return discord.File(
source.fp.name,
filename=name,
spoiler=source.spoiler,
description=source.description,
)
if source.fp.closed:
raise DPYCacheError(
f"discord.File source for {name!r} is already closed and can't be safely "
f"reused; pass the raw bytes or path instead of a spent discord.File"
)
try:
source.fp.seek(source._original_pos)
data = source.fp.read()
source.fp.seek(source._original_pos)
except (OSError, ValueError) as err:
raise DPYCacheError(
f"discord.File source for {name!r} could not be re-read (opaque or "
f"already-consumed stream); pass the raw bytes instead of a discord.File"
) from err
return discord.File(
io.BytesIO(data),
filename=name,
spoiler=source.spoiler,
description=source.description,
)
async def _resolve_message(self, message: "str | int | discord.Message") -> discord.Message:
"""turn a jump-url / id / Message into a fetched Message on the injected channel"""
"""turn a jump-url / id / digit-only id string / Message into a fetched Message on
the injected channel"""
if isinstance(message, discord.Message):
return message
if isinstance(message, int):
return await self._channel.fetch_message(message)
if isinstance(message, str):
if message.isdigit():
return await self._channel.fetch_message(int(message))
channel_id, message_id = self._parse_jump_url(message)
if channel_id is not None and channel_id != self._channel.id:
if channel_id != self._channel.id:
raise DPYCacheError(
f"lookup jump-url targets channel {channel_id} but the injected channel is "
f"{self._channel.id}; this lib holds no client and can only fetch its own channel"
@@ -252,13 +302,13 @@ class DPYCache:
raise ValueError(f"unsupported lookup input type: {type(message).__name__}")
@staticmethod
def _parse_jump_url(url: str) -> "tuple[int | None, int]":
def _parse_jump_url(url: str) -> "tuple[int, int]":
"""parse (channel_id, message_id) out of a discord jump url; raises DPYCacheError on
a malformed url"""
parts = url.rstrip("/").split("/")
try:
message_id = int(parts[-1])
channel_id = int(parts[-2])
except (IndexError, ValueError):
raise DPYCacheError(f"lookup: not a valid discord jump url: {url}")
except (IndexError, ValueError) as err:
raise DPYCacheError(f"lookup: not a valid discord jump url: {url}") from err
return channel_id, message_id