Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
0421fc734d | ||
|
|
c6dd2669e4 | ||
|
|
59c0258aff | ||
|
|
d89a91e2d9 | ||
|
|
fbd94d6c81 | ||
|
|
aedc8f3b84 | ||
|
|
0a68b65937 | ||
|
|
30f0e44097 | ||
|
|
9da851629d | ||
|
|
ebd808f326 | ||
|
|
e20368287d | ||
|
|
1d55efbcdd | ||
|
|
fe8f2da480 | ||
|
|
9ad3458cf9 |
+1
-1
@@ -1,5 +1,5 @@
|
|||||||
# claude
|
# claude
|
||||||
CLAUDE.md
|
.claude/
|
||||||
|
|
||||||
# python
|
# python
|
||||||
__pycache__/
|
__pycache__/
|
||||||
|
|||||||
@@ -8,31 +8,39 @@ helpers for the common paths, with a raw escape hatch for everything else.
|
|||||||
`requirements.txt`:
|
`requirements.txt`:
|
||||||
|
|
||||||
```
|
```
|
||||||
mongo @ git+ssh://git@git.rethinkstudios.io/rethink-public/mongo.git@v0.1.2
|
mongo @ git+ssh://git@git.rethinkstudios.io/rethink-public/mongo.git@v1.0.0
|
||||||
```
|
```
|
||||||
|
|
||||||
Direct:
|
Direct:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
pip install "mongo @ git+ssh://git@git.rethinkstudios.io/rethink-public/mongo.git@v0.1.2"
|
pip install "mongo @ git+ssh://git@git.rethinkstudios.io/rethink-public/mongo.git@v1.0.0"
|
||||||
```
|
```
|
||||||
|
|
||||||
Requires `motor` and `pymongo` (pulled transitively).
|
Requires `motor` and `pymongo` (pulled transitively).
|
||||||
|
|
||||||
Drop the `@v0.1.2` suffix from the line above to install the latest unpinned.
|
Drop the `@v1.0.0` suffix from the line above to install the latest unpinned.
|
||||||
|
|
||||||
## Usage
|
## Usage
|
||||||
|
|
||||||
**Object (preferred)** — one client per process:
|
**Object (preferred)** — one client per process:
|
||||||
|
|
||||||
```python
|
```python
|
||||||
from mongo import Mongo
|
from mongo import MongoDB
|
||||||
|
|
||||||
db = Mongo(conn_string, database) # attach as bot.db / app.db
|
db = MongoDB(conn_string, database) # attach as bot.db / app.db
|
||||||
|
await db.connect() # optional: ping to fail-early on a bad URI
|
||||||
users = await db.get_documents("users", {"active": True})
|
users = await db.get_documents("users", {"active": True})
|
||||||
db.close() # on shutdown (sync)
|
db.close() # on shutdown (sync)
|
||||||
|
|
||||||
|
# or with guaranteed cleanup:
|
||||||
|
async with MongoDB(conn_string, database) as db:
|
||||||
|
await db.get_documents("users", {"active": True})
|
||||||
```
|
```
|
||||||
|
|
||||||
|
The class is `MongoDB`; **`Mongo` remains a back-compat alias** (`Mongo = MongoDB`), so
|
||||||
|
existing `Mongo(...)` call sites keep working unchanged.
|
||||||
|
|
||||||
**Module proxy (back-compat)** — arm once, then call bare:
|
**Module proxy (back-compat)** — arm once, then call bare:
|
||||||
|
|
||||||
```python
|
```python
|
||||||
@@ -44,10 +52,26 @@ users = await mongo.get_documents("users", {"active": True})
|
|||||||
Both styles share one client. The proxy exists so legacy call sites keep working
|
Both styles share one client. The proxy exists so legacy call sites keep working
|
||||||
after a one-line `init()`; new code should use the object.
|
after a one-line `init()`; new code should use the object.
|
||||||
|
|
||||||
|
## Naming consistency with the datastore trio
|
||||||
|
|
||||||
|
mongo predates the `redis`/`psql`/`mysql` trio; this version makes it surface-consistent
|
||||||
|
**without breaking anything** (all additive, every old name preserved):
|
||||||
|
|
||||||
|
- class is **`MongoDB`** (with `Mongo` kept as an alias)
|
||||||
|
- **`connect()`** + **`async with`** like the trio (motor connects lazily, so `connect()`
|
||||||
|
just pings to validate early)
|
||||||
|
- **`exists()`** aliases `check_document_exists()`; **`delete()`** aliases
|
||||||
|
`delete_document()` — old names still work, the trio-consistent names are now available
|
||||||
|
|
||||||
|
The one deliberate difference that remains: **mongo swallows** (see below) where the trio
|
||||||
|
is **fail-loud**. That's intentional — flipping it would break existing consumers' branch-
|
||||||
|
on-result control flow.
|
||||||
|
|
||||||
## Error contract
|
## Error contract
|
||||||
|
|
||||||
- **Wrapped methods** log-and-swallow exceptions and return a safe default
|
- **Wrapped methods** log-and-swallow exceptions and return a safe default
|
||||||
(`False` / `[]` / `{}` / `0` / `None`). Branch on the result.
|
(`False` / `[]` / `{}` / `0` / `None`). Branch on the result. (`connect()` is the
|
||||||
|
exception — it raises on a bad connection so you can fail-early.)
|
||||||
- **`db.collection(name)`** (or `db[name]`) returns the raw motor collection:
|
- **`db.collection(name)`** (or `db[name]`) returns the raw motor collection:
|
||||||
full driver surface, no swallowing, **raises**. Use it for anything not wrapped
|
full driver surface, no swallowing, **raises**. Use it for anything not wrapped
|
||||||
(`find_one_and_*` beyond what's exposed, change streams, complex bulk ops).
|
(`find_one_and_*` beyond what's exposed, change streams, complex bulk ops).
|
||||||
@@ -63,6 +87,9 @@ are included.
|
|||||||
|
|
||||||
- `from mongo import func` won't see the proxy (resolved at import, before `init`).
|
- `from mongo import func` won't see the proxy (resolved at import, before `init`).
|
||||||
Use `import mongo` then `mongo.func(...)`.
|
Use `import mongo` then `mongo.func(...)`.
|
||||||
|
- `create_collection(collection, index=...)` seeds **one** index (the param is `index`,
|
||||||
|
renamed from `indexes` in v0.1.6). A key-spec like `[("a",1),("b",1)]` builds one
|
||||||
|
compound index, not two — call `create_index` per index for multiples.
|
||||||
- `find_one_and_update` returns the **after** image by default (`return_after=True`).
|
- `find_one_and_update` returns the **after** image by default (`return_after=True`).
|
||||||
- `bulk_write` takes pymongo ops the caller builds:
|
- `bulk_write` takes pymongo ops the caller builds:
|
||||||
```python
|
```python
|
||||||
@@ -73,4 +100,4 @@ are included.
|
|||||||
|
|
||||||
## Versioning
|
## Versioning
|
||||||
|
|
||||||
Releases are tagged `vX.Y.Z`. The install line above pins a release; drop the `@vX.Y.Z` suffix to install the latest unpinned. Pin deliberately for reproducible installs.
|
Releases are tagged `vX.Y.Z`. The install line above pins a release; drop the `@vX.Y.Z` suffix to install the latest unpinned. Pin deliberately for reproducible installs.
|
||||||
|
|||||||
+1
-1
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
|
|||||||
|
|
||||||
[project]
|
[project]
|
||||||
name = "mongo"
|
name = "mongo"
|
||||||
version = "0.1.2"
|
version = "1.1.0"
|
||||||
description = "async mongodb wrapper over motor with a raw escape hatch"
|
description = "async mongodb wrapper over motor with a raw escape hatch"
|
||||||
requires-python = ">=3.10"
|
requires-python = ">=3.10"
|
||||||
dependencies = [
|
dependencies = [
|
||||||
|
|||||||
+12
-12
@@ -1,18 +1,18 @@
|
|||||||
from .mongo import Mongo, init, instance
|
from importlib.metadata import PackageNotFoundError, version
|
||||||
|
|
||||||
__all__ = ["Mongo", "init", "instance"]
|
from .mongo import MongoDB, Mongo, init, instance
|
||||||
|
|
||||||
|
__all__ = ["MongoDB", "Mongo", "init", "instance"]
|
||||||
|
|
||||||
|
try:
|
||||||
|
__version__ = version("mongo")
|
||||||
|
except PackageNotFoundError:
|
||||||
|
__version__ = "0.0.0+unknown"
|
||||||
|
|
||||||
|
|
||||||
def __getattr__(name: str):
|
def __getattr__(name: str):
|
||||||
"""proxy bare package attribute access to the default instance (PEP 562)
|
"""proxy bare attribute access to the default instance (PEP 562); needs
|
||||||
|
`import mongo`, not `from mongo import func` (resolved before init)"""
|
||||||
lets `import mongo; await mongo.get_documents(...)` work after init().
|
if not name.startswith("_") and hasattr(MongoDB, name):
|
||||||
`from mongo import func` still won't see this (resolved before init).
|
|
||||||
note: the bare-proxy raw escape hatch is `mongo.collection(name)`, not
|
|
||||||
`mongo[name]` — module-level subscripting isn't a thing in python, so the proxy
|
|
||||||
can only forward named attributes. on a Mongo instance both `db[name]` and
|
|
||||||
`db.collection(name)` work.
|
|
||||||
"""
|
|
||||||
if not name.startswith("_") and hasattr(Mongo, name):
|
|
||||||
return getattr(instance(), name)
|
return getattr(instance(), name)
|
||||||
raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
|
raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
|
||||||
|
|||||||
+146
-96
@@ -1,43 +1,62 @@
|
|||||||
"""
|
"""
|
||||||
async mongodb wrapper over motor
|
async mongodb wrapper over motor, with a raw escape hatch for anything not wrapped
|
||||||
|
|
||||||
object (preferred), one client per process:
|
from mongo import MongoDB
|
||||||
from mongo import Mongo
|
db = MongoDB(conn_string, database)
|
||||||
bot.db = Mongo(conn_string, database)
|
await db.get_documents("users", {"active": True})
|
||||||
await bot.db.get_documents("users", {"active": True})
|
|
||||||
bot.db.close() # on shutdown (sync)
|
|
||||||
|
|
||||||
module proxy (back-compat), arm once then call bare:
|
`Mongo` is a back-compat alias of `MongoDB`. A module-proxy path also exists
|
||||||
import mongo # not `from mongo import ...`
|
(`mongo.init(...)` then bare `mongo.func(...)`) - see README for both usage patterns.
|
||||||
mongo.init(conn_string, database)
|
|
||||||
await mongo.get_documents("users", {"active": True})
|
|
||||||
|
|
||||||
errors:
|
errors: wrapped methods log and swallow, returning a safe default (False / [] / {} / 0 /
|
||||||
wrapped methods log and swallow, returning a safe default
|
None). .collection(name) / db[name] return the raw motor collection: full driver surface,
|
||||||
(False / [] / {} / 0 / None). .collection(name) / db[name] return
|
raises, nothing swallowed. mongo SWALLOWS by design, unlike the fail-loud redis/psql/mysql
|
||||||
the raw motor collection: full driver surface, raises, nothing swallowed.
|
trio - kept so existing consumers' branch-on-result control flow doesn't break.
|
||||||
|
|
||||||
notes:
|
|
||||||
- the proxy needs `import mongo`; `from mongo import func` resolves before init
|
|
||||||
- find_one_and_update returns the after-image by default
|
|
||||||
- bulk_write takes caller-built pymongo ops (UpdateOne/DeleteOne/...)
|
|
||||||
"""
|
"""
|
||||||
|
|
||||||
import logging
|
import logging
|
||||||
from typing import Any, List, Optional
|
from typing import Any, List, Optional
|
||||||
|
|
||||||
from pymongo import ReturnDocument
|
from pymongo import ReturnDocument
|
||||||
from motor.motor_asyncio import AsyncIOMotorClient, AsyncIOMotorCollection
|
from pymongo.errors import BulkWriteError
|
||||||
|
from motor.motor_asyncio import AsyncIOMotorClient, AsyncIOMotorCollection, AsyncIOMotorDatabase
|
||||||
|
|
||||||
log = logging.getLogger(__name__)
|
log = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
class Mongo:
|
class MongoDB:
|
||||||
"""async mongodb wrapper; one client per process, attach to bot as bot.db"""
|
"""async mongodb wrapper; one client per process, attach to bot as bot.db"""
|
||||||
|
|
||||||
def __init__(self, connection_string: str, database: str):
|
def __init__(self, connection_string: str, database: str):
|
||||||
self._client = AsyncIOMotorClient(connection_string)
|
self._client = AsyncIOMotorClient(connection_string)
|
||||||
self._db = self._client[database]
|
try:
|
||||||
|
self._db = self._client[database]
|
||||||
|
except BaseException:
|
||||||
|
self._client.close()
|
||||||
|
raise
|
||||||
|
|
||||||
|
async def connect(self) -> "MongoDB":
|
||||||
|
"""validate the connection with a ping and return self; raises, unlike the
|
||||||
|
swallowing wrapped methods
|
||||||
|
|
||||||
|
an instance is single-use: pymongo 4.x `close()` is irreversible and
|
||||||
|
connect() doesn't rebuild the client. construct a fresh MongoDB instead of
|
||||||
|
reusing one after close()/`async with` exit.
|
||||||
|
"""
|
||||||
|
await self._client.admin.command("ping")
|
||||||
|
return self
|
||||||
|
|
||||||
|
async def __aenter__(self) -> "MongoDB":
|
||||||
|
try:
|
||||||
|
return await self.connect()
|
||||||
|
except BaseException:
|
||||||
|
# __aexit__ isn't called when __aenter__ raises; close here so the
|
||||||
|
# client built in __init__ doesn't leak
|
||||||
|
self.close()
|
||||||
|
raise
|
||||||
|
|
||||||
|
async def __aexit__(self, exc_type, exc, tb) -> None:
|
||||||
|
self.close()
|
||||||
|
|
||||||
def __getitem__(self, collection: str) -> AsyncIOMotorCollection:
|
def __getitem__(self, collection: str) -> AsyncIOMotorCollection:
|
||||||
"""raw collection access via subscript: bot.db['users'].aggregate(...)"""
|
"""raw collection access via subscript: bot.db['users'].aggregate(...)"""
|
||||||
@@ -48,7 +67,7 @@ class Mongo:
|
|||||||
return self._db[name]
|
return self._db[name]
|
||||||
|
|
||||||
@property
|
@property
|
||||||
def database(self):
|
def database(self) -> AsyncIOMotorDatabase:
|
||||||
"""raw motor database handle"""
|
"""raw motor database handle"""
|
||||||
return self._db
|
return self._db
|
||||||
|
|
||||||
@@ -62,32 +81,36 @@ class Mongo:
|
|||||||
return False
|
return False
|
||||||
|
|
||||||
def close(self) -> None:
|
def close(self) -> None:
|
||||||
"""close the client pool on shutdown (sync — motor's close() is synchronous)"""
|
"""close the client pool on shutdown (sync - motor's close() is synchronous)"""
|
||||||
self._client.close()
|
self._client.close()
|
||||||
|
|
||||||
# -------------------------------------------------------------------------
|
# -------------------------------------------------------------------------
|
||||||
# collection / index management
|
# collection / index management
|
||||||
|
|
||||||
async def create_collection(self, collection: str, indexes=None) -> bool:
|
async def create_collection(self, collection: str, index=None) -> bool:
|
||||||
"""create a collection, optionally seeding an index"""
|
"""create a collection, optionally seeding ONE index
|
||||||
|
|
||||||
|
`index` is a single key-spec (e.g. `[("a", 1), ("b", 1)]` builds one compound
|
||||||
|
index, not two) - call create_index per index for multiples
|
||||||
|
"""
|
||||||
try:
|
try:
|
||||||
await self._db.create_collection(collection)
|
await self._db.create_collection(collection)
|
||||||
if indexes:
|
if index:
|
||||||
await self._db[collection].create_index(indexes)
|
await self._db[collection].create_index(index)
|
||||||
log.info(f"created indexes for {collection}")
|
log.info("created index for %s", collection)
|
||||||
return True
|
return True
|
||||||
except Exception:
|
except Exception:
|
||||||
log.exception(f"db.create_collection() for {collection}")
|
log.exception("db.create_collection() for %s", collection)
|
||||||
return False
|
return False
|
||||||
|
|
||||||
async def create_index(self, collection: str, index, **kwargs) -> bool:
|
async def create_index(self, collection: str, index, **kwargs) -> bool:
|
||||||
"""create an index; kwargs pass through (unique=True, name=..., etc)"""
|
"""create an index; kwargs pass through (unique=True, name=..., etc)"""
|
||||||
try:
|
try:
|
||||||
await self._db[collection].create_index(index, **kwargs)
|
await self._db[collection].create_index(index, **kwargs)
|
||||||
log.info(f"created index for {collection}")
|
log.info("created index for %s", collection)
|
||||||
return True
|
return True
|
||||||
except Exception:
|
except Exception:
|
||||||
log.exception(f"db.create_index() for {collection}")
|
log.exception("db.create_index() for %s", collection)
|
||||||
return False
|
return False
|
||||||
|
|
||||||
async def drop_collection(self, collection: str) -> bool:
|
async def drop_collection(self, collection: str) -> bool:
|
||||||
@@ -96,7 +119,7 @@ class Mongo:
|
|||||||
await self._db.drop_collection(collection)
|
await self._db.drop_collection(collection)
|
||||||
return True
|
return True
|
||||||
except Exception:
|
except Exception:
|
||||||
log.exception(f"db.drop_collection() for {collection}")
|
log.exception("db.drop_collection() for %s", collection)
|
||||||
return False
|
return False
|
||||||
|
|
||||||
async def check_collection_exists(self, collection: str) -> bool:
|
async def check_collection_exists(self, collection: str) -> bool:
|
||||||
@@ -104,7 +127,7 @@ class Mongo:
|
|||||||
try:
|
try:
|
||||||
return collection in await self._db.list_collection_names()
|
return collection in await self._db.list_collection_names()
|
||||||
except Exception:
|
except Exception:
|
||||||
log.exception(f"db.check_collection_exists() for {collection}")
|
log.exception("db.check_collection_exists() for %s", collection)
|
||||||
return False
|
return False
|
||||||
|
|
||||||
async def list_collections(self) -> List[str]:
|
async def list_collections(self) -> List[str]:
|
||||||
@@ -124,18 +147,27 @@ class Mongo:
|
|||||||
await self._db[collection].insert_one(document)
|
await self._db[collection].insert_one(document)
|
||||||
return True
|
return True
|
||||||
except Exception:
|
except Exception:
|
||||||
log.exception(f"db.create_document() on {collection}")
|
log.exception("db.create_document() on %s", collection)
|
||||||
return False
|
return False
|
||||||
|
|
||||||
async def create_documents(
|
async def create_documents(
|
||||||
self, collection: str, documents: List[dict], ordered: bool = True
|
self, collection: str, documents: List[dict], ordered: bool = True
|
||||||
) -> int:
|
) -> int:
|
||||||
"""insert many documents, returning the inserted count"""
|
"""insert many documents, returning the inserted count
|
||||||
|
|
||||||
|
on a partial failure (e.g. ordered=False with one duplicate key) the driver
|
||||||
|
raises BulkWriteError AFTER inserting the rest; count comes from
|
||||||
|
`e.details['nInserted']` so a caller retrying on a wrongly-reported 0 doesn't
|
||||||
|
re-insert documents that already landed.
|
||||||
|
"""
|
||||||
try:
|
try:
|
||||||
result = await self._db[collection].insert_many(documents, ordered=ordered)
|
result = await self._db[collection].insert_many(documents, ordered=ordered)
|
||||||
return len(result.inserted_ids)
|
return len(result.inserted_ids)
|
||||||
|
except BulkWriteError as e:
|
||||||
|
log.exception("db.create_documents() on %s", collection)
|
||||||
|
return e.details.get("nInserted", 0)
|
||||||
except Exception:
|
except Exception:
|
||||||
log.exception(f"db.create_documents() on {collection}")
|
log.exception("db.create_documents() on %s", collection)
|
||||||
return 0
|
return 0
|
||||||
|
|
||||||
# -------------------------------------------------------------------------
|
# -------------------------------------------------------------------------
|
||||||
@@ -148,7 +180,7 @@ class Mongo:
|
|||||||
try:
|
try:
|
||||||
return await self._db[collection].find_one(target, fields)
|
return await self._db[collection].find_one(target, fields)
|
||||||
except Exception:
|
except Exception:
|
||||||
log.exception(f"db.get_document() on {collection}")
|
log.exception("db.get_document() on %s", collection)
|
||||||
return None
|
return None
|
||||||
|
|
||||||
async def get_documents(
|
async def get_documents(
|
||||||
@@ -159,7 +191,7 @@ class Mongo:
|
|||||||
cursor = self._db[collection].find(target, fields)
|
cursor = self._db[collection].find(target, fields)
|
||||||
return [doc async for doc in cursor]
|
return [doc async for doc in cursor]
|
||||||
except Exception:
|
except Exception:
|
||||||
log.exception(f"db.get_documents() on {collection}")
|
log.exception("db.get_documents() on %s", collection)
|
||||||
return []
|
return []
|
||||||
|
|
||||||
async def get_documents_sorted(
|
async def get_documents_sorted(
|
||||||
@@ -177,7 +209,7 @@ class Mongo:
|
|||||||
)
|
)
|
||||||
return [doc async for doc in cursor]
|
return [doc async for doc in cursor]
|
||||||
except Exception:
|
except Exception:
|
||||||
log.exception(f"db.get_documents_sorted() on {collection}")
|
log.exception("db.get_documents_sorted() on %s", collection)
|
||||||
return []
|
return []
|
||||||
|
|
||||||
async def get_document_hashmap(self, collection: str, target: dict, key: str) -> dict:
|
async def get_document_hashmap(self, collection: str, target: dict, key: str) -> dict:
|
||||||
@@ -190,7 +222,7 @@ class Mongo:
|
|||||||
cursor = self._db[collection].find(target)
|
cursor = self._db[collection].find(target)
|
||||||
return {doc[key]: doc async for doc in cursor if key in doc}
|
return {doc[key]: doc async for doc in cursor if key in doc}
|
||||||
except Exception:
|
except Exception:
|
||||||
log.exception(f"db.get_document_hashmap() on {collection}")
|
log.exception("db.get_document_hashmap() on %s", collection)
|
||||||
return {}
|
return {}
|
||||||
|
|
||||||
async def get_document_fields(
|
async def get_document_fields(
|
||||||
@@ -205,7 +237,7 @@ class Mongo:
|
|||||||
cursor = self._db[collection].find(target, fields)
|
cursor = self._db[collection].find(target, fields)
|
||||||
return [doc[key] async for doc in cursor if key in doc]
|
return [doc[key] async for doc in cursor if key in doc]
|
||||||
except Exception:
|
except Exception:
|
||||||
log.exception(f"db.get_document_fields() on {collection}")
|
log.exception("db.get_document_fields() on %s", collection)
|
||||||
return []
|
return []
|
||||||
|
|
||||||
async def count_documents(self, collection: str, target: dict) -> int:
|
async def count_documents(self, collection: str, target: dict) -> int:
|
||||||
@@ -213,7 +245,7 @@ class Mongo:
|
|||||||
try:
|
try:
|
||||||
return await self._db[collection].count_documents(target)
|
return await self._db[collection].count_documents(target)
|
||||||
except Exception:
|
except Exception:
|
||||||
log.exception(f"db.count_documents() on {collection}")
|
log.exception("db.count_documents() on %s", collection)
|
||||||
return 0
|
return 0
|
||||||
|
|
||||||
async def check_document_exists(self, collection: str, target: dict) -> bool:
|
async def check_document_exists(self, collection: str, target: dict) -> bool:
|
||||||
@@ -221,15 +253,19 @@ class Mongo:
|
|||||||
try:
|
try:
|
||||||
return await self._db[collection].count_documents(target, limit=1) > 0
|
return await self._db[collection].count_documents(target, limit=1) > 0
|
||||||
except Exception:
|
except Exception:
|
||||||
log.exception(f"db.check_document_exists() on {collection}")
|
log.exception("db.check_document_exists() on %s", collection)
|
||||||
return False
|
return False
|
||||||
|
|
||||||
|
async def exists(self, collection: str, target: dict) -> bool:
|
||||||
|
"""trio-consistent alias of check_document_exists"""
|
||||||
|
return await self.check_document_exists(collection, target)
|
||||||
|
|
||||||
async def count_value_in_array(self, collection: str, array_field: str, value: str) -> int:
|
async def count_value_in_array(self, collection: str, array_field: str, value: str) -> int:
|
||||||
"""count documents whose array field contains value"""
|
"""count documents whose array field contains value"""
|
||||||
try:
|
try:
|
||||||
return await self._db[collection].count_documents({array_field: value})
|
return await self._db[collection].count_documents({array_field: value})
|
||||||
except Exception:
|
except Exception:
|
||||||
log.exception(f"db.count_value_in_array() on {collection}")
|
log.exception("db.count_value_in_array() on %s", collection)
|
||||||
return 0
|
return 0
|
||||||
|
|
||||||
async def exists_in_array(self, collection: str, array_field: str, value: str) -> bool:
|
async def exists_in_array(self, collection: str, array_field: str, value: str) -> bool:
|
||||||
@@ -238,7 +274,7 @@ class Mongo:
|
|||||||
doc = await self._db[collection].find_one({array_field: value}, {"_id": 1})
|
doc = await self._db[collection].find_one({array_field: value}, {"_id": 1})
|
||||||
return doc is not None
|
return doc is not None
|
||||||
except Exception:
|
except Exception:
|
||||||
log.exception(f"db.exists_in_array() on {collection}")
|
log.exception("db.exists_in_array() on %s", collection)
|
||||||
return False
|
return False
|
||||||
|
|
||||||
async def distinct(self, collection: str, field: str, target: Optional[dict] = None) -> List:
|
async def distinct(self, collection: str, field: str, target: Optional[dict] = None) -> List:
|
||||||
@@ -246,7 +282,7 @@ class Mongo:
|
|||||||
try:
|
try:
|
||||||
return await self._db[collection].distinct(field, target or {})
|
return await self._db[collection].distinct(field, target or {})
|
||||||
except Exception:
|
except Exception:
|
||||||
log.exception(f"db.distinct() on {collection}")
|
log.exception("db.distinct() on %s", collection)
|
||||||
return []
|
return []
|
||||||
|
|
||||||
async def run_aggregation(
|
async def run_aggregation(
|
||||||
@@ -257,7 +293,7 @@ class Mongo:
|
|||||||
stages = [{"$match": match_filter}, *pipeline] if match_filter else pipeline
|
stages = [{"$match": match_filter}, *pipeline] if match_filter else pipeline
|
||||||
return await self._db[collection].aggregate(stages).to_list(length=None)
|
return await self._db[collection].aggregate(stages).to_list(length=None)
|
||||||
except Exception:
|
except Exception:
|
||||||
log.exception(f"db.run_aggregation() on {collection}")
|
log.exception("db.run_aggregation() on %s", collection)
|
||||||
return []
|
return []
|
||||||
|
|
||||||
async def search_documents(
|
async def search_documents(
|
||||||
@@ -268,7 +304,7 @@ class Mongo:
|
|||||||
cursor = self._db[collection].find(search_query).sort(sort_key, sort_order)
|
cursor = self._db[collection].find(search_query).sort(sort_key, sort_order)
|
||||||
return [doc async for doc in cursor]
|
return [doc async for doc in cursor]
|
||||||
except Exception:
|
except Exception:
|
||||||
log.exception(f"db.search_documents() on {collection}")
|
log.exception("db.search_documents() on %s", collection)
|
||||||
return []
|
return []
|
||||||
|
|
||||||
async def search_indexes(self, collection: str, search_term: str) -> List[dict]:
|
async def search_indexes(self, collection: str, search_term: str) -> List[dict]:
|
||||||
@@ -280,7 +316,7 @@ class Mongo:
|
|||||||
).sort([("score", {"$meta": "textScore"})])
|
).sort([("score", {"$meta": "textScore"})])
|
||||||
return [doc async for doc in cursor]
|
return [doc async for doc in cursor]
|
||||||
except Exception:
|
except Exception:
|
||||||
log.exception(f"db.search_indexes() on {collection}")
|
log.exception("db.search_indexes() on %s", collection)
|
||||||
return []
|
return []
|
||||||
|
|
||||||
# -------------------------------------------------------------------------
|
# -------------------------------------------------------------------------
|
||||||
@@ -294,7 +330,7 @@ class Mongo:
|
|||||||
response = await self._db[collection].replace_one(target, document, upsert=upsert)
|
response = await self._db[collection].replace_one(target, document, upsert=upsert)
|
||||||
return bool(response.matched_count or response.upserted_id)
|
return bool(response.matched_count or response.upserted_id)
|
||||||
except Exception:
|
except Exception:
|
||||||
log.exception(f"db.update_document() on {collection}")
|
log.exception("db.update_document() on %s", collection)
|
||||||
return False
|
return False
|
||||||
|
|
||||||
async def update_documents(self, collection: str, target: dict, values: dict) -> int:
|
async def update_documents(self, collection: str, target: dict, values: dict) -> int:
|
||||||
@@ -303,21 +339,21 @@ class Mongo:
|
|||||||
response = await self._db[collection].update_many(target, values)
|
response = await self._db[collection].update_many(target, values)
|
||||||
return response.modified_count
|
return response.modified_count
|
||||||
except Exception:
|
except Exception:
|
||||||
log.exception(f"db.update_documents() on {collection}")
|
log.exception("db.update_documents() on %s", collection)
|
||||||
return 0
|
return 0
|
||||||
|
|
||||||
async def update_document_field(self, collection: str, target: dict, updates: dict) -> bool:
|
async def update_document_field(self, collection: str, target: dict, updates: dict) -> bool:
|
||||||
"""$set one or more fields on a single document
|
"""$set one or more fields on a single document
|
||||||
|
|
||||||
returns True when a document matched, even if the write changed nothing (a
|
returns True when a document matched, even if `$set` to an identical value
|
||||||
`$set` to the identical value leaves `modified_count=0`); use `matched_count`
|
leaves `modified_count=0` - uses `matched_count` so an idempotent no-op isn't
|
||||||
so an idempotent no-op on an existing doc isn't misread as a failure.
|
misread as a failure.
|
||||||
"""
|
"""
|
||||||
try:
|
try:
|
||||||
response = await self._db[collection].update_one(target, {"$set": updates})
|
response = await self._db[collection].update_one(target, {"$set": updates})
|
||||||
return response.matched_count > 0
|
return response.matched_count > 0
|
||||||
except Exception:
|
except Exception:
|
||||||
log.exception(f"db.update_document_field() on {collection}")
|
log.exception("db.update_document_field() on %s", collection)
|
||||||
return False
|
return False
|
||||||
|
|
||||||
async def update_document_operator(
|
async def update_document_operator(
|
||||||
@@ -325,26 +361,30 @@ class Mongo:
|
|||||||
) -> bool:
|
) -> bool:
|
||||||
"""apply raw update operators ($set/$inc/$unset/...) to a single document
|
"""apply raw update operators ($set/$inc/$unset/...) to a single document
|
||||||
|
|
||||||
returns True when a document matched, even if the operators changed nothing
|
returns True when a document matched, even if the operators changed nothing -
|
||||||
(e.g. `$set` to an identical value) — use `matched_count` so an idempotent
|
uses `matched_count` so an idempotent write isn't misread as a failure.
|
||||||
write on an existing doc isn't misread as a failure.
|
|
||||||
"""
|
"""
|
||||||
try:
|
try:
|
||||||
response = await self._db[collection].update_one(target, update_query)
|
response = await self._db[collection].update_one(target, update_query)
|
||||||
return response.matched_count > 0
|
return response.matched_count > 0
|
||||||
except Exception:
|
except Exception:
|
||||||
log.exception(f"db.update_document_operator() on {collection}")
|
log.exception("db.update_document_operator() on %s", collection)
|
||||||
return False
|
return False
|
||||||
|
|
||||||
async def upsert_document_field(self, collection: str, target: dict, updates: dict) -> bool:
|
async def upsert_document_field(self, collection: str, target: dict, updates: dict) -> bool:
|
||||||
"""$set fields on a single document, creating it if absent"""
|
"""$set fields on a single document, creating it if absent
|
||||||
|
|
||||||
|
returns True when a document matched or was upserted, even if `$set` to an
|
||||||
|
identical value leaves `modified_count=0` - mirrors update_document_field's
|
||||||
|
matched_count-over-modified_count contract.
|
||||||
|
"""
|
||||||
try:
|
try:
|
||||||
response = await self._db[collection].update_one(
|
response = await self._db[collection].update_one(
|
||||||
target, {"$set": updates}, upsert=True
|
target, {"$set": updates}, upsert=True
|
||||||
)
|
)
|
||||||
return bool(response.modified_count or response.upserted_id)
|
return bool(response.matched_count or response.upserted_id)
|
||||||
except Exception:
|
except Exception:
|
||||||
log.exception(f"db.upsert_document_field() on {collection}")
|
log.exception("db.upsert_document_field() on %s", collection)
|
||||||
return False
|
return False
|
||||||
|
|
||||||
async def update_documents_pipeline(
|
async def update_documents_pipeline(
|
||||||
@@ -355,33 +395,40 @@ class Mongo:
|
|||||||
response = await self._db[collection].update_many(target, pipeline)
|
response = await self._db[collection].update_many(target, pipeline)
|
||||||
return response.modified_count
|
return response.modified_count
|
||||||
except Exception:
|
except Exception:
|
||||||
log.exception(f"db.update_documents_pipeline() on {collection}")
|
log.exception("db.update_documents_pipeline() on %s", collection)
|
||||||
return 0
|
return 0
|
||||||
|
|
||||||
async def document_push_array(
|
async def document_push_array(
|
||||||
self, collection: str, target: dict, array: str, value: Any
|
self, collection: str, target: dict, array: str, value: Any
|
||||||
) -> bool:
|
) -> bool:
|
||||||
"""$push a value onto an array field"""
|
"""$push a value onto an array field
|
||||||
|
|
||||||
|
returns True when a document matched; $push always mutates, so matched_count
|
||||||
|
and modified_count agree here in practice.
|
||||||
|
"""
|
||||||
try:
|
try:
|
||||||
response = await self._db[collection].update_one(target, {"$push": {array: value}})
|
response = await self._db[collection].update_one(target, {"$push": {array: value}})
|
||||||
return response.modified_count > 0
|
return response.matched_count > 0
|
||||||
except Exception:
|
except Exception:
|
||||||
log.exception(f"db.document_push_array() on {collection}")
|
log.exception("db.document_push_array() on %s", collection)
|
||||||
return False
|
return False
|
||||||
|
|
||||||
async def document_push_and_set(
|
async def document_push_and_set(
|
||||||
self, collection: str, target: dict, array: str, value: Any,
|
self, collection: str, target: dict, array: str, value: Any,
|
||||||
field_to_set: str, set_value: Any,
|
field_to_set: str, set_value: Any,
|
||||||
) -> bool:
|
) -> bool:
|
||||||
"""$push to an array and $set a field in one update"""
|
"""$push to an array and $set a field in one update
|
||||||
|
|
||||||
|
returns True when a document matched (matched_count, not modified_count).
|
||||||
|
"""
|
||||||
try:
|
try:
|
||||||
response = await self._db[collection].update_one(
|
response = await self._db[collection].update_one(
|
||||||
target,
|
target,
|
||||||
{"$push": {array: value}, "$set": {field_to_set: set_value}},
|
{"$push": {array: value}, "$set": {field_to_set: set_value}},
|
||||||
)
|
)
|
||||||
return response.modified_count > 0
|
return response.matched_count > 0
|
||||||
except Exception:
|
except Exception:
|
||||||
log.exception(f"db.document_push_and_set() on {collection}")
|
log.exception("db.document_push_and_set() on %s", collection)
|
||||||
return False
|
return False
|
||||||
|
|
||||||
async def document_pop_array(
|
async def document_pop_array(
|
||||||
@@ -389,15 +436,14 @@ class Mongo:
|
|||||||
) -> bool:
|
) -> bool:
|
||||||
"""$pull a value from an array field
|
"""$pull a value from an array field
|
||||||
|
|
||||||
returns True when a document matched, even if nothing was pulled (the value
|
returns True when a document matched, even if the value was absent and
|
||||||
was absent) — use `matched_count` so a no-op pull on an existing doc isn't
|
nothing was pulled - uses `matched_count`, not `modified_count`.
|
||||||
misread as a failure.
|
|
||||||
"""
|
"""
|
||||||
try:
|
try:
|
||||||
response = await self._db[collection].update_one(target, {"$pull": {array: value}})
|
response = await self._db[collection].update_one(target, {"$pull": {array: value}})
|
||||||
return response.matched_count > 0
|
return response.matched_count > 0
|
||||||
except Exception:
|
except Exception:
|
||||||
log.exception(f"db.document_pop_array() on {collection}")
|
log.exception("db.document_pop_array() on %s", collection)
|
||||||
return False
|
return False
|
||||||
|
|
||||||
async def find_one_and_update(
|
async def find_one_and_update(
|
||||||
@@ -413,7 +459,7 @@ class Mongo:
|
|||||||
return_document=ReturnDocument.AFTER if return_after else ReturnDocument.BEFORE,
|
return_document=ReturnDocument.AFTER if return_after else ReturnDocument.BEFORE,
|
||||||
)
|
)
|
||||||
except Exception:
|
except Exception:
|
||||||
log.exception(f"db.find_one_and_update() on {collection}")
|
log.exception("db.find_one_and_update() on %s", collection)
|
||||||
return None
|
return None
|
||||||
|
|
||||||
async def find_one_and_replace(
|
async def find_one_and_replace(
|
||||||
@@ -429,7 +475,7 @@ class Mongo:
|
|||||||
return_document=ReturnDocument.AFTER if return_after else ReturnDocument.BEFORE,
|
return_document=ReturnDocument.AFTER if return_after else ReturnDocument.BEFORE,
|
||||||
)
|
)
|
||||||
except Exception:
|
except Exception:
|
||||||
log.exception(f"db.find_one_and_replace() on {collection}")
|
log.exception("db.find_one_and_replace() on %s", collection)
|
||||||
return None
|
return None
|
||||||
|
|
||||||
async def find_one_and_delete(
|
async def find_one_and_delete(
|
||||||
@@ -439,7 +485,7 @@ class Mongo:
|
|||||||
try:
|
try:
|
||||||
return await self._db[collection].find_one_and_delete(target, projection=fields)
|
return await self._db[collection].find_one_and_delete(target, projection=fields)
|
||||||
except Exception:
|
except Exception:
|
||||||
log.exception(f"db.find_one_and_delete() on {collection}")
|
log.exception("db.find_one_and_delete() on %s", collection)
|
||||||
return None
|
return None
|
||||||
|
|
||||||
# -------------------------------------------------------------------------
|
# -------------------------------------------------------------------------
|
||||||
@@ -451,26 +497,29 @@ class Mongo:
|
|||||||
response = await self._db[collection].delete_one(target)
|
response = await self._db[collection].delete_one(target)
|
||||||
return response.deleted_count
|
return response.deleted_count
|
||||||
except Exception:
|
except Exception:
|
||||||
log.exception(f"db.delete_document() on {collection}")
|
log.exception("db.delete_document() on %s", collection)
|
||||||
return 0
|
return 0
|
||||||
|
|
||||||
|
async def delete(self, collection: str, target: dict) -> int:
|
||||||
|
"""trio-consistent alias of delete_document (single-document delete)"""
|
||||||
|
return await self.delete_document(collection, target)
|
||||||
|
|
||||||
async def delete_documents(self, collection: str, target: dict) -> int:
|
async def delete_documents(self, collection: str, target: dict) -> int:
|
||||||
"""delete all matching documents, returning deleted count"""
|
"""delete all matching documents, returning deleted count"""
|
||||||
try:
|
try:
|
||||||
response = await self._db[collection].delete_many(target)
|
response = await self._db[collection].delete_many(target)
|
||||||
return response.deleted_count
|
return response.deleted_count
|
||||||
except Exception:
|
except Exception:
|
||||||
log.exception(f"db.delete_documents() on {collection}")
|
log.exception("db.delete_documents() on %s", collection)
|
||||||
return 0
|
return 0
|
||||||
|
|
||||||
# -------------------------------------------------------------------------
|
# -------------------------------------------------------------------------
|
||||||
# bulk / checks
|
# bulk / checks
|
||||||
|
|
||||||
async def bulk_write(self, collection: str, operations: list, ordered: bool = True) -> dict:
|
async def bulk_write(self, collection: str, operations: list, ordered: bool = True) -> dict:
|
||||||
"""
|
"""run a batch of pymongo write ops (InsertOne/UpdateOne/DeleteOne/...)
|
||||||
run a batch of pymongo write ops (InsertOne/UpdateOne/DeleteOne/...)
|
|
||||||
|
|
||||||
caller builds the ops from pymongo, e.g.:
|
caller builds the ops, e.g.:
|
||||||
from pymongo import UpdateOne
|
from pymongo import UpdateOne
|
||||||
ops = [UpdateOne({'_id': i}, {'$set': {...}}, upsert=True) for i in ids]
|
ops = [UpdateOne({'_id': i}, {'$set': {...}}, upsert=True) for i in ids]
|
||||||
returns a summary dict of counts, or an empty dict on failure
|
returns a summary dict of counts, or an empty dict on failure
|
||||||
@@ -485,14 +534,13 @@ class Mongo:
|
|||||||
"upserted": result.upserted_count,
|
"upserted": result.upserted_count,
|
||||||
}
|
}
|
||||||
except Exception:
|
except Exception:
|
||||||
log.exception(f"db.bulk_write() on {collection}")
|
log.exception("db.bulk_write() on %s", collection)
|
||||||
return {}
|
return {}
|
||||||
|
|
||||||
async def document_check_multi(
|
async def document_check_multi(
|
||||||
self, collection: str, target: dict, checks: List[str], value: str
|
self, collection: str, target: dict, checks: List[str], value: str
|
||||||
) -> bool:
|
) -> bool:
|
||||||
"""
|
"""return whether a doc matching target has value in any of the given arrays
|
||||||
return whether a doc matching target has value in any of the given arrays
|
|
||||||
|
|
||||||
target is a normal filter dict (merged into the query), not a bare _id
|
target is a normal filter dict (merged into the query), not a bare _id
|
||||||
"""
|
"""
|
||||||
@@ -503,29 +551,31 @@ class Mongo:
|
|||||||
}
|
}
|
||||||
return await self._db[collection].find_one(query) is not None
|
return await self._db[collection].find_one(query) is not None
|
||||||
except Exception:
|
except Exception:
|
||||||
log.exception(f"db.document_check_multi() on {collection}")
|
log.exception("db.document_check_multi() on %s", collection)
|
||||||
return False
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
# back-compat alias: the class was historically `Mongo`; existing `Mongo(...)` call
|
||||||
|
# sites keep working unchanged
|
||||||
|
Mongo = MongoDB
|
||||||
|
|
||||||
|
|
||||||
# -----------------------------------------------------------------------------
|
# -----------------------------------------------------------------------------
|
||||||
# backwards-compat module proxy
|
# backwards-compat module proxy: `import mongo; mongo.init(conn, db)` then
|
||||||
# - lets legacy call sites keep using `await mongo.get_documents(...)`
|
# bare `await mongo.func(...)`. needs `import mongo` - `from mongo import func`
|
||||||
# without an object, after a one-time `mongo.init(conn, db)`
|
# resolves at import time, before init() runs.
|
||||||
# - works with `import mongo; mongo.func(...)` (resolved at call time)
|
|
||||||
# - does NOT work with `from mongo import func` (resolved at import,
|
|
||||||
# before init runs) — switch those sites to `import mongo`
|
|
||||||
|
|
||||||
_default: Optional[Mongo] = None
|
_default: Optional[MongoDB] = None
|
||||||
|
|
||||||
|
|
||||||
def init(connection_string: str, database: str) -> Mongo:
|
def init(connection_string: str, database: str) -> MongoDB:
|
||||||
"""arm the module-level default instance and return it"""
|
"""arm the module-level default instance and return it"""
|
||||||
global _default
|
global _default
|
||||||
_default = Mongo(connection_string, database)
|
_default = MongoDB(connection_string, database)
|
||||||
return _default
|
return _default
|
||||||
|
|
||||||
|
|
||||||
def instance() -> Mongo:
|
def instance() -> MongoDB:
|
||||||
"""return the default instance, raising if init() has not run"""
|
"""return the default instance, raising if init() has not run"""
|
||||||
if _default is None:
|
if _default is None:
|
||||||
raise RuntimeError("mongo not initialized; call mongo.init(conn, db) first")
|
raise RuntimeError("mongo not initialized; call mongo.init(conn, db) first")
|
||||||
|
|||||||
Reference in New Issue
Block a user