fix: close() nulls _pool + connect() lock (psql-7/8 twin), % in identifiers, doc caveats

close() now nulls self._pool so a closed instance reports not-connected instead of
masquerading as live (twin of psql-7); connect() is guarded by an internal asyncio.Lock
so concurrent connect() calls serialize instead of racing to create and orphan multiple
live pools (twin of psql-8). _quote_ident now escapes a literal % in identifiers (mysql-7:
PyMySQL's query % args substitution otherwise breaks any Layer-1 call against a %-bearing
table/column name). fetchval/exists tolerate a caller-overridden tuple cursorclass via a
new _first_value helper (mysql-8). Documents the %%-escaping rule for literal % in raw SQL
text (mysql-6) and the upsert() VALUES() deprecation on MySQL 8.0.20+ (mysql-9).

Signed-off-by: disqualifier <dev@disqualifier.me>
This commit is contained in:
2026-07-02 23:32:53 -04:00
parent 50388de22c
commit be683fcc5e
4 changed files with 88 additions and 52 deletions
+14 -5
View File
@@ -11,13 +11,13 @@ wire-compatible and share the driver, so this covers both.
`requirements.txt`:
```
mysql @ git+ssh://git@git.rethinkstudios.io/rethink-public/mysql.git@v0.1.3
mysql @ git+ssh://git@git.rethinkstudios.io/rethink-public/mysql.git@v0.1.4
```
Direct:
```bash
pip install "mysql @ git+ssh://git@git.rethinkstudios.io/rethink-public/mysql.git@v0.1.3"
pip install "mysql @ git+ssh://git@git.rethinkstudios.io/rethink-public/mysql.git@v0.1.4"
```
Pulls `aiomysql` (which pulls `PyMySQL`).
@@ -99,7 +99,12 @@ For anything even Layer 2 doesn't model (`executemany`, server-side cursors), us
- **Values are always parameterized.** Layer 1 builds `%s` internally; Layer 2 takes your
`%s` placeholders + a params sequence. Never f-string/`%`-format a value into SQL (the
`%s` is the DBAPI placeholder, not Python string formatting). Only identifiers
(table/column names) are interpolated, and they're backtick-quoted.
(table/column names) are interpolated, and they're backtick-quoted — Layer 1 also
escapes a literal `%` inside an identifier for you.
- **A literal `%` in Layer-2 SQL text must be written `%%`.** PyMySQL builds the query via
`query % args` whenever you pass params, so `"... LIKE '%foo%'"` with params raises
`TypeError`/`ValueError` — write `"... LIKE '%%foo%%'"` instead. Calls with no params are
unaffected (no substitution happens), and Layer 1 handles this for you internally.
## Dialect vs psql
@@ -115,7 +120,8 @@ Layer 1 is portable — you never see these. A Layer-2 raw-SQL author does:
`upsert()`'s returned rowcount follows MySQL's `ON DUPLICATE KEY UPDATE` convention (1 for
an insert, 2 for an update, 0 for a no-op) — a per-row count that differs from psql's; treat
it as "affected", not "rows matched".
it as "affected", not "rows matched". It emits the `col = VALUES(col)` update form, which
MySQL 8.0.20+ deprecates (warning 1287, still functional; MariaDB is unaffected).
## Error contract — fail loud
@@ -134,7 +140,10 @@ result (no row, empty table) — never a swallowed failure.
`get`/`delete`/`exists`/`get_one` take **equality** conditions only (`col = val AND ...`);
anything richer goes through Layer 2. The pool runs with `autocommit=True`; `transaction()`
opens an explicit transaction for atomic multi-statement blocks.
opens an explicit transaction for atomic multi-statement blocks. `cursorclass` is
overridable via a constructor kwarg (default `DictCursor`); `fetchval`/`exists` work under
either a dict or tuple cursorclass, everything else stays dict-shaped only under the
default `DictCursor`.
## Versioning
+1 -1
View File
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
[project]
name = "mysql"
version = "0.1.3"
version = "0.1.4"
description = "async mysql/mariadb wrapper over aiomysql: two-layer API (friendly verbs + raw escape hatch), fail-loud, config-free"
requires-python = ">=3.10"
dependencies = [
+3 -1
View File
@@ -1,3 +1,5 @@
from .mysql import MysqlDB
__all__ = ["MysqlDB"]
__version__ = "0.1.4"
__all__ = ["MysqlDB", "__version__"]
+55 -30
View File
@@ -33,18 +33,17 @@ call-site changes; only the dialect below differs, hidden inside the wrapper):
dialect (mysql-specific; visible only in raw layer-2 SQL you write):
- placeholders are `%s` (here) vs `$1` (psql). the `%s` is the DBAPI placeholder the
driver consumes, NOT python `%` string formatting.
- upsert emits `ON DUPLICATE KEY UPDATE` (here) vs `ON CONFLICT` (psql); layer-1
upsert() hides this.
driver consumes, NOT python `%` string formatting — but PyMySQL builds it via
`query % args` whenever params are passed, so any OTHER literal `%` in the query
text (e.g. `LIKE '%foo%'`) must be written `%%` or it raises. no params -> no
substitution -> a lone `%` is fine. layer 1 escapes `%` in identifiers for you.
- upsert emits `ON DUPLICATE KEY UPDATE` (here) vs `ON CONFLICT` (psql), via the
deprecated `col = VALUES(col)` form (MySQL 8.0.20+ warns; still works, MariaDB is
unaffected). layer-1 upsert() hides both.
rows:
layer 1 and fetch/fetchone return plain dicts ({column: value}) via aiomysql's
DictCursor — identical shape to the psql lib.
placeholders / injection safety:
values are ALWAYS parameterized — layer 1 builds `%s` internally; layer 2 takes your
`%s` placeholders + a params sequence. never f-string/%-format a value into SQL. only
identifiers (table/column names) are interpolated, and they are backtick-quoted.
layer 1 and fetch/fetchone return plain dicts via aiomysql's DictCursor (identical
shape to psql). fetchval/exists tolerate a caller-overridden tuple cursorclass too.
errors (FAIL LOUD — unlike the mongo lib's swallow-and-default):
every method catches the driver error (pymysql.err.MySQLError, OSError on connection
@@ -53,6 +52,7 @@ errors (FAIL LOUD — unlike the mongo lib's swallow-and-default):
wrapped, use the raw `.pool` property (the aiomysql.Pool).
"""
import asyncio
import logging
from typing import Any, Dict, List, Optional, Sequence
@@ -64,13 +64,27 @@ log = logging.getLogger(__name__)
_DRIVER_ERRORS = (MySQLError, OSError)
def _first_value(row) -> Any:
"""return the first column's value from a dict or sequence row
fetchval/exists read a single scalar column and work under either cursorclass: the
default DictCursor (dict row) or a caller override to a tuple/sequence cursor via
pool_kwargs — the only two row shapes aiomysql cursors produce.
"""
return next(iter(row.values())) if isinstance(row, dict) else row[0]
def _quote_ident(identifier: str) -> str:
"""backtick-quote a sql identifier (table/column), escaping embedded backticks
"""backtick-quote a sql identifier (table/column), escaping backticks and `%`
identifiers can't be parameterized, so they are interpolated — backtick-quoting +
doubling any embedded backtick is the mysql-safe way to do that for caller names.
doubling any embedded backtick is the mysql-safe way to do that for caller names. an
embedded `%` is also doubled: PyMySQL applies `query % args` whenever a call passes
params, and an unescaped `%` inside an interpolated identifier would hit that same
substitution and raise, not just literal `%` written by the caller (see the module
docstring's `%%`-escaping rule).
"""
return "`" + identifier.replace("`", "``") + "`"
return "`" + identifier.replace("`", "``").replace("%", "%%") + "`"
class MysqlDB:
@@ -92,10 +106,15 @@ class MysqlDB:
host/port/db/user/password/minsize/maxsize are injected by the caller. extra
pool_kwargs pass through to aiomysql.create_pool (unix_socket, ssl, charset, ...).
rows come back as dicts (DictCursor), overridable via pool_kwargs. autocommit is
always on: layer-1 verbs and raw execute() rely on it to commit on their own, so
`autocommit=False` in pool_kwargs is rejected (use transaction() for atomic
multi-statement blocks instead — it disables autocommit for just that block).
rows come back as dicts (DictCursor), overridable via pool_kwargs to a tuple
cursorclass — fetchval/exists work either way, everything else stays dict-shaped
only under DictCursor. autocommit is always on: layer-1 verbs and raw execute()
rely on it to commit their own writes without an explicit commit, so
`autocommit=False` in pool_kwargs is rejected outright — silently accepting it
would mean every write sits uncommitted until something else commits or the
connection is returned to the pool and rolled back, losing data with no error.
use transaction() for an atomic multi-statement block instead (it disables
autocommit for just that block).
"""
if pool_kwargs.get("autocommit") is False:
raise ValueError(
@@ -116,6 +135,7 @@ class MysqlDB:
self._config.setdefault("cursorclass", aiomysql.cursors.DictCursor)
self._config.update(pool_kwargs)
self._pool: Optional[aiomysql.Pool] = None
self._connect_lock = asyncio.Lock()
async def connect(self) -> "MysqlDB":
"""build the pool and validate it with SELECT 1; fail loud on bad config
@@ -123,8 +143,11 @@ class MysqlDB:
returns self so callers can write `db = await MysqlDB(...).connect()`. if called
again on an already-connected instance the previous pool is closed first (no
orphaned pool); if the SELECT-1 validation fails the freshly-built pool is torn
down before re-raising, so a failed connect() never leaks a live pool.
down before re-raising, so a failed connect() never leaks a live pool. concurrent
callers serialize on an internal lock so racing connect() calls build exactly one
pool instead of each leaking the losers' live pools.
"""
async with self._connect_lock:
if self._pool is not None:
await self.close()
pool = await aiomysql.create_pool(**self._config)
@@ -151,6 +174,8 @@ class MysqlDB:
except _DRIVER_ERRORS:
log.exception("mysql.close()")
raise
finally:
self._pool = None
async def __aenter__(self) -> "MysqlDB":
return await self.connect()
@@ -234,7 +259,7 @@ class MysqlDB:
raise
if not row:
return None
return next(iter(row.values()))
return _first_value(row)
def transaction(self):
"""async context manager running a block of statements atomically
@@ -349,16 +374,19 @@ class MysqlDB:
except _DRIVER_ERRORS:
log.exception("mysql.exists(%s)", table)
raise
return bool(next(iter(row.values()))) if row else False
return bool(_first_value(row)) if row else False
async def upsert(self, table: str, values: Dict[str, Any], conflict: Sequence[str]) -> int:
"""INSERT ... ON DUPLICATE KEY UPDATE — insert or update on unique/pk clash
`conflict` is the list of columns forming the unique/pk constraint. mysql upserts
on ANY unique-key clash (it does not name the constraint the way postgres does), so
`conflict` is accepted for a call-signature identical to psql but the constraint is
whatever unique/pk key the row violates. the wrapper emits ON DUPLICATE KEY UPDATE
(psql emits ON CONFLICT for the identical call). returns the affected rowcount.
`conflict` is accepted for call-signature parity with psql but the constraint is
whatever unique/pk key the row violates. returns the affected rowcount. emits the
`col = VALUES(col)` update form, deprecated since MySQL 8.0.20 (warning 1287, still
functional) and unaffected on MariaDB; a future MySQL-only build could probe
`SELECT VERSION()` and switch to alias syntax, not done here as a documented cost/
benefit call, not an oversight.
"""
cols = list(values.keys())
placeholders = ", ".join(["%s"] * len(cols))
@@ -392,9 +420,8 @@ class _Transaction:
try:
await self._conn.begin()
except BaseException:
# begin() failing after acquire would otherwise leak the pooled connection —
# __aexit__ is not called when __aenter__ raises. release it and reset so a
# failed transaction start never burns a pool slot.
# __aexit__ never runs if __aenter__ raises, so begin() failing after acquire
# would otherwise leak the pooled connection — release it here instead.
await self._pool.release(self._conn)
self._conn = None
raise
@@ -407,10 +434,8 @@ class _Transaction:
else:
await self._conn.rollback()
finally:
# aiomysql's release() returns a task (it schedules a _wakeup); await it so we
# don't leave an un-awaited task each transaction matches psql's awaited
# release and aiomysql's own idiom. the connection is returned synchronously
# inside release() regardless, so pool accounting is never at risk.
# release() returns a task (schedules a _wakeup); await it to avoid leaving
# it un-awaited each transaction, matching psql's and aiomysql's own idiom.
await self._pool.release(self._conn)