2 Commits
Author SHA1 Message Date
dsql 2e837da7df fix: close() nulls _pool + connect() lock (psql-7/8 twin), % in identifiers, doc caveats
close() now nulls self._pool so a closed instance reports not-connected instead of
masquerading as live (twin of psql-7); connect() is guarded by an internal asyncio.Lock
so concurrent connect() calls serialize instead of racing to create and orphan multiple
live pools (twin of psql-8). _quote_ident now escapes a literal % in identifiers (mysql-7:
PyMySQL's query % args substitution otherwise breaks any Layer-1 call against a %-bearing
table/column name). fetchval/exists tolerate a caller-overridden tuple cursorclass via a
new _first_value helper (mysql-8). Documents the %%-escaping rule for literal % in raw SQL
text (mysql-6) and the upsert() VALUES() deprecation on MySQL 8.0.20+ (mysql-9).

Signed-off-by: disqualifier <dev@disqualifier.me>
2026-07-02 23:32:53 -04:00
dsql 50388de22c fix: reject autocommit=False override and IS NULL condition handling
_where() rendered None conditions as col = %s bound to NULL, which sql
never matches, so get/get_one/exists/delete silently missed NULL rows
despite insert() writing NULL fine — fixed to emit col IS NULL, in
lockstep with the psql lib's identical fix.

Separately, __init__ now rejects autocommit=False in pool_kwargs: with
it, _run/_fetchall never commit, so on pool release aiomysql closes the
in-transaction connection and MySQL rolls back server-side while
insert()/delete()/upsert()/execute() still return success signals
(lastrowid/rowcount) for writes that were silently discarded.

Signed-off-by: disqualifier <dev@disqualifier.me>
2026-07-02 16:44:34 -04:00
4 changed files with 107 additions and 54 deletions
+15 -6
View File
@@ -11,18 +11,18 @@ wire-compatible and share the driver, so this covers both.
`requirements.txt`: `requirements.txt`:
``` ```
mysql @ git+ssh://git@git.rethinkstudios.io/rethink-public/mysql.git@v0.1.2 mysql @ git+ssh://git@git.rethinkstudios.io/rethink-public/mysql.git@v0.1.4
``` ```
Direct: Direct:
```bash ```bash
pip install "mysql @ git+ssh://git@git.rethinkstudios.io/rethink-public/mysql.git@v0.1.2" pip install "mysql @ git+ssh://git@git.rethinkstudios.io/rethink-public/mysql.git@v0.1.4"
``` ```
Pulls `aiomysql` (which pulls `PyMySQL`). Pulls `aiomysql` (which pulls `PyMySQL`).
Drop the `@v0.1.2` suffix from the line above to install the latest unpinned. Drop the `@v0.1.3` suffix from the line above to install the latest unpinned.
## The two-layer API ## The two-layer API
@@ -99,7 +99,12 @@ For anything even Layer 2 doesn't model (`executemany`, server-side cursors), us
- **Values are always parameterized.** Layer 1 builds `%s` internally; Layer 2 takes your - **Values are always parameterized.** Layer 1 builds `%s` internally; Layer 2 takes your
`%s` placeholders + a params sequence. Never f-string/`%`-format a value into SQL (the `%s` placeholders + a params sequence. Never f-string/`%`-format a value into SQL (the
`%s` is the DBAPI placeholder, not Python string formatting). Only identifiers `%s` is the DBAPI placeholder, not Python string formatting). Only identifiers
(table/column names) are interpolated, and they're backtick-quoted. (table/column names) are interpolated, and they're backtick-quoted — Layer 1 also
escapes a literal `%` inside an identifier for you.
- **A literal `%` in Layer-2 SQL text must be written `%%`.** PyMySQL builds the query via
`query % args` whenever you pass params, so `"... LIKE '%foo%'"` with params raises
`TypeError`/`ValueError` — write `"... LIKE '%%foo%%'"` instead. Calls with no params are
unaffected (no substitution happens), and Layer 1 handles this for you internally.
## Dialect vs psql ## Dialect vs psql
@@ -115,7 +120,8 @@ Layer 1 is portable — you never see these. A Layer-2 raw-SQL author does:
`upsert()`'s returned rowcount follows MySQL's `ON DUPLICATE KEY UPDATE` convention (1 for `upsert()`'s returned rowcount follows MySQL's `ON DUPLICATE KEY UPDATE` convention (1 for
an insert, 2 for an update, 0 for a no-op) — a per-row count that differs from psql's; treat an insert, 2 for an update, 0 for a no-op) — a per-row count that differs from psql's; treat
it as "affected", not "rows matched". it as "affected", not "rows matched". It emits the `col = VALUES(col)` update form, which
MySQL 8.0.20+ deprecates (warning 1287, still functional; MariaDB is unaffected).
## Error contract — fail loud ## Error contract — fail loud
@@ -134,7 +140,10 @@ result (no row, empty table) — never a swallowed failure.
`get`/`delete`/`exists`/`get_one` take **equality** conditions only (`col = val AND ...`); `get`/`delete`/`exists`/`get_one` take **equality** conditions only (`col = val AND ...`);
anything richer goes through Layer 2. The pool runs with `autocommit=True`; `transaction()` anything richer goes through Layer 2. The pool runs with `autocommit=True`; `transaction()`
opens an explicit transaction for atomic multi-statement blocks. opens an explicit transaction for atomic multi-statement blocks. `cursorclass` is
overridable via a constructor kwarg (default `DictCursor`); `fetchval`/`exists` work under
either a dict or tuple cursorclass, everything else stays dict-shaped only under the
default `DictCursor`.
## Versioning ## Versioning
+1 -1
View File
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
[project] [project]
name = "mysql" name = "mysql"
version = "0.1.2" version = "0.1.4"
description = "async mysql/mariadb wrapper over aiomysql: two-layer API (friendly verbs + raw escape hatch), fail-loud, config-free" description = "async mysql/mariadb wrapper over aiomysql: two-layer API (friendly verbs + raw escape hatch), fail-loud, config-free"
requires-python = ">=3.10" requires-python = ">=3.10"
dependencies = [ dependencies = [
+3 -1
View File
@@ -1,3 +1,5 @@
from .mysql import MysqlDB from .mysql import MysqlDB
__all__ = ["MysqlDB"] __version__ = "0.1.4"
__all__ = ["MysqlDB", "__version__"]
+73 -31
View File
@@ -33,18 +33,17 @@ call-site changes; only the dialect below differs, hidden inside the wrapper):
dialect (mysql-specific; visible only in raw layer-2 SQL you write): dialect (mysql-specific; visible only in raw layer-2 SQL you write):
- placeholders are `%s` (here) vs `$1` (psql). the `%s` is the DBAPI placeholder the - placeholders are `%s` (here) vs `$1` (psql). the `%s` is the DBAPI placeholder the
driver consumes, NOT python `%` string formatting. driver consumes, NOT python `%` string formatting — but PyMySQL builds it via
- upsert emits `ON DUPLICATE KEY UPDATE` (here) vs `ON CONFLICT` (psql); layer-1 `query % args` whenever params are passed, so any OTHER literal `%` in the query
upsert() hides this. text (e.g. `LIKE '%foo%'`) must be written `%%` or it raises. no params -> no
substitution -> a lone `%` is fine. layer 1 escapes `%` in identifiers for you.
- upsert emits `ON DUPLICATE KEY UPDATE` (here) vs `ON CONFLICT` (psql), via the
deprecated `col = VALUES(col)` form (MySQL 8.0.20+ warns; still works, MariaDB is
unaffected). layer-1 upsert() hides both.
rows: rows:
layer 1 and fetch/fetchone return plain dicts ({column: value}) via aiomysql's layer 1 and fetch/fetchone return plain dicts via aiomysql's DictCursor (identical
DictCursor — identical shape to the psql lib. shape to psql). fetchval/exists tolerate a caller-overridden tuple cursorclass too.
placeholders / injection safety:
values are ALWAYS parameterized — layer 1 builds `%s` internally; layer 2 takes your
`%s` placeholders + a params sequence. never f-string/%-format a value into SQL. only
identifiers (table/column names) are interpolated, and they are backtick-quoted.
errors (FAIL LOUD — unlike the mongo lib's swallow-and-default): errors (FAIL LOUD — unlike the mongo lib's swallow-and-default):
every method catches the driver error (pymysql.err.MySQLError, OSError on connection every method catches the driver error (pymysql.err.MySQLError, OSError on connection
@@ -53,6 +52,7 @@ errors (FAIL LOUD — unlike the mongo lib's swallow-and-default):
wrapped, use the raw `.pool` property (the aiomysql.Pool). wrapped, use the raw `.pool` property (the aiomysql.Pool).
""" """
import asyncio
import logging import logging
from typing import Any, Dict, List, Optional, Sequence from typing import Any, Dict, List, Optional, Sequence
@@ -64,13 +64,27 @@ log = logging.getLogger(__name__)
_DRIVER_ERRORS = (MySQLError, OSError) _DRIVER_ERRORS = (MySQLError, OSError)
def _first_value(row) -> Any:
"""return the first column's value from a dict or sequence row
fetchval/exists read a single scalar column and work under either cursorclass: the
default DictCursor (dict row) or a caller override to a tuple/sequence cursor via
pool_kwargs — the only two row shapes aiomysql cursors produce.
"""
return next(iter(row.values())) if isinstance(row, dict) else row[0]
def _quote_ident(identifier: str) -> str: def _quote_ident(identifier: str) -> str:
"""backtick-quote a sql identifier (table/column), escaping embedded backticks """backtick-quote a sql identifier (table/column), escaping backticks and `%`
identifiers can't be parameterized, so they are interpolated — backtick-quoting + identifiers can't be parameterized, so they are interpolated — backtick-quoting +
doubling any embedded backtick is the mysql-safe way to do that for caller names. doubling any embedded backtick is the mysql-safe way to do that for caller names. an
embedded `%` is also doubled: PyMySQL applies `query % args` whenever a call passes
params, and an unescaped `%` inside an interpolated identifier would hit that same
substitution and raise, not just literal `%` written by the caller (see the module
docstring's `%%`-escaping rule).
""" """
return "`" + identifier.replace("`", "``") + "`" return "`" + identifier.replace("`", "``").replace("%", "%%") + "`"
class MysqlDB: class MysqlDB:
@@ -92,9 +106,22 @@ class MysqlDB:
host/port/db/user/password/minsize/maxsize are injected by the caller. extra host/port/db/user/password/minsize/maxsize are injected by the caller. extra
pool_kwargs pass through to aiomysql.create_pool (unix_socket, ssl, charset, ...). pool_kwargs pass through to aiomysql.create_pool (unix_socket, ssl, charset, ...).
rows come back as dicts (DictCursor) and autocommit is on by default; both can be rows come back as dicts (DictCursor), overridable via pool_kwargs to a tuple
overridden via pool_kwargs. cursorclass — fetchval/exists work either way, everything else stays dict-shaped
only under DictCursor. autocommit is always on: layer-1 verbs and raw execute()
rely on it to commit their own writes without an explicit commit, so
`autocommit=False` in pool_kwargs is rejected outright — silently accepting it
would mean every write sits uncommitted until something else commits or the
connection is returned to the pool and rolled back, losing data with no error.
use transaction() for an atomic multi-statement block instead (it disables
autocommit for just that block).
""" """
if pool_kwargs.get("autocommit") is False:
raise ValueError(
"mysql: autocommit=False is not supported via pool_kwargs — layer-1 verbs and "
"raw execute() need autocommit to persist their writes; use db.transaction() "
"for an atomic multi-statement block instead"
)
self._config = dict( self._config = dict(
host=host, host=host,
port=port, port=port,
@@ -108,6 +135,7 @@ class MysqlDB:
self._config.setdefault("cursorclass", aiomysql.cursors.DictCursor) self._config.setdefault("cursorclass", aiomysql.cursors.DictCursor)
self._config.update(pool_kwargs) self._config.update(pool_kwargs)
self._pool: Optional[aiomysql.Pool] = None self._pool: Optional[aiomysql.Pool] = None
self._connect_lock = asyncio.Lock()
async def connect(self) -> "MysqlDB": async def connect(self) -> "MysqlDB":
"""build the pool and validate it with SELECT 1; fail loud on bad config """build the pool and validate it with SELECT 1; fail loud on bad config
@@ -115,8 +143,11 @@ class MysqlDB:
returns self so callers can write `db = await MysqlDB(...).connect()`. if called returns self so callers can write `db = await MysqlDB(...).connect()`. if called
again on an already-connected instance the previous pool is closed first (no again on an already-connected instance the previous pool is closed first (no
orphaned pool); if the SELECT-1 validation fails the freshly-built pool is torn orphaned pool); if the SELECT-1 validation fails the freshly-built pool is torn
down before re-raising, so a failed connect() never leaks a live pool. down before re-raising, so a failed connect() never leaks a live pool. concurrent
callers serialize on an internal lock so racing connect() calls build exactly one
pool instead of each leaking the losers' live pools.
""" """
async with self._connect_lock:
if self._pool is not None: if self._pool is not None:
await self.close() await self.close()
pool = await aiomysql.create_pool(**self._config) pool = await aiomysql.create_pool(**self._config)
@@ -143,6 +174,8 @@ class MysqlDB:
except _DRIVER_ERRORS: except _DRIVER_ERRORS:
log.exception("mysql.close()") log.exception("mysql.close()")
raise raise
finally:
self._pool = None
async def __aenter__(self) -> "MysqlDB": async def __aenter__(self) -> "MysqlDB":
return await self.connect() return await self.connect()
@@ -226,7 +259,7 @@ class MysqlDB:
raise raise
if not row: if not row:
return None return None
return next(iter(row.values())) return _first_value(row)
def transaction(self): def transaction(self):
"""async context manager running a block of statements atomically """async context manager running a block of statements atomically
@@ -341,16 +374,19 @@ class MysqlDB:
except _DRIVER_ERRORS: except _DRIVER_ERRORS:
log.exception("mysql.exists(%s)", table) log.exception("mysql.exists(%s)", table)
raise raise
return bool(next(iter(row.values()))) if row else False return bool(_first_value(row)) if row else False
async def upsert(self, table: str, values: Dict[str, Any], conflict: Sequence[str]) -> int: async def upsert(self, table: str, values: Dict[str, Any], conflict: Sequence[str]) -> int:
"""INSERT ... ON DUPLICATE KEY UPDATE — insert or update on unique/pk clash """INSERT ... ON DUPLICATE KEY UPDATE — insert or update on unique/pk clash
`conflict` is the list of columns forming the unique/pk constraint. mysql upserts `conflict` is the list of columns forming the unique/pk constraint. mysql upserts
on ANY unique-key clash (it does not name the constraint the way postgres does), so on ANY unique-key clash (it does not name the constraint the way postgres does), so
`conflict` is accepted for a call-signature identical to psql but the constraint is `conflict` is accepted for call-signature parity with psql but the constraint is
whatever unique/pk key the row violates. the wrapper emits ON DUPLICATE KEY UPDATE whatever unique/pk key the row violates. returns the affected rowcount. emits the
(psql emits ON CONFLICT for the identical call). returns the affected rowcount. `col = VALUES(col)` update form, deprecated since MySQL 8.0.20 (warning 1287, still
functional) and unaffected on MariaDB; a future MySQL-only build could probe
`SELECT VERSION()` and switch to alias syntax, not done here as a documented cost/
benefit call, not an oversight.
""" """
cols = list(values.keys()) cols = list(values.keys())
placeholders = ", ".join(["%s"] * len(cols)) placeholders = ", ".join(["%s"] * len(cols))
@@ -384,9 +420,8 @@ class _Transaction:
try: try:
await self._conn.begin() await self._conn.begin()
except BaseException: except BaseException:
# begin() failing after acquire would otherwise leak the pooled connection — # __aexit__ never runs if __aenter__ raises, so begin() failing after acquire
# __aexit__ is not called when __aenter__ raises. release it and reset so a # would otherwise leak the pooled connection — release it here instead.
# failed transaction start never burns a pool slot.
await self._pool.release(self._conn) await self._pool.release(self._conn)
self._conn = None self._conn = None
raise raise
@@ -399,19 +434,26 @@ class _Transaction:
else: else:
await self._conn.rollback() await self._conn.rollback()
finally: finally:
# aiomysql's release() returns a task (it schedules a _wakeup); await it so we # release() returns a task (schedules a _wakeup); await it to avoid leaving
# don't leave an un-awaited task each transaction matches psql's awaited # it un-awaited each transaction, matching psql's and aiomysql's own idiom.
# release and aiomysql's own idiom. the connection is returned synchronously
# inside release() regardless, so pool accounting is never at risk.
await self._pool.release(self._conn) await self._pool.release(self._conn)
def _where(conditions: Optional[Dict[str, Any]]) -> tuple: def _where(conditions: Optional[Dict[str, Any]]) -> tuple:
"""build a parameterized `WHERE col = %s AND ...` clause + the params list """build a parameterized `WHERE col = %s AND ...` clause + the params list
returns ("", []) when there are no conditions. equality only. returns ("", []) when there are no conditions. equality only. a None value renders as
`col IS NULL` (not `col = %s` bound to NULL, which sql never matches) and does not
consume a placeholder.
""" """
if not conditions: if not conditions:
return "", [] return "", []
clause = " AND ".join(f"{_quote_ident(c)} = %s" for c in conditions) parts = []
return f" WHERE {clause}", list(conditions.values()) params = []
for col, val in conditions.items():
if val is None:
parts.append(f"{_quote_ident(col)} IS NULL")
else:
params.append(val)
parts.append(f"{_quote_ident(col)} = %s")
return f" WHERE {' AND '.join(parts)}", params