10 Commits
Author SHA1 Message Date
dsql d827dca30f fix: is_file drift on decrypt_aes_key_with_rsa and non-str dict key coercion
self_test(is_file=False) only forwarded is_file to the public-key wrap;
decrypt_aes_key_with_rsa had no is_file parameter and always open()'d its
argument, so a PEM string was opened as a filename, misdiagnosing a good
keypair as non-pairing and leaking the private key PEM into the
FileNotFoundError traceback. decrypt_aes_key_with_rsa now takes is_file
(default True, preserving current callers), and self_test threads it
through to both key loads.

encrypt_data json.dumps a dict without checking key types, silently
stringifying int/float/bool/None keys (e.g. snowflake-int-keyed dicts),
so a decrypt round-trip silently lost the original key. encrypt_data now
raises TypeError on a non-str key instead of coercing it.

Signed-off-by: disqualifier <dev@disqualifier.me>
2026-07-02 17:17:44 -04:00
dsql 49af2a1143 fix: reencrypt/is_encrypted_record/decrypt_record miss self-blob and deep-nested records
Two silent-data-loss paths in the record-level functions: (1) reencrypt's
traversal_level cutoff silently left blobs nested deeper than the default
under the old key with no signal, contradicting its own documented fail-loud
rotation contract, and is_encrypted_record shared the cutoff so a
post-rotation audit couldn't detect the leftover; (2) all three record
functions inspected only record.values(), never the record itself, so a bare
{secure, iv, data} blob used as the whole document (the README's file-storage
pattern) was invisible to is_encrypted_record and passed through reencrypt
unchanged under the old key.

reencrypt now raises when a blob sits deeper than traversal_level instead of
silently truncating, and detects/handles the record-itself-is-a-blob case;
is_encrypted_record falls back to an unbounded-depth scan past
traversal_level so it reliably flags leftovers regardless of nesting depth;
decrypt_record likewise handles a record that is itself a blob. Bumped to
v0.1.4.

Signed-off-by: disqualifier <dev@disqualifier.me>
2026-07-02 16:40:37 -04:00
dsql e03622b175 chore: ignore .claude/ dir (CLAUDE.md now lives under .claude/)
Signed-off-by: disqualifier <dev@disqualifier.me>
2026-06-29 21:55:13 -04:00
dsql 306e5b8057 fix: EC-1 single key load for fingerprint; encrypt/decrypt type guards
EC-1: factor _fingerprint_of(public_key) so encrypt_aes_key_with_rsa fingerprints the
already-loaded key instead of re-opening/parsing the file. encrypt_data rejects a
non-dict/str with a clear TypeError (was: opaque .encode() AttributeError); decrypt_data
raises ValueError on a malformed blob (was: raw KeyError); a wrong-password PEM load gives
a clearer message; reencrypt's dict-only traversal (list-nested blobs skipped) documented.

Signed-off-by: disqualifier <dev@disqualifier.me>
2026-06-29 21:35:04 -04:00
dsql 254826f86c docs: pin install line to release, note unpinned-latest option
Signed-off-by: disqualifier <dev@disqualifier.me>
2026-06-29 18:13:53 -04:00
dsql 113a3e6949 docs: show unpinned install line; note tag-pinning for reproducibility
Signed-off-by: disqualifier <dev@disqualifier.me>
2026-06-29 18:07:38 -04:00
dsql 72c7aa936e fix: normalize key-load exceptions; type-faithful decrypt_data (v0.1.3)
- encrypted OpenSSH private key with no password now raises ValueError (not a raw
  TypeError from load_ssh_private_key), matching the PEM path and the docstring (L14)
- a non-PEM/non-SSH public key raises a clear ValueError instead of cryptography's
  UnsupportedAlgorithm, consistent with the private-key paths (L15)
- decrypt_data only treats a json-OBJECT plaintext as a dict, so json-shaped strings
  ('123','true','[1,2]') round-trip as strings; existing dict blobs unaffected (L16)
- both key loads route through shared _load_private_key/_load_public_key helpers
- document reencrypt's fail-loud (vs decrypt_record's per-field swallow) asymmetry (nit).

Signed-off-by: disqualifier <dev@disqualifier.me>
2026-06-29 17:58:26 -04:00
dsql 5de8b5d736 fix: OpenSSH private-key fingerprint fallback + clean error on missing password
get_rsa_key_fingerprint(is_private=True) only loaded PEM private keys, so an OpenSSH-format private key raised — unlike decrypt_aes_key_with_rsa, which already had the fallback. mirrored it: on a PEM load failure, an OPENSSH-marked key is loaded via load_ssh_private_key. also normalized the encrypted-key-without-password case: cryptography raises TypeError there, which now becomes a clear ValueError('private key is encrypted but no password was provided') in both methods instead of leaking the raw TypeError.

Signed-off-by: disqualifier <dev@disqualifier.me>
2026-06-29 01:39:21 -04:00
dsql 16205e810a fix: deepcopy in reencrypt/decrypt_record so input is not mutated
both used record.copy() (shallow), leaving unencrypted mutable fields shared between the input and the returned dict, violating the documented 'input is not mutated' contract. switched to copy.deepcopy.

Signed-off-by: disqualifier <dev@disqualifier.me>
2026-06-28 17:18:28 -04:00
dsql 313b0c7d56 fix: forward password to private-key fingerprinting (v0.1.1)
get_rsa_key_fingerprint(is_private=True) called load_pem_private_key(password=None),
so an encrypted private key raised a raw TypeError. add an optional password param
forwarded to the load; unencrypted keys ignore it.

verified: encrypted private key fingerprints with its password and matches the
public key's fingerprint; missing password still raises.

Signed-off-by: disqualifier <dev@disqualifier.me>
2026-06-28 15:53:04 -04:00
4 changed files with 272 additions and 59 deletions
+1 -1
View File
@@ -1,5 +1,5 @@
# claude # claude
CLAUDE.md .claude/
# python # python
__pycache__/ __pycache__/
+33 -5
View File
@@ -11,17 +11,19 @@ and storage-agnostic.
`requirements.txt`: `requirements.txt`:
``` ```
envelope_crypto @ git+ssh://git@git.rethinkstudios.io/rethink-public/envelope_crypto.git@v0.1.0 envelope_crypto @ git+ssh://git@git.rethinkstudios.io/rethink-public/envelope_crypto.git@v0.1.5
``` ```
Direct: Direct:
```bash ```bash
pip install "envelope_crypto @ git+ssh://git@git.rethinkstudios.io/rethink-public/envelope_crypto.git@v0.1.0" pip install "envelope_crypto @ git+ssh://git@git.rethinkstudios.io/rethink-public/envelope_crypto.git@v0.1.5"
``` ```
Requires `cryptography` (pulled transitively). Requires `cryptography` (pulled transitively).
Drop the `@v0.1.5` suffix from the line above to install the latest unpinned.
## First-time setup ## First-time setup
Run once, ever, to create the data key and authorize the first system. You need an Run once, ever, to create the data key and authorize the first system. You need an
@@ -64,6 +66,13 @@ bot.crypto = crypto
The `keys` schema (`_id` = fingerprint, `key` = wrapped) is the **caller's** choice; The `keys` schema (`_id` = fingerprint, `key` = wrapped) is the **caller's** choice;
this lib only produces `(fingerprint, wrapped_key)`. this lib only produces `(fingerprint, wrapped_key)`.
`decrypt_aes_key_with_rsa` (like `encrypt_aes_key_with_rsa` and
`get_rsa_key_fingerprint`) takes `is_file` (default `True`). Pass `is_file=False` to
hand it PEM/OpenSSH key data directly — e.g. a private key sourced from a vault —
instead of a file path. `self_test` threads the same `is_file` through to both the
public and private key it loads, so `self_test(pub_pem, priv_pem, is_file=False)`
round-trips two in-memory PEM strings rather than treating them as paths.
## Encrypt / decrypt ## Encrypt / decrypt
```python ```python
@@ -71,9 +80,16 @@ enc = crypto.encrypt_data({"ssn": "..."}) # -> {"secure": True, "iv": ...,
plain = crypto.decrypt_data(enc) # -> {"ssn": "..."} plain = crypto.decrypt_data(enc) # -> {"ssn": "..."}
``` ```
Dict keys must be `str`. `encrypt_data` raises `TypeError` on a non-str key (e.g. an
int-keyed dict of Discord snowflakes) instead of silently stringifying it — the
underlying JSON encoding has no other key type, so a coerced key would come back out
of `decrypt_data` as a `str` and no longer match the original lookup key.
For whole records: `decrypt_record(crypto, doc)` decrypts every `{secure, iv, data}` For whole records: `decrypt_record(crypto, doc)` decrypts every `{secure, iv, data}`
field (nested up to `traversal_level`, default 2); `is_encrypted_record(doc)` reports field (nested up to `traversal_level`, default 2); `is_encrypted_record(doc)` reports
whether any encrypted field exists. whether any encrypted field exists. Both also detect `doc` itself being a bare
`{secure, iv, data}` blob (the file-storage pattern below, where the blob IS the whole
document) — not just blobs nested under a key.
```python ```python
from envelope_crypto import is_encrypted_record, decrypt_record from envelope_crypto import is_encrypted_record, decrypt_record
@@ -82,6 +98,11 @@ if is_encrypted_record(doc):
doc = decrypt_record(crypto, doc) doc = decrypt_record(crypto, doc)
``` ```
`is_encrypted_record` falls back to an unbounded-depth scan once `traversal_level` is
exhausted, so it reliably reports `True` for a blob left behind by a shallower
`decrypt_record`/`reencrypt` call — safe to use as a leftover-detecting audit after
rotation, regardless of how deep the blob is nested.
Naming aliases (same objects): `EnvelopeCrypto` = `DocumentCrypto` = `RecordCrypto` Naming aliases (same objects): `EnvelopeCrypto` = `DocumentCrypto` = `RecordCrypto`
= `PCICrypto` (deprecated legacy alias). `decrypt_record` = `decrypt_document` = = `PCICrypto` (deprecated legacy alias). `decrypt_record` = `decrypt_document` =
`decrypt_dict`; `is_encrypted_record` = `is_encrypted_document` = `is_encrypted_dict`. `decrypt_dict`; `is_encrypted_record` = `is_encrypted_document` = `is_encrypted_dict`.
@@ -127,7 +148,14 @@ for fingerprint, wrapped_key in wrapped.items():
`reencrypt(source_crypto, record)` is a method on the **destination** (new-key) `reencrypt(source_crypto, record)` is a method on the **destination** (new-key)
instance: it decrypts each encrypted field with `source_crypto` (old key) and instance: it decrypts each encrypted field with `source_crypto` (old key) and
re-encrypts with itself. Only `{secure, ...}` fields are touched. re-encrypts with itself. Only `{secure, ...}` fields are touched — including `record`
itself if it IS a `{secure, iv, data}` blob (the file-storage pattern).
Rotation must fail loud: a per-field decrypt failure raises, and so does a blob nested
deeper than `traversal_level` — silently leaving it under the old key would strand it
once the old key's wrapped-key record is deleted below. If you nest blobs deeper than
the default `traversal_level=2`, pass a higher `traversal_level` or flatten the record
first.
## Storage patterns ## Storage patterns
@@ -152,4 +180,4 @@ The lib never touches a database; only the caller's storage layer differs.
## Versioning ## Versioning
Tagged `vX.Y.Z`. Pin the tag in `requirements.txt`. Releases are tagged `vX.Y.Z`. The install line above pins a release; drop the `@vX.Y.Z` suffix to install the latest unpinned. Pin deliberately for reproducible installs.
+1 -1
View File
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
[project] [project]
name = "envelope_crypto" name = "envelope_crypto"
version = "0.1.0" version = "0.1.5"
description = "Envelope encryption (RSA-OAEP wrapped AES-256-GCM) for dict records — config-free, storage-agnostic, installable." description = "Envelope encryption (RSA-OAEP wrapped AES-256-GCM) for dict records — config-free, storage-agnostic, installable."
requires-python = ">=3.10" requires-python = ">=3.10"
dependencies = [ dependencies = [
+232 -47
View File
@@ -44,22 +44,36 @@ set, then re-encrypt existing records old -> new:
for record in caller_iter(): for record in caller_iter():
caller_update(new_crypto.reencrypt(crypto, record)) caller_update(new_crypto.reencrypt(crypto, record))
reencrypt/is_encrypted_record/decrypt_record all detect a bare {secure, iv, data}
blob used AS the whole record (the file-storage pattern), not just blobs nested
under a key. reencrypt fails loud (raises) rather than silently leaving a field
under the old key — including when a blob sits deeper than `traversal_level`;
is_encrypted_record falls back to an unbounded-depth scan past traversal_level so
it reliably catches leftovers as a post-rotation audit.
config-free: the host supplies the DEK and RSA key paths; this lib never imports config-free: the host supplies the DEK and RSA key paths; this lib never imports
config, configures logging, or touches a database. storage-agnostic — the config, configures logging, or touches a database. storage-agnostic — the
encrypted blob is a plain dict; store it in mongo, a sql json column, or a file. encrypted blob is a plain dict; store it in mongo, a sql json column, or a file.
encrypt_data/decrypt_data round-trip a dict only when every key is a str —
json (the wire format under the hood) has no other key type, so encrypt_data
raises TypeError on a non-str key (e.g. an int-keyed dict of discord snowflakes)
rather than silently stringifying it and losing the original key on decrypt.
naming: EnvelopeCrypto is canonical. PCICrypto / DocumentCrypto / RecordCrypto are naming: EnvelopeCrypto is canonical. PCICrypto / DocumentCrypto / RecordCrypto are
aliases (PCICrypto is a deprecated legacy alias). the document/record/dict aliases (PCICrypto is a deprecated legacy alias). the document/record/dict
function variants are the same functions — use whichever fits your storage. function variants are the same functions — use whichever fits your storage.
""" """
import os import os
import copy
import json import json
import base64 import base64
import hashlib import hashlib
import logging import logging
from typing import Any, Dict, List, Optional, Tuple, Union from typing import Any, Dict, List, Optional, Tuple, Union
from cryptography.exceptions import UnsupportedAlgorithm
from cryptography.hazmat.primitives import hashes, serialization from cryptography.hazmat.primitives import hashes, serialization
from cryptography.hazmat.primitives.asymmetric import padding from cryptography.hazmat.primitives.asymmetric import padding
from cryptography.hazmat.primitives.ciphers.aead import AESGCM from cryptography.hazmat.primitives.ciphers.aead import AESGCM
@@ -68,6 +82,83 @@ from cryptography.hazmat.primitives.serialization import load_ssh_public_key
_log = logging.getLogger(__name__) _log = logging.getLogger(__name__)
def _load_private_key(key_data: bytes, pw: Optional[bytes]):
"""load a PEM or OpenSSH private key, normalizing the missing-password error
cryptography raises TypeError when a key is encrypted but no password was given
(PEM raises it on the first call; OpenSSH raises it inside the openssh fallback).
both are normalized to a clear ValueError so callers see one error type.
"""
try:
return serialization.load_pem_private_key(key_data, password=pw)
except ValueError as error:
if b"BEGIN OPENSSH PRIVATE KEY" in key_data:
try:
return serialization.load_ssh_private_key(key_data, password=pw)
except TypeError as ssh_error:
raise ValueError(
"private key is encrypted but no password was provided"
) from ssh_error
if pw is not None:
# a password was given but the PEM load still failed — most likely a wrong
# password; give a clearer message than cryptography's raw "Bad decrypt"
raise ValueError("could not load private key (wrong password or malformed key)") from error
raise error
except TypeError as error:
raise ValueError(
"private key is encrypted but no password was provided"
) from error
def _fingerprint_of(public_key) -> str:
"""base64 SHA-256 fingerprint of an already-loaded public key
factored so callers that already hold a loaded key (e.g. encrypt_aes_key_with_rsa)
don't re-open and re-parse the key file just to fingerprint it.
"""
key_bytes = public_key.public_bytes(
encoding=serialization.Encoding.DER,
format=serialization.PublicFormat.SubjectPublicKeyInfo,
)
digest = hashes.Hash(hashes.SHA256())
digest.update(key_bytes)
return base64.b64encode(digest.finalize()).decode()
def _is_blob(value: Any) -> bool:
"""return whether value has the {secure, iv, data} encrypted-blob shape"""
return isinstance(value, dict) and value.get("secure") is True and "iv" in value and "data" in value
def _has_encrypted_field(record: Any) -> bool:
"""unbounded-depth scan: does record (or anything nested under it) contain a blob
used to detect a blob left behind by a depth-limited traversal — no traversal_level
cutoff here, since the whole point is to catch what a bounded pass would miss.
"""
if not isinstance(record, dict):
return False
if _is_blob(record):
return True
return any(_has_encrypted_field(value) for value in record.values())
def _load_public_key(key_data: bytes):
"""load a PEM or OpenSSH public key, normalizing non-key input to ValueError
load_ssh_public_key raises UnsupportedAlgorithm (not ValueError) on non-SSH/garbage
input; normalize it so a bad public key always surfaces as a clear ValueError,
consistent with the private-key path.
"""
try:
return serialization.load_pem_public_key(key_data)
except ValueError:
try:
return load_ssh_public_key(key_data)
except (UnsupportedAlgorithm, ValueError) as error:
raise ValueError("not a valid PEM or OpenSSH public key") from error
class EnvelopeCrypto: class EnvelopeCrypto:
"""hybrid RSA/AES-256-GCM envelope encryption for dict records """hybrid RSA/AES-256-GCM envelope encryption for dict records
@@ -102,7 +193,11 @@ class EnvelopeCrypto:
round-trips sample data through this instance's DEK, then wraps the DEK round-trips sample data through this instance's DEK, then wraps the DEK
with the public key and unwraps with the private key, confirming they with the public key and unwraps with the private key, confirming they
match. run after bootstrap (or anytime as a health check) to catch a bad match. run after bootstrap (or anytime as a health check) to catch a bad
keypair or wrong key path before relying on it. returns True on success. keypair or wrong key path before relying on it. is_file (default True) is
threaded through to both the public and private key loads, so
is_file=False correctly treats both rsa_public_key and rsa_private_key as
in-memory PEM/OpenSSH data rather than opening either as a file path.
returns True on success.
""" """
if not self.master_key: if not self.master_key:
raise ValueError("self_test: not initialized with a key") raise ValueError("self_test: not initialized with a key")
@@ -113,7 +208,7 @@ class EnvelopeCrypto:
_, wrapped = self.encrypt_aes_key_with_rsa(self.master_key, rsa_public_key, is_file=is_file) _, wrapped = self.encrypt_aes_key_with_rsa(self.master_key, rsa_public_key, is_file=is_file)
try: try:
recovered = self.decrypt_aes_key_with_rsa(wrapped, rsa_private_key, password=password) recovered = self.decrypt_aes_key_with_rsa(wrapped, rsa_private_key, is_file=is_file, password=password)
except Exception as error: except Exception as error:
raise RuntimeError( raise RuntimeError(
"self_test: key unwrap failed (public/private keys do not pair, " "self_test: key unwrap failed (public/private keys do not pair, "
@@ -147,9 +242,18 @@ class EnvelopeCrypto:
return key return key
def get_rsa_key_fingerprint( def get_rsa_key_fingerprint(
self, key_path_or_data: str, is_private: bool = False, is_file: bool = True self, key_path_or_data: str, is_private: bool = False, is_file: bool = True,
password: Optional[str] = None,
) -> str: ) -> str:
"""return a base64 SHA-256 fingerprint of an RSA key for identification""" """return a base64 SHA-256 fingerprint of an RSA key for identification
for an encrypted private key (is_private=True), pass its `password`; an
unencrypted key ignores it. fingerprinting always uses the public half, so a
private and its public key produce the same fingerprint. PEM and OpenSSH
private-key formats are both accepted (mirrors decrypt_aes_key_with_rsa). an
encrypted key with no/wrong password raises ValueError with a clear message
(cryptography raises TypeError for the missing-password case — normalized here).
"""
if is_file: if is_file:
with open(key_path_or_data, "rb") as key_file: with open(key_path_or_data, "rb") as key_file:
key_data = key_file.read() key_data = key_file.read()
@@ -161,21 +265,13 @@ class EnvelopeCrypto:
) )
if is_private: if is_private:
private_key = serialization.load_pem_private_key(key_data, password=None) pw = password.encode() if password else None
private_key = _load_private_key(key_data, pw)
public_key = private_key.public_key() public_key = private_key.public_key()
else: else:
try: public_key = _load_public_key(key_data)
public_key = serialization.load_pem_public_key(key_data)
except ValueError:
public_key = load_ssh_public_key(key_data)
key_bytes = public_key.public_bytes( fingerprint = _fingerprint_of(public_key)
encoding=serialization.Encoding.DER,
format=serialization.PublicFormat.SubjectPublicKeyInfo,
)
digest = hashes.Hash(hashes.SHA256())
digest.update(key_bytes)
fingerprint = base64.b64encode(digest.finalize()).decode()
_log.info("generated %s key fingerprint", "private" if is_private else "public") _log.info("generated %s key fingerprint", "private" if is_private else "public")
return fingerprint return fingerprint
@@ -189,10 +285,7 @@ class EnvelopeCrypto:
else: else:
key_data = rsa_key.encode() if isinstance(rsa_key, str) else rsa_key key_data = rsa_key.encode() if isinstance(rsa_key, str) else rsa_key
try: public_key = _load_public_key(key_data)
public_key = serialization.load_pem_public_key(key_data)
except ValueError:
public_key = load_ssh_public_key(key_data)
wrapped = public_key.encrypt( wrapped = public_key.encrypt(
aes_key, aes_key,
@@ -202,29 +295,30 @@ class EnvelopeCrypto:
label=None, label=None,
), ),
) )
fingerprint = self.get_rsa_key_fingerprint(rsa_key, is_private=False, is_file=is_file) # fingerprint from the already-loaded public_key — no second open/parse of the file
fingerprint = _fingerprint_of(public_key)
wrapped_b64 = base64.b64encode(wrapped).decode() wrapped_b64 = base64.b64encode(wrapped).decode()
_log.info("wrapped data key for fingerprint %s", fingerprint[:8]) _log.info("wrapped data key for fingerprint %s", fingerprint[:8])
return fingerprint, wrapped_b64 return fingerprint, wrapped_b64
def decrypt_aes_key_with_rsa( def decrypt_aes_key_with_rsa(
self, encrypted_key_base64: str, rsa_private_key_path: str, self, encrypted_key_base64: str, rsa_private_key: str,
password: Optional[str] = None, is_file: bool = True, password: Optional[str] = None,
) -> bytes: ) -> bytes:
"""unwrap an AES key with an RSA private key""" """unwrap an AES key with an RSA private key
with open(rsa_private_key_path, "rb") as key_file:
is_file defaults to True (rsa_private_key is a path), matching
encrypt_aes_key_with_rsa / get_rsa_key_fingerprint; pass is_file=False to
supply the PEM/OpenSSH key data directly (e.g. an in-memory or vault-sourced
key) instead of a file path.
"""
if is_file:
with open(rsa_private_key, "rb") as key_file:
key_data = key_file.read() key_data = key_file.read()
try:
private_key = serialization.load_pem_private_key(
key_data, password=password.encode() if password else None
)
except ValueError as error:
if b"BEGIN OPENSSH PRIVATE KEY" in key_data:
private_key = serialization.load_ssh_private_key(
key_data, password=password.encode() if password else None
)
else: else:
raise error key_data = rsa_private_key.encode() if isinstance(rsa_private_key, str) else rsa_private_key
pw = password.encode() if password else None
private_key = _load_private_key(key_data, pw)
wrapped = base64.b64decode(encrypted_key_base64) wrapped = base64.b64decode(encrypted_key_base64)
aes_key = private_key.decrypt( aes_key = private_key.decrypt(
@@ -267,9 +361,30 @@ class EnvelopeCrypto:
return new_key, wrapped return new_key, wrapped
def encrypt_data(self, data: Union[Dict[str, Any], str]) -> Dict[str, str]: def encrypt_data(self, data: Union[Dict[str, Any], str]) -> Dict[str, str]:
"""encrypt a dict or string under the data key with a unique IV""" """encrypt a dict or string under the data key with a unique IV
dict keys must be str: json.dumps silently stringifies int/float/bool/None
keys (e.g. a snowflake-int-keyed dict), which would make decrypt_data return
a dict that no longer matches the original by key type or identity — a
non-str key is rejected here instead, so the failure is loud at encrypt
time rather than a silent lookup miss after decrypt.
"""
if not self.master_key: if not self.master_key:
raise ValueError("not initialized with data key") raise ValueError("not initialized with data key")
if not isinstance(data, (dict, str)):
# a non-dict/non-str would .encode()-fail with an opaque AttributeError below;
# reject it clearly (decrypt_data only round-trips dict or str anyway)
raise TypeError(f"encrypt_data expects a dict or str, got {type(data).__name__}")
if isinstance(data, dict):
non_str_keys = [key for key in data if not isinstance(key, str)]
if non_str_keys:
# json.dumps would silently stringify these (int/float/bool/None keys),
# corrupting the round-trip (decrypt_data would return a dict keyed by
# the stringified value) — fail loud instead of coercing
raise TypeError(
"encrypt_data requires str dict keys, got non-str key(s): "
f"{[type(key).__name__ for key in non_str_keys]}"
)
data_str = json.dumps(data) if isinstance(data, dict) else data data_str = json.dumps(data) if isinstance(data, dict) else data
iv = os.urandom(12) iv = os.urandom(12)
@@ -282,18 +397,36 @@ class EnvelopeCrypto:
} }
def decrypt_data(self, encrypted_data: Dict[str, str]) -> Union[Dict[str, Any], str]: def decrypt_data(self, encrypted_data: Dict[str, str]) -> Union[Dict[str, Any], str]:
"""decrypt a {secure, iv, data} blob; returns the original dict or string""" """decrypt a {secure, iv, data} blob; returns the original dict or string
`encrypt_data` only json-encodes dicts (a string is stored verbatim), so decrypt
only treats a json-OBJECT plaintext as a dict and returns everything else as the
raw string. this keeps the type faithful for the common cases: a string that is
json-shaped but NOT an object ('123'->'123', 'true'->'true', '[1,2]'->'[1,2]')
round-trips as a STRING, not an int/bool/list. the one irreducible ambiguity: a
string whose exact value is a json OBJECT ('{"a":1}') decrypts to a dict, because
without a type marker it is indistinguishable from a stored dict — don't store a
bare string that is a json object if you need it back as a string. existing stored
blobs are unaffected — a dict was stored as a json object and still parses to a dict.
dict KEYS round-trip faithfully only because encrypt_data now requires str keys —
json has no other key type, so this is the only shape decrypt_data ever sees.
"""
if not self.master_key: if not self.master_key:
raise ValueError("not initialized with data key") raise ValueError("not initialized with data key")
if not isinstance(encrypted_data, dict) or "iv" not in encrypted_data or "data" not in encrypted_data:
# a structurally-malformed blob would raise a raw KeyError/TypeError; surface
# a clear ValueError instead, matching the documented {secure, iv, data} shape
raise ValueError("decrypt_data expects a {secure, iv, data} blob")
iv = base64.b64decode(encrypted_data["iv"]) iv = base64.b64decode(encrypted_data["iv"])
ciphertext = base64.b64decode(encrypted_data["data"]) ciphertext = base64.b64decode(encrypted_data["data"])
aesgcm = AESGCM(self.master_key) aesgcm = AESGCM(self.master_key)
plaintext = aesgcm.decrypt(iv, ciphertext, None).decode() plaintext = aesgcm.decrypt(iv, ciphertext, None).decode()
try: try:
return json.loads(plaintext) parsed = json.loads(plaintext)
except json.JSONDecodeError: except json.JSONDecodeError:
return plaintext return plaintext
return parsed if isinstance(parsed, dict) else plaintext
def reencrypt(self, source_crypto: "EnvelopeCrypto", record: dict, traversal_level: int = 2) -> dict: def reencrypt(self, source_crypto: "EnvelopeCrypto", record: dict, traversal_level: int = 2) -> dict:
"""re-encrypt a record's encrypted fields from source_crypto's key to this one's """re-encrypt a record's encrypted fields from source_crypto's key to this one's
@@ -301,16 +434,43 @@ class EnvelopeCrypto:
self holds the destination (new) key; source_crypto holds the source (old) self holds the destination (new) key; source_crypto holds the source (old)
key. only {secure, iv, data} fields are touched; plaintext fields are left key. only {secure, iv, data} fields are touched; plaintext fields are left
as-is. returns a new dict; the input is not mutated. used during rotation. as-is. returns a new dict; the input is not mutated. used during rotation.
if `record` itself is a {secure, iv, data} blob (the file-storage pattern, where
the blob IS the whole document) it is re-encrypted directly and the result is
returned in place of `record` — not nested under a key.
unlike decrypt_record (which logs a failed field and leaves it encrypted), a
per-field decrypt failure here RAISES — rotation must fail loud, since silently
keeping a field under the old key would lose it once the old key is retired. for
the same reason, a blob nested DEEPER than `traversal_level` also RAISES instead
of being silently left under the old key: the caller either needs a higher
`traversal_level` or must flatten the record before rotation.
traversal recurses into nested DICTS only; a blob nested inside a LIST is NOT
re-encrypted and is NOT covered by the depth-limit raise above. records in this
scheme key blobs by field name, not inside arrays, so this doesn't arise in
practice — but if you store list-nested blobs, flatten them to dict fields before
rotation or they'll be silently left under the old key.
""" """
if not self.master_key: if not self.master_key:
raise ValueError("destination not initialized with data key") raise ValueError("destination not initialized with data key")
result = record.copy() if _is_blob(record):
return self.encrypt_data(source_crypto.decrypt_data(record))
result = copy.deepcopy(record)
for key, value in record.items(): for key, value in record.items():
if isinstance(value, dict) and value.get("secure") is True and "iv" in value and "data" in value: if _is_blob(value):
result[key] = self.encrypt_data(source_crypto.decrypt_data(value)) result[key] = self.encrypt_data(source_crypto.decrypt_data(value))
elif traversal_level > 0 and isinstance(value, dict): elif isinstance(value, dict):
if traversal_level > 0:
result[key] = self.reencrypt(source_crypto, value, traversal_level - 1) result[key] = self.reencrypt(source_crypto, value, traversal_level - 1)
elif _has_encrypted_field(value):
raise ValueError(
f"reencrypt: field {key!r} contains an encrypted blob nested deeper "
"than traversal_level; increase traversal_level or flatten the record "
"before rotation — leaving it would strand the field under the old key"
)
return result return result
@@ -323,14 +483,24 @@ PCICrypto = EnvelopeCrypto # deprecated legacy alias; remove after all systems
def is_encrypted_record(record, traversal_level: int = 2) -> bool: def is_encrypted_record(record, traversal_level: int = 2) -> bool:
"""return whether a record has any encrypted ({secure, iv, data}) fields """return whether a record has any encrypted ({secure, iv, data}) fields
checks `record` itself (the file-storage pattern stores the blob AS the whole
document, not nested under a key) as well as fields up to `traversal_level` deep.
beyond that bounded pass, this ALSO does an unbounded-depth scan before giving up —
so a blob left behind by a shallower decrypt_record/reencrypt call (nested deeper
than their traversal_level) is still reported as encrypted. this makes the function
safe to use as a leftover-detecting post-rotation audit: it never returns False for
a record that still contains a blob, at any depth.
aliases: is_encrypted_document, is_encrypted_dict — same function aliases: is_encrypted_document, is_encrypted_dict — same function
""" """
if not isinstance(record, dict): if not isinstance(record, dict):
return False return False
if _is_blob(record):
return True
for value in record.values(): for value in record.values():
if isinstance(value, dict) and value.get("secure") is True: if _is_blob(value):
if "iv" in value and "data" in value:
return True return True
if traversal_level > 0: if traversal_level > 0:
@@ -339,12 +509,20 @@ def is_encrypted_record(record, traversal_level: int = 2) -> bool:
return True return True
return False return False
return any(_has_encrypted_field(value) for value in record.values())
def decrypt_record(crypto: EnvelopeCrypto, record, traversal_level: int = 2) -> dict:
def decrypt_record(crypto: EnvelopeCrypto, record, traversal_level: int = 2) -> Union[dict, Any]:
"""decrypt a record's encrypted fields into a new dict (up to traversal_level deep) """decrypt a record's encrypted fields into a new dict (up to traversal_level deep)
if `record` itself is a {secure, iv, data} blob (the file-storage pattern, where the
blob IS the whole document) it is decrypted directly and the decrypted value
(dict or string — see decrypt_data) is returned in place of `record`.
failures on a single field are logged and that field is left encrypted, so a failures on a single field are logged and that field is left encrypted, so a
partial failure is visible (the {secure,...} blob remains) rather than silent. partial failure is visible (the {secure,...} blob remains) rather than silent. a
failure decrypting `record` itself (the self-blob case above) is likewise logged
and the still-encrypted blob is returned unchanged.
aliases: decrypt_document, decrypt_dict — same function aliases: decrypt_document, decrypt_dict — same function
""" """
@@ -353,9 +531,16 @@ def decrypt_record(crypto: EnvelopeCrypto, record, traversal_level: int = 2) ->
if not isinstance(record, dict): if not isinstance(record, dict):
return record return record
result = record.copy() if _is_blob(record):
try:
return crypto.decrypt_data(record)
except Exception:
_log.exception("failed to decrypt record")
return copy.deepcopy(record)
result = copy.deepcopy(record)
for key, value in record.items(): for key, value in record.items():
if isinstance(value, dict) and value.get("secure") is True and "iv" in value and "data" in value: if _is_blob(value):
try: try:
result[key] = crypto.decrypt_data(value) result[key] = crypto.decrypt_data(value)
except Exception: except Exception: