2 Commits
Author SHA1 Message Date
dsqlandClaude Opus 4.8 8a220a8810 add package: pyproject + src
EnvelopeCrypto: hybrid envelope encryption for dict records — a random
AES-256-GCM data key (DEK) encrypts the data, wrapped per-system via
RSA-OAEP (SHA-256) for distribution. config-free (DEK + key paths
injected), storage-agnostic, object-only. covers bootstrap/self_test,
authorize/deauthorize, rotate + reencrypt, and record-level decrypt.
src/ layout, hatchling build, cryptography backend.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: disqualifier <dev@disqualifier.me>
2026-06-24 17:07:06 -04:00
dsql 1864612d64 init: envelope encryption (RSA-OAEP + AES-256-GCM) for dict records
Signed-off-by: disqualifier <dev@disqualifier.me>
2026-06-24 17:07:06 -04:00
4 changed files with 90 additions and 374 deletions
+1 -1
View File
@@ -1,5 +1,5 @@
# claude
.claude/
CLAUDE.md
# python
__pycache__/
+9 -49
View File
@@ -11,19 +11,17 @@ and storage-agnostic.
`requirements.txt`:
```
envelope_crypto @ git+ssh://git@git.rethinkstudios.io/rethink-public/envelope_crypto.git@v0.1.6
envelope_crypto @ git+ssh://git@git.rethinkstudios.io/rethink-public/envelope_crypto.git@v0.1.0
```
Direct:
```bash
pip install "envelope_crypto @ git+ssh://git@git.rethinkstudios.io/rethink-public/envelope_crypto.git@v0.1.6"
pip install "envelope_crypto @ git+ssh://git@git.rethinkstudios.io/rethink-public/envelope_crypto.git@v0.1.0"
```
Requires `cryptography` (pulled transitively).
Drop the `@v0.1.6` suffix from the line above to install the latest unpinned.
## First-time setup
Run once, ever, to create the data key and authorize the first system. You need an
@@ -39,10 +37,10 @@ openssl rsa -in local_priv.pem -pubout -out local_pub.pem
from envelope_crypto import EnvelopeCrypto
# generate the DEK and wrap it for this system in one call
crypto, fingerprint, wrapped = EnvelopeCrypto.bootstrap("public_key.pem")
crypto, fingerprint, wrapped = EnvelopeCrypto.bootstrap(cfg.local_pub)
# verify the keypair actually round-trips BEFORE storing anything
crypto.self_test("public_key.pem", "private_key.pem") # raises if keys don't pair
crypto.self_test(cfg.local_pub, cfg.local_priv) # raises if keys don't pair
# store the wrapped key — this is now the ONLY record of the DEK
await db.create_document("keys", {"_id": fingerprint, "key": wrapped})
@@ -55,24 +53,17 @@ re-derived each boot by unwrapping. **Never persist the plaintext key.**
```python
crypto = EnvelopeCrypto()
fingerprint = crypto.get_rsa_key_fingerprint("public_key.pem")
fingerprint = crypto.get_rsa_key_fingerprint(cfg.local_pub)
record = await db.get_document("keys", {"_id": fingerprint})
if not record:
raise RuntimeError("this system is not authorized")
crypto.initialize(crypto.decrypt_aes_key_with_rsa(record["key"], "private_key.pem"))
crypto.initialize(crypto.decrypt_aes_key_with_rsa(record["key"], cfg.local_priv))
bot.crypto = crypto
```
The `keys` schema (`_id` = fingerprint, `key` = wrapped) is the **caller's** choice;
this lib only produces `(fingerprint, wrapped_key)`.
`decrypt_aes_key_with_rsa` (like `encrypt_aes_key_with_rsa` and
`get_rsa_key_fingerprint`) takes `is_file` (default `True`). Pass `is_file=False` to
hand it PEM/OpenSSH key data directly — e.g. a private key sourced from a vault —
instead of a file path. `self_test` threads the same `is_file` through to both the
public and private key it loads, so `self_test(pub_pem, priv_pem, is_file=False)`
round-trips two in-memory PEM strings rather than treating them as paths.
## Encrypt / decrypt
```python
@@ -80,16 +71,9 @@ enc = crypto.encrypt_data({"ssn": "..."}) # -> {"secure": True, "iv": ...,
plain = crypto.decrypt_data(enc) # -> {"ssn": "..."}
```
Dict keys must be `str`. `encrypt_data` raises `TypeError` on a non-str key (e.g. an
int-keyed dict of Discord snowflakes) instead of silently stringifying it — the
underlying JSON encoding has no other key type, so a coerced key would come back out
of `decrypt_data` as a `str` and no longer match the original lookup key.
For whole records: `decrypt_record(crypto, doc)` decrypts every `{secure, iv, data}`
field (nested up to `traversal_level`, default 2); `is_encrypted_record(doc)` reports
whether any encrypted field exists. Both also detect `doc` itself being a bare
`{secure, iv, data}` blob (the file-storage pattern below, where the blob IS the whole
document) — not just blobs nested under a key.
whether any encrypted field exists.
```python
from envelope_crypto import is_encrypted_record, decrypt_record
@@ -98,11 +82,6 @@ if is_encrypted_record(doc):
doc = decrypt_record(crypto, doc)
```
`is_encrypted_record` falls back to an unbounded-depth scan once `traversal_level` is
exhausted, so it reliably reports `True` for a blob left behind by a shallower
`decrypt_record`/`reencrypt` call — safe to use as a leftover-detecting audit after
rotation, regardless of how deep the blob is nested.
Naming aliases (same objects): `EnvelopeCrypto` = `DocumentCrypto` = `RecordCrypto`
= `PCICrypto` (deprecated legacy alias). `decrypt_record` = `decrypt_document` =
`decrypt_dict`; `is_encrypted_record` = `is_encrypted_document` = `is_encrypted_dict`.
@@ -148,14 +127,7 @@ for fingerprint, wrapped_key in wrapped.items():
`reencrypt(source_crypto, record)` is a method on the **destination** (new-key)
instance: it decrypts each encrypted field with `source_crypto` (old key) and
re-encrypts with itself. Only `{secure, ...}` fields are touched — including `record`
itself if it IS a `{secure, iv, data}` blob (the file-storage pattern).
Rotation must fail loud: a per-field decrypt failure raises, and so does a blob nested
deeper than `traversal_level` — silently leaving it under the old key would strand it
once the old key's wrapped-key record is deleted below. If you nest blobs deeper than
the default `traversal_level=2`, pass a higher `traversal_level` or flatten the record
first.
re-encrypts with itself. Only `{secure, ...}` fields are touched.
## Storage patterns
@@ -177,19 +149,7 @@ The lib never touches a database; only the caller's storage layer differs.
- The scheme is envelope/hybrid encryption (AES-256-GCM data key wrapped by RSA-OAEP).
Using it does not by itself confer PCI-DSS or any other compliance — that is a
whole-system property.
- `initialize(master_key)` requires exactly 32 bytes (`bytes`, `len == 32`) — a
16/24-byte key or a `str` raises `ValueError` instead of silently downgrading to
AES-128/192 or failing late at first encrypt.
- Wrap/unwrap (`encrypt_aes_key_with_rsa`, `decrypt_aes_key_with_rsa`,
`authorize_system`, `bootstrap`, `rotate_master_key`) is **RSA-only**. A non-RSA
key (e.g. Ed25519/EC) loads and fingerprints fine but raises a clear `ValueError`
at wrap/unwrap rather than a raw `AttributeError`.
- `fingerprint_data` serializes non-JSON-native values (datetime/date/time,
bytes/bytearray, ObjectId-like objects) via a stable `default=` handler instead of
raising `TypeError`, and type-tags dict keys internally so `{1: "a"}` and
`{"1": "a"}` no longer collide to the same fingerprint. Never logs the data it
fingerprints.
## Versioning
Releases are tagged `vX.Y.Z`. The install line above pins a release; drop the `@vX.Y.Z` suffix to install the latest unpinned. Pin deliberately for reproducible installs.
Tagged `vX.Y.Z`. Pin the tag in `requirements.txt`.
+1 -1
View File
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
[project]
name = "envelope_crypto"
version = "0.1.6"
version = "0.1.0"
description = "Envelope encryption (RSA-OAEP wrapped AES-256-GCM) for dict records — config-free, storage-agnostic, installable."
requires-python = ">=3.10"
dependencies = [
+79 -323
View File
@@ -1,41 +1,40 @@
"""
envelope encryption for dict records
hybrid encryption: a random AES-256-GCM data key (DEK) encrypts the data, wrapped
(RSA-OAEP) per authorized system's public key (KEK) for distribution. the wrapped
key is stored by the caller, keyed by fingerprint; each system unwraps its own copy
with its private key. same pattern KMS-style systems use. RSA-envelope only — a
non-RSA key (e.g. Ed25519/EC) loads and fingerprints fine but raises ValueError at
wrap/unwrap. never logs key material (DEK, PEM, wrapped key) — only fingerprints
and counts.
hybrid encryption: a random AES-256-GCM data key (DEK) encrypts the data, and
that key is wrapped (RSA-OAEP) per authorized system's public key (KEK) for
distribution. the wrapped key is stored by the caller, keyed by fingerprint;
each system unwraps its own copy with its private key. this is the same
envelope-encryption pattern used by KMS-style systems.
from envelope_crypto import EnvelopeCrypto
crypto = EnvelopeCrypto()
crypto.initialize(master_key) # exactly 32 bytes (AES-256)
crypto.initialize(master_key) # 32-byte AES DEK
enc = crypto.encrypt_data({"ssn": "..."}) # -> {secure, iv, data}
plain = crypto.decrypt_data(enc) # -> original
first-time setup: generate the DEK and wrap it for the first system in one call,
then verify the pipeline before storing anything:
crypto, fingerprint, wrapped = EnvelopeCrypto.bootstrap("public_key.pem")
crypto.self_test("public_key.pem", "private_key.pem") # raises if anything is wrong
crypto, fingerprint, wrapped = EnvelopeCrypto.bootstrap(cfg.local_pub)
crypto.self_test(cfg.local_pub, cfg.local_priv) # raises if anything is wrong
caller_store({"_id": fingerprint, "key": wrapped}) # the only record of the DEK
boot (already set up): fingerprint own pubkey, fetch the wrapped DEK, unwrap:
fp = crypto.get_rsa_key_fingerprint("public_key.pem")
fp = crypto.get_rsa_key_fingerprint(cfg.local_pub)
record = caller_lookup(fp)
crypto.initialize(crypto.decrypt_aes_key_with_rsa(record["key"], "private_key.pem"))
crypto.initialize(crypto.decrypt_aes_key_with_rsa(record["key"], cfg.local_priv))
authorize another system (this instance must already hold the DEK):
fp, wrapped = crypto.authorize_system(other_pub_path)
caller_store({"_id": fp, "key": wrapped})
deauthorize: caller deletes that fingerprint's record. stops future unwraps but
does not revoke a DEK already in a running system's memory — rotate if compromised.
deauthorize: caller deletes that fingerprint's record. note this stops future
unwraps but does not revoke a DEK already in a running system's memory — rotate
if compromised.
rotate (new DEK + re-encrypt): generate a new DEK, wrap for the still-authorized
set, then re-encrypt existing records old -> new:
@@ -45,141 +44,30 @@ set, then re-encrypt existing records old -> new:
for record in caller_iter():
caller_update(new_crypto.reencrypt(crypto, record))
reencrypt/is_encrypted_record/decrypt_record all detect a bare {secure, iv, data}
blob used AS the whole record (file-storage pattern), not just blobs nested under a
key. reencrypt fails loud (raises) rather than silently leaving a field under the
old key — including a blob nested deeper than `traversal_level`; is_encrypted_record
falls back to an unbounded-depth scan past traversal_level to reliably catch
leftovers as a post-rotation audit.
config-free: the host supplies the DEK and RSA key paths; this lib never imports
config, configures logging, or touches a database. storage-agnostic — the
encrypted blob is a plain dict; store it in mongo, a sql json column, or a file.
encrypt_data/decrypt_data round-trip a dict only when every key is a str — json
(the wire format) has no other key type, so encrypt_data raises TypeError on a
non-str key (e.g. an int-keyed dict of discord snowflakes) rather than silently
stringifying it and losing the original key on decrypt.
naming: EnvelopeCrypto is canonical. PCICrypto / DocumentCrypto / RecordCrypto are
aliases (PCICrypto is a deprecated legacy alias). the document/record/dict function
variants are the same functions — use whichever fits your storage.
aliases (PCICrypto is a deprecated legacy alias). the document/record/dict
function variants are the same functions — use whichever fits your storage.
"""
import os
import copy
import json
import base64
import hashlib
import logging
from typing import Any, Dict, List, Optional, Tuple, Union
from cryptography.exceptions import UnsupportedAlgorithm
from cryptography.hazmat.primitives import hashes, serialization
from cryptography.hazmat.primitives.asymmetric import padding, rsa
from cryptography.hazmat.primitives.asymmetric import padding
from cryptography.hazmat.primitives.ciphers.aead import AESGCM
from cryptography.hazmat.primitives.serialization import load_ssh_public_key
_log = logging.getLogger(__name__)
def _password_mismatch_message(pw: Optional[bytes]) -> str:
"""clear ValueError text for a TypeError raised by a password/encryption mismatch
cryptography raises TypeError for both directions: encrypted key + no password,
and unencrypted key + a password given. branch on which the caller supplied so
the message matches the actual case instead of always claiming "encrypted".
"""
if pw is not None:
return "password was given but private key is not encrypted"
return "private key is encrypted but no password was provided"
def _load_private_key(key_data: bytes, pw: Optional[bytes]):
"""load a PEM or OpenSSH private key, normalizing the password/encryption mismatch error
cryptography raises TypeError for a password/encryption mismatch (PEM raises it on
the first call; OpenSSH raises it inside the openssh fallback), normalized here to a
clear ValueError so callers see one error type with a message matching the actual case.
"""
try:
return serialization.load_pem_private_key(key_data, password=pw)
except ValueError as error:
if b"BEGIN OPENSSH PRIVATE KEY" in key_data:
try:
return serialization.load_ssh_private_key(key_data, password=pw)
except TypeError as ssh_error:
raise ValueError(_password_mismatch_message(pw)) from ssh_error
if pw is not None:
# a password was given but the PEM load still failed — most likely a wrong
# password; give a clearer message than cryptography's raw "Bad decrypt"
raise ValueError("could not load private key (wrong password or malformed key)") from error
raise error
except TypeError as error:
raise ValueError(_password_mismatch_message(pw)) from error
def _fingerprint_of(public_key) -> str:
"""base64 SHA-256 fingerprint of an already-loaded public key
factored so callers that already hold a loaded key (e.g. encrypt_aes_key_with_rsa)
don't re-open and re-parse the key file just to fingerprint it.
"""
key_bytes = public_key.public_bytes(
encoding=serialization.Encoding.DER,
format=serialization.PublicFormat.SubjectPublicKeyInfo,
)
digest = hashes.Hash(hashes.SHA256())
digest.update(key_bytes)
return base64.b64encode(digest.finalize()).decode()
def _is_blob(value: Any) -> bool:
"""return whether value has the {secure, iv, data} encrypted-blob shape"""
return isinstance(value, dict) and value.get("secure") is True and "iv" in value and "data" in value
def _has_encrypted_field(record: Any) -> bool:
"""unbounded-depth scan: does record (or anything nested under it) contain a blob
used to detect a blob left behind by a depth-limited traversal — no traversal_level
cutoff here, since the whole point is to catch what a bounded pass would miss.
"""
if not isinstance(record, dict):
return False
if _is_blob(record):
return True
return any(_has_encrypted_field(value) for value in record.values())
def _require_rsa(key) -> None:
"""raise ValueError unless key is an RSA public or private key
this lib is RSA-envelope only: get_rsa_key_fingerprint accepts any key type
(fingerprinting is algorithm-agnostic), but wrap/unwrap calls RSA-OAEP methods
that don't exist on e.g. Ed25519/EC keys and would otherwise crash raw with an
AttributeError far from a clear cause.
"""
if not isinstance(key, (rsa.RSAPublicKey, rsa.RSAPrivateKey)):
raise ValueError(f"RSA key required for envelope wrap/unwrap, got {type(key).__name__}")
def _load_public_key(key_data: bytes):
"""load a PEM or OpenSSH public key, normalizing non-key input to ValueError
load_ssh_public_key raises UnsupportedAlgorithm (not ValueError) on non-SSH/garbage
input; normalize it so a bad public key always surfaces as a clear ValueError,
consistent with the private-key path.
"""
try:
return serialization.load_pem_public_key(key_data)
except ValueError:
try:
return load_ssh_public_key(key_data)
except (UnsupportedAlgorithm, ValueError) as error:
raise ValueError("not a valid PEM or OpenSSH public key") from error
class EnvelopeCrypto:
"""hybrid RSA/AES-256-GCM envelope encryption for dict records
@@ -211,11 +99,10 @@ class EnvelopeCrypto:
) -> bool:
"""verify the full pipeline against a keypair; raises on any mismatch
round-trips sample data through this instance's DEK, then wraps and unwraps
the DEK with the given keypair, confirming they match. run after bootstrap
(or as a health check) to catch a bad keypair or wrong path before relying
on it. is_file threads through to both key loads (is_file=False treats both
as in-memory PEM/OpenSSH data, not paths). returns True on success.
round-trips sample data through this instance's DEK, then wraps the DEK
with the public key and unwraps with the private key, confirming they
match. run after bootstrap (or anytime as a health check) to catch a bad
keypair or wrong key path before relying on it. returns True on success.
"""
if not self.master_key:
raise ValueError("self_test: not initialized with a key")
@@ -226,7 +113,7 @@ class EnvelopeCrypto:
_, wrapped = self.encrypt_aes_key_with_rsa(self.master_key, rsa_public_key, is_file=is_file)
try:
recovered = self.decrypt_aes_key_with_rsa(wrapped, rsa_private_key, is_file=is_file, password=password)
recovered = self.decrypt_aes_key_with_rsa(wrapped, rsa_private_key, password=password)
except Exception as error:
raise RuntimeError(
"self_test: key unwrap failed (public/private keys do not pair, "
@@ -239,15 +126,7 @@ class EnvelopeCrypto:
return True
def initialize(self, master_key: bytes) -> None:
"""arm the instance with the AES data key (DEK)
requires exactly 32 bytes (AES-256): a shorter key would silently downgrade
to AES-128/192 with no warning, and a non-bytes value would otherwise fail
late and opaquely at first encrypt/decrypt — both rejected here instead.
never logs the key material itself.
"""
if not isinstance(master_key, bytes) or len(master_key) != 32:
raise ValueError("master_key must be exactly 32 bytes (AES-256)")
"""arm the instance with the AES data key (DEK)"""
self.master_key = master_key
_log.info("crypto initialized with data key")
@@ -268,17 +147,9 @@ class EnvelopeCrypto:
return key
def get_rsa_key_fingerprint(
self, key_path_or_data: str, is_private: bool = False, is_file: bool = True,
password: Optional[str] = None,
self, key_path_or_data: str, is_private: bool = False, is_file: bool = True
) -> str:
"""return a base64 SHA-256 fingerprint of an RSA key for identification
for an encrypted private key (is_private=True), pass its `password`; an
unencrypted key ignores it. always fingerprints the public half, so a
private key and its public key match. PEM and OpenSSH accepted (mirrors
decrypt_aes_key_with_rsa). a password/encryption mismatch raises a clear
ValueError (cryptography's raw TypeError normalized here).
"""
"""return a base64 SHA-256 fingerprint of an RSA key for identification"""
if is_file:
with open(key_path_or_data, "rb") as key_file:
key_data = key_file.read()
@@ -290,32 +161,38 @@ class EnvelopeCrypto:
)
if is_private:
pw = password.encode() if password else None
private_key = _load_private_key(key_data, pw)
private_key = serialization.load_pem_private_key(key_data, password=None)
public_key = private_key.public_key()
else:
public_key = _load_public_key(key_data)
try:
public_key = serialization.load_pem_public_key(key_data)
except ValueError:
public_key = load_ssh_public_key(key_data)
fingerprint = _fingerprint_of(public_key)
key_bytes = public_key.public_bytes(
encoding=serialization.Encoding.DER,
format=serialization.PublicFormat.SubjectPublicKeyInfo,
)
digest = hashes.Hash(hashes.SHA256())
digest.update(key_bytes)
fingerprint = base64.b64encode(digest.finalize()).decode()
_log.info("generated %s key fingerprint", "private" if is_private else "public")
return fingerprint
def encrypt_aes_key_with_rsa(
self, aes_key: bytes, rsa_key: str, is_file: bool = True
) -> Tuple[str, str]:
"""wrap an AES key with an RSA public key; returns (fingerprint, wrapped_b64)
raises ValueError if rsa_key is not an RSA key (this lib is RSA-envelope only;
e.g. an Ed25519/EC key loads and fingerprints fine but cannot wrap).
"""
"""wrap an AES key with an RSA public key; returns (fingerprint, wrapped_b64)"""
if is_file:
with open(rsa_key, "rb") as key_file:
key_data = key_file.read()
else:
key_data = rsa_key.encode() if isinstance(rsa_key, str) else rsa_key
public_key = _load_public_key(key_data)
_require_rsa(public_key)
try:
public_key = serialization.load_pem_public_key(key_data)
except ValueError:
public_key = load_ssh_public_key(key_data)
wrapped = public_key.encrypt(
aes_key,
@@ -325,31 +202,29 @@ class EnvelopeCrypto:
label=None,
),
)
# fingerprint from the already-loaded public_key — no second open/parse of the file
fingerprint = _fingerprint_of(public_key)
fingerprint = self.get_rsa_key_fingerprint(rsa_key, is_private=False, is_file=is_file)
wrapped_b64 = base64.b64encode(wrapped).decode()
_log.info("wrapped data key for fingerprint %s", fingerprint[:8])
return fingerprint, wrapped_b64
def decrypt_aes_key_with_rsa(
self, encrypted_key_base64: str, rsa_private_key: str,
is_file: bool = True, password: Optional[str] = None,
self, encrypted_key_base64: str, rsa_private_key_path: str,
password: Optional[str] = None,
) -> bytes:
"""unwrap an AES key with an RSA private key
is_file defaults to True (rsa_private_key is a path), matching
encrypt_aes_key_with_rsa / get_rsa_key_fingerprint; pass is_file=False to
supply the PEM/OpenSSH key data directly (e.g. an in-memory or vault-sourced
key) instead of a file path.
"""
if is_file:
with open(rsa_private_key, "rb") as key_file:
key_data = key_file.read()
else:
key_data = rsa_private_key.encode() if isinstance(rsa_private_key, str) else rsa_private_key
pw = password.encode() if password else None
private_key = _load_private_key(key_data, pw)
_require_rsa(private_key)
"""unwrap an AES key with an RSA private key"""
with open(rsa_private_key_path, "rb") as key_file:
key_data = key_file.read()
try:
private_key = serialization.load_pem_private_key(
key_data, password=password.encode() if password else None
)
except ValueError as error:
if b"BEGIN OPENSSH PRIVATE KEY" in key_data:
private_key = serialization.load_ssh_private_key(
key_data, password=password.encode() if password else None
)
else:
raise error
wrapped = base64.b64decode(encrypted_key_base64)
aes_key = private_key.decrypt(
@@ -392,30 +267,9 @@ class EnvelopeCrypto:
return new_key, wrapped
def encrypt_data(self, data: Union[Dict[str, Any], str]) -> Dict[str, str]:
"""encrypt a dict or string under the data key with a unique IV
dict keys must be str: json.dumps silently stringifies int/float/bool/None
keys (e.g. a snowflake-int-keyed dict), which would make decrypt_data return
a dict that no longer matches the original by key type or identity — a
non-str key is rejected here instead, so the failure is loud at encrypt
time rather than a silent lookup miss after decrypt.
"""
"""encrypt a dict or string under the data key with a unique IV"""
if not self.master_key:
raise ValueError("not initialized with data key")
if not isinstance(data, (dict, str)):
# a non-dict/non-str would .encode()-fail with an opaque AttributeError below;
# reject it clearly (decrypt_data only round-trips dict or str anyway)
raise TypeError(f"encrypt_data expects a dict or str, got {type(data).__name__}")
if isinstance(data, dict):
non_str_keys = [key for key in data if not isinstance(key, str)]
if non_str_keys:
# json.dumps would silently stringify these (int/float/bool/None keys),
# corrupting the round-trip (decrypt_data would return a dict keyed by
# the stringified value) — fail loud instead of coercing
raise TypeError(
"encrypt_data requires str dict keys, got non-str key(s): "
f"{[type(key).__name__ for key in non_str_keys]}"
)
data_str = json.dumps(data) if isinstance(data, dict) else data
iv = os.urandom(12)
@@ -428,73 +282,35 @@ class EnvelopeCrypto:
}
def decrypt_data(self, encrypted_data: Dict[str, str]) -> Union[Dict[str, Any], str]:
"""decrypt a {secure, iv, data} blob; returns the original dict or string
encrypt_data only json-encodes dicts (a string is stored verbatim), so decrypt
treats a json-OBJECT plaintext as a dict and everything else as a raw string —
a json-shaped but non-object string ('123', 'true', '[1,2]') round-trips as a
STRING, not int/bool/list. one irreducible ambiguity: a string whose exact
value is a json object ('{"a":1}') decrypts to a dict, indistinguishable
without a type marker from a stored dict — don't store a bare json-object
string if you need it back as a string. dict keys round-trip faithfully
because encrypt_data requires str keys (json's only key type).
"""
"""decrypt a {secure, iv, data} blob; returns the original dict or string"""
if not self.master_key:
raise ValueError("not initialized with data key")
if not isinstance(encrypted_data, dict) or "iv" not in encrypted_data or "data" not in encrypted_data:
# a structurally-malformed blob would raise a raw KeyError/TypeError; surface
# a clear ValueError instead, matching the documented {secure, iv, data} shape
raise ValueError("decrypt_data expects a {secure, iv, data} blob")
iv = base64.b64decode(encrypted_data["iv"])
ciphertext = base64.b64decode(encrypted_data["data"])
aesgcm = AESGCM(self.master_key)
plaintext = aesgcm.decrypt(iv, ciphertext, None).decode()
try:
parsed = json.loads(plaintext)
return json.loads(plaintext)
except json.JSONDecodeError:
return plaintext
return parsed if isinstance(parsed, dict) else plaintext
def reencrypt(self, source_crypto: "EnvelopeCrypto", record: dict, traversal_level: int = 2) -> dict:
"""re-encrypt a record's encrypted fields from source_crypto's key to this one's
self holds the destination (new) key; source_crypto holds the source (old) key.
only {secure, iv, data} fields are touched; plaintext fields are left as-is.
returns a new dict; the input is not mutated. used during rotation. if `record`
itself is a bare blob (file-storage pattern) it is re-encrypted directly and
returned in place of `record`, not nested under a key.
fails loud, unlike decrypt_record: a per-field decrypt failure RAISES (silently
keeping a field under the old key would lose it once that key is retired), and
so does a blob nested DEEPER than `traversal_level` — raise a higher
traversal_level or flatten the record before rotation instead.
traversal recurses into nested DICTS only; a blob nested inside a LIST is not
re-encrypted and not covered by the depth-limit raise. this scheme keys blobs
by field name, not inside arrays, so this shouldn't arise in practice — but
flatten list-nested blobs to dict fields before rotation or they'll be
silently left under the old key.
self holds the destination (new) key; source_crypto holds the source (old)
key. only {secure, iv, data} fields are touched; plaintext fields are left
as-is. returns a new dict; the input is not mutated. used during rotation.
"""
if not self.master_key:
raise ValueError("destination not initialized with data key")
if _is_blob(record):
return self.encrypt_data(source_crypto.decrypt_data(record))
result = copy.deepcopy(record)
result = record.copy()
for key, value in record.items():
if _is_blob(value):
if isinstance(value, dict) and value.get("secure") is True and "iv" in value and "data" in value:
result[key] = self.encrypt_data(source_crypto.decrypt_data(value))
elif isinstance(value, dict):
if traversal_level > 0:
result[key] = self.reencrypt(source_crypto, value, traversal_level - 1)
elif _has_encrypted_field(value):
raise ValueError(
f"reencrypt: field {key!r} contains an encrypted blob nested deeper "
"than traversal_level; increase traversal_level or flatten the record "
"before rotation — leaving it would strand the field under the old key"
)
elif traversal_level > 0 and isinstance(value, dict):
result[key] = self.reencrypt(source_crypto, value, traversal_level - 1)
return result
@@ -507,41 +323,28 @@ PCICrypto = EnvelopeCrypto # deprecated legacy alias; remove after all systems
def is_encrypted_record(record, traversal_level: int = 2) -> bool:
"""return whether a record has any encrypted ({secure, iv, data}) fields
checks `record` itself (the file-storage pattern stores the blob AS the whole
document) as well as fields up to `traversal_level` deep. beyond that bounded
pass, falls back to an unbounded-depth scan, so a blob left behind by a
shallower decrypt_record/reencrypt call is still reported — safe to use as a
leftover-detecting post-rotation audit; never returns False for a record that
still contains a blob, at any depth.
aliases: is_encrypted_document, is_encrypted_dict — same function
"""
if not isinstance(record, dict):
return False
if _is_blob(record):
return True
for value in record.values():
if _is_blob(value):
return True
if isinstance(value, dict) and value.get("secure") is True:
if "iv" in value and "data" in value:
return True
if traversal_level > 0:
for value in record.values():
if isinstance(value, dict) and is_encrypted_record(value, traversal_level - 1):
return True
return False
return any(_has_encrypted_field(value) for value in record.values())
return False
def decrypt_record(crypto: EnvelopeCrypto, record, traversal_level: int = 2) -> Union[dict, Any]:
def decrypt_record(crypto: EnvelopeCrypto, record, traversal_level: int = 2) -> dict:
"""decrypt a record's encrypted fields into a new dict (up to traversal_level deep)
if `record` itself is a bare {secure, iv, data} blob (file-storage pattern) it is
decrypted directly and the value (dict or string — see decrypt_data) is returned
in place of `record`. a failure on a single field (or on `record` itself) is
logged and left encrypted, so a partial failure stays visible rather than silent.
failures on a single field are logged and that field is left encrypted, so a
partial failure is visible (the {secure,...} blob remains) rather than silent.
aliases: decrypt_document, decrypt_dict — same function
"""
@@ -550,16 +353,9 @@ def decrypt_record(crypto: EnvelopeCrypto, record, traversal_level: int = 2) ->
if not isinstance(record, dict):
return record
if _is_blob(record):
try:
return crypto.decrypt_data(record)
except Exception:
_log.exception("failed to decrypt record")
return copy.deepcopy(record)
result = copy.deepcopy(record)
result = record.copy()
for key, value in record.items():
if _is_blob(value):
if isinstance(value, dict) and value.get("secure") is True and "iv" in value and "data" in value:
try:
result[key] = crypto.decrypt_data(value)
except Exception:
@@ -569,49 +365,9 @@ def decrypt_record(crypto: EnvelopeCrypto, record, traversal_level: int = 2) ->
return result
def _fingerprint_default(value: Any) -> str:
"""json.dumps default= handler for values fingerprint_data can't natively serialize
covers datetime/date/time (isoformat), bytes/bytearray (hex), and anything else
(e.g. ObjectId) via a type-tagged repr — never raises, never logs the value.
"""
if hasattr(value, "isoformat"):
return f"isoformat:{value.isoformat()}"
if isinstance(value, (bytes, bytearray)):
return f"hex:{value.hex()}"
return f"{type(value).__name__}:{value!r}"
def _fingerprint_normalize(value: Any) -> Any:
"""recursively tag dict keys with their type to avoid cross-type key collisions
json object keys are always strings, so {1: "a"} and {"1": "a"} would otherwise
serialize identically and collide to the same fingerprint; prefixing each key with
its type name keeps them distinct. non-dict containers/values pass through
untouched (lists recurse; leaf values are handled by _fingerprint_default).
"""
if isinstance(value, dict):
return {
f"{type(key).__name__}:{key!r}": _fingerprint_normalize(sub_value)
for key, sub_value in value.items()
}
if isinstance(value, (list, tuple)):
return [_fingerprint_normalize(item) for item in value]
return value
def fingerprint_data(data: dict) -> str:
"""return a deterministic, collision-free SHA-256 hex fingerprint of a dict
dict keys of different types that would otherwise coerce to the same JSON string
(e.g. {1: "a"} vs {"1": "a"}) are kept distinct via a type-tagged pre-pass. values
that aren't JSON-native (datetime/date/time, bytes/bytearray, ObjectId-like
objects) are serialized via a stable default= handler instead of raising. never
logs the data being fingerprinted.
"""
normalized = _fingerprint_normalize(data)
encoded = json.dumps(normalized, sort_keys=True, default=_fingerprint_default)
return hashlib.sha256(encoded.encode()).hexdigest()
"""return a deterministic SHA-256 hex fingerprint of a dict"""
return hashlib.sha256(json.dumps(data, sort_keys=True).encode()).hexdigest()
# function aliases — same functions, naming preference only