envelope_crypto/README.md

# envelope_crypto

Envelope encryption for dict records. A random AES-256-GCM data key (DEK) encrypts
the data; that key is wrapped (RSA-OAEP) per authorized system's public key (KEK)
and stored by the caller, keyed by fingerprint. Each system unwraps its own copy
with its private key. The same envelope pattern KMS-style systems use — config-free
and storage-agnostic.

## Install

`requirements.txt`:

```
envelope_crypto @ git+ssh://git@git.rethinkstudios.io/rethink-public/envelope_crypto.git@v0.1.10
```

Direct:

```bash
pip install "envelope_crypto @ git+ssh://git@git.rethinkstudios.io/rethink-public/envelope_crypto.git@v0.1.10"
```

Requires `cryptography` (pulled transitively).

Drop the `@v0.1.10` suffix from the line above to install the latest unpinned.

## First-time setup

Run once, ever, to create the data key and authorize the first system. You need an
RSA keypair first:

```bash
# generate an RSA keypair for the first system (PEM)
openssl genrsa -out local_priv.pem 4096
openssl rsa -in local_priv.pem -pubout -out local_pub.pem
```

```python
from envelope_crypto import EnvelopeCrypto

# generate the DEK and wrap it for this system in one call
crypto, fingerprint, wrapped = EnvelopeCrypto.bootstrap("public_key.pem")

# verify the keypair actually round-trips BEFORE storing anything
crypto.self_test("public_key.pem", "private_key.pem")        # raises if keys don't pair

# store the wrapped key — this is now the ONLY record of the DEK
await db.create_document("keys", {"_id": fingerprint, "key": wrapped})
```

The plaintext DEK is never stored. It survives only as the RSA-wrapped copy, and is
re-derived each boot by unwrapping. **Never persist the plaintext key.**

## Boot (already set up)

```python
crypto = EnvelopeCrypto()
fingerprint = crypto.get_rsa_key_fingerprint("public_key.pem")
record = await db.get_document("keys", {"_id": fingerprint})
if not record:
    raise RuntimeError("this system is not authorized")
crypto.initialize(crypto.decrypt_aes_key_with_rsa(record["key"], "private_key.pem"))
bot.crypto = crypto
```

The `keys` schema (`_id` = fingerprint, `key` = wrapped) is the **caller's** choice;
this lib only produces `(fingerprint, wrapped_key)`.

`decrypt_aes_key_with_rsa` (like `encrypt_aes_key_with_rsa` and
`get_rsa_key_fingerprint`) takes `is_file` (default `True`). Pass `is_file=False` to
hand it PEM/OpenSSH key data directly — e.g. a private key sourced from a vault —
instead of a file path. `self_test` threads the same `is_file` through to both the
public and private key it loads, so `self_test(pub_pem, priv_pem, is_file=False)`
round-trips two in-memory PEM strings rather than treating them as paths.

## Encrypt / decrypt

```python
enc = crypto.encrypt_data({"ssn": "..."})      # -> {"secure": True, "iv": ..., "data": ...}
plain = crypto.decrypt_data(enc)               # -> {"ssn": "..."}
```

Dict keys must be `str`, at any nesting depth (including a dict nested inside a list
or tuple). `encrypt_data` raises `TypeError` on a non-str key anywhere in the payload
(e.g. an int-keyed dict of Discord snowflakes, even nested a few levels down) instead
of silently stringifying it — the underlying JSON encoding has no other key type, so a
coerced key would come back out of `decrypt_data` as a `str` and no longer match the
original lookup key.

For whole records: `decrypt_record(crypto, doc)` decrypts every `{secure, iv, data}`
field (nested up to `traversal_level`, default 2); `is_encrypted_record(doc)` reports
whether any encrypted field exists. Both also detect `doc` itself being a bare
`{secure, iv, data}` blob (the file-storage pattern below, where the blob IS the whole
document) — not just blobs nested under a key.

```python
from envelope_crypto import is_encrypted_record, decrypt_record

if is_encrypted_record(doc):
    doc = decrypt_record(crypto, doc)
```

`is_encrypted_record` always falls back to an unbounded-depth scan whenever its bounded
pass finds nothing, so it reliably reports `True` for a blob left behind by a shallower
`decrypt_record`/`reencrypt` call — safe to use as a leftover-detecting audit after
rotation, regardless of how deep the blob is nested (including inside a list or tuple
at any depth) and regardless of which `traversal_level` you pass; it never false-negatives.

Naming aliases (same objects): `EnvelopeCrypto` = `DocumentCrypto` = `RecordCrypto`
= `PCICrypto` (deprecated legacy alias). `decrypt_record` = `decrypt_document` =
`decrypt_dict`; `is_encrypted_record` = `is_encrypted_document` = `is_encrypted_dict`.

## Authorize another system

An initialized system wraps the DEK for another system's public key. Returns the
record to store.

```python
fingerprint, wrapped = crypto.authorize_system(other_pub_path)
await db.create_document("keys", {"_id": fingerprint, "key": wrapped})
```

Only a system that already holds the DEK can authorize others.

## Deauthorize

Delete that fingerprint's key record — the system can no longer unwrap at boot. This
does **not** revoke a DEK already held in memory by a running system; rotate if a
system is compromised.

## Rotate (new key + re-encrypt)

Generate a new DEK, wrap for the still-authorized set, then re-encrypt existing data.

```python
new_key, wrapped = crypto.rotate_master_key([pub_a, pub_b])   # omit a system to drop it

new_crypto = EnvelopeCrypto()
new_crypto.initialize(new_key)

# re-encrypt every record (caller owns the DB loop)
for doc in await db.get_documents("settings", {}):
    fresh = new_crypto.reencrypt(crypto, doc)                 # decrypt(old) -> encrypt(new)
    await db.update_document("settings", {"_id": doc["_id"]}, fresh)

# replace the key records
await db.delete_documents("keys", {})
for fingerprint, wrapped_key in wrapped.items():
    await db.create_document("keys", {"_id": fingerprint, "key": wrapped_key})
```

`reencrypt(source_crypto, record)` is a method on the **destination** (new-key)
instance: it decrypts each encrypted field with `source_crypto` (old key) and
re-encrypts with itself. Only `{secure, ...}` fields are touched — including `record`
itself if it IS a `{secure, iv, data}` blob (the file-storage pattern).

Rotation must fail loud: a per-field decrypt failure raises, and so does a blob nested
deeper than `traversal_level` — silently leaving it under the old key would strand it
once the old key's wrapped-key record is deleted below. If you nest blobs deeper than
the default `traversal_level=2`, pass a higher `traversal_level` or flatten the record
first.

## Storage patterns

The encrypted blob is just a dict — store it wherever:

- **Mongo** — store the dict directly (Mongo is dict-native).
- **MariaDB / Postgres** — `json.dumps(enc)` into a `JSON` (or `TEXT`) column;
  `json.loads` on read, then `decrypt_data`.
- **File** — `json.dump(enc, f)`.

The lib never touches a database; only the caller's storage layer differs.

## Notes

- `shutdown()` drops the key reference but cannot guarantee zeroing it from RAM
  (Python immutable bytes).
- A failed field decryption in `decrypt_record` is logged and left encrypted (the
  blob stays visible) rather than silently dropped.
- The scheme is envelope/hybrid encryption (AES-256-GCM data key wrapped by RSA-OAEP).
  Using it does not by itself confer PCI-DSS or any other compliance — that is a
  whole-system property.
- `initialize(master_key)` requires exactly 32 bytes (`bytes`, `len == 32`) — a
  16/24-byte key or a `str` raises `ValueError` instead of silently downgrading to
  AES-128/192 or failing late at first encrypt.
- Wrap/unwrap (`encrypt_aes_key_with_rsa`, `decrypt_aes_key_with_rsa`,
  `authorize_system`, `bootstrap`, `rotate_master_key`) is **RSA-only**. A non-RSA
  key (e.g. Ed25519/EC) loads and fingerprints fine but raises a clear `ValueError`
  at wrap/unwrap rather than a raw `AttributeError`.
- `fingerprint_data` serializes non-JSON-native values (datetime/date/time,
  bytes/bytearray, ObjectId-like objects) via a stable `default=` handler instead of
  raising `TypeError`, and type-tags dict keys internally so `{1: "a"}` and
  `{"1": "a"}` no longer collide to the same fingerprint. Never logs the data it
  fingerprints.

## Versioning

Releases are tagged `vX.Y.Z`. The install line above pins a release; drop the `@vX.Y.Z` suffix to install the latest unpinned. Pin deliberately for reproducible installs.