Operator Guide

Recovery Encryption Keys

Keep one long-lived operator recovery identity off the VPSs, install only its public recipient, and rotate it only through the guarded consolidation workflow.

Know the three credentials

Credential	Location	Purpose
Public snapshot recipient (`age1...`)	Both HA nodes	Encrypts recovery snapshots. It is not secret.
Private snapshot identity (`AGE-SECRET-KEY-1...`)	Trusted operator storage only	Deep-verifies and restores recovery snapshots.
Node-local HA replication identity	`/etc/mp-opt-ha` on exactly one node	Decrypts peer replication. Never copy it to the peer.

Generate and protect the key

Workstation

On Windows Command Prompt, from the exact server checkout:

conda activate mp-opt-soak-test
where age-keygen
deploy\ha\create-recovery-key.cmd --output "%USERPROFILE%\Documents\MP-OPT-Recovery\production.agekey"

The helper refuses destinations inside the repository and refuses to overwrite. It creates the private identity plus a public recipient receipt containing the recipient and its SHA-256.

Store one complete copy in a protected password-manager attachment or secure note.
Store a second copy on independently encrypted offline media.
Retrieve each copy to a temporary location and verify it derives the recorded recipient.

Workstation

python deploy\ha\recovery_key_setup.py verify --identity "D:\Temporary\restored.agekey" --recipient "age1..."

Accept only Result: MATCH and the expected 64-character recipient SHA-256.

Configure the public recipient

MP-OPT

On the current holder, select Configuration → Configure or safely rotate snapshot recovery encryption and paste only the public age1... value.

In HA mode the holder obtains a writer permit, stages the recipient on the peer, installs it atomically on both nodes and compares the fingerprints. Check High availability → Show lease, peer and replication state; local and peer SHA-256 must both match the workstation receipt.

Planned rotation with the old key

Generate and independently back up the new identity first.
Disable automatic failover and ensure the peer is reachable.
Open the same recovery-encryption action on the current holder.
Provide the new public recipient and new private identity.
Provide the old private identity and enter ROTATE RECOVERY KEY.
MP-OPT decrypts, validates, re-encrypts and deep-verifies every managed local/peer copy before replacing originals.
It synchronises the new public recipient and creates a deeply verified complete baseline.
With manual portable storage, export the exact named baseline and confirm the workstation SHA-256 before rotation finalises.

If the old private identity is missing

Generate and verify two protected copies of a new identity.
Start the recovery-encryption action and leave the old-identity prompt blank.
Enter the exact phrase ROTATE WITHOUT OLD KEY.
MP-OPT keeps old ciphertext catalogued as unavailable and creates a new deeply verified baseline.
Export that baseline immediately and verify its portable package SHA-256.

See the Emergency & Lost-Key Guide for the complete decision path.