Operator Guide

Emergency & Lost-Key Guide

Identify exactly what was lost before changing anything. Different MP-OPT keys protect different data, and substituting one for another can make recovery worse.

I lost the private snapshot recovery identity

Generate a new identity and verify two protected copies.
On the holder open Configuration → Configure or safely rotate snapshot recovery encryption.
Provide the new recipient/identity and leave the old-identity prompt blank.
Enter ROTATE WITHOUT OLD KEY.
Create, deep-verify and export the required new baseline.
Keep old ciphertext marked unavailable; delete it only under an explicit retention decision.

I lost the public recipient, but still have the private identity

No encrypted data is lost. Use the trusted workstation verification helper to derive/verify the public recipient from the private identity. Reinstall only the public age1... value through the guarded recovery-encryption action and require matching node fingerprints.

I lost the root passkey

MP-OPT

Select Root administrator recovery → Reset root passkeys and sessions. The action creates and deep-verifies a rollback snapshot, preserves application data and shows one replacement bootstrap token.

Browser

Register the new root passkey with that token, verify login, then return to MP-OPT and select Disable bootstrap after registering a root passkey.

I lost one node's HA identity or witness token

Do not copy /etc/mp-opt-ha from the peer. The private replication identity and node token define that one node. If a separately encrypted, node-labelled infrastructure backup cannot restore the original values safely, replace the node and commission a fresh cluster ID with new identities/tokens from the surviving durable data.

Automatic failover remains disabled until replication, switchovers and certification pass again.

I lost an operational secret

Lost item	Safe response	Expected effect
SMTP token	Revoke/reissue at the provider, then MP-OPT Configuration → Configure or update SMTP and send a token-free test.	No application data loss.
VAPID private key	Configuration → Rotate VAPID and clear push subscriptions.	Users must enable push again.
Application secret	Configuration → Rotate the application secret.	All sessions are revoked.
Database password	Configuration → Rotate the internal database password.	Role, protected configuration and containers are changed together.
Origin TLS private key	Issue fresh Origin CA material in Cloudflare and reinstall it independently on the affected node.	Temporary origin outage until Caddy validates/reloads.

I lost a VPS—or both VPSs

Follow Disaster Recovery. One surviving current holder can seed a replacement. Total loss requires a portable recovery snapshot and its separate private identity, followed by standalone restore and a fresh HA cluster.

The witness or peer is unreachable

Peer unreachable: holder remains available; do not hand off; repair SSH/network and send a manual copy.
Witness unreachable: writes fail closed; do not bypass fencing; restore the Worker path.
Former primary returns: keep it fenced until it accepts the current generation.