Technical Reference
High-Availability Protocol
The two server nodes are symmetric. An external Durable Object witness serialises ownership, while Cloudflare routing follows the ownership decision.
Symmetric two-VPS high availability
The witness grants ownership. The load balancer only sends traffic to the node that can prove it is ready.
Cloudflare Load Balancer
Public HTTPS traffic and /ha/ready health checks
VPS A
Application, PostgreSQL, local node identity
VPS B
Application, PostgreSQL, local node identity
Durable Object witness
One writer lease, generation, heartbeat, and short write permits
Independent external snapshot
Encrypted recovery copy plus an operator-held private identity kept away from both VPSs
Either VPS can become holder. The labels A and B identify nodes, not permanent primary and secondary roles.
Lease and write fencing
The holder sends a heartbeat every 15 seconds and obtains short-lived write permits of approximately 10 seconds. FastAPI validates authority before each mutating request, and the persistence layer checks it again before committing ORM work. A node receiving traffic without a current permit cannot become writer.
| Probe or state | Proves | Use |
|---|---|---|
/health | Process and local database are alive | Local diagnosis only |
/ha/ready | Health, current ownership, and generation | Cloudflare pool monitor and readiness |
| Generation | Monotonic ownership epoch | Invalidates stale authority after promotion |
| Write permit | Current short-lived authority | Request and commit fencing |
Replication acceptance
- The holder builds and encrypts a complete point-in-time bundle.
- Authenticated SSH transfers it without replacing the peer's accepted copy.
- The peer validates the outer hash, receipt, encrypted manifest, cluster identity, generation, and release.
- It restores the database into staging and validates the staged state.
- Only after successful checks and witness authorisation does it atomically accept the new recovery point.
- The prior local database remains available for guarded rollback.
Automatic failover sequence
Promotion is deliberately gated to prevent both servers from accepting writes.
1
Heartbeat stops
The current holder disappears
2
Witness waits
Approximately five minutes
3
Peer qualifies
Healthy, matching release, accepted current bundle
4
Generation advances
Old write authority becomes invalid
5
Peer promotes
Database and local ownership activate
6
Routing follows
Cloudflare sees /ha/ready
Promotion security effects
Promotion preserves registered passkeys and durable application data. It invalidates sessions, exchange codes, incomplete passkey ceremonies, activation and reset links, public schedule links, and desktop publishing credentials. These bearer-style values are intentionally not trusted across the ownership boundary.
For commands and checkpoints, use HA Setup and Active HA Verification.