Survives any one VPS
A load balancer, an API server, a database member: any single machine can stop while the apps keep working.
Level 4 · ResilientArchitecture 10 of 13
High availability
A spare for every tier: two Caddy load balancers on a floating IP, three API servers and replicated databases.
Any single VPS can stop and the apps keep working. The public IP floats between two Caddy load balancers, health checks steer around failed servers, and every database has a copy ready to take over. Upgrades roll through the servers one at a time.
See it live
Every request is drawn as it travels. Slow it down, pause, go step by step, or open the full canvas and zoom in.
api-maker/architectures/high-availabilityLive- Floating IP
- lb-1on
- Servers
- 3/3in the pool
- Database
- db-1primary
- Capacity
- ~22,269req/s
Two of everything, three where a vote is needed. A floating IP on two Caddy load balancers, three API servers, a replica set of three and a database with a standby.
How it works
The same steps as the diagram, in more detail.
Two of everything, three where a vote is needed.
Every tier has a spare. The public IP floats between two load balancers running Caddy with the same certificates, three API servers share the traffic, Redis runs as a cluster, API Maker DB runs on a replica set of three and your database has a standby. Any single VPS can stop, and the apps are still served.
The active load balancer fails, the standby takes the IP.
Both load balancers run Caddy and keepalived. The active one holds the floating IP and sends heartbeats to the standby over VRRP. When they stop, the standby claims the IP, through the API of your VPS provider where needed, and the apps reconnect to the same address. Its Caddy already holds the certificates, kept in a storage both share, so HTTPS works from the first request. DNS never changes.
A server fails, the others take its share.
Caddy, on both load balancers, calls GET /ping on every server every few seconds. When server 2 stops answering, Caddy takes it out of the pool: the requests it was answering fail, new ones go to the other servers. The keys of its schedulers expire in Redis and server 1 takes them over, and its WebSocket clients reconnect through Caddy.
The database fails, its standby is promoted.
Your database streams every change to a standby, and a failover manager such as Patroni or repmgr watches the primary. When it stops, the standby is promoted and the virtual IP moves to it. API Maker opens new connections to the same address: only the queries running at the moment of the failure return an error.
Upgrade API Maker without downtime.
Upgrade one server at a time: API Maker Launchpad updates the API Maker servers one after the other. Each one leaves the upstreams of Caddy on both load balancers, keeps its .env, is started by systemd with the new version, and comes back once GET /ping answers. The other servers carry the traffic meanwhile, and the definitions stay in API Maker DB: nothing to redeploy.
Why choose this architecture
One address, always
The floating IP moves to the standby load balancer within seconds. Apps and DNS never see a change.
Health checks built in
Every API Maker server answers GET /ping, which tells Caddy on the load balancers where to send traffic, and where not to.
Schedulers never stop
The server which runs a scheduler can fail: another one takes it over within about twenty seconds.
Upgrades in office hours
Roll a new API Maker version through the servers one at a time, with API Maker Launchpad (Cloud mode), without a maintenance window. The Deploy page of the admin panel upgrades every server at once instead.
Every database has a copy
API Maker DB elects a new primary by itself, and your database fails over to its standby.
On video
Design Clusters on a Canvas: Replica Sets, Galera, Gateways and HTTPS
Design a cluster on a canvas: API Maker on several servers behind a gateway, databases clustered over the servers you pick, a MongoDB replica set, MariaDB as a Galera cluster, Caddy, Ferron or Pingap as reverse proxies with their own domains and Let's Encrypt certificates, and every rule checked before anything is deployed.
Every video of API Maker
Server configuration
Plain VPS from any provider, or your own servers, with Ubuntu 24.04 LTS, installed by API Maker Launchpad. Sizes are a starting point: measure and adjust.
Caddy load balancers
×2: active, standby- CPU
- 2 vCPU
- Memory
- 2 GB
- Storage
- 20 GB SSD
Runs
- Caddy: HTTPS, WSS and health checks, the same certificates on both
- keepalived: the floating IP
Ports
- 443 and 80: on the floating IP
- VRRP: between the two only
API servers
×3 or more- CPU
- 4 – 8 vCPU
- Memory
- 8 – 16 GB
- Storage
- 40 GB SSD
Runs
- API Maker, run by systemd, the same .env on every server
- The admin panel, served by API Maker where .env names its folder
Ports
- 38246 (APIs and WebSockets) and 4626 (admin panel): from the load balancers only
Data servers
×3- CPU
- 4 vCPU
- Memory
- 16 GB
- Storage
- 200 GB NVMe
Runs
- A MongoDB member of rs0: API Maker DB and logs
- A Redis master, and the replica of another master
Ports
- 27017, 6379 – 6380, 16379 – 16380: from the API servers and each other
Three servers hold both clusters: when one stops, its Redis master has a replica on another server.
Database servers
×2: primary, standby- CPU
- 8 vCPU
- Memory
- 32 GB
- Storage
- 500 GB NVMe
Runs
- PostgreSQL with streaming replication
- Patroni or repmgr, and a virtual IP
Ports
- 5432 on the virtual IP: from the API servers only
api.example.com { reverse_proxy 10.0.1.11:38246 10.0.1.12:38246 10.0.1.13:38246 { lb_policy least_conn lb_try_duration 5s # a failed server: the request goes to another one health_uri /ping health_interval 2s }}ws.example.com { # the WebSockets: the same port of API Maker reverse_proxy 10.0.1.11:38246 10.0.1.12:38246 10.0.1.13:38246 { lb_policy least_conn health_uri /ping health_interval 2s }}The same Caddyfile on lb-1 and lb-2. Both need the certificates: give them a shared storage, or the DNS challenge of your DNS provider, so the standby serves HTTPS the moment it takes the floating IP.
vrrp_script caddy { script "pidof caddy" interval 2}vrrp_instance api { state MASTER # BACKUP on lb-2 priority 150 # 100 on lb-2 interface eth1 virtual_router_id 51 advert_int 1 unicast_src_ip 10.0.0.2 # lb-1 unicast_peer { 10.0.0.3 # lb-2 } virtual_ipaddress { 203.0.113.10 } track_script { caddy } notify_master /etc/keepalived/claim-ip.sh # asks your provider to route the IP here}VPS networks rarely carry multicast, hence the unicast peers.
Good to know
- Twice the machines of a simple setup, to patch and to watch.
- A floating IP moves within one location of one provider. To survive the loss of a data center, see Multi-cloud.
- Requests running on a server or database at the moment it fails return an error: let clients retry safe requests.
- Test your failovers on purpose, before a real failure tests them for you.
Questions
Why a floating IP and not DNS with two addresses?
A floating IP moves within seconds and every client follows it at once. DNS answers are cached by resolvers and devices, often for minutes, so users keep reaching a failed address. DNS is the right tool between regions, as the Geo-routing architecture shows, with a setup like this one in each region.
Does my VPS provider support floating IPs?
Most do, under names like floating, reserved, failover or additional IP. keepalived decides which load balancer holds it, and a small script calls the API of the provider when the IP has to be routed to the other one.
What happens to WebSocket clients when a server fails?
Their connections close. The clients reconnect through Caddy, land on another server and subscribe again. Plan for events missed while reconnecting, for example by reloading the data after a reconnect.