Survives any one VPS
A load balancer, an API server, a database member: any single machine can stop while the apps keep working.
The framework for AI era
The framework for AI era
Level 4 · ResilientArchitecture 10 of 13
A spare for every tier: two load balancers on a floating IP, three API servers and replicated databases.
Any single VPS can stop and the apps keep working. The public IP floats between two load balancers, health checks steer around failed servers, and every database has a copy ready to take over. Upgrades roll through the servers one at a time.
Every request is drawn as it travels. Slow it down, pause, go step by step, or open the full canvas and zoom in.
api-maker/architectures/high-availabilityLiveTwo of everything, three where a vote is needed. A floating IP on two load balancers, three API servers, a replica set of three and a database with a standby.
The same steps as the diagram, in more detail.
Every tier has a spare. The public IP floats between two load balancers, three API servers share the traffic, Redis runs as a cluster, API Maker DB runs on a replica set of three and your database has a standby. Any single VPS can stop, and the apps are still served.
Both load balancers run HAProxy and keepalived. The active one holds the floating IP and sends heartbeats to the standby over VRRP. When they stop, the standby claims the IP, through the API of your VPS provider where needed, and the apps reconnect to the same address. DNS never changes.
The load balancers call GET /ping on every server every few seconds. When server 2 stops answering, they take it out of the pool: the requests it was answering fail, new ones go to the other servers. Its lock of the schedulers expires and server 1 takes them over, and its WebSocket clients reconnect through the load balancer.
Your database streams every change to a standby, and a failover manager such as Patroni or repmgr watches the primary. When it stops, the standby is promoted and the virtual IP moves to it. API Maker opens new connections to the same address: only the queries running at the moment of the failure return an error.
Upgrade one server at a time: disable it on the load balancers, run the installer with the new version, which keeps the existing .env, and enable it again once GET /ping answers. The other servers carry the traffic meanwhile, and the definitions stay in API Maker DB: nothing to redeploy.
A load balancer, an API server, a database member: any single machine can stop while the apps keep working.
The floating IP moves to the standby load balancer within seconds. Apps and DNS never see a change.
Every API Maker server answers GET /ping, which tells the load balancers where to send traffic, and where not to.
The server holding the scheduler lock can fail: another one takes the lock and the jobs run on time.
Roll a new API Maker version through the servers one at a time, without a maintenance window.
API Maker DB elects a new primary by itself, and your database fails over to its standby.
Plain VPS from any provider, or your own servers, with Ubuntu 22.04 LTS. Sizes are a starting point: measure and adjust.
Runs
Ports
Runs
Ports
Runs
Ports
Three servers hold both clusters: when one stops, its Redis master has a replica on another server.
Runs
Ports
frontend api bind :443 ssl crt /etc/ssl/api.pem acl is_websocket hdr(Upgrade) -i websocket use_backend websocket if is_websocket default_backend httpbackend http balance leastconn option httpchk GET /ping default-server check inter 2s fall 3 rise 2 server api-1 10.0.1.11:38246 server api-2 10.0.1.12:38246 server api-3 10.0.1.13:38246backend websocket balance leastconn timeout tunnel 1h option httpchk GET /ping default-server check port 38246 inter 2s fall 3 rise 2 server api-1 10.0.1.11:38245 server api-2 10.0.1.12:38245 server api-3 10.0.1.13:38245vrrp_script haproxy { script "pidof haproxy" interval 2}vrrp_instance api { state MASTER # BACKUP on lb-2 priority 150 # 100 on lb-2 interface eth1 virtual_router_id 51 advert_int 1 unicast_src_ip 10.0.0.2 # lb-1 unicast_peer { 10.0.0.3 # lb-2 } virtual_ipaddress { 203.0.113.10 } track_script { haproxy } notify_master /etc/keepalived/claim-ip.sh # asks your provider to route the IP here}VPS networks rarely carry multicast, hence the unicast peers.
# 1. On both load balancers: take api-1 out of the poolecho "disable server http/api-1; disable server websocket/api-1" | socat stdio /run/haproxy/admin.sock# 2. On api-1: install the new version, the existing .env is keptcurl -fsSL https://apimaker.dev/v1/install.sh > install.sh && bash install.sh --version=3.2.1# 3. Once it answers, back in the pool. Then api-2, then api-3curl -fs http://10.0.1.11:38246/pingecho "enable server http/api-1; enable server websocket/api-1" | socat stdio /run/haproxy/admin.sockA floating IP moves within seconds and every client follows it at once. DNS answers are cached by resolvers and devices, often for minutes, so users keep reaching a failed address. DNS is the right tool between regions, as the Geo-routing architecture shows, with a setup like this one in each region.
Most do, under names like floating, reserved, failover or additional IP. keepalived decides which load balancer holds it, and a small script calls the API of the provider when the IP has to be routed to the other one.
Their connections close. The clients reconnect through the load balancer, land on another server and subscribe again. Plan for events missed while reconnecting, for example by reloading the data after a reconnect.