Capacity that adds up
Each 8-core server adds about 4,500 requests per second in our benchmark. Add or remove one in minutes.
The framework for AI era
The framework for AI era
Level 3 · ScaleArchitecture 05 of 13
Several identical API Maker servers behind a load balancer, sharing one data tier.
Add capacity by adding servers. Every server runs API Maker with the same .env, so any of them answers any request. Schedulers run once, cache and events are shared through Redis, and one git pull updates them all.
Every request is drawn as it travels. Slow it down, pause, go step by step, or open the full canvas and zoom in.
api-maker/architectures/load-balancedLiveAny server answers any request. The load balancer sends each request to the least busy server. They share the same data, so no sticky sessions are needed.
The same steps as the diagram, in more detail.
Every API Maker server has the same .env: the same API Maker DB, the same Redis, the same secrets. They all serve every API, and tokens signed by one are valid on the others. The load balancer simply sends each request to the least busy server, with no sticky sessions. Three 8-core servers answer about 13,500 requests per second in our benchmark.
Cached responses live in the shared Redis, not in one server: a response cached while server 1 answered is served to the next caller by whichever server gets the request. A write through any server resets the cache of the table for everyone, so no server returns stale data.
Each scheduler runs on one server of the fleet only: the servers take it through a lock in Redis. If that server stops, its lock expires and another server takes the schedulers over, without any configuration. Jobs run once, never twice, never zero times.
A git pull on any server writes the new definitions into the shared API Maker DB. It then publishes a cluster event through Redis: every worker of every server reloads the changes and recycles its sandboxes. There is nothing to deploy server by server.
To grow, install API Maker on one more VPS with the same .env and add it to the load balancer. It reads the same definitions and connects to the same Redis, so it serves every API from its first request. Remove a server the same way: the others take its share.
The Analytics dashboard of the admin panel shows every server and worker of the fleet, with their traffic, latency, memory and CPU, and restarts any of them remotely. Restart servers one at a time: health checks take each one out of the load balancer until it is back, and the other servers keep serving the apps.
Each 8-core server adds about 4,500 requests per second in our benchmark. Add or remove one in minutes.
Tokens, the shared Redis and the shared API Maker DB let any server answer any user and any request.
A lock in Redis gives each scheduler to one server of the fleet, and another one takes over if it stops.
Responses cached through one server are served by all of them, and a write resets them everywhere.
A git pull on any server reaches every worker of every server through a cluster event.
The Analytics dashboard shows traffic, latency and health of every server and worker, and restarts them remotely.
Plain VPS from any provider, or your own servers, with Ubuntu 22.04 LTS. Sizes are a starting point: measure and adjust.
Runs
Ports
Runs
Ports
Runs
Ports
Runs
Ports
Runs
Ports
am__serverName="api-2" # the only line that differs, shown in the dashboardsam__cpuCount="AUTO"# The same on every serveram__passJWT="***"am__passDBEncryptDecrypt="***"am__passCommunication="***"am__mongo_db_connection="mongodb://api_maker:***@10.0.0.20:27017/api_maker_db?authSource=admin&replicaSet=rs0"am__redisInternal='{"nodes": [{host: "10.0.0.30", port: 6379, pass: "***"}]}'am__redisExternal='{"nodes": [{host: "10.0.0.30", port: 6390, pass: "***"}]}'frontend api bind :443 ssl crt /etc/ssl/api.pem acl is_websocket hdr(Upgrade) -i websocket use_backend websocket if is_websocket default_backend httpbackend http balance leastconn server api-1 10.0.0.11:38246 check server api-2 10.0.0.12:38246 check server api-3 10.0.0.13:38246 checkbackend websocket balance leastconn timeout tunnel 1h # WebSockets can stay quiet for a long time server api-1 10.0.0.11:38245 check server api-2 10.0.0.12:38245 check server api-3 10.0.0.13:38245 checkAny load balancer that forwards HTTP and WebSocket traffic works the same way.
No. Authentication uses tokens, the cache and the WebSocket subscriptions live in the shared Redis, and the definitions in the shared API Maker DB, so any server can answer any request.
Each WebSocket stays on the server it connected to. Events raised on any other server reach it through Redis. Give the load balancer a long idle timeout: API Maker keeps quiet connections open with pings.
As many as your data tier can serve. Each server opens its own connections to MongoDB, Redis and your databases, so size those for the whole fleet, or make them clusters as the Resilient architectures show.