API Maker

The backend engine for the AI era

Scale outEpisode 73 7:03 12 chapters November 2026

Scale Out a Backend: Many API Maker Servers Behind One Load Balancer

Two API Maker servers on the same MongoDB and Redis, behind Caddy: any server answers any call, a save is live on every server at once, WebSocket clients are told whichever server holds them, each scheduler runs once, and a server which stops hands over its clients and schedulers. Shown live, with two servers.

In the playlists The complete course Performance Run in production

Chapters

What you will learn

  • A second server: the same .env, its own name, the same MongoDB and Redis
  • One address in front with Caddy: round robin, a retry with the body, /ping health checks
  • No sticky sessions: a token one server gives is taken by the other
  • A save of code live on every server, and one cache in Redis
  • WebSocket clients on both servers, told of every change
  • Schedulers spread over the servers, one run per tick, and what happens when a server stops
  • Upgrades, and how many servers to run
In the docsScale to N users : one server or many
Transcript1008 words · click a time to play from there

Intro

Many API Maker servers can sit behind one address. A second server shares the calls, the WebSocket clients, the schedulers, and the cache with the first one. You need no sticky sessions, no cron server, and no message broker.

More servers, the usual way

Scaling out a backend usually means sticky sessions, so that each user stays on the server that knows them. One server for the cron jobs, so that no job runs twice. A message broker, so that the other servers hear about a change. And WebSocket clients that hear only the events of their own server. With API Maker, any server answers any call. Each scheduler runs on one server of the cluster.

A save reaches every server through Redis. And every client is told, whichever server holds it. In this video: a second server, one address in front, a save on every server, WebSocket clients on both, the schedulers, a server that stops, the cache, and upgrades, and what to know.

A second server

The autumn sale of Calm Home is coming, with more calls than one server should take. So a second server joins. It runs the same API Maker, with the same .env, so the same secrets, and the same MongoDB and Redis. Only its name and its address are its own. That is all a cluster of API Maker is: servers that share MongoDB and Redis. On a new machine, the installer does it with one answer: secondary.

On the Servers page, both servers are listed, video1 and video2, each with its threads. video2 has its own server id. It started seconds ago, on the same version. The page asks every server through Redis, and each one says what it is.

One address in front

In front of them is one address: Caddy, with both servers. Each call goes to the next server. If a call finds its server down, it goes to the other one, with its body. And /ping says which servers answer. We get a token of the API user, through that address. Then we make four calls of the custom API Sale Banner. Caddy names the server of each answer: one, then the other.

The token came from one server, and both accept it. No sticky sessions. In Analytics, open the Live tab, while the shop calls Sale Banner eight times a second. Two servers are responding. Each one takes half of the calls. The page asks every server, wherever it is served from.

A save on every server

The sale grows, and so does the banner. Open Sale Banner, and its code. It says twenty percent. Make it thirty. Save it, on the panel, which video1 serves. The next calls go through the one address, and both servers answer with the new banner, at once. The save told every server, through Redis, to drop its copy of the code. No build, no restart, and no deploy to each server.

WebSocket clients on both servers

The app of the shop shows its prices live. It keeps a WebSocket open, subscribed to the price changes of the products. Two clients connect through the one address, and Caddy gives one to each server. Now a price changes, through the one address. Both clients are told, at the same moment. The server that saved publishes on Redis, and each server tells its own clients.

Schedulers, once per cluster

Here are the schedulers of the shop, every ten seconds. The Server column says which server runs each one, and both servers run some. Each server works out the owner of every scheduler from the same list of servers. They agree without talking. Pick one of video2. Open its runs. There is one run every ten seconds, never two, because its server holds a key in Redis while it runs it.

A server stops

Now video2 stops. It could be an upgrade, a crash, or a machine that goes away. Its client is told that the server is going away, and it connects again, through the one address. video1 takes it. The calls go on, and video1 answers all of them. And the next price change reaches both clients. The schedulers of video2 run on video1 within seconds. A server that stops gives its schedulers back.

If a server is killed, its schedulers are taken over once its keys run out, within twenty seconds. When video2 comes back, it takes its share again.

One cache

The cache is shared too, because it is in Redis. The lamp is read on video1, from the database. The same read on video2 comes from the cache. Now its price changes on video2. The cached answers of the table are cleared, for every server. A read on video1 again comes from the database, with the new price.

Upgrades

Now upgrades. Launchpad switches API Maker one server at a time, while its gateway sends the calls to the others. The page Upgrade/Downgrade Engine upgrades every server from one page. Each one is checked, and rolled back by itself if needed. Video forty seven shows it. And your code needs no upgrade at all, because a save is live on every server.

Good to know

Good to know. A server full of WebSocket clients answers one more with 503 and Retry-After. It keeps connections for the calls of its APIs. Count the servers by memory, a few kilobytes for each client. And keep one spare, for the clients of a server that stops. Smaller servers lose fewer clients at once. In the benchmarks, when one server of four stopped, a quarter of the clients dropped, and all came back in under a minute.

And work done by one server, like the scan of Index Maker each minute, runs on just one of them, through Redis.

Recap

To sum up. Servers that share MongoDB, Redis, and their secrets are one cluster. One address sits in front. Any server answers any call, with no sticky sessions. A save, a WebSocket event, and the cache reach every server. Each scheduler runs on one server, and moves when that server stops. And a client of a server that stops comes back on another. Next: clusters drawn and deployed by Launchpad.

API Maker. Build faster, spend smarter, deliver better.