Every subscriber, every server
An order written through server 1 reaches dashboards connected to servers 2 and 3 within milliseconds.
Level 3 · ScaleArchitecture 06 of 13
Real-time events across servers
WebSocket clients on any server, changes made on any server: Redis connects them.
Dashboards, apps and devices keep a WebSocket open to whichever server they reached. An API call answered by any server is published in Redis and pushed to every matching subscriber, on every server, within milliseconds.
See it live
Every request is drawn as it travels. Slow it down, pause, go step by step, or open the full canvas and zoom in.
api-maker/architectures/realtime-websocketsLive- Servers
- 3/3up
- Events
- 0pushed
- WebSocket port
- 38246wss through Caddy
- Capacity
- ~22,269req/s
Clients subscribe once, on any server. Dashboards, driver apps and screens open a secure WebSocket to Caddy, which passes it to a server. Their subscriptions are kept in Redis.
How it works
The same steps as the diagram, in more detail.
Clients subscribe once, on any server.
A frontend or a device opens a WebSocket to wss://ws.example.com. Caddy holds the certificate and passes the connection, as a plain WebSocket, to port 38246 of one server, where it stays. The client registers the events it wants: calls of an API, like the insert of orders, with conditions on the response, or custom events. Every subscription becomes a key in the internal Redis, which is how the other servers find it.
A change arrives on any server.
Requests keep flowing through Caddy to the least busy server. A new order may be written by server 1 while the dashboards that want to see it are connected to server 2 or 3. No server needs to know where the subscribers are.
Redis delivers it to every server.
The call is published as an event in the internal Redis. Redis delivers it to every server that registered a matching subscription, and each one pushes it down the WebSockets it holds, through Caddy. A dashboard connected to server 3 sees an order written by server 1 within milliseconds.
Only the subscribers who asked for it.
Subscriptions carry conditions on the response and a list of fields to send. An event only goes where a condition matches, so servers without matching subscribers receive nothing, and clients only get the fields they asked for. Access is checked when the client subscribes, with the token of the right auth provider.
Your code emits events too.
Besides database changes, custom code can emit WebSocket events of your own, like the position of a driver or the progress of a job. They travel the same way: through Redis, to every server holding a subscriber, with the same conditions and access checks.
A server stops, its clients move.
When a server stops, the WebSockets it held close. The clients reconnect to wss://ws.example.com, Caddy sends them to a healthy server, and they register their subscriptions again. Events raised anywhere reach them through their new server.
Why choose this architecture
Only what each client asked for
Conditions on the response decide who gets an event, and select sends only the fields a client needs.
Checked like any API
Clients subscribe with a token of the right auth provider, and your code can accept or refuse each one.
Events of your own
Custom code emits custom WebSocket events, delivered with the same conditions and checks.
Survives a server loss
Clients of a stopped server reconnect to another one and subscribe again. Nothing to configure.
No extra service
The WebSocket server is part of API Maker, on the same port as its APIs, and Redis is the one you already run.
On video
WebSockets Without a Realtime Server: Push Live Data from Your APIs
Live data for your apps from the same server as your APIs: a save, a status, pushed the moment it happens. Follow Kabir, a customer of Calm Home: one WebSocket, his orders live, a status pushed from code with emitEventWS, a rule for each subscription, and how far it scales.
Every video of API Maker
Server configuration
Plain VPS from any provider, or your own servers, with Ubuntu 24.04 LTS, installed by API Maker Launchpad. Sizes are a starting point: measure and adjust.
Caddy load balancer
×1- CPU
- 2 vCPU
- Memory
- 2 GB
- Storage
- 20 GB SSD
Runs
- Caddy: HTTPS in, plain HTTP to port 38246 of the servers
- Secure WebSockets (wss) in, plain WebSockets to the same port
- Certificates of api. and ws.example.com, renewed by itself
Ports
- 443 and 80: from the internet
API servers
×3 or more- CPU
- 4 – 8 vCPU
- Memory
- 8 – 16 GB
- Storage
- 40 GB SSD
Runs
- API Maker, run by systemd: HTTP APIs and the WebSocket server, on one port
- Docker: sandboxes of custom code
Ports
- 38246 (APIs and WebSockets) and 4626 (admin panel): from Caddy only
A client which waits costs about 9 KB of memory and almost no CPU: a server of 8 GB held 795,000 clients, one of 16 GB 1.73 million. Count the servers from the memory, plus one spare. The installer gives API Maker and Caddy open files and room in the firewall from the memory of each server: Caddy holds two connections for every client it passes on.
Redis server
×1- CPU
- 2 – 4 vCPU
- Memory
- 8 GB
- Storage
- 20 GB SSD
Runs
- Redis internal: a key per subscription, pub/sub of the events
- Redis cache: responses of your APIs
Ports
- 6379 and 6390: from the API servers only
About 1.0 to 1.1 KB of Redis per subscription: 100,000 clients with one subscription each are about 100 MB.
Data servers
2+- CPU
- 4 – 8 vCPU
- Memory
- 16 – 32 GB
- Storage
- 200 GB+ NVMe
Runs
- API Maker DB, MongoDB replica set
- Your databases
{ "objType": "REGISTER", "onEvents": [{ "eventType": "INSTANCES", "apiName": "GEN_POST_BULK_INSERT", "instance": "shop", "database": "public", "collection": "orders", "condition": { "conditionType": "RESPONSE", "criteria": { "store": "42" } }, "select": { "id": 1, "items": 1, "total": 1 }, "getEventData": true }]}Sent over wss://ws.example.com, through Caddy, with the token of the client. Every insert of an order of store 42, through any server, is pushed to it.
Good to know
- Every WebSocket is a long-lived connection: size the servers and Caddy for their number, not only for requests.
- Keep the servers under three quarters full: the clients of a server which stops need room on the others, and after a restart they all come back in the same seconds.
- The internal Redis carries every event: keep it close to the servers, with enough memory for the subscriptions.
- Clients must reconnect on their own when a connection closes: every WebSocket client library can.
Questions
How many WebSocket clients can this architecture hold?
About 9 KB of memory per client, so the memory of the API servers decides: ~2.39 million WebSocket clients on 3 × 8 GB, ~5.20 million WebSocket clients on 3 × 16 GB. Measured with one Redis for all of them, for the same price: 2.02 million on 1 server, 1.92 million on 2, 1.62 million on 4. The WebSocket benchmarks of the docs give every server size: https://docs.apimaker.dev/v4/docs/benchmarks/websockets/index.html
One big API server or several small ones?
For the same price they hold about the same number of clients. Several servers lose only their share of the clients when one of them stops (25 % on 4 servers, all back on the others after 38 seconds), and can be restarted one at a time. Start with one, split as soon as the clients must not all drop together.
Do WebSockets need sticky sessions?
No. A WebSocket is one long connection, so it stays on the server it reached by itself. Events raised on any other server reach it through Redis.
What can a client subscribe to?
Calls of the generated APIs of your tables, like the insert or the update of orders, with conditions on the response and the fields to receive. Also calls of custom APIs and system APIs, and custom WebSocket events emitted by your code.
How are subscriptions secured?
A client connects with the token of an auth provider and each WebSocket event accepts the tokens of the provider chosen for it. Custom code can also decide, for each client, whether it may subscribe.