Why Rust: API Maker Load-Tested Live on a $5 Server

Intro

0:00 API Maker is written in Rust. In this video, we put it on the smallest server Linode sells, five dollars a month, and load it live, to see what that changes.

The usual way

0:12 A backend on a heavy runtime usually means a bigger server than its traffic needs, pauses when a garbage collector runs, memory that climbs with the load, and seconds to start again. With API Maker, one native program, compiled for the machine, no garbage collector : memory is freed the moment it is no longer used, so it stays flat under load, and a restart answers in a fraction of a second.

0:38 In this video : the server, a minute of load, memory and CPU while it runs, the cache, a restart, what Rust does underneath, and the published numbers, WebSockets included.

The server

0:50 The server : one shared virtual CPU, and one gigabyte of memory, for API Maker, its MongoDB, its Redis and two sandboxes. API Maker is one program. At rest, it holds a few dozen megabytes, on two threads.

A minute of load

1:07 From a second server on the private network, autocannon : a hundred connections for sixty seconds, on get all of a MongoDB table, read from the database every time. Over a hundred thousand requests : close to two thousand a second, half of them answered within about fifty milliseconds, and almost all within a hundred.

Memory and CPU

1:28 Meanwhile on the server, every five seconds. The memory of API Maker : about fifty megabytes, the same from the first sample to the last. Its CPU : under half of the one core. No pause, and no growth. And the rest of the core? MongoDB, on the same machine, and the proxy of Docker in front of it. Here the database is the limit, not API Maker.

The cache

1:54 Now a table with caching on : get by id, answered from Redis. Several times more requests a second, almost every one within twenty milliseconds, on the same five dollar server.

A restart

2:07 A restart : stop, start, and the first answer of an API after a fraction of a second. An upgrade or a restart costs that, not a warm-up.

Underneath

2:17 Why it holds. API Maker is compiled to native code for the machine, and its requests run on Tokio : a few threads serve thousands of connections, two threads here. Rust frees memory when its owner is done with it, at a known place in the code : there is no garbage collector to pause the server. The compiler refuses code which could use freed memory or race on data.

2:41 And the database drivers, the TypeScript compiler and the JavaScript engine of lean code run inside the same program.

The published numbers

2:49 The published benchmarks go further : Linode servers of every size, each loaded the same way. Get all grows from over two thousand requests per second on the five dollar server to over twelve thousand on eight dedicated cores. Today's run is a little lower : this Nanode landed on another processor, slower per core than the one of the published runs. WebSockets too. This same kind of server held tens of thousands of clients at once, each with its tokens and a subscription, until its memory was full.

3:23 One server of eight cores and sixteen gigabytes held millions, a few kilobytes a client.

Good to know

3:30 A shared CPU is shared : runs differ by a few percent, and a neighbour can take some. The database ran on the same machine : on a server of its own, API Maker has the core to itself. Every published page has the whole output of autocannon and the memory of each process : docs.apimaker.dev, Benchmarks.

Built on standards

3:52 The core is Rust : the compiler refuses use of freed memory and data races before the server ever runs. One program, the same on every server, checked before an upgrade uses it. And your code, written by your team or by an AI assistant, runs on top of it, in sandboxes and with timeouts.

Recap

4:13 To sum up. One native program on a five dollar server. About two thousand requests a second from MongoDB, several times more from the cache. Memory flat under load. A restart in a fraction of a second. And published numbers over twelve thousand requests per second, and millions of WebSocket clients. Next : API Maker on a server of your own, with one command. API Maker. Build faster, spend smarter, deliver better.