API Maker

The backend engine for the AI era

Level 2 · GrowArchitecture 03 of 13

Dedicated data tier

A stateless API server, with the MongoDB, logs and Redis of API Maker on servers of their own.

Everything API Maker keeps moves off the API server: definitions to its MongoDB, cache and events to Redis, logs to their own database. The API server becomes disposable, ready to be rebuilt in minutes or joined by others.

Watch it live Server configuration

See it live

Every request is drawn as it travels. Slow it down, pause, go step by step, or open the full canvas and zoom in.

api-maker/architectures/dedicated-data-tierLive
  • Request
  • Response
  • Cache hit
  • Write
  • Logs and events
  • HTTPS and WSS up to Caddy
Servers
4VPS
Capacity
~7,423req/s
Cache hits
35%of reads
State
noneon the API server

The API server keeps nothing. Definitions, cache, events and logs all live on the data tier. The API server only runs Caddy, API Maker under systemd and its sandboxes.

How it works

The same steps as the diagram, in more detail.

  1. The API server keeps nothing.

    Everything API Maker remembers moves off the API server. What you build in the admin panel lives in API Maker DB, cached responses in one Redis, events and locks in another, logs in their own database. The API server runs code only: Caddy, API Maker with its admin panel under systemd, and the sandbox containers of custom code.

  2. APIs are defined once, in API Maker DB.

    Instances, schemas, custom APIs, hooks, schedulers, users and roles are stored in API Maker DB, a MongoDB replica set. API Maker keeps them in memory, so serving a request needs no round trip to it. When something changes, an event goes through the internal Redis and every server reloads it.

  3. Events and cache in separate Redis.

    redisInternal is what API Maker needs to work: events between the servers, the keys of the schedulers, WebSocket subscriptions and auto increment values. redisExternal holds cached responses of your APIs, capped by maxCharsResToCache and expired after redisValueExpireInSeconds. Apart, cache pressure never touches the rest.

  4. Every call is logged, out of the way.

    API Maker logs every call: request, response, duration, errors and console output of custom code. With a logs database of its own, this write-heavy traffic never competes with your data. A daily scheduler keeps the newest maxLogsCount entries, and the log explorer of the admin panel searches them.

  5. Rebuild the API server in minutes.

    Because nothing lives on it, the API server can be replaced at any time: to recover from a failure, to move to another provider or to a bigger machine. Install API Maker on a fresh VPS with the same .env file and point the domains at it: systemd starts API Maker, Caddy gets its certificates again by itself, and every API is served at once. Nothing to restore, nothing to migrate.

Why choose this architecture

A disposable API server

Nothing lives on it but code: rebuild, resize or move it in minutes with the same .env. Caddy gets its certificates again by itself.

Each store sized for its job

MongoDB gets fast disks, Redis gets memory, logs get room to grow: every server is sized for what it does.

Logs out of the way

Every call is logged to a database of its own and trimmed daily, without touching your data.

Cache apart from events

Responses live in their own Redis, so evicting cache never touches locks, events or WebSocket subscriptions.

Ready to scale out

A second API server with the same .env serves the same APIs at once. The next architectures do exactly that.

Less exposed

The apps only reach Caddy, on the API server. The data servers accept its private IP and nothing else.

On video

Design Clusters on a Canvas: Replica Sets, Galera, Gateways and HTTPS

Design a cluster on a canvas: API Maker on several servers behind a gateway, databases clustered over the servers you pick, a MongoDB replica set, MariaDB as a Galera cluster, Caddy, Ferron or Pingap as reverse proxies with their own domains and Let's Encrypt certificates, and every rule checked before anything is deployed.

Every video of API Maker

Server configuration

Plain VPS from any provider, or your own servers, with Ubuntu 24.04 LTS, installed by API Maker Launchpad. Sizes are a starting point: measure and adjust.

API server

×1
CPU
4 vCPU
Memory
8 GB
Storage
40 GB SSD

Runs

  • Caddy: HTTPS and WSS, in front of the APIs and the admin panel
  • API Maker, run by systemd, on every CPU core
  • Docker: sandbox containers of custom code

Ports

  • 443 and 80: Caddy
  • 38246 (APIs and WebSockets) and 4626 (admin panel): API Maker, behind Caddy only

MongoDB server

×1
CPU
2 – 4 vCPU
Memory
8 GB
Storage
100 GB NVMe

Runs

  • MongoDB, replica set rs0
  • api_maker_db: APIs, schemas, settings, users
  • api_maker_logs: a log of every call

Ports

  • 27017: from the API server only

Three members make it highly available: see the High availability architecture.

Redis server

×1
CPU
2 vCPU
Memory
4 – 8 GB
Storage
20 GB SSD

Runs

  • Redis 7 internal: events, locks, WebSocket subscriptions, auto increments
  • Redis 7 cache: responses of your APIs

Ports

  • 6379 and 6390: from the API server only

Database server

1 per database
CPU
4 – 8 vCPU
Memory
16 – 32 GB
Storage
200 GB+ NVMe

Runs

  • Your databases: any of the 8 supported types

Ports

  • From the API server only
Settings of the API server.env
am__serverName="api-1"# Definitions and logs: the MongoDB serveram__mongo_db_connection="mongodb://api_maker:***@10.0.0.20:27017/api_maker_db?authSource=admin&replicaSet=rs0"am__logs__mongo_db_connection_logs="mongodb://api_maker:***@10.0.0.20:27017/api_maker_logs?authSource=admin&replicaSet=rs0"am__logs__maxLogsCount="100000"                     # newest logs keptam__logs__logRemoveSchedulerInterval="0 0 0 * * *"  # trimmed every day# Events and locks, and the cache: the Redis serveram__redisInternal='{"nodes": [{host: "10.0.0.30", port: 6379, pass: "***"}]}'am__redisExternal='{"nodes": [{host: "10.0.0.30", port: 6390, pass: "***"}], redisValueExpireInSeconds: 7200, maxCharsResToCache: 1000000}'

The "Encrypt Strings" tool of the dashboard encrypts these connection strings before they go into .env.

Good to know

  • Still one API server: when it stops, the apps wait for a new one. The next architecture runs several.
  • MongoDB and Redis are single servers here: back them up, or make them clusters as the Resilient architectures show.
  • Four servers to run instead of one or two.
When you outgrow itOne API server is no longer enough for your traffic, or for your uptime. Load-balanced API servers

Questions

What exactly is stored in the MongoDB of API Maker?

Everything you build in the admin panel: instances and their encrypted connection strings, schemas, custom APIs, hooks, schedulers, WebSocket events, users, roles and settings. That is why any API Maker server that points to it serves the same APIs.

Why two Redis?

redisInternal carries what API Maker needs to work: events between the servers, the keys of the schedulers, WebSocket subscriptions and auto increment values. redisExternal holds the cached responses of your APIs, which can be evicted at any time. On small setups both can be the same Redis.

Can I turn logs off or keep fewer of them?

Yes. logs.enableLogs turns them off, logs.maxLogsCount sets how many of the newest logs are kept, and logs.logRemoveSchedulerInterval sets when older ones are removed, every day by default.