API Maker

The framework for AI era

Level 2 · GrowArchitecture 03 of 13

Dedicated data tier

A stateless API server, with the MongoDB, logs and Redis of API Maker on servers of their own.

Everything API Maker keeps moves off the API server: definitions to its MongoDB, cache and events to Redis, logs to their own database. The API server becomes disposable, ready to be rebuilt in minutes or joined by others.

See it live

Every request is drawn as it travels. Slow it down, pause, go step by step, or open the full canvas and zoom in.

api-maker/architectures/dedicated-data-tierLive
  • Request
  • Response
  • Cache hit
  • Write
  • Logs and events
Servers
4VPS
Capacity
~2,759req/s
Cache hits
35%of reads
State
noneon the API server

The API server keeps nothing. Definitions, cache, events and logs all live on the data tier. The API server only runs API Maker and its sandboxes.

How it works

The same steps as the diagram, in more detail.

  1. The API server keeps nothing.

    Everything API Maker remembers moves off the API server. What you build in the admin panel lives in API Maker DB, cached responses in one Redis, events and locks in another, logs in their own database. The API server runs code only: API Maker, its admin panel and the sandbox containers of custom code.

  2. APIs are defined once, in API Maker DB.

    Instances, schemas, custom APIs, hooks, schedulers, users and roles are stored in API Maker DB, a MongoDB replica set. Workers keep them in memory, so serving a request needs no round trip to it. When something changes, an event goes through the internal Redis and every worker of every server reloads it.

  3. Events and cache in separate Redis.

    redisInternal is what API Maker needs to work: events between workers and servers, locks of the schedulers, WebSocket subscriptions and auto increment values. redisExternal holds cached responses of your APIs, capped by maxCharsResToCache and expired after redisValueExpireInSeconds. Apart, cache pressure never touches the rest.

  4. Every call is logged, out of the way.

    API Maker logs every call: request, response, duration, errors and console output of custom code. With a logs database of its own, this write-heavy traffic never competes with your data. A daily scheduler keeps the newest maxLogsCount entries, and the log explorer of the admin panel searches them.

  5. Rebuild the API server in minutes.

    Because nothing lives on it, the API server can be replaced at any time: to recover from a failure, to move to another provider or to a bigger machine. Install API Maker on a fresh VPS with the same .env file, and it serves every API at once. Nothing to restore, nothing to migrate.

Why choose this architecture

A disposable API server

Nothing lives on it but code: rebuild, resize or move it in minutes with the same .env file.

Each store sized for its job

MongoDB gets fast disks, Redis gets memory, logs get room to grow: every server is sized for what it does.

Logs out of the way

Every call is logged to a database of its own and trimmed daily, without touching your data.

Cache apart from events

Responses live in their own Redis, so evicting cache never touches locks, events or WebSocket subscriptions.

Ready to scale out

A second API server with the same .env serves the same APIs at once. The next architectures do exactly that.

Less exposed

Only the API server faces the internet. The data servers accept its private IP and nothing else.

Server configuration

Plain VPS from any provider, or your own servers, with Ubuntu 22.04 LTS. Sizes are a starting point: measure and adjust.

API server

×1
CPU
4 vCPU
Memory
8 GB
Storage
40 GB SSD

Runs

  • API Maker backend, cpuCount AUTO
  • Admin panel
  • Docker: sandbox containers of custom code

Ports

  • 38246: HTTP APIs
  • 38245: WebSocket
  • 4626: admin panel, your IPs only

MongoDB server

×1
CPU
2 – 4 vCPU
Memory
8 GB
Storage
100 GB NVMe

Runs

  • MongoDB 6, replica set rs0
  • api_maker_db: APIs, schemas, settings, users
  • api_maker_logs: a log of every call

Ports

  • 27017: from the API server only

Three members make it highly available: see the High availability architecture.

Redis server

×1
CPU
2 vCPU
Memory
4 – 8 GB
Storage
20 GB SSD

Runs

  • Redis 7 internal: events, locks, WebSocket subscriptions, auto increments
  • Redis 7 cache: responses of your APIs

Ports

  • 6379 and 6390: from the API server only

Database server

1 per database
CPU
4 – 8 vCPU
Memory
16 – 32 GB
Storage
200 GB+ NVMe

Runs

  • Your databases: any of the 8 supported types

Ports

  • From the API server only
Settings of the API server.env
am__serverName="api-1"am__cpuCount="AUTO"# Definitions and logs: the MongoDB serveram__mongo_db_connection="mongodb://api_maker:***@10.0.0.20:27017/api_maker_db?authSource=admin&replicaSet=rs0"am__logs__mongo_db_connection_logs="mongodb://api_maker:***@10.0.0.20:27017/api_maker_logs?authSource=admin&replicaSet=rs0"am__logs__maxLogsCount="100000"                     # newest logs keptam__logs__logRemoveSchedulerInterval="0 0 0 * * *"  # trimmed every day# Events and locks, and the cache: the Redis serveram__redisInternal='{"nodes": [{host: "10.0.0.30", port: 6379, pass: "***"}]}'am__redisExternal='{"nodes": [{host: "10.0.0.30", port: 6390, pass: "***"}], redisValueExpireInSeconds: 7200, maxCharsResToCache: 1000000}'

The "Encrypt Strings" tool of the dashboard encrypts these connection strings before they go into .env.

Good to know

  • Still one API server: when it stops, the apps wait for a new one. The next architecture runs several.
  • MongoDB and Redis are single servers here: back them up, or make them clusters as the Resilient architectures show.
  • Four servers to run instead of one or two.
When you outgrow itOne API server is no longer enough for your traffic, or for your uptime. Load-balanced API servers

Questions

What exactly is stored in the MongoDB of API Maker?

Everything you build in the admin panel: instances and their encrypted connection strings, schemas, custom APIs, hooks, schedulers, WebSocket events, users, roles and settings. That is why any API Maker server that points to it serves the same APIs.

Why two Redis?

redisInternal carries what API Maker needs to work: events between workers and servers, locks of the schedulers, WebSocket subscriptions and auto increment values. redisExternal holds the cached responses of your APIs, which can be evicted at any time. On small setups both can be the same Redis.

Can I turn logs off or keep fewer of them?

Yes. logs.enableLogs turns them off, logs.maxLogsCount sets how many of the newest logs are kept, and logs.logRemoveSchedulerInterval sets when older ones are removed, every day by default.