# API Maker > The framework for AI era. API Maker connects to your databases and gives every table secure REST APIs at once. You add your own logic in TypeScript: custom APIs, pre and post hooks, schedulers and events. Caching, access control, logging, monitoring and Git deployment are built in, all run from one admin panel on your own servers. - Databases: MongoDB, MySQL, MariaDB, PostgreSQL, SQL Server, Oracle, TiDB and Percona XtraDB. - Generated APIs: 17 per MongoDB collection and 14 per SQL table under /api/gen, and the same again under /api/schema for tables with a schema. Every API needs a token unless you make it public. - Install: on a blank Ubuntu VPS with the install script, from npm (@sava-info-systems/api-maker), or on your computer with the API Maker Local Run desktop app for macOS, Windows and Linux (https://apimaker.dev/download). - Documentation: https://docs.apimaker.dev - Made by SAVA Info Systems. Contact: contact@apimaker.dev Short index of this site: https://apimaker.dev/llms.txt # Features ## Auto Generated APIs URL: https://apimaker.dev/auto-generated-apis-from-database Connect a database. Every table and collection gets its REST APIs at once. API Maker reads your databases and serves APIs for every table: list, read, save, update, delete, query, count, distinct values and streams. Use them straight on the table, or as schema APIs that check every write. No code to write for any of it. ### At a glance - Databases: MongoDB, MySQL, MariaDB, PostgreSQL, SQL Server, Oracle, TiDB, Percona XtraDB - APIs per table: 17 on MongoDB, 14 on SQL, twice with a schema - Paths: /api/gen/… and /api/schema/… - Access: Token required by default ### How it works 1. Connect any of 8 databases. Pick the connection string from your Secret. API Maker connects and lists every database, table and collection. 2. Every table gets its APIs at once. A MongoDB collection gets 17 generated APIs, from get all and streams to aggregate. A SQL table gets 14. No code. 3. Schema APIs double them. The schema is detected from the data (the first 100 documents, or the table columns). Its 17 schema APIs validate every write. 4. Ask for exactly the data you need. find, sort, limit and select shape the answer on the server. skip, deep and getTotalCount are there too. 5. Invalid data never reaches the table. A schema API answers 400 with a clear message for every field, and saves valid data with 201. 6. Stream millions of rows. The stream APIs read and send 1000 rows at a time and wait for the client, so memory stays flat whatever the size. ### What you get - Every operation you need: Get all, get by id, save one or many, master save, update by id, update many, remove by id or by query, query, count and distinct. On MongoDB also replace by id, array operations and aggregate. - Generated or schema APIs: Generated APIs work straight on the table. Schema APIs use the schema of the table to convert and validate every write, and to follow relations. The schema is detected for you and stays editable. - One query language: find, sort, skip, limit and select work the same on every database, with operators like $gt, $in, $and and $or. getTotalCount adds the total for paging. - Streams for big results: The stream APIs read 1000 rows at a time and wait for the client, so exporting millions of rows keeps memory flat. - Related data in one call: Populate related rows with deep, or filter on fields of related tables with find, even when they live in another database. - Secure from the start: Every API needs a token unless you make it public, and groups decide which APIs and fields each user can use. ### The APIs of every table Every table gets these APIs under /api/gen. Tables with a schema get them again under /api/schema, where each request goes through the schema first. The examples use the products table of the main database of an instance named shop, and admin as the API path of the account. Base path: `/api/gen///` | API | Method | Path | What it does | |---|---|---|---| | Get all | GET | `/:table` | Rows of the table, filtered, sorted, paged and trimmed to some fields by the query string. | | Get all by stream | GET | `/:table/stream` | The same filters as get all. The rows are sent as a stream, for exports of any size. | | Get by id | GET | `/:table/get-by-id/:id` | One row by its id. Add a column name after the id to find the row by that column instead. | | Save single or multiple | POST | `/:table/save-single-or-multiple` | Insert one row, or an array of rows in one call. | | Master save | POST | `/:table/master-save` | A row with its related rows, following the relations of the schema: rows with an id are updated, the others inserted. | | Array operations (MongoDB only) | PUT | `/:table/array-operations` | Push, add to set, pull, pop or set items of an array field, in the documents that match find. | | Update many | PUT | `/:table/update-many` | Set the same fields on every row that matches find. | | Update by id | PUT | `/:table/update-by-id/:id` | Change some fields of one row. upsert and returnDocument go in the query string. | | Replace by id (MongoDB only) | PUT | `/:table/replace-by-id/:id` | Replace a whole document with the body. | | Remove by id | DELETE | `/:table/:id` | Remove one row by its id. | | Query | POST | `/:table/query` | find, sort, skip, limit, select and deep in a JSON body, for filters too long for a URL. | | Query by stream | POST | `/:table/query-stream` | The query API, with the rows sent as a stream. | | Remove by query | POST | `/:table/query/delete` | Remove every row that matches find. A missing or empty find is refused. | | Aggregate (MongoDB only) | POST | `/:table/aggregate` | Run a MongoDB aggregation pipeline on the collection. | | Count | POST | `/:table/count` | How many rows match find. | | Distinct | GET | `/:table/distinct/:field` | Unique values of one or more fields. Add /asc or /desc after the fields to choose the order. | | Distinct with query | POST | `/:table/distinct/:field` | Unique values of the fields, among the rows that match find. | Get all: ```http GET /api/gen/admin/shop/main/products?find={stock:{$gt:0}}&sort=-price&limit=10&select=name,price ``` Get all by stream: ```http GET /api/gen/admin/shop/main/products/stream?find={category:'mouse'} ``` Get by id: ```http GET /api/gen/admin/shop/main/products/get-by-id/1042 ``` Save single or multiple: ```http POST /api/gen/admin/shop/main/products/save-single-or-multiple [{ "name": "Mouse", "price": 19 }, { "name": "Keyboard", "price": 49 }] ``` Master save: ```http POST /api/schema/admin/shop/main/products/master-save { "name": "Mouse", "price": 19, "brand_id": { "name": "Logi" } } ``` Array operations: ```http PUT /api/gen/admin/shop/main/products/array-operations { "find": { "_id": 1042 }, "operations": [{ "operation": "push", "path": "tags", "dataToPush": ["wireless"] }] } ``` Update many: ```http PUT /api/gen/admin/shop/main/products/update-many { "find": { "category": "mouse" }, "updateData": { "discount": 10 } } ``` Update by id: ```http PUT /api/gen/admin/shop/main/products/update-by-id/1042 { "price": 17 } ``` Replace by id: ```http PUT /api/gen/admin/shop/main/products/replace-by-id/1042 { "name": "Mouse", "price": 17, "stock": 40 } ``` Remove by id: ```http DELETE /api/gen/admin/shop/main/products/1042 ``` Query: ```http POST /api/gen/admin/shop/main/products/query { "find": { "price": { "$lt": 50 } }, "sort": { "price": 1 }, "limit": 10 } ``` Query by stream: ```http POST /api/gen/admin/shop/main/products/query-stream { "find": { "stock": { "$gt": 0 } } } ``` Remove by query: ```http POST /api/gen/admin/shop/main/products/query/delete { "find": { "stock": 0 } } ``` Aggregate: ```http POST /api/gen/admin/shop/main/products/aggregate [{ "$match": { "stock": { "$gt": 0 } } }, { "$group": { "_id": "$category", "count": { "$sum": 1 } } }] ``` Count: ```http POST /api/gen/admin/shop/main/products/count { "find": { "category": "mouse" } } ``` Distinct: ```http GET /api/gen/admin/shop/main/products/distinct/category ``` Distinct with query: ```http POST /api/gen/admin/shop/main/products/distinct/brand { "find": { "category": "mouse" } } ``` ### Example: The 10 cheapest products in stock, with two fields and the total ```http GET /api/gen/admin/shop/main/products?find={"stock":{"$gt":0}}&sort=price&limit=10&select=name,price&getTotalCount=true x-am-authorization: ``` admin is the API path of your account, shop the instance, main the database and products the table. ### Example: The same filter in the body of the query API ```http POST /api/schema/admin/shop/main/products/query x-am-authorization: { "find": { "stock": { "$gt": 0 }, "category": { "$in": ["mouse", "keyboard"] } }, "sort": { "price": 1 }, "limit": 10 } ``` ### Example: The answer of every API has the same shape ```json { "success": true, "statusCode": 200, "data": [ { "_id": "66f1c2…", "name": "Mouse", "price": 19 }, { "_id": "66f1c3…", "name": "Keyboard", "price": 49 } ], "totalCount": 42 } ``` ### Good to know - Replace by id, array operations and aggregate are MongoDB only, which is why a SQL table gets 14 APIs instead of 17. - Schema APIs need a schema for the table. Generated APIs work without one, and do not validate what you save. - TiDB and Percona XtraDB are connected as MySQL instances. ### Questions **What is the difference between generated and schema APIs?** Generated APIs (/api/gen) run your request on the table as it is. Schema APIs (/api/schema) first apply the schema of the table: type conversions, default values, validations, encryption and relations. Both exist for every table that has a schema. **Where does the schema come from?** API Maker detects it: from the columns of a SQL table, or from the first 100 documents of a MongoDB collection. You can edit it afterwards to add validations and relations. **Do I have to restart anything when I add a table?** No. The generated APIs are generic routes, so a new table is served by the same handlers. **Can I add my own logic to these APIs?** Yes. Pre and post hooks run your TypeScript before and after any generated API, and custom APIs cover everything else. ### Documentation - [Generated APIs](https://docs.apimaker.dev/v1/docs/apis-all/generated-apis/auto-generated-get-all-api.html) - [Schema APIs](https://docs.apimaker.dev/v1/docs/apis-all/schema-apis/auto-generated-schema-based-get-all-api.html) - [Query parameters](https://docs.apimaker.dev/v1/docs/apis-all/query-params/query-params.html) - [find](https://docs.apimaker.dev/v1/docs/apis-all/query-params/find.html) - [Connection strings](https://docs.apimaker.dev/v1/docs/Database-connection-string/mongodb-connection-strings.html) ## Schema & Validation URL: https://apimaker.dev/schema-validation Describe a table once. Every write through the schema APIs is converted and checked for you. The schema of a table is a TypeScript object: the type of each field, how to clean it, its default, its rules and its relations. API Maker detects a first version from your data. From then on, the schema APIs trim, convert, validate, encrypt and number your data before it reaches the database. ### At a glance - Written in: TypeScript, in the admin panel - First version: Detected from your table - Types: string, number, boolean, date, objectId, arrays, objects - Applied by: The schema APIs (/api/schema/…) ### How it works 1. Your table, described in TypeScript. API Maker detects a first schema from your table. You add the types, conversions, rules and relations, and every write through the schema APIs follows them. 2. The data is cleaned first. Keys that are not in the schema are refused. Values get their type, "25" becomes 25, and strings are trimmed or cased. 3. Your own conversion function. conversionFun(value, all) gets the value and the whole object, and its return is stored: "Wireless Mouse" becomes the slug wireless-mouse. 4. Ids, numbers and secrets are filled in. Fields are encrypted or hashed with your secret, ids are generated, numbers incremented, defaults set. Then the rules pass and the row is saved. 5. Your own rules, and every error at once. validatorFun(value, all) refuses a value by returning false or throwing: the message you throw goes to the caller, next to the errors of the other rules. 6. Safe concurrent updates. The field marked isConcurrencyControlField is a version. An update sends the version it read: when the row changed meanwhile, API Maker refuses it instead of overwriting the other change. ### What you get - Conversions: trim, trimStart, trimEnd, toLowerCase and toUpperCase, default values or default functions, and conversionFun to compute a value, like a slug from a name. - Validations: required, min, max, minLength, maxLength, email and enum, plus validatorFun for rules that need the whole object. - Relations: instance, database, collection or table, and column on a field point to another table, even in another database. Deep populate, find and join and master save follow them. - Numbers and ids: isAutoIncrementByAM counts from your start value with your step. isAutoGenerateByAM creates a UUID, ObjectID, ULID or short UUID. - Encryption and hashing: encryption stores a field encrypted with the key of your secret (AES, RC4 or TripleDES); decrypt it with g.sys.system.decrypt when you need it. hashing stores a one-way HMAC SHA-256 hash. - Safe concurrent updates: Mark a version field with isConcurrencyControlField: an update carrying an old version fails instead of overwriting someone else's change. ### A real life example One set of rules for every client: A web app, a mobile app and a partner integration all save products. The rules live in the schema, so each of them gets the same checks, the same defaults and the same error messages, and none of them can store a product without a category. ### Example: Schema of a products table ```ts import { EType, ISchemaType, ISchemaProperty } from 'types'; const schema: ISchemaType = { _id: { __type: EType.objectId, isPrimaryKey: true, isAutoGenerateByAM: { valueGeneratorType: 'ObjectID' } }, product_no: { __type: EType.number, isAutoIncrementByAM: { start: 1000, step: 1 } }, name: { __type: EType.string, validations: { required: true, minLength: 2, maxLength: 120 }, conversions: { trim: true } }, status: { __type: EType.string, validations: { enum: ['DRAFT', 'ACTIVE', 'ARCHIVED'] }, conversions: { toUpperCase: true, defaults: { defaultValue: 'DRAFT' } } }, category: { __type: EType.objectId, validations: { required: true }, database: 'shop', collection: 'categories', column: '_id' }, slug: { __type: EType.string }, supplier_tax_id: { __type: EType.string, conversions: { encryption: true } }, version: { __type: EType.number, isConcurrencyControlField: true, conversions: { conversionFun: () => new Date().getTime() } }, }; module.exports = { schema }; ``` ### Example: What a bad save gets back ```json { "success": false, "statusCode": 400, "errors": [ { "type": "required", "field": "category", "code": 400, "message": "Please provide valid 'category' field with type 'objectId'." }, { "type": "enumValidation", "field": "status", "code": 400, "message": "Property 'status' should have any value from [DRAFT, ACTIVE, ARCHIVED]." } ] } ``` Messages come from the language pack of the caller, in the language it asks for. ### Good to know - The rules are applied by the schema APIs. Generated APIs (/api/gen) write what they get. - An encrypted field can not be searched by its value. Store a hashed copy next to it when you need to find rows by it. - Optimistic concurrency control does not apply to update many, which updates rows directly in the database. ### Questions **Can I check data without saving it?** Yes. The system API is-valid-data-for-table (g.sys.system.isValidDataForTable) runs the same conversions and validations and returns the errors. **What happens to a field that is not in the schema?** The schema APIs refuse it with a schemaKeyNotFound error, "Property not found in schema for key …", instead of storing it. **Does the schema also drive relations?** Yes. instance, database, collection or table, and column on a field point to another table. Deep populate, find and join and master save follow them, even into another database. **Is optimistic concurrency control on for my own code?** Not by default: calls from custom code skip it. Pass skipConcurrencyControl: false to use it there too. ### Documentation - [Table schema](https://docs.apimaker.dev/v1/docs/schema/schema.html) - [Auto increment](https://docs.apimaker.dev/v1/docs/features/auto-increment.html) - [Optimistic concurrency control](https://docs.apimaker.dev/v1/docs/features/optimistic-concurrency-control.html) - [Is valid data for table API](https://docs.apimaker.dev/v1/docs/apis-all/system-apis/system-generated-is-valid-data-for-table-api.html) ## Deep Populate URL: https://apimaker.dev/deep-populate Get a row with its related rows in one request, even from other databases. Add deep to a request and API Maker replaces the ids in the answer with the rows they point to, level after level. The ids of all rows are read together, with one query per level, whether the related table is in the same database or on another server. ### At a glance - Parameter: deep - Levels: As many as you nest - Queries: One per level, not per row - Across databases: Any instance of your account ### How it works 1. Say what to populate, in one request. deep names the key to follow and where its rows live: another instance, database and type. Levels nest. 2. The first table is read as usual. API Maker reads the cities from PostgreSQL. Each row holds a state_id: a key into Oracle. 3. Keys are collected, then read in one query. The state ids of all rows go in a single $in query to Oracle. The rows come back keyed by id and replace the numbers. 4. Every level repeats it, in any database. Inside each state, country_id is read from MySQL the same way: one query with $in, however many rows point to it. 5. One query per level, not per row. Three levels cost three queries. Reading row by row would have cost seven here, and thousands on real data. 6. Filter and shape every level. Each deep item takes find, select, sort, skip, limit and isMultiple. With relations in the schema, deep=state_id is enough. ### What you get - No N+1 queries: The ids of every row of a level are collected and read with a single $in query. Three levels cost three queries, whatever the number of rows. - Across databases: Cities in PostgreSQL, states in Oracle, countries in MySQL: each level can point to another instance and database type. - Shape every level: Each deep item takes find, select, sort, skip and limit, and isMultiple to get an array instead of one object. - Short with a schema: When the schema holds the relation, deep=state_id is enough. Without a schema, name the target with t_instance, t_db, t_col and t_key. - Parent to children too: Virtual fields in the schema bring the rows pointing to a row, like the states of a country, in chunks of 1000 by default. - Rules still apply: The field access of the API user is applied to populated rows as well, so hidden fields stay hidden. ### A real life example An order screen: An order page needs the customer, the city of the customer and the product of every order line. One query API call with deep on customer_id and on the lines returns all of it, instead of one request per customer and per product. ### Example: Cities with their state, and the country of each state ```http POST /api/gen/admin/postgresql/geo/cities/query x-am-authorization: { "find": {}, "deep": [{ "s_key": "state_id", "t_instance": "oracle", "t_db": "inventory", "t_col": "states", "t_key": "id", "deep": [{ "s_key": "country_id", "t_instance": "mysql", "t_db": "inventory", "t_col": "countries", "t_key": "id", "select": "country_name" }] }] } ``` ### Example: The same with relations in the schema ```http GET /api/schema/admin/postgresql/geo/cities?deep=state_id ``` With schema APIs the table state_id points to comes from the schema of cities. ### Example: Answer ```json { "success": true, "statusCode": 200, "data": [{ "id": 101, "city_name": "AHMEDABAD", "state_id": { "id": 201, "state_name": "GUJARAT", "country_id": { "id": 301, "country_name": "INDIA" } } }] } ``` ### Good to know - deep works on get all, get by id, query and their streams, and on the answers of save, master save, update, replace and remove by id. - skip and limit inside a deep item are applied after the rows of the level are read, because all ids are read at once. - deep only reaches the instances of your own account. ### Questions **How many database queries does deep make?** One per deep item and level. The ids of all the rows of a level are sent in one $in query. **Can I populate two fields at once?** Yes. deep is an array: add one item per field, each with its own target and its own nested deep. **Do I need a schema?** No. With generated APIs you name the target table in the deep item. With schema APIs the relations of the schema are used, and the target tables need a schema too. ### Documentation - [deep parameter](https://docs.apimaker.dev/v1/docs/apis-all/query-params/deep.html) - [Table schema and relations](https://docs.apimaker.dev/v1/docs/schema/schema.html) ## Find and Join URL: https://apimaker.dev/find-and-join Filter a table on the fields of its related tables, even when they live in another database. With schema APIs, a find can follow relations: orders whose customer lives in Surat is one condition, customer_id.city_id.name = "Surat". API Maker reads the last table of the path first and walks the ids back, one $in query per table, across databases and database types. ### At a glance - How: Dotted paths in find - APIs: Schema APIs - Queries: One per table of the path - Across: Instances and database types ### How it works 1. Relations live in the schema. The schema of orders says customer_id points to customers.id in MySQL; customers.city_id points to cities in PostgreSQL. 2. Filter on a field of another table. One find on orders: customer_id.city_id.name = "Surat". API Maker cuts the path at every relation. 3. The last table of the path is read first. cities is queried with name = "Surat" and only its primary key is kept: [7]. 4. The ids walk back, one $in per table. customers with city_id $in [7] give [12, 15]; orders with customer_id $in [12, 15] are the answer. 5. One request, three databases, one answer. The orders of customers living in Surat, although orders, customers and cities are in MongoDB, MySQL and PostgreSQL. 6. Mix it with everything else. The same path works inside $and and $or, next to plain conditions, with sort, limit, getTotalCount and deep. ### What you get - Relations from the schema: The schema says where customer_id points. API Maker cuts the path at every relation and knows which table and database to query at each step. - Innermost table first: The condition runs on the last table of the path and keeps only its keys. Each table before it is then read with an $in of those keys. - No joins in the database: Because each step is its own query, the tables can be in MongoDB, MySQL and PostgreSQL at the same time. - Inside $and and $or: Dotted paths work next to plain conditions and inside $and and $or, with sort, limit, getTotalCount and deep. - On every reading API: Get all, query, both streams, count, distinct and distinct with query, and the writes that take a find: update many and remove by query. - Access checked on each table: The groups of the API user must allow every table of the path. A user without access to customers cannot filter orders by customers. ### A real life example A sales report by region: Sales wants the paid orders of the customers of one state, while orders are in MongoDB and customers are in a SQL database. One query with customer_id.state_id.name in find answers it, with no reporting database to build. ### Example: Orders of customers who live in Surat ```http POST /api/schema/admin/mongodb/shop/orders/query x-am-authorization: { "find": { "status": "paid", "customer_id.city_id.name": "Surat" }, "sort": { "createdAt": -1 }, "getTotalCount": true } ``` orders.customer_id points to customers in MySQL, customers.city_id to cities in PostgreSQL, as written in the schemas. ### Example: What API Maker runs for it ```text 1. PostgreSQL · cities name = "Surat" → ids [7] 2. MySQL · customers city_id $in [7] → ids [12, 15] 3. MongoDB · orders status = "paid", customer_id $in [12, 15] → the answer ``` ### Good to know - Dotted paths that cross tables need schema APIs: the relations come from the schemas of the tables. - Each table of the path is one query. A condition that matches many rows sends that many keys in the next $in. ### Questions **Is this a SQL JOIN?** No. API Maker runs one query per table and passes the keys from one to the next. That is why it also works between different databases and database types. **Can I write the join myself?** Yes. The query body of schema APIs also accepts find_join items with the target table, its find, and where to put the keys it returns. **Can I combine it with deep?** Yes. find decides which rows come back, deep fills in their related rows. ### Documentation - [Find and join](https://docs.apimaker.dev/v1/docs/apis-all/query-params/find.html#find-and-join-fields-of-related-tables) - [find parameter](https://docs.apimaker.dev/v1/docs/apis-all/query-params/find.html) - [Table schema and relations](https://docs.apimaker.dev/v1/docs/schema/schema.html) ## Master Save URL: https://apimaker.dev/master-save Save a whole tree of related objects in one call, across tables and databases. Send a city with its state and the country of that state as nested objects. API Maker saves the deepest object first, puts its new id in the parent, and goes up to the top. Objects that carry their key are updated instead. If a step fails, what was saved is removed and what was updated is restored. ### At a glance - Path: /api/schema/…/master-save - Payload: One object or an array, nested - Across: Tables, databases and database types - On failure: Saved rows removed, updates restored - Success: 201 Created ### How it works 1. Send the whole tree in one call. A city, its state and the country of that state, as nested objects. The schema knows which table, database and type each goes to. 2. The deepest object is saved first. The country goes to PostgreSQL and gets id 303. In the payload, the object is replaced by that id. 3. Each parent is saved with the ids of its children. The state is saved in SQL Server with country_id 303 and gets id 204, which takes its place in turn. 4. Three databases, one call, one answer. The city is saved in MySQL with state_id 204 and the API answers 201 Created with the saved row. 5. Objects with their key are updated. The state carries id 201 and its row exists: it is updated, its old values kept as a backup, and the new city points to it. 6. A failure reverts everything. The city has no city_name: after its country and state were saved, it fails validation. Both are deleted again and the API answers 400. ### What you get - Children first, parents after: The deepest object is saved first. Its new id replaces the object in its parent, which is saved next, and so on. - Insert or update, per object: An object without its primary key is inserted. An object with its key is updated, and its old values are kept to revert it if needed. - All or nothing: When any object fails, for example on validation, the rows saved by the call are deleted and the updated ones restored, then the error is returned. - Any mix of databases: The schema says where each nested object goes, so one payload can write to MySQL, SQL Server and PostgreSQL. - Validated on the way: Every object is converted and validated with the schema of its own table before it is written. - Children lists too: With virtual fields, a parent can carry the array of its children, like an invoice with its lines, in the same payload. ### A real life example An invoice form: A form creates an invoice with its lines and a new customer at once. One master save writes the customer, the invoice and every line in the right order. If one line is invalid, nothing of it stays in the database. ### Example: A city, its new state and an existing country, in one call ```http POST /api/schema/admin/mysql/geo/cities/master-save x-am-authorization: { "city_name": "Wembley", "state_id": { "state_name": "London", "country_id": { "id": 302, "country_name": "UK" } } } ``` The country has its id, so it is updated. The state has none, so it is inserted and its new id goes into the city. ### Good to know - Master save follows the relations of the schema: use it on schema APIs. Without a schema, the call only saves the top objects, like save single or multiple. - The revert is done by API Maker, not by a database transaction: other requests can read the new rows until they are reverted, and a revert that fails is reported in the errors. ### Questions **How is it different from save single or multiple?** Save inserts the objects you send into one table. Master save walks the nested objects into their own tables, inserts or updates each one, and links them with the new ids. **Can I send many trees at once?** Yes. Send an array: every item is saved with its own nested objects. **What does the answer contain?** The saved top object with 201 Created. Add select or deep to the call to shape what comes back. ### Documentation - [Master save API](https://docs.apimaker.dev/v1/docs/apis-all/schema-apis/auto-generated-schema-based-master-save-api.html) - [Table schema and relations](https://docs.apimaker.dev/v1/docs/schema/schema.html) ## Multi-Tenant URL: https://apimaker.dev/multi-tenant One set of APIs for all your customers, each with a database of its own. Mark an instance as a multi-tenant structure and point it to a tenants table. Each request names its tenant, and API Maker runs the same APIs, schemas, hooks and permissions on that tenant's database, whose connection string it reads from the tenants table. Customers share the servers and the APIs, not the databases. ### At a glance - Pick a tenant: crm::acme in the path, or a header - Tenants: Rows of a table of your own - Connection strings: Can be stored encrypted - Pools and cache: One per tenant - Databases: MongoDB, MySQL, MariaDB, PostgreSQL, SQL Server, Oracle ### How it works 1. The request names its tenant. crm::acme in the path: the instance crm, the tenant acme. The same API serves every customer. 2. API Maker reads the row of the tenant. It finds acme in the tenants table, with the find of the secret entry, and decrypts its connection string. 3. The database of the tenant answers. A pool opens for crm::acme and the query runs on the database of Acme. Only the customers of Acme come back. 4. Same API, another customer. globex comes in a header. Its row gives another connection string, another pool and another database. 5. The next requests reuse the pool. Acme asks again. Its pool is open, so the query goes straight to its database, without reading the tenants table. ### What you get - Isolation by design: Each customer has its own database, so there is no tenant_id filter to add to every query, or to forget. - One product to maintain: Schemas, hooks, settings, permissions and custom code are defined once and apply to every tenant. - Users per tenant: Your users can live in a users table inside each tenant database. A user token made for a tenant works for that tenant only. - Cache kept apart: The tenant is part of every cache key, so one customer never receives the cached answer of another. - Move a tenant any time: Copy its database, update its row, and call the multi-tenant-instance-updated system API: every server drops the pool of that tenant and clears the cache of the instance. - Tenants in the admin panel: Pick a tenant in the databases panel and on the API testing page to work with its data. Logs record the tenant of each call. ### A real life example A B2B SaaS: A CRM sold to companies keeps each company in its own PostgreSQL database. Small customers share a database server, a large one gets its own, and a regulated one keeps its database in its own data center. The product has one set of APIs, and onboarding a customer is creating its database and adding one row to the tenants table. ### Example: In the default secret: where the tenants are ```ts import * as T from 'types'; let Secret: T.ISecretType | any = { // … common and your other keys multiTenant: { crm: { instanceName: 'catalog', databaseName: 'public', collectionName: 'tenants', usernameColumn: 'username', connectionStringColumn: 'connection_string', find: { active: true }, }, }, }; module.exports = Secret; ``` Then choose multiTenant.crm as "Connection String Multi Tenant" on the instance crm. For Oracle, add the columns of the username and password of each tenant, and of its privilege when it connects with one. ### Example: The same API, the database of each customer ```http GET /api/gen/admin/crm::acme/crm/customers x-am-authorization: GET /api/gen/admin/crm/crm/customers x-am-authorization: x-am-tenant-username: globex ``` The first request runs on the database of Acme, the second on the database of Globex. The instance finds the tenant in its own tenants table, so the request names the tenant only. ### Example: After moving a tenant to another server ```ts await g.sys.system.multiTenantInstanceUpdated({ instanceName: 'crm', username: 'globex' }); ``` ### Good to know - A user token made for a tenant is refused on every other tenant and on the structure database. Tokens from a users table outside the multi-tenant instance are not tied to a tenant: to keep a caller to its own tenant, compare g.req.params.tenantUsername with the caller in a pre hook. - All tenants of a multi-tenant instance use the same database type as the instance. - A schema change has to run on every tenant database: plan your migrations for all of them. - Each API Maker process keeps a pool per active tenant. Size the connection limits of your database servers for it. ### Questions **How do I set it up?** Keep your tenants in a table that has a schema in API Maker, with a username column and a connection string column. Add an entry for that table to the default secret. Then check "Is Multi Tenant Structure Instance" on the instance and choose that entry in "Connection String Multi Tenant". **What does the instance itself connect to?** Its own connection string, the structure database. It holds the tables whose schemas every tenant uses. A request without a tenant works on it. **How does a user get a token for its tenant?** Call the token API with the tenant header and the name of a DB token generator on the users table of the multi-tenant instance. The user is read from the database of that tenant, and the token works for that tenant only, also after a refresh. **Where does the tenant come from in my frontend?** From the account of the signed-in user. Put it in the path (crm::acme) or in the x-am-tenant-username header of every request. **Can I have several multi-tenant instances?** Yes. Each one has its own secret entry and tenants table, and its own database type. **How are new tenants picked up?** The first request of a new tenant reads its row and opens its pool. No restart and no deployment. ### Documentation - [Multi-tenant](https://docs.apimaker.dev/v1/docs/features/multi-tenant.html) - [Secrets](https://docs.apimaker.dev/v1/docs/secrets/secrets.html) - [Table schema](https://docs.apimaker.dev/v1/docs/schema/schema.html) ## JSON, XML & YAML Output URL: https://apimaker.dev/get-output The client picks the format and the shape of the response with a header. Every API answers JSON by default. Send x-am-content-type-response: text/xml or text/yaml and the same response comes back as XML or YAML. Two more headers change the case of the keys and flatten nested objects, so each client gets data the way it reads it. ### At a glance - Formats: JSON, XML, YAML - Format header: x-am-content-type-response - Key case: x-am-response-case - Flatten: x-am-response-object-type: make_flat ### How it works 1. JSON by default. Every API answers { success, statusCode, data } in JSON: generated, custom and system APIs alike. 2. XML with one header. Send x-am-content-type-response: text/xml and the same response comes as XML, each array item in an _el element. 3. YAML the same way. text/yaml gives YAML, for tools and configuration systems that read it. No conversion code on either side. 4. Flat objects for flat consumers. x-am-response-object-type: make_flat joins nested keys with an underscore: address.city becomes address_city. 5. Keys in the case your app uses. x-am-response-case renames every key of the data: camelCase, PascalCase, snake_case, CONSTANT_CASE, param-case and more. 6. Combine them freely. API Maker flattens first, then changes the case, then writes the format. Here: flat, camelCase keys, in YAML. ### What you get - XML and YAML on request: text/xml (or application/xml) returns XML with a root element and each array item in an _el element. text/yaml returns YAML. JSON stays the default. - Keys in your case: camelCase, PascalCase, snake_case, CONSTANT_CASE, param-case, dot.case, path/case and more. Every key of the data is renamed, nested ones too. - Flat objects: make_flat joins nested keys with an underscore: address.city becomes address_city. Handy for grids, CSV and spreadsheets. - Every kind of API: Generated, schema, custom and system APIs all read the same headers. No conversion code in any of them. - Text, HTML and files from your code: A custom API sets g.res.contentType to answer plain text or an HTML page, or returns a file to download. - Applied in order: Flatten first, then the key case, then the format. The headers combine freely. ### A real life example One API, three clients: A web app reads JSON, a legacy ERP only imports XML, and an operations tool keeps its data in YAML. They all call the same API with their own header, and nobody writes a converter. ### Example: Ask for XML ```http GET /api/gen/admin/shop/main/products?limit=1&select=name,price x-am-authorization: x-am-content-type-response: text/xml ``` ### Example: The same response in XML ```http true 200 <_el> <_id>66f1c2… Mouse 19 ``` ### Example: Flat, camelCase, in YAML ```http GET /api/gen/admin/crm/main/customers?limit=1&select=name,address x-am-authorization: x-am-response-object-type: make_flat x-am-response-case: camelCase x-am-content-type-response: text/yaml ``` { "address": { "zip_code": "395007" } } becomes addressZipCode: "395007" in the answer. ### Good to know - Stream APIs always answer JSON. - The key case and flattening change the data of the response, not success, statusCode and the other fields around it. ### Questions **Which values does x-am-content-type-response take?** application/json (default), text/xml or application/xml, text/yaml, and text/plain or text/html for responses whose data is a string. **Which cases does x-am-response-case support?** noChange (default), camelCase, capitalCase, constantCase, dotCase, headerCase, noCase, paramCase, pascalCase, pathCase, sentenceCase and snakeCase. **Are these responses cached separately?** Yes. With caching on, these headers are part of the cache key, so a JSON client never gets a cached XML answer. ### Documentation - [Response content types](https://docs.apimaker.dev/v1/examples/res/contentType/contentType.html) - [Request headers](https://docs.apimaker.dev/v1/docs/apis-all/header/requestHeader.html) ## Custom APIs URL: https://apimaker.dev/custom-api Write a TypeScript function, save it, and the API is live. For logic that goes beyond the generated APIs, write a main function in the editor of API Maker. It gets the request in g.req and every database, cache and system API in g.sys. Save it and it answers at /api/custom-api/, with tokens, validation, hooks and logs like every other API. ### At a glance - Language: TypeScript - Path: /api/custom-api// - Methods: GET, POST, PUT, DELETE - Runs in: The sandbox, or the native process - Time limit: 13 s by default, per API ### How it works 1. Your logic is a TypeScript function. main(g) gets the request in g.req and API Maker in g.sys: databases, cache, system APIs, events. Return what the API answers. 2. Save it and the API is live. The TypeScript is compiled on save and POST /api/custom-api/admin/order-total answers at once. No build, no deployment. 3. Every call is checked first. The token is verified, the pre hooks run and the body is validated against the schema of the API, before your code. 4. Your code runs in the sandbox. In a Docker container, apart from API Maker, it queries MongoDB through g.sys.db like any API, and computes the total. 5. The answer has the usual shape. After the post hooks, the returned object becomes { success, statusCode, data }. The call is logged with its execution time. 6. Errors are yours to shape. No orders for customer 99: g.res.errors sets the message and the status code, and the API answers 404. 7. Keep versions, run the active one. Save a new version next to the old one, each with its own pre and post hooks. Activate it and the next call runs it; activate the old one to roll back. ### What you get - All your data through g.sys: g.sys.db calls every generated and schema API of every instance, g.sys.cache works with Redis, and g.sys.system encrypts, hashes, emits events and calls external APIs. - Input checked before your code: Give reqBodySchema and reqQueryParametersSchema in the settings: the body and query are converted and validated before main runs. - Uploads and downloads: Accept files with size and extension rules per field, and return a file or a zip of folders for the client to download. - Secured like the rest: A token is required by default. Make an API public, or callable only from your own code, and choose its auth providers. - Your npm packages: Add packages in the sandbox settings and import them. A heavy API can get a sandbox group of its own, with its own packages and Dockerfile. - Caching when it helps: Turn on enableCaching for a custom API and clear it when the tables or APIs it depends on change. - Versions: Keep several versions of a custom API, each with its own pre and post hooks. Calls run the active one: activate another to switch, or the old one to roll back. ### A real life example Checkout: A checkout API reads the cart, checks stock in one database, saves the order in another, calls the payment provider with a key from your secret and emits an event for the receipt email, all in one TypeScript function. ### Example: POST /api/custom-api/admin/order-total ```ts import * as T from 'types'; async function main(g: T.IAMGlobal) { const orders = await g.sys.db.query<{ total: number }>({ instance: 'mongodb', database: 'shop', collection: 'orders', find: { customer_id: g.req.body.customerId, status: 'paid' }, select: { total: 1 }, }); if (!orders.length) { g.res.errors = [{ code: 404, message: 'No paid orders for this customer.' }]; return; } return { orders: orders.length, total: orders.reduce((sum, o) => sum + o.total, 0) }; } module.exports = main; ``` The returned object becomes { success, statusCode, data }. g.sys.db calls throw on an error, or return the full response when you pass true as second argument. ### Example: Its settings ```ts import * as T from 'types'; import { EType } from 'types'; let customApi: T.ICustomApiSettingsTypes = { name: 'Order Total', requestMethod: T.ERequestMethod.POST, path: '/order-total', apiAccessType: T.EAPIAccessType.TOKEN_ACCESS, customApiTimeoutInSeconds: 5, reqBodySchema: { customerId: { __type: EType.number, validations: { required: true } }, }, errorList: ['No paid orders for this customer.'], }; module.exports = customApi; ``` ### Good to know - runOnNativeProcess skips the sandbox: faster and with the packages of API Maker itself, but code that blocks or leaks affects the whole server. Use it only for code you trust. - A call that passes its time limit is answered with a timeout error. Raise customApiTimeoutInSeconds for long work, or move it to a scheduler. ### Questions **Do I need to deploy after saving?** No. The TypeScript is compiled when you save and the next call runs the new code. Git deployment moves your custom APIs between environments. **Can a custom API call other custom APIs?** Yes, through g.sys, and it can import your utility classes to share code with them. **How do I test it?** Run it from the API testing page of API Maker with any body and headers, write test cases with mocks for its logic, and read its console output in the logs. ### Documentation - [Custom API](https://docs.apimaker.dev/v1/docs/apis-all/custom-apis/user-created-custom-api.html) - [Custom API settings](https://docs.apimaker.dev/v1/docs/settings/customApiSettings.html) - [Custom API examples](https://docs.apimaker.dev/v1/examples/custom-apis/custom-api.html) - [The global object g](https://docs.apimaker.dev/v1/docs/pre-defined-terms/global-object-g.html) ## Pre & Post Hooks URL: https://apimaker.dev/hooks Run your own code before and after any API, without touching the API itself. A pre hook runs before an API: it can check the request, change the body, or answer on its own. A post hook runs after it and can change the output. Hooks are TypeScript, work on generated, custom and system APIs, and apply on the next request after you save them. ### At a glance - Kinds: Pre hooks and post hooks - Database APIs: Instance, database, table and API levels - Other APIs: Custom and system APIs - Language: TypeScript, with g - Changes: Live on the next request ### How it works 1. Hooks on four levels. Put a hook on an instance and it runs for every table in it. On a table, for all its APIs. Or on one API. Generated, schema, custom and system APIs all have hooks. 2. Pre hooks run from the outside in. Instance, database, table, then the API. They read and change g.req.body. A hook with group names runs only for API users in those groups. 3. The API runs with the request as the hooks left it. The order is saved with created_by, stamped by the table hook: no client can forget it or fake it. 4. Post hooks run from the inside out. API level first, instance last. They read the result and can replace it with g.res.output: the internal cost never leaves the server. 5. Throw to stop a call. A pre hook that throws ends the call with your message. The API does not run, nothing is saved. 6. Return a value to answer the call. When a hook returns something, that is the response: the API and the post hooks do not run. A plain return only leaves the hook. ### What you get - Set once, apply everywhere: A hook on an instance runs for every table in it. A hook on a table runs for all its generated and schema APIs. - Only for some groups: Give a hook group names and it runs only for API users in one of those groups, for example audit logs for partners. - Full access to g: Hooks query other tables, read secrets, call external APIs, emit events and import your utility classes. - Switch on and off: Every hook has its own active flag. Turn one off to test without it: no restart, no deployment. - Know who is calling: g.req.isApiRequestFromUser tells a real client call from an internal one, so a hook can skip calls made by your own code. - Where it runs: Hooks run in the sandbox like custom APIs, or on the native process when you allow it for a trusted hook. ### A real life example Rules the database cannot hold: Orders must have lines, prices must come from the product table and not from the client, and every change must record who made it. Three pre hooks on the orders table enforce it for the generated APIs, the schema APIs and every app that calls them. ### Example: Pre hook on the orders table: stamp the user, refuse empty orders ```ts import * as T from 'types'; async function main(g: T.IAMGlobal) { if (!g.req.isApiRequestFromUser) return; // calls from your own code pass through const order = g.req.body; if (!order.lines?.length) throw 'An order needs at least one line.'; order.created_by = g.req.auth.authAMDB?.username; } module.exports = main; ``` ### Example: Post hook: hide the internal cost from the answer ```ts import * as T from 'types'; async function main(g: T.IAMGlobal) { const output = g.res.output; const rows = Array.isArray(output) ? output : [output]; for (const row of rows) delete row.internal_cost; g.res.output = output; // the answer the client gets } module.exports = main; ``` ### Good to know - Stream APIs run pre hooks only: their response is sent while it is read, so post hooks cannot change it. - A response served from the cache does not run the hooks. - Calls made from your code with g.sys.db skip hooks by default. Pass skipHookRunning: false to run them. ### Questions **Which APIs can have hooks?** Every generated and schema API, at the instance, database, table or API level, and every system API. Custom APIs get hooks per version. **In which order do several hooks run?** Pre hooks: instance, database, table, API, and from top to bottom inside a level. Post hooks: the other way round, API first and instance last. **How do I stop a request with an error?** Throw in the pre hook. The call stops and the client gets your message in errors. ### Documentation - [Pre hooks](https://docs.apimaker.dev/v1/docs/apis-all/hooks/preHook-api.html) - [Post hooks](https://docs.apimaker.dev/v1/docs/apis-all/hooks/postHook-api.html) - [The global object g](https://docs.apimaker.dev/v1/docs/pre-defined-terms/global-object-g.html) ## Utility Classes & Process Initializers URL: https://apimaker.dev/utility-classes Write shared code once and import it everywhere. Run setup code as soon as a sandbox starts. A utility class is TypeScript that your custom APIs, hooks, events, schedulers and tests import by its path. A process initializer is code that runs when a sandbox starts, before any request, to prepare what your code needs, like a subscription to Redis or MQTT. ### At a glance - Import by: Its folder path and name - Versions: Many per class, one active - Initializers run: When a sandbox starts - Order: You choose it ### How it works 1. Shared code in a utility class. Give it a folder and a name, write TypeScript and export an instance. Price rules, formatting and API clients live in one place. 2. Import it by its folder and name. Custom APIs, hooks, event listeners, schedulers and test cases import it like a module, with autocompletion in the editor. 3. Keep versions, activate one. Save a new version and activate it: every importer uses it at once, on every server. Go back by activating the old one. 4. Setup code for every sandbox. When a sandbox starts, the active process initializers run one after another, in your order, before it takes any request. 5. Ready before the first call. Ten seconds after API Maker starts it creates the sandboxes of accounts with initializers. A restarted sandbox runs them again: make them safe to run twice. ### What you get - Write it once: Price rules, formatting, API clients and validation helpers live in one place instead of being copied into every API. - Types included: Interfaces exported by a utility class can be used by every importer, and the editor suggests its functions. - Changes apply everywhere: Save or activate a version and the sandboxes load the new code, on every server of the cluster. - Listen to other systems: An initializer can connect to a message broker or a Redis channel and react to its messages, for as long as the sandbox lives. - Ready before the first call: Shortly after API Maker starts it warms up the sandboxes, so initializers do not wait for the first request. ### A real life example One tax rule for the whole shop: The checkout API, the invoice scheduler and the order hooks all need the same tax calculation. It lives in a utility class, and when the rate changes you save one new version. ### Example: A utility class ```ts class Calculator { add(x: number, y: number) { return x + y; } multiply(x: number, y: number) { return x * y; } } let temp = new Calculator(); export = temp; ``` ### Example: Used in a custom API ```ts import * as T from 'types'; import * as calc from 'utils/Calculator'; async function main(g: T.IAMGlobal) { return { sum: calc.add(3, 6), product: calc.multiply(3, 6) }; } module.exports = main; ``` ### Good to know - Initializers run in the shared sandboxes of the account. Sandboxes of a separate sandbox group do not run them. - An initializer runs again every time a sandbox is created or restarted: make it safe to run more than once. ### Questions **What is the difference between an initializer and a scheduler?** A scheduler runs on its intervals, once in the cluster. An initializer runs once in every new sandbox, without any trigger, to set that sandbox up. **Can utility classes use g?** Yes, when the importer passes it: call your functions with g from the custom API, hook or scheduler that uses them. ### Documentation - [Utility classes](https://docs.apimaker.dev/v1/docs/utility-class/utility-class.html) - [Process initializers](https://docs.apimaker.dev/v1/docs/features/process-initializers.html) ## System APIs URL: https://apimaker.dev/inbuilt-system Ready APIs for the jobs every backend has: encryption, secrets, Redis, cache, validation, tokens and events. API Maker comes with 21 system APIs. Your code calls them with g.sys, and you can open any of them to clients as a REST API under /api/system-api. They have settings and hooks like every other API. ### At a glance - System APIs: 21 - In code: g.sys.system and g.sys.cache - Over REST: POST /api/system-api// - From outside: Closed by default, except get token ### How it works 1. 21 system APIs, ready to use. Encryption, secrets and Redis, cache reset, data validation, events and utilities. Call them as REST APIs, or with g.sys in your code. 2. Encrypt, decrypt and hash, one line each. The algorithm and key come from your Secret: AES by default, RC4 or TripleDES. The hash is an HMAC SHA-256, it can not be reversed. 3. Secrets and Redis at hand. Read any key of your encrypted Secret. Set, read and remove Redis keys with a TTL: OTPs, sessions, locks. 4. Reset a cache when your logic needs it. The cache of a table, of a custom API, of system APIs or of third party APIs. You get back how many Redis keys were removed. 5. Check data without saving it. The validation of a save runs against the schema of a table, a custom API or a third party API, and every error comes back. 6. Events, tokens and external APIs. Emit events to their listeners or to WebSockets, get tokens for your users, call external APIs in parallel or in sequence. ### What you get - Encrypt, decrypt, hash: Encrypt and decrypt with the algorithm and key of your secret (AES by default, RC4 or TripleDES) or with your own. Hash with HMAC SHA-256. - Secrets and Redis: Read keys of your secret. Get, set and remove Redis keys with a TTL, and set only if absent: OTPs, sessions, locks. - Cache resets: Clear the cache of a table, a custom API or the system APIs. The answer says how many keys were removed. - Validation without saving: Run the validation of a table or a custom API on some data and get every error back. - Tokens: get token signs in API users and the users of your own tables, and returns a token, a refresh token and its expiry. - External APIs, events and more: Call external APIs in parallel or in sequence and pass values from one answer to the next request. Emit events and WebSocket events. Read table metadata, check a connection string, reset the pools of a moved tenant. ### Example: A one-time password: kept in Redis, sent with a key from the secret ```ts import * as T from 'types'; async function main(g: T.IAMGlobal) { const otp = String(Math.floor(100000 + Math.random() * 900000)); await g.sys.cache.setKey(`otp:${g.req.body.phone}`, otp, 300); // expires in 5 minutes const smsKey = await g.sys.system.getSecret('sms.apiKey'); const sent = await g.sys.system.callExternalApi({ url: 'https://sms.example.com/v1/messages', method: 'POST', headers: { Authorization: `Bearer ${smsKey}` }, body: { to: g.req.body.phone, text: `Your code is ${otp}` }, }); return { sent: sent.statusCode === 200 }; } module.exports = main; ``` ### Example: A system API over REST, once opened in its settings ```http POST /api/system-api/admin/hash-data x-am-authorization: { "name": "Joseph" } ``` The answer is the HMAC SHA-256 of the body, in hex, in data. ### Good to know - System APIs other than get token can not be called from outside until their settings give them an access type. From your code they can be called. - Encryption and hashing use the keys of your secret: changing them later does not re-encrypt data already stored. ### Questions **Which system APIs are there?** encrypt-data, decrypt-data, hash-data, token, call-external-api, get-secret-by-name, get-redis-key, set-redis-key, remove-redis-key, the four reset caches (database, custom APIs, system APIs, third party APIs), get-table-meta, emit-event, emit-event-ws, the three is-valid-data checks (table, custom API, third party API), multi-tenant-instance-updated and is-valid-connection-string. **Can my code run a raw query?** Yes. g.sys.system.executeQuery runs SQL on a SQL instance, or a command on MongoDB. **Can I cache or hook a system API?** Yes. Each system API has settings with enableCaching, its access type and auth providers, and pre and post hooks. ### Documentation - [All system APIs](https://docs.apimaker.dev/v1/docs/apis-all/overview.html#system-apis) - [System API settings](https://docs.apimaker.dev/v1/docs/settings/systemApiSettings.html) - [System API examples](https://docs.apimaker.dev/v1/examples/sys/system/system.html) - [Call external API](https://docs.apimaker.dev/v1/docs/apis-all/system-apis/system-generated-call-external-api.html) ## Schedulers URL: https://apimaker.dev/schedule Run your code on a timetable, once per cluster, without a cron server. A scheduler is TypeScript plus one or more intervals: every few seconds, every day at midnight, or your own cron. With several servers, one process owns each scheduler through a Redis lock, so every run happens once. If that server goes down, another one takes over. ### At a glance - Intervals: Presets or cron with seconds - Time zone: Per interval - Runs: Once per cluster - Time limit: 10 minutes by default ### How it works 1. A scheduler is your code and its intervals. Write TypeScript with g, add one or more intervals and press Ctrl + S. It is compiled and saved: no cron server to set up. 2. Presets or cron, in any time zone. From every 5 seconds to every year, or your own cron with seconds. Each interval has its time zone and can be switched off. 3. Many servers, one owner. Every process tries to pick schedulers under a Redis lock. The first one starts the jobs and marks the scheduler running, so each run happens once in the cluster. 4. On time, in the sandbox, logged. The owner runs your code in its sandbox with the scheduler timeout, 10 minutes by default, and saves the log of each run. It refreshes its Redis key every minute. 5. A server goes down, another takes over. Its key is not refreshed and expires after 75 seconds. A check every minute marks the scheduler as not running, and another process picks it up, a pickup runs every 5 minutes. 6. Save, and the next run uses the new code. The owner stops its jobs, a run in progress finishes, and the scheduler is picked up again 5 seconds later with the new code. No redeploy. ### What you get - Presets for the usual: Every 1, 5, 10, 15 or 30 seconds, every few minutes or hours, every day, working days, every month or every year. - Cron when you need it: Write your own cron expression with seconds. A scheduler can have several intervals, each with its time zone and an on/off switch. - Once, whatever the servers: Processes pick schedulers under a Redis lock. The owner runs them and refreshes its claim every minute. - Failover built in: When the owner stops refreshing its claim, it expires and another process picks the scheduler up. - Everything g offers: Query and update any database, call external APIs, emit events and use your utility classes, like in a custom API. - Every run logged: Runs appear in the logs like API calls, with their output or error when the log profile saves them. ### A real life example Nightly jobs without extra infrastructure: Expire old carts, send a daily sales report by email, sync prices from a supplier API every 15 minutes. Each one is a scheduler in API Maker, versioned with the rest of your project and deployed with it. ### Example: Every night: close carts left open for a day ```ts import * as T from 'types'; async function main(g: T.IAMGlobal) { const dayAgo = new Date(Date.now() - 24 * 60 * 60 * 1000); const result = await g.sys.db.updateMany({ instance: 'mongodb', database: 'shop', collection: 'carts', find: { status: 'open', updatedAt: { $lt: dayAgo } }, updateData: { status: 'expired' }, }); g.logger.log(`${result.updatedRowsCount} carts expired`); } module.exports = main; ``` Interval: every day, at midnight, Asia/Kolkata. Press Ctrl + S and it is scheduled. ### Good to know - Schedulers of developer accounts never run in the background. Developers run them by hand from API Maker. - A run that passes schedulerTimeoutInMinutes (10 by default) is stopped. Raise it for long jobs. ### Questions **What happens to a run when I save new code?** The run in progress finishes. The scheduler is then picked up again with the new code: no restart, no deployment. **Can I pause a scheduler?** Yes. Switch the scheduler off, or switch off one of its intervals. **Where does it run?** In the sandbox of the process that owns it, or on the native process when you allow it. ### Documentation - [Schedulers](https://docs.apimaker.dev/v1/docs/apis-all/schedulers/user-created-schedulers-api.html) ## Events & Listeners URL: https://apimaker.dev/events-management React to API calls with your own listeners, or emit events from any code. An event runs a list of TypeScript listeners after an API call, or when your code emits it. The caller always gets its response first. Listeners can emit more events, API Maker stops loops for you, and the code that emits an event gets the output of every listener back. ### At a glance - Events: Listeners in TypeScript - Triggered by: API calls, code or REST - When: After the response is sent - Loops: Stopped at once ### How it works 1. The response first, then the events. An event can run on every call of an API: generated, schema, custom or system. The caller gets its response first, the event gets that response as its data. 2. Listeners run one after another. Each listener is TypeScript run in the sandbox, with its own timeout. It reads the data in g.req.eventData: send a receipt, update the stock, call any API. 3. Events can emit events. Any listener can run another event with g.sys.system.emitEvent. Workflows grow step by step, each step in its own listener. 4. A loop is stopped at once. API Maker keeps the chain of events of each call. An event that is already in it is not run again, the caller gets the chain in the error. 5. Emit an event from any code, or over REST. Name the listeners to run only those. emitEvent returns the output of each listener. Other systems use the emit-event system API. ### What you get - Listeners in order: An event holds listeners that run one after another, each with its own time limit, in the sandbox or on the native process. They read the data in g.req.eventData. - Triggered by any API: Attach an event to generated, schema, custom or system APIs: it runs after every call, with the response as its data. - Emitted from code: g.sys.system.emitEvent(name, data, listeners) runs an event from any code, all its listeners or only the ones you name. - Outputs come back: emitEvent returns the output of each listener that ran, so a custom API can use the result of the workflow it started. - No endless loops: API Maker keeps the chain of events of each call. An event already in the chain is not run again, and the error shows the chain. - From other systems: The emit-event system API runs an event over REST, with the same data and the same choice of listeners. ### A real life example After an order is placed: The order API answers the shop at once. Then the event order-placed records a notification for the customer, sends the receipt and updates the stock, and a low stock emits stock-low, whose listener orders more from the supplier. ### Example: A listener of the event order-paid ```ts import * as T from 'types'; async function main(g: T.IAMGlobal) { const order = g.req.eventData; await g.sys.db.saveSingleOrMultiple({ instance: 'mongodb', database: 'shop', collection: 'notifications', saveData: { customer_id: order.customer_id, text: `Order ${order._id} is paid.` }, }); } module.exports = main; ``` ### Example: Resend a receipt: run one listener of an event ```ts import * as T from 'types'; async function main(g: T.IAMGlobal) { const order = g.req.body; // Only send-receipt runs : update-stock does not take the stock twice const out = await g.sys.system.emitEvent('order-placed', order, ['send-receipt']); return out.outputArr[0].output; // { listenerName, output } of each listener that ran } module.exports = main; ``` ### Good to know - Events run after the response, so they can not change it. Use a post hook for that. - Each listener has a time limit in minutes: split long work into several listeners or events. ### Questions **Do events slow down my APIs?** No. The response is written first. Events, WebSocket notifications and logs run after it. **Can I run only some listeners of an event?** Yes. Pass their names to g.sys.system.emitEvent or to the emit-event system API. A name that does not exist is an error. **How do I push an event to web and mobile apps?** With WebSockets: apps register for an API or a custom WebSocket event, and g.sys.system.emitEventWS pushes to them. ### Documentation - [Events](https://docs.apimaker.dev/v1/docs/apis-all/events/user-created-events-api.html) - [Emit event API](https://docs.apimaker.dev/v1/docs/apis-all/system-apis/system-generated-emit-event-api.html) ## WebSockets URL: https://apimaker.dev/websockets Live updates for web and mobile apps: subscribe to any API or your own events, get only what matches. Apps connect over WebSocket with their tokens and register for an API, a table or a custom event, with a condition on the response. After each successful call, API Maker pushes the data to the registrations that match, on whichever server holds the socket. No polling, and no socket server to build. ### At a glance - Connect with: API user token and user token - Register for: Tables, custom and system APIs, your events - Delivery: Only when the condition matches - Scale: Any server, through Redis ### How it works 1. Apps connect with their tokens. Web and mobile apps open wss:// with the API user token and the user token. Caddy ends TLS, API Maker checks the tokens and answers CONNECTED, or closes the socket. 2. Register for an API, a table and a condition. Each registration is checked: the WebSocket event of that API, the groups of the API user, the auth provider. It gets an eventId. 3. Only matching sockets get the data. After a successful call, the response goes to the caller first. Then the registrations whose criteria match it get a NOTIFICATION with the fields they selected. 4. One socket, a whole cluster. Registrations live in Redis. The process that ran the API publishes the notification, the one that holds the socket pushes it, on any server. 5. Push your own events from code. Create a custom WebSocket event, and emit it from a custom API, a hook, a listener or a scheduler with g.sys.system.emitEventWS. It returns how many sockets it reached. 6. Your code decides who may listen. The can user connect code of a WebSocket event refuses a registration with your own message. Apps stop listening with UNREGISTER and the eventId. ### What you get - Subscribe to any API: Register for the generated and schema APIs of a table, a custom API, a system API or a custom WebSocket event. Each registration gets an eventId. - Only what matters: A condition on the response picks the calls that notify a socket, and select picks the fields it receives. A kitchen screen gets the orders of its own store only. - Your own events: Create a custom WebSocket event and push it from any code with g.sys.system.emitEventWS(name, data), or from another system with the emit-event-ws system API. It returns how many sockets it reached. - Your access rules: A WebSocket event accepts the tokens of its auth providers and follows the groups of the API user. Its can user connect code refuses a registration with your own message. - Built for clusters: Registrations are kept in Redis. The process that runs the API publishes the notification, the one that holds the socket pushes it, on any server. - wss through Caddy: API Maker listens for WebSockets on port 38245. The installer puts it behind Caddy, which serves it as wss with certificates it renews by itself. ### A real life example A live kitchen screen: When an order is saved, the screen of its store shows it at once, the other stores see nothing, and the manager app gets "order ready" as soon as the kitchen marks it. Nothing polls, and the web servers can be many. ### Example: A kitchen screen: connect, register, listen ```ts const ws = new WebSocket('wss://ws.example.com/' + '?x-am-authorization=' + encodeURIComponent(apiUserToken) + '&x-am-user-authorization=' + encodeURIComponent(userToken)); ws.onmessage = e => { const msg = JSON.parse(e.data); if (msg.type === 'CONNECTED') { ws.send(JSON.stringify({ objType: 'REGISTER', onEvents: [{ eventType: 'INSTANCES', apiName: 'SCHEMA_POST_BULK_INSERT', instance: 'shop', database: 'main', collection: 'orders', condition: { conditionType: 'RESPONSE', criteria: { store_id: 7 } }, select: { _id: 1, items: 1 }, getEventData: true, }], })); } if (msg.type === 'REGISTER') console.log(msg.response.invalidOnEvents); // refused ones, with their errors if (msg.type === 'NOTIFICATION') showOrder(msg.response.eventData); }; ``` First add the WebSocket event of this API in API Info → WebSocket Events: registering for an API without one is refused. ### Example: Push your own event from a custom API ```ts import * as T from 'types'; async function main(g: T.IAMGlobal) { const { store_id, order_id } = g.req.body; // Reaches the sockets registered for order-ready with criteria { store_id } const reached = await g.sys.system.emitEventWS('order-ready', { store_id, order_id }); return { reached }; } module.exports = main; ``` The app registers with eventType: 'CUSTOM_WS_EVENTS' and apiName: 'order-ready'. Delivery is an exact match on the criteria: to reach several users, register them with the same criteria or emit once per user. ### Good to know - A condition compares flat values of the response: nested objects are not supported in its criteria. - No notification is sent for a call that fails before its response, for example when a pre hook throws. - Stream APIs can not be registered for notifications. ### Questions **Do WebSocket notifications slow down my APIs?** No. The response is written first. Events, WebSocket notifications and logs run after it. **Can a client connected to one server get notifications from another server?** Yes. Registrations are kept in Redis, so the server that runs the API finds the sockets to notify, wherever they are connected. **Are calls made from my code notified too?** Yes. A save made by a custom API, a hook or a listener notifies the sockets just like a call from an app. **How does an app stop listening?** It sends { objType: 'UNREGISTER', onEvents: [eventId] } with the eventIds it got, or closes the socket. ### Documentation - [WebSocket events](https://docs.apimaker.dev/v1/docs/pages/web-socket-event-page.html) - [Emit event WS API](https://docs.apimaker.dev/v1/docs/apis-all/system-apis/system-generated-emit-event-ws-api.html) ## Internationalization URL: https://apimaker.dev/internationalization Every error message in the language of the caller. Create a language pack in API Maker and translate the messages you want: validation messages, system errors and the errors of your own custom APIs. A client sends x-am-internationalization with the name of the pack and gets its errors in that language. ### At a glance - Header: x-am-internationalization - Value: Name or id of a language pack - Missing text: Falls back to the default English - Changes: Used on the next request ### How it works 1. Errors in the language of the caller. Send x-am-internationalization with the name of a language pack. The same request gets its error in English, Hindi, Spanish, Japanese or Chinese. 2. A pack maps error codes to your words. Each system error has a code and a template. A pack gives your own template for the codes you want, in any script: Unicode just works. 3. Missing translations never break. A code the pack does not have uses the Default English template, so a half done pack is already safe to use. 4. Even field names read well. The COMMON map of a pack replaces words in the message data: cust_nm becomes customer name. A pack can also reword the English messages for a region. 5. Your own API errors too. Errors your custom APIs return are listed in each pack to translate. Save a pack and the next request uses it: no restart. ### What you get - Every kind of message: Message templates of API Maker, constant errors, and the errors your custom APIs list in their errorList. - Starts filled in: A new pack comes with every message in English. Translate the ones you need; the others keep working. - Field names too: The COMMON map of a pack replaces words inside messages, so cust_nm can read as customer name, in any language. - Any language, any script: Packs are plain text mappings, so Hindi, Japanese, Arabic or Chinese work like English. ### A real life example One backend, users in many countries: A mobile app sends the language of the phone in x-am-internationalization. Validation errors and the messages of your own APIs come back in that language, and the app shows them as they are. ### Example: A Spanish pack, with only what you changed ```ts let mappings = { COMMON: { cust_nm: 'nombre del cliente' }, AM_00001: { template: `Proporcione un campo '{field}' válido de tipo '{type}'.` }, CUSTOM_API_ERRORS: { 'No paid orders for this customer.': 'No hay pedidos pagados para este cliente.', }, }; module.exports = mappings; ``` AM_00001 is the "required" message of the schema APIs. A new pack lists every key with its English text. ### Example: The client asks for Spanish ```http POST /api/schema/admin/crm/main/customers/save-single-or-multiple x-am-authorization: x-am-internationalization: es { "email": "ana@example.com" } ``` ### Good to know - Packs translate messages, not your data: the values stored in your tables are returned as they are. ### Questions **What if the header is missing?** The default English messages are used. **How do my own errors get translated?** List them in the errorList of your custom API, throw them from your code, and give them a translation in the pack. ### Documentation - [Internationalization](https://docs.apimaker.dev/v1/docs/i18/i18.html) ## Access Management URL: https://apimaker.dev/access-management Role-based access down to the field: who can call which API, and which fields they read or write. Every call carries an API user token, and often a user token for the person behind it. Groups say what they may do: which instances, databases, tables and APIs, and which fields they can read or write. API Maker checks it on every request, and a change to a group applies to the next one. ### At a glance - Application: API user token in x-am-authorization - Person: User token in x-am-user-authorization - Permissions: Groups, combined - Down to: Read and write per field ### How it works 1. Users get their access from groups. A group allows APIs: generated APIs of some tables or all of them, custom and system APIs. A user can be in several groups. 2. Only the APIs a group allows. priya's groups allow reading orders, not deleting them. The delete stops with 403 before anything runs. 3. Hidden fields leave the response. Analysts can read name and email of employees, not salary. salary is removed from every response omar gets. 4. No filtering or writing around the rules. Querying on salary, or writing email without write access, returns 403 with the field and the group in the message. 5. No token, no data. Changes are instant. A request without a valid token gets 401. Give Analysts read on salary and omar's next request already includes it: no restart. ### What you get - Grants that cascade: A group grants an instance, a database, a table, one generated API of a table, or single fields. Custom and system APIs, events, schedulers and WebSocket events are granted by name. - Field level read and write: Fields without read access leave every response. Writing a field, or filtering on it, without access is refused with 403. - Your users, your table: Sign in the users of your own users table and get a user token. Its groups column says which groups apply; * gives all the groups of the API user. - Other sign-in providers: Accept Google, Azure AD and AWS Cognito tokens, or your own token logic, and map those users to groups too. - Row level rules: Pre hooks restrict the rows a user reaches, for example to their own company, and can be skipped for a group of managers. The APIs Security Report finds tables without one and adds it in one click. - No restart: Change a group and the next request uses it. 401 means the token is missing or invalid, 403 that no group grants the call. ### A real life example An HR app: Everybody reads names and emails of employees, managers also read salaries, and only HR can change them. Three groups express it, the same APIs serve every screen, and nobody writes permission checks in code. ### Example: A call made for a signed-in person ```http GET /api/schema/admin/hr/main/employees?select=name,email,salary x-am-authorization: x-am-user-authorization: ``` If the groups of the person can not read salary, the answer comes without it. ### Example: Sign in a user of your own table ```http POST /api/system-api/admin/token { "name": "app_users", "u": "priya@example.com", "p": "••••••••" } ``` app_users is the name of the auth provider that points to your users table. The answer holds the token, a refresh token and its expiry. ### Good to know - Broad flags like all tables also grant the tables you add later. Prefer naming what a group needs. - Groups decide which APIs and fields, not which rows: rows are limited by pre hooks. The APIs Security Report shows the tables where no pre hook does it. ### Questions **What is the difference between the two tokens?** x-am-authorization identifies the application with an API user created in API Maker. x-am-user-authorization identifies the person, a row of your users table, whose groups column decides which groups apply to the call. **Can an API be public?** Yes. Set its access type to IS_PUBLIC. NO_ACCESS keeps an API for your own code only, TOKEN_ACCESS (the default) needs a token. **Where do API user passwords live?** In API Maker, or in your secret: an API user can read its password from a path of the secret. ### Documentation - [Handle role based permissions](https://docs.apimaker.dev/v1/docs/authorization/handle-role-based-permissions.html) - [API group permission](https://docs.apimaker.dev/v1/docs/apis-security/api-group-permission.html) - [API user permission](https://docs.apimaker.dev/v1/docs/apis-security/api-user-permission.html) - [Users of your database](https://docs.apimaker.dev/v1/docs/authorization/AMDB.html) - [Get token API](https://docs.apimaker.dev/v1/docs/apis-all/system-apis/system-generated-token-api.html) ## APIs Security Report & Actions URL: https://apimaker.dev/api-security-report Find where a person could read data that is not theirs, and fix it in one click. API Maker reads your groups, API users, auth providers, settings and the code of your pre-hooks, and shows where a person could read or change rows that are not theirs. A leak comes with the request that proves it, and many findings with an action that fixes them in one click. The report lives in your Git repository, where your AI assistant can read it too. ### At a glance - Reads: Groups, API users, auth providers, settings, hooks - Score: 100 minus the open findings, grade A to F - Fixes: One click, then a new scan - Report: Markdown and YAML in your Git repository ### How it works 1. It reads the whole account. Groups, API users, auth providers, settings and the code of every pre-hook, read and never run. Each open finding takes points off 100: 20 for a critical one, 10 high, 4 medium, 1 low. 2. Allowed is not the same as scoped. The group lets the shop app read orders and alice signs in with her own token, but no pre-hook limits the rows: she gets the orders of every customer. The finding comes with the request that proves it. 3. One click installs the fix. A row scoping pre-hook: the owner column goes into every filter with $and, the owner is stamped on every write, aggregate is refused. You can edit the code first. The account is scanned again right after. 4. Only her own rows now. The same request returns the two orders of alice. Asking for the orders of customer 23 matches nothing: her filter is joined to the owner with $and. Aggregate, which no filter can scope, gets 403. 5. Every finding has its action. Revoke a sensitive system API, block inline SQL, or skip a finding with a reason, like reference data everyone may read. A skipped finding leaves the score and its reason is kept in git. 6. It keeps watching while you work. With a local client connected, a change in the account is scanned within minutes and the report is committed to git. Your AI assistant reads it there, fixes the finding, and the next scan confirms it. ### What you get - Rows of the caller only: For every table your groups reach, it checks that an active pre-hook limits every API to the rows of the person behind the request, merges the filter with $and, stamps the owner on writes and refuses aggregate and distinct. - Groups that reach too far: Broad flags like all collections or all system APIs, sensitive system APIs such as EXECUTE_PLAIN_QUERY and GET_SECRET, bulk update and delete grants, and groups nobody uses. - Tokens and public APIs: Passwords inside tokens or stored in clear, providers without a groups column, tokens that live longer than 30 days, and APIs that need no token at all. - Actions that fix it: Add a row scoping pre-hook, add a column to a token, require a person token, freeze a group to an explicit allow-list, revoke grants or block inline SQL. You see the code before it is saved. - Skip with a reason: Some findings are fine, like reference data everyone may read. Skip them with a reason: they leave the score, and the reason is kept in git for your team. - Fresh for your AI assistant: While a local client is connected, changes are scanned on their own and the report is committed to git: README.md, report.md, report.yaml and actions.yaml. ### A real life example A shop with customer logins: Customers sign in with their own token and the app calls the generated APIs of orders. The report sees that no pre-hook limits orders to the rows of the caller, shows the request any customer could send to read every order, and installs the pre-hook in one click. The next scan confirms it. ### Example: The request that proves a finding ```bash # every row of "orders", with the token of any person allowed on it : curl "$BASE/api/schema/$ADMIN/shop/main/orders?limit=1000" \ -H "x-am-authorization: $APP_TOKEN" \ -H "x-am-user-authorization: $PERSON_TOKEN" ``` A finding about rows shows how it is exploited. Here any customer gets every order, because no pre-hook of orders reads the person behind the request. ### Example: The pre-hook that one click installs ```ts const OWNER_COLUMN = 'customer_id'; const IDENTITY_FIELD = 'id'; async function main(g: T.IAMGlobal) { // Custom APIs, schedulers and events run under their own rules : only real callers are scoped. if (!g.req.isApiRequestFromUser) return; const person = g.req.auth?.authAMDB; const me = person ? person[IDENTITY_FIELD] : undefined; // ... 401 without a person token const scope = { [OWNER_COLUMN]: me }; // MERGE, never replace : $and keeps what the caller asked for and adds the owner on top const merge = (existing: any) => (existing && typeof existing === 'object' && Object.keys(existing).length ? { $and: [existing, scope] } : { ...scope }); const apiId = g.req.reqInfo?.apiInfo?.id; if (QUERY_FIND_APIS.has(apiId)) { g.req.query.find = merge(g.req.query.find); stamp(g.req.body); // update-by-id and replace-by-id : the owner of a row can not be handed over return; } if (BODY_FIND_APIS.has(apiId)) { g.req.body = g.req.body || {}; g.req.body.find = merge(g.req.body.find); if (g.req.body.updateData) stamp(g.req.body.updateData); // update-many return; } // ... save and master-save : the owner is stamped on every row // Fail closed : aggregate and distinct can not be scoped generically, so they are refused. g.res.statusCode = T.EStatusCode.FORBIDDEN; throw new Error(`API ${apiId} is not available for row scoped users.`); } module.exports = main; ``` Generated for the owner column and the field of the token you pick, shortened here. Edit it before saving if you want: the report reads the code again on every scan. ### Example: The report in your repository ```text # API Security Report > Generated by API Maker. Read only : work on the findings below, then rescan from the admin panel. See README.md. - Generated at : 2026-09-29 10:00 UTC - Score : **52 / 100** (grade D, Data at risk) - Findings : 5 open, 0 skipped, 0 fixed ... ## Findings ### Row gate : does every collection limit persons to their own rows ? #### [CRITICAL] "orders" is not scoped to the rows of the caller ``` Next to it: README.md for people and AI assistants, report.yaml with the same content, and actions.yaml with the skipped findings and their reasons. ### Good to know - Nothing in a database says who owns a row: the owner column is a scored guess, from a relation to the users table, a column named after it, or names like user_id and created_by. Pick another column in the fix dialog when needed. - Aggregate and distinct can not be scoped by a filter: the generated pre-hook refuses them for row scoped callers. Serve those needs from a custom API. - When the token of a database auth provider does not carry the owner column, an action adds it to the token first. People get it at their next sign-in. - Automatic scans run while a local client is connected, and commit only when the account has a Git repository. ### Questions **Does the scan run my hooks or call my APIs?** No. It reads the settings of the account and parses the code of your pre-hooks as TypeScript, without running it. Nothing changes until you click an action. **How is the score computed?** Every open finding takes points off 100: 20 for a critical one, 10 for high, 4 for medium and 1 for low. Skipped and fixed findings take nothing. 90 and more is grade A, then B from 75, C from 60, D from 40, and F below. **What happens to a skipped finding?** It leaves the score and keeps its reason. The reason is written to actions.yaml in git, so your team, or an AI assistant working on the repository, knows the decision. You can open it again at any time. **How often do the automatic scans run?** While a local client is connected, the next scan comes 100 times the duration of the last one later, between 1 and 30 minutes. When nothing changed in the account, nothing is scanned, saved or committed. **Can an AI assistant work with the report?** Yes. It reads report.md in src/API Security Report/, fixes the hooks, groups or providers in their own folders and commits. The next scan shows the result. ### Documentation - [APIs Security Report & Actions](https://docs.apimaker.dev/v1/docs/apis-security/api-security-report.html) - [Handle role based permissions](https://docs.apimaker.dev/v1/docs/authorization/handle-role-based-permissions.html) - [API group permission](https://docs.apimaker.dev/v1/docs/apis-security/api-group-permission.html) - [Pre-hook](https://docs.apimaker.dev/v1/docs/apis-all/hooks/preHook-api.html) - [Users of your database](https://docs.apimaker.dev/v1/docs/authorization/AMDB.html) ## Single Sign-On URL: https://apimaker.dev/single-sign Let people sign in with Google, Microsoft or AWS, and use your API Maker permissions for them. Your app signs users in with Google, Azure AD or AWS Cognito and sends the token it gets. API Maker checks it with the keys of the provider, finds the user in your own table, and runs the call with the groups of that user. You store no passwords. For anything else, write your own token provider. ### At a glance - Providers: Google, Azure AD, AWS Cognito, custom - Headers: x-google-, x-azure-, x-aws-, x-custom-authorization - Permissions: Groups from a table of yours - In code: g.req.auth.authGoogle and others ### How it works 1. Users sign in with the account they have. Google, Microsoft Azure AD or AWS Cognito signs the user in and gives your app a token. No password for you to store. 2. The token goes in a header. x-google-authorization, x-azure-authorization or x-aws-authorization. The settings of each API name its auth providers, else the secret does. 3. Verified with the keys of the provider. For Google: the signature, the expiry, and that the token was made for your client id. Azure uses the keys of your tenant and app, AWS your Cognito user pool. 4. Mapped to your groups. A field of the token, here the email, finds the user in your own table. Its groups column gives the API Maker groups, * gives all of them. 5. The call runs with those permissions. Tables, columns and APIs follow the groups of the user. Your code reads the verified token in g.req.auth.authGoogle. 6. Everything else gets 401. No token, a token that is invalid or expired, a user without a row or without groups: the request stops before any data is read. ### What you get - Google: Give the OAuth client id. The signature, the expiry and the audience of the token are checked. - Azure AD: Give the application id, the tenant, the audience and the issuer. Tokens are checked with the keys of your tenant. - AWS Cognito: Give the user pool id and the region, and whether you send access or id tokens. - Mapped to your groups: A field of the token, like the email, finds the user in your table. Its groups column gives the API Maker groups, * gives all of them. - Custom providers: Write a token generator and a token validator in TypeScript, with any npm package, for tokens of any other system. The validator's result lands in g.req.auth.authCustom. - Per API or for all: The auth providers of your secret apply to every API, and the settings of an API, a table or a database can name others. ### A real life example Company sign-in for an internal tool: Employees open an internal dashboard and sign in with their Microsoft account. The dashboard sends the Azure AD token, API Maker finds each employee in the staff table, and their department decides which APIs and fields they can use. ### Example: A Google auth provider ```ts import * as T from 'types'; let googleTokenGenerator: T.IAuthTokenGoogle & { name: string } = { name: 'google_users', clientId: '1234567890-abc.apps.googleusercontent.com', sourceFieldOfUniqueId: 'email', groupsDataSource: { instance: 'mysql', database: 'crm', table: 'users', targetFieldForUniqueId: 'email', groupsColumn: 'groups', }, }; module.exports = googleTokenGenerator; ``` ### Example: The app calls with the Google token ```http GET /api/schema/admin/mysql/crm/orders x-am-authorization: x-google-authorization: ``` ### Good to know - A signed-in user needs a row in your groups table, with groups, to reach APIs that require their token. ### Questions **Do I still need an API user token?** Yes for token protected APIs: x-am-authorization identifies your app, the provider token identifies the person. **Where does my code find who is calling?** In g.req.auth: authGoogle, authAzure, authAWS, authCustom, and authAMDB for users of your own table. **What happens with an invalid token?** The call stops with 401 before any data is read: no token, a wrong or expired token, or a user without a row or groups. ### Documentation - [Google](https://docs.apimaker.dev/v1/docs/authorization/Google.html) - [Azure AD](https://docs.apimaker.dev/v1/docs/authorization/Azure.html) - [AWS Cognito](https://docs.apimaker.dev/v1/docs/authorization/AWS.html) - [Custom auth provider](https://docs.apimaker.dev/v1/examples/req/auth/authCustom.html) - [Single sign-on](https://docs.apimaker.dev/v1/docs/features/single-sign-on-authentication.html) ## Secrets Management URL: https://apimaker.dev/secrets-management Keys, passwords and connection strings in one encrypted place, never in your code or in Git. A secret is a TypeScript object of keys: database connection strings, encryption keys, sign-in settings and the API keys of the services you use. API Maker stores it encrypted, instances point to paths in it, and your code reads it with getSecret. Each environment and each developer has its own. ### At a glance - Written as: A TypeScript object - Stored: Encrypted in the database - Git: Never pushed - Pick one per request: x-am-secret header, else the default ### How it works 1. Every key in one secret. A secret is TypeScript: encryption and hashing keys, database connection strings, sign in settings, keys of Stripe or any service. Your code reads them with getSecret. 2. Encrypted on the way and at rest. The browser sends it encrypted. API Maker compiles it, runs it in the sandbox to get the keys, and stores all of it encrypted with the database secret. 3. Instances keep a path, not the password. The connection string of an instance is a path in the secret. API Maker reads the value from the secret when it connects. 4. One secret per environment. DEV, QA, UAT and PROD get their own secret, and every developer too. A request picks one with the x-am-secret header, else the default one is used. Secrets never go to Git. 5. Change it, it applies at once. Saving a secret resets every cache. The next request already uses the new connection string or key: no restart, no deployment. ### What you get - Everything sensitive in one place: Connection strings, the keys used for encryption and hashing, auth providers, API user passwords and the keys of third party services. - Instances point, not paste: The connection string of an instance is a path in the secret. The password never sits in the instance itself. - Read from code: g.sys.system.getSecret('stripe.secretKey') returns a value, anywhere you write code in API Maker. - One secret per environment: DEV, QA, UAT and PROD each get their own values, and so does every developer account. - Changes apply at once: Saving a secret resets the caches, so the next request already uses the new connection string or key. - Kept out of Git: Git deployment moves APIs, schemas and settings between environments, never the secrets. ### A real life example Same code, different environments: The payment key of production and the one of the test account live in two secrets. The code calls getSecret and each server gets its own key, so nothing has to change when a release moves from QA to production. ### Example: A secret ```ts import * as T from 'types'; let Secret: T.ISecretType | any = { common: { encryptionAlgorithm: 'AES', secret: '…', // used by encryption conversions and encrypt-data hashingAlgorithm: 'SHA256', connectionString: { mongodb: 'mongodb://user:password@10.0.0.5:27017/?authSource=admin', }, }, stripe: { secretKey: 'sk_live_…' }, }; module.exports = Secret; ``` An instance then uses common.connectionString.mongodb as its connection string. ### Good to know - The encryption keys of common are used for data already stored: changing them does not re-encrypt existing values. ### Questions **Can I have more than one secret?** Yes. One is the default. A request can pick another with the x-am-secret header, which carries the id of the secret. **Who can read a secret?** Code running in API Maker, and users of the admin panel with access to it. The get-secret-by-name system API is closed to outside callers unless you open it in its settings. ### Documentation - [Secrets](https://docs.apimaker.dev/v1/docs/secrets/secrets.html) - [Get secret API](https://docs.apimaker.dev/v1/docs/apis-all/system-apis/system-generated-get-secret-by-name-api.html) ## Secure Sandbox URL: https://apimaker.dev/secure-sandbox Your code runs in Docker containers, apart from API Maker, with time limits and automatic replacement. Custom APIs, hooks, event listeners, schedulers, migrations and test cases run in sandbox containers, not in the API Maker process. Every run has a time limit, a stuck sandbox is replaced, and each account builds its own image with the npm packages it needs. ### At a glance - Runs in: Docker containers - Time limit: 13 s by default - Packages: Your npm packages and Dockerfile - Debugging: Chrome DevTools, with breakpoints ### How it works 1. Your code runs outside API Maker. Custom APIs, hooks, event listeners, schedulers, migrations and test cases run in Docker containers. The worker sends the code and the request over TCP and gets the output back. 2. Data comes through API Maker. When your code calls g.sys.db.query, the call goes back to the worker, which runs the real API with its checks, and returns the rows to the sandbox. 3. Every run has a time limit. 13 seconds by default. Change it per request with the x-am-sandbox-timeout header, or in the settings of a custom API, a scheduler or a listener. 4. A stuck sandbox is replaced. It gets no new requests, the ones inside it finish, then it is removed and a new sandbox takes its place. The time to get a sandbox is not counted in the logs. 5. Your own image, your own packages. Each admin user gets a sandbox image built from a Dockerfile and the npm packages you add. Restart all sandboxes and they start from the new image. 6. Tune it to your load. Sandboxes per worker, a restart every N seconds, a separate sandbox for a heavy API, or the native process for code you trust. ### What you get - Apart from API Maker: An endless loop, a crash or a memory leak in your code stays in its container. API Maker keeps serving the other requests. - Time and memory limits: Every run has a time limit, per request or per API. Each sandbox has a heap limit: sandboxMaxOldSpaceMB, or the larger of 2500 MB and the memory of the server divided by its CPU count. - Replaced, not restarted in place: A sandbox that times out gets no new requests, the ones inside it finish, and a new one takes its place. You can also restart sandboxes every N seconds. - Your image: Add npm packages in the sandbox settings, edit the Dockerfile, and the image of your account is built with them. - Separate sandbox groups: Give a heavy custom API a sandbox group of its own, with its own packages, Dockerfile and restart interval. - Real debugging: Turn on debugging and attach Chrome DevTools to the sandbox: breakpoints, steps and variables in your TypeScript. ### A real life example A heavy report next to fast APIs: Generating PDFs needs a big package and a lot of memory. In its own sandbox group, it gets its package and restarts every hour, while every other API keeps its small, fast sandboxes. ### Example: A custom API with a sandbox group of its own ```ts import * as T from 'types'; let customApi: T.ICustomApiSettingsTypes = { name: 'Invoice PDF', requestMethod: T.ERequestMethod.POST, path: '/invoice-pdf', customApiTimeoutInSeconds: 60, separateSandboxSettings: { enableSeparateSandboxForThis: 'pdf_group', packages: { allowAllPackagesOfAdmin: false, sandboxPackages: [{ name: 'pdfkit', version: '0.15.0' }] }, autoRestart: { afterTheseMuchSeconds: 3600 }, }, errorList: [], }; module.exports = customApi; ``` ### Good to know - runOnNativeProcess skips the sandbox: code then runs inside API Maker, with its packages, and can slow the whole server. - Sandbox containers need Docker on the server. The installer sets it up. - The containers are started privileged, so treat the sandbox as a guard against faulty code, not against hostile code: only give code access to people you trust. ### Questions **How many sandboxes run?** Each API Maker process keeps a pool per account, sandboxCountForAdmin containers, and a pool per sandbox group. Requests wait in a queue when all of them are busy. **How does code in the sandbox reach my databases?** Through g.sys.db. The call goes back to API Maker, which runs the real API with its checks and returns the rows. **Can I change the time limit of one call?** Yes. Send x-am-sandbox-timeout in milliseconds, or set the limit in the settings of the custom API, scheduler or listener. ### Documentation - [Sandbox settings](https://docs.apimaker.dev/v1/docs/settings/sandboxSettings.html) - [Custom API settings](https://docs.apimaker.dev/v1/docs/settings/customApiSettings.html) - [Playwright in the sandbox](https://docs.apimaker.dev/v1/docs/guides/browser-automation.html#playwright-in-the-sandbox) ## Security Features URL: https://apimaker.dev/security Encrypted payloads, allowed origins, two-factor sign-in and package audits, on top of tokens and groups. Beyond who may call what, API Maker can require encrypted request bodies and return encrypted responses, refuse browsers from other origins, ask a second factor when people sign in to the admin panel, and check your npm packages for known vulnerabilities. ### At a glance - Payloads: Encrypted in and out, on request - Browsers: Allowed origins list - Admin sign-in: Authenticator app or email code - Packages: Checked for known vulnerabilities ### How it works 1. Only the browsers of your apps. List the origins of your web apps. A browser request from any other origin gets 403 before anything runs. Calls without an Origin header, like servers, are not affected. 2. Bodies stay unreadable on the way. The app encrypts { data, createdAt } with the transfer key of your secret and sends dataEncFE. Turn on acceptOnlyEncryptedData and plain bodies are refused. 3. A captured request expires. The payload carries its creation time. Sent again after feTransferDataValidityInSeconds, 300 seconds here, it is refused: Payload expired. 4. The answer can go back encrypted. x-am-get-encrypted-data asks for the answer in encryptedData, alone or next to data. 5. Two-factor sign-in for the admin panel. The root user can require a code from an authenticator app, a code sent by email, or both. Recovery codes are shown once and stored hashed. 6. Packages checked for known vulnerabilities. The vulnerabilities page audits the packages of API Maker and the npm packages you add to your sandboxes. ### What you get - Encrypted request payloads: Turn on acceptOnlyEncryptedData for a database, a table, an API, a custom or system API, and plain bodies and query strings are refused. - Short replay window: An encrypted payload carries its creation time. Payloads older than feTransferDataValidityInSeconds are refused. - Allowed origins: List the web origins of your apps. Requests from other browser origins get 403 before anything runs. - Two-factor sign-in: The root user can require a code from an authenticator app, a code sent by email, or both, when people sign in to the admin panel. Recovery codes are shown once and stored hashed. - Package audits: The vulnerabilities page audits the packages of API Maker and the npm packages of your sandboxes. - Encrypted and hashed fields: Schema conversions store sensitive fields encrypted or as HMAC SHA-256 hashes, with the keys of your secret. ### A real life example A banking app: The mobile app encrypts every transfer before sending it, so the body stays unreadable even where TLS is ended by a proxy, and a captured request can not be replayed after a few minutes. Admin users need their authenticator app to sign in. ### Example: An encrypted request ```http POST /api/schema/admin/bank/main/transfers/save-single-or-multiple x-am-authorization: x-am-encrypted-payload: true x-am-get-encrypted-data: get_only_encryption { "dataEncFE": "U2FsdGVkX1+q3n…" } ``` dataEncFE is { data, createdAt } encrypted with encryptionAlgorithmFETransfer and secretFETransfer of the secret, the key you share with your frontend or mobile app. ### Example: The transfer keys in the secret ```ts common: { encryptionAlgorithmFETransfer: 'AES', secretFETransfer: '…', feTransferDataValidityInSeconds: 300, // older payloads are refused }, ``` ### Good to know - API Maker itself speaks plain HTTP. Run it behind Caddy or another proxy that serves HTTPS: the installer sets up Caddy. - Email codes need SMTP settings in the root user settings. ### Questions **Do encrypted payloads replace HTTPS?** No. They add a second layer on top of HTTPS, useful when TLS ends before API Maker or when the body must stay opaque to intermediaries. **Who sets the two-factor policy?** The root user: email codes, authenticator codes, recovery codes, code length and expiry, attempts and resend delay. ### Documentation - [Encrypt data API](https://docs.apimaker.dev/v1/docs/apis-all/system-apis/system-generated-encrypt-data-api.html) - [Decrypt data API](https://docs.apimaker.dev/v1/docs/apis-all/system-apis/system-generated-decrypt-data-api.html) - [Table settings (acceptOnlyEncryptedData)](https://docs.apimaker.dev/v1/docs/settings/collectionSettings.html) - [Security features](https://docs.apimaker.dev/v1/docs/features/security-features.html) ## Automatic Caching URL: https://apimaker.dev/automatic-caching One setting, and repeat reads come from Redis instead of your database. Turn on enableCaching for a database, a table or an API. API Maker keeps each response in Redis, answers the same request from there, and removes the cached reads of a table as soon as it is written through API Maker. You write no caching code. ### At a glance - Switch on: enableCaching: true - Database APIs: Per database or per table - Other APIs: Per custom or system API - Keys expire: After 7200 s by default - Stored in: Redis ### How it works 1. Caching is one setting. Set enableCaching: true in the settings of a database or a table, or of a custom or system API. No caching code to write. 2. The first read fills the cache. The key is the table plus a hash of the request and its API user. On a miss, API Maker reads the database and keeps the response in Redis. 3. Repeat reads skip the database. The same request is answered straight from Redis, with the header x-am-data-source: cache. The database is not touched. 4. A write resets the cache of its table. Insert, update or delete through API Maker, and every cached read of products is removed. The next read gets the new price. 5. Every API user has its own entry. The API user is part of the key, so users with different field access never share a response. Keys expire after 7200 s. 6. Reset it yourself when you need to. Send x-am-cache-control: reset_cache to force a fresh read, or call g.sys.cache.resetCacheDB from your code. ### What you get - Writes reset the cache: Save, update or delete rows through API Maker and every cached read of that table is removed. An update or delete also clears cached tables whose schema points to it. - One entry per API user: The API user is part of the cache key, so users with different field access never get each other's response. - Dependencies you declare: A cached custom API can list the tables and APIs it depends on in resetCacheOnModificationOf. When one of them changes, its cache is cleared. - See where it came from: Every response carries x-am-data-source: cache or api, so you always know if the database was read. - Reset it when you want: Send x-am-cache-control: reset_cache to force a fresh read, or call g.sys.cache.resetCacheDB, resetCacheCustomApis or resetCacheSystemApis from your code. - Separate per tenant: With multi-tenant instances the tenant is part of the key, so one customer never receives the cache of another. ### A real life example A product catalog: Thousands of visitors open the same category pages, while prices change a few times a day. With caching on the products table, the pages are served from Redis, and the moment someone updates a price through the API the cached pages of that table are dropped. The next visitor sees the new price. ### Example: Table settings : cache every read API of this table ```ts import * as T from 'types'; let instanceColSetting: T.IInstanceApiSettingsTypes = { enableCaching: true, }; module.exports = instanceColSetting; ``` ### Example: Custom API settings : cache it, and clear it when products change ```ts import * as T from 'types'; let customApi: T.ICustomApiSettingsTypes = { name: 'Top Products', requestMethod: T.ERequestMethod.GET, path: '/top-products', enableCaching: true, resetCacheOnModificationOf: [ 'DB:shop:main:products', // a write to this table clears the cache of this API ], errorList: [], }; module.exports = customApi; ``` ### Example: Reset a table cache from your code ```ts await g.sys.cache.resetCacheDB({ instance: 'shop', database: 'main', collection: 'products', }); ``` ### Good to know - Caching needs the external Redis of API Maker (redisExternal), set in package.json or in the settings of the root user. - Only writes made through API Maker reset the cache. A change made directly in the database is seen after the key expires (7200 s by default, redisValueExpireInSeconds) or after you reset the cache. - Responses longer than 1,000,000 characters are not cached (maxCharsResToCache), so large exports do not fill Redis. - For database APIs, caching is a database or table setting, not a setting of a single generated API. ### Questions **Which requests are cached?** The read APIs of a table with caching on: get all, get by id, query, aggregate, count and distinct. Custom and system APIs are cached when their own settings have enableCaching: true. **What is the cache key made of?** The table (or API), the tenant if any, and a hash of the API user, the request body, query and parameters, and the headers that change the answer: content type, secret, response format, key case, flattening and encryption. **How do I know a response came from the cache?** Look at the x-am-data-source response header: cache means Redis answered, api means the API ran. **Can I keep keys longer or shorter?** Yes. redisValueExpireInSeconds in the redisExternal settings sets how long keys live. The default is 7200 seconds. ### Documentation - [Automatic caching](https://docs.apimaker.dev/v1/docs/features/automatic-caching.html) - [Database settings](https://docs.apimaker.dev/v1/docs/settings/databaseSettings.html) - [Table settings](https://docs.apimaker.dev/v1/docs/settings/collectionSettings.html) - [Custom API settings](https://docs.apimaker.dev/v1/docs/settings/customApiSettings.html) - [Reset database cache API](https://docs.apimaker.dev/v1/docs/apis-all/system-apis/system-generated-reset-redis-cache-db-api.html) ## Low Memory Footprint URL: https://apimaker.dev/low-memory-footprint Built to run well on small servers: 1 CPU core and 1 GB of RAM is enough to start. API Maker loads only the database drivers you use, serves every table with the same handlers, streams big results in small pieces, and keeps its queues bounded. Your own code runs in sandbox containers, so a leak there does not grow the API Maker process. ### At a glance - Minimum server: 1 CPU core, 1 GB RAM, 20 GB storage - Benchmark: 788 req/s, get all, $5 Linode - Streams: 1000 rows at a time - Workers: cpuCount, or AUTO ### How it works 1. Only the drivers you use are loaded. A database driver is loaded the first time an instance of that database is used. Connect only MySQL and PostgreSQL, SQL Server and Oracle drivers never load. 2. Five thousand tables, the same handlers. Generated APIs are generic routes. A new table adds no code to load: the same handlers serve orders, customers and every other table. 3. Millions of rows, a few in memory. Stream APIs read 1000 rows at a time and write them in 64 KB pieces, waiting when the client is slow. At most two bunches are in memory, whatever the size. 4. Leaky code can not fill the server. Your code runs in sandbox containers. Set a restart interval and each one is replaced on time, the requests inside finish first. 5. A crash is not an outage. When a worker process dies, the cluster starts a new one. PM2 keeps the main process running. 6. A $5 server, 788 requests per second. Linode 1 GB RAM, 1 vCPU, MongoDB and Redis on the same server: the get all API averaged 788.89 requests per second, 47k in 60 seconds. ### What you get - Drivers on demand: The driver of a database type is loaded the first time an instance of that type is used. The others never load. - No code per table: Generated APIs are generic routes: five tables or five thousand, the same handlers serve them. - Streams with backpressure: Stream APIs read 1000 rows at a time and write them in pieces of about 64 KB, waiting for slow clients. A request holds at most two bunches. - Bounded queues: API logs are written in batches of up to 1000, and at most 10,000 wait in memory. Responses over 1,000,000 characters are not cached. - User code outside: Custom code runs in sandbox containers with their own heap limit, and they can be restarted on a timer. - Workers sized to the server: Set cpuCount, or AUTO for one worker per 2 GB of RAM, up to the number of cores. A worker that dies is started again. ### A real life example API Maker, MongoDB and Redis on a $5 server: In the benchmark of the docs (April 2023), a $5 Linode with 1 vCPU and 1 GB of RAM ran API Maker, MongoDB 6 and Redis together. The schema get all API averaged 788.89 requests per second over 60 seconds, 47k requests in all. ### Good to know - Memory use depends on your traffic, your data and your code: measure with your own load. The benchmark ran API Maker, MongoDB and Redis on the same 1 GB server. - Sandbox containers use memory of their own, next to the API Maker process. ### Questions **How many workers does API Maker start?** As many as cpuCount says. With AUTO it takes the smaller of the CPU count and the RAM in GB divided by two, and at least one. **Where are the benchmarks?** In the docs, with the server, the API and the numbers of every run, from 1 to 8 CPUs. ### Documentation - [Benchmark summary](https://docs.apimaker.dev/v1/docs/performance/api-maker/Linode/summary.html) - [Get all on 1 CPU, 1 GB](https://docs.apimaker.dev/v1/docs/performance/api-maker/Linode/5-Server/GetAll.html) - [API Maker configurations](https://docs.apimaker.dev/v1/docs/am-resources/api-maker-configurations.html) ## Git Deployment URL: https://apimaker.dev/git-deployment Branches, commits and pull requests for everything you build in API Maker. A Git pull is the deployment. Connect a Git repository and API Maker writes your project into it: APIs, schemas, settings, hooks, events, tests and migrations, as files. Work on a branch, review a pull request, then pull the branch on the server of an environment. The pull is applied all at once, or not at all. ### At a glance - Repository: Any Git host: GitHub, GitLab, Bitbucket… - Deploy with: Git pull, from the panel or a hook - Failed pull: Changes nothing - Secrets: Never in Git ### How it works 1. Work on a branch, from API Maker. Create feature_1 from PROD, change APIs, schemas or settings, commit and push. Everything you build goes to Git, secrets never do. 2. Review it where you review code. Open a pull request on GitHub, GitLab or Bitbucket, review the changes and merge them into the branch of the environment. 3. On the server, a pull is the deployment. Press Git pull in the panel, or let your pipeline call the deployment hook with its token and secret, from allowed IPs only if you want. 4. Cloned, written, committed, live. The branch is cloned in memory and written in one database transaction. After the commit, pending migration scripts run, caches and sandboxes are renewed. 5. A failed pull changes nothing. If anything fails before the commit, the transaction is dropped: nothing of the pull is stored and the server keeps serving the version it had. ### What you get - Git inside API Maker: Create branches, see the status, commit, push, sync and revert without leaving the admin panel. - Your project as files: Custom APIs, schemas, settings, hooks, groups, events, schedulers, test cases, i18n packs, utility classes and more are files in the repository, easy to review. - History of every item: See the versions of one custom API or schema in Git and bring an older one back. - Deployment hook: Your CI calls a URL with its access token and secret to pull, optionally only from allowed IP addresses. The last hits are listed. - All or nothing: The branch is written in one database transaction. Until it commits nothing is stored, so a failure leaves the server on the version it had. - Upgrade API Maker too: Deploy a new API Maker version from the panel: every server of the cluster installs it, with live progress, and keeps its .env and license. ### A real life example DEV, QA, UAT, PROD: Each environment has its server and its branch. A developer builds on feature_1, opens a pull request into QA, and the QA server pulls it. The same review and pull promote it to UAT and then to production, where the pending migration scripts run on the way. ### Good to know - Secrets stay on each server. Set the secret of every environment on its own server. - Data is not in Git. Use migration scripts to change the structure or seed data of each environment. ### Questions **What happens after a pull commits?** Pending migration scripts run, every cache is reset and the sandboxes of the account are renewed, so the next request runs the new code. **Can two pulls run at the same time?** No. A cluster wide lock keeps the pulls of an account one after another. **Can I see the progress of a pull?** Yes. The panel shows each stage as it happens: cloning, writing, committing, migrations and cache reset. ### Documentation - [Git feature](https://docs.apimaker.dev/v1/docs/Git/git.html) - [Deploy API Maker](https://docs.apimaker.dev/v1/docs/features/deploy-api-maker.html) ## Developer Accounts URL: https://apimaker.dev/developer-accounts A whole team works on one development server, each developer in a workspace of their own. The admin creates an account for each developer. Every account has its own login, API path, secrets and Git branch, and database name masking gives it its own copy of each database under the same name. Nobody overwrites anybody, and nothing has to be renamed before a pull request. ### At a glance - Accounts: Created by the admin, as many as needed - Each has: Its login, API path and secrets - Databases: Masked: same name, own copy - Git: A branch per developer ### How it works 1. A workspace for every developer. The admin creates as many developer accounts as needed. Each one has its own login and API path, so people never overwrite each other. 2. Same names, their own database. Everyone writes database main. Database name masking runs priya's calls on main_priya and omar's on main_omar, so nothing is renamed before a pull request. 3. Secrets stay with each developer. Each account has its own secrets, with its own connection strings and keys. They are never pushed to Git. 4. Pull, push, diff and revert in API Maker. Each developer works on a branch without leaving API Maker. What reaches Git says main, like every other branch. 5. Nothing runs twice behind your back. Schedulers of developer accounts never run in the background, only the admin's do. Developers run them from the API testing page when they want. ### What you get - A workspace each: Developers sign in with their own account and API path, and change APIs, schemas and settings without touching the work of others. - Database name masking: Everyone writes the database main in code and requests. For priya it runs on main_priya, for omar on main_omar. - Own secrets: Each account has its own secrets, with its own connection strings and keys. They never go to Git. - Git from the panel: Each developer pulls, commits, pushes and reverts on a branch. What reaches Git uses the real database names. - Safe by default: Schedulers of developer accounts never run in the background. Developers run them by hand when they need to. - One server for the team: Development happens on a shared server, so there is nothing to install on each machine. API Maker Local Run is there for those who want it locally. ### A real life example Three developers, one sprint: Priya adds an orders API, Omar changes the product schema and Lin writes a migration, at the same time on the same server. Each works on their own branch and their own copy of the database, then opens a pull request as usual. ### Good to know - Masked databases are separate databases: create and fill the copy of each developer, for example with migration scripts. ### Questions **Who can create developer accounts?** The admin user, from Dev Accounts in the Utility menu. The API path and the email of each account must be unique. **Do developers need a server each?** No. Developer accounts share one server. Each one works in its own space on it. ### Documentation - [Developer accounts](https://docs.apimaker.dev/v1/docs/dev-accounts/dev-accounts.html) - [Mask database](https://docs.apimaker.dev/v1/docs/features/mask-database.html) - [Git feature](https://docs.apimaker.dev/v1/docs/Git/git.html) ## Database Migrations URL: https://apimaker.dev/database-migrations Change the structure and data of every environment with TypeScript scripts that run once, in order. A migration script is TypeScript with g: it reads table metadata, runs SQL or MongoDB commands on any of your databases, and moves data. Scripts go to Git with the rest of the project. After a deployment, the ones an environment has not run yet run one by one, in your order. ### At a glance - Written in: TypeScript, with g - Order: A number per script - Runs: After a Git pull, or from the panel - Once: Recorded per account ### How it works 1. A migration is a script in order. TypeScript run in the sandbox with g: read metadata, run SQL on any of your databases, import data. Each script has a number that sets its turn. 2. A deploy runs what is pending. After a Git pull is committed, the scripts not executed yet on this account run one by one, in order. create-tables and seed-countries ran before, add-description is pending. 3. It checks, then changes only what is missing. It reads the columns of order_transactions: description is missing, so it runs ALTER TABLE and adds it after qty, then the script is marked done. A script that fails is reported and runs again next time. 4. Run it again: nothing breaks. The next pull finds nothing pending. Even run by hand, the script sees the column already there and changes nothing. ### What you get - Any of your databases: getTableMeta reads the columns of a table, executeQuery runs SQL or a MongoDB command, and g.sys.db moves data with the usual APIs. - Deployed with Git: Scripts are part of the project in Git. Pull a branch on a server and its pending scripts run after the pull commits. - Once and in order: API Maker records which scripts ran on each account, so every script runs once, following its order number. - Failures come back: A script that fails is reported and not marked as done: it runs again the next time. - Run on demand: The admin panel shows the scripts still pending, and you can run the ones you pick. - In the sandbox: Scripts run in the sandbox like the rest of your code, with your packages. lodash and moment come with every sandbox. ### A real life example A new column in every environment: A feature needs a description column. The developer writes the script next to the API that uses it, and when QA, UAT and production pull the branch, each server adds the column before the new code serves its first request. ### Example: Add a column only if it is missing ```ts import * as T from 'types'; import * as _ from 'lodash'; async function main(g: T.IAMGlobal) { const columns = await g.sys.system.getTableMeta({ instance: 'mysql8', database: 'inventory', table: 'order_transactions', }); if (_.find(columns, { name: 'description' })) return 'already there'; await g.sys.system.executeQuery({ instance: 'mysql8', query: 'ALTER TABLE `inventory`.`order_transactions` ADD COLUMN `description` varchar(255) NULL AFTER `qty`', }); return 'added'; } module.exports = main; ``` ### Good to know - Make scripts safe to run twice: check before you change, as a failed script runs again. - Scripts run on the database servers of the environment that pulls them. They do not undo themselves: write a new script to revert. ### Questions **Which databases can a script change?** Any instance of the account: MongoDB, MySQL, MariaDB, PostgreSQL, SQL Server or Oracle. **How long may a script run?** Up to 10 minutes each when run after a pull. ### Documentation - [Database migration](https://docs.apimaker.dev/v1/docs/features/database-migration.html) - [Get table meta API](https://docs.apimaker.dev/v1/docs/apis-all/system-apis/system-generated-get-table-meta-api.html) ## Testing Framework URL: https://apimaker.dev/inbuilt-testing Test the business logic of your custom APIs and utility classes, inside API Maker. A test case is TypeScript: a list of small tests, each an async function with node:assert. Mocks stand in for the database and system APIs, so a test checks your logic without touching real data. Run all of them, or the ones you pick, and see each result with the line that failed. ### At a glance - Written in: TypeScript with node:assert - Tests: Custom APIs and utility classes - Mocks: For the methods of g.sys - Stored: In Git with your project ### How it works 1. Test cases live next to your code. A test case is TypeScript: an array of small tests, each with a name and an async function using node:assert. Import your utility classes and call them with g. 2. Mocks stand in for the database. Pick an API method, here g.sys.db.query, the values its arguments must have and what it returns. A matching call gets that value, no database needed. 3. They run in the sandbox. Each test calls your real code. The query is answered by the mock, the database is not called, and every assert is checked. 4. Run them all, or the ones you pick. Execute runs every test of the test case you are on. Run lets you check the test cases you want. Each small test shows its own result. 5. A change that breaks a rule is caught. Someone moves the discount threshold. The failing test shows the assert message and its line in your TypeScript, before anything is deployed. ### What you get - Small tests, one file: A test case exports an array of { name, code } objects. Each code is an async function that gets g. - Mocks without a database: Pick a method like g.sys.db.query, the arguments it must receive and the value it returns. Matching calls get that value. - Your real code: Import your utility classes and call them as your APIs do. The test runs the same code the API runs. - Execute or pick: Execute runs every test of the test case you are on. Run lets you tick the tests you want. - Failures you can read: A failing test shows the assert message, the actual and expected values, and its line in your TypeScript. - Reviewed like code: Test cases go to Git with the rest of the project, so a pull request shows the tests next to the change. ### A real life example Catch a broken rule before release: Someone changes the discount threshold in a utility class. The test case fails with its message and line, before the change reaches a pull request. ### Example: A test case for a discount rule ```ts import * as T from 'types'; import * as assert from 'node:assert'; import * as pricing from 'utils/Pricing'; module.exports = [ { name: 'No discount under 1000', code: async (g: T.IAMGlobal) => { assert.strictEqual(await pricing.discountFor(g, 999), 0); }, }, { name: '10% from 1000', code: async (g: T.IAMGlobal) => { assert.strictEqual(await pricing.discountFor(g, 1000), 100); }, }, ]; ``` ### Good to know - Calls that no mock matches reach the real API, so keep test data in mind when a test is not fully mocked. ### Questions **Where do tests run?** In the sandbox, like your custom APIs. **Can I test an API end to end?** Yes, from the API testing page: send a request with any body and headers, read the response, and keep it as a saved state. ### Documentation - [Test cases](https://docs.apimaker.dev/v1/docs/test-cases/test-cases.html) - [API testing page](https://docs.apimaker.dev/v1/docs/features/developer-tools.html#api-testing-page) ## Developer Tools URL: https://apimaker.dev/developer-tools Swagger docs, TypeScript types, code snippets, a debugger and code search, built into API Maker. Everything around the APIs is there too. Each API user gets Swagger docs of the APIs it may call, your schemas become TypeScript interfaces for your code, the API testing page writes client code in 21 languages, and you can debug custom code with breakpoints, search all your code at once, or edit it in your own editor. ### At a glance - API docs: Swagger, per API user - Types: Interfaces from your schemas - Snippets: 21 languages - Debugger: Chrome DevTools on the sandbox ### How it works 1. Try any API in the panel. Pick an API, get a sample payload, set the token of a test user and send it. Save the request as a state, it goes to Git with your code. 2. The same request as code. Copy it for cURL, JavaScript, Node.js, Python, Java, Go, C#, PHP, Swift, Dart and more: 21 languages with their variants. 3. Swagger docs per API user. Turn on Swagger docs for an API user and share its URL: it lists only the APIs its groups allow. Turn them off and the URL stops answering. 4. Your schemas become TypeScript interfaces. db-interfaces has an interface for every table, grouped by instance and database. Your code imports them and the editor completes the fields. 5. Breakpoints in the sandbox. Enable debugging and attach Chrome DevTools to the sandbox: it stops on your breakpoints, in your TypeScript, with the values of every variable. 6. Search all your code, edit it where you like. Find a word in custom APIs, hooks, events, schedulers, migrations, utility classes, schemas and secrets at once, or sync the code with your own editor. ### What you get - Swagger per API user: Each API user can have a Swagger URL with a random part. The docs list the system, custom and database APIs that user can call, and the URL answers only while Swagger docs are on for it. - Interfaces from schemas: Your table schemas become TypeScript interfaces, grouped by instance and database, and your code imports them from db-interfaces. - API testing page: Every API of the account in one place: sample payloads, headers, tokens of test users, the response, and saved states that go to Git. - Code finder: Search a word in all your code at once: custom APIs, hooks, events, schedulers, migrations, utility classes, schemas, secrets and more, and edit it from the results. - Your own editor: localClient.js in your Git repository syncs your files with API Maker both ways, with a sync token from your profile. - Logs and notes: Read the console output of your code and of the native process from the panel, and keep notes for the team inside API Maker. ### A real life example Hand over an API to a partner: Create an API user for the partner, give its group the three APIs it needs, and enable its Swagger docs. The partner gets a URL that documents exactly those three APIs, and copies client code for its own language from them. ### Example: Typed code with the generated interfaces ```ts import * as T from 'types'; import * as db from 'db-interfaces'; async function main(g: T.IAMGlobal) { const products = await g.sys.db.getAll({ instance: 'shop', database: 'main', collection: 'products', }); return products.map(p => p.name); // p is typed } module.exports = main; ``` One namespace per instance and database, and an interface I + the table name in PascalCase: open Generated Interfaces to see yours. ### Example: Settings of the local client ```ts module.exports = () => { return { webSocketURL: 'wss://ws.example.com', // WebSocket URL of your API Maker syncToken: '…', // generate it in your user profile adminUserPath: 'admin', }; }; ``` ### Good to know - The debugger works for code that runs in the sandbox, not for code that runs on the native process. - Open debugging ports to the internet only through an SSH tunnel: the debug settings show the command. ### Questions **Which languages do snippets support?** C, C#, cURL, Dart, Go, HTTP, Java, JavaScript, Kotlin, Node.js, Objective-C, OCaml, PHP, Postman CLI, PowerShell, Python, R, Ruby, Rust, Shell and Swift, with their variants. **Can I run API Maker on my computer?** Yes. API Maker Local Run is a desktop app for macOS, Windows and Linux: it installs and runs API Maker, with MongoDB, Redis and your databases in Docker. Get it from the download page. ### Documentation - [Code finder](https://docs.apimaker.dev/v1/docs/features/code-finder.html) - [Notes](https://docs.apimaker.dev/v1/docs/notes/notes.html) - [Developer tools](https://docs.apimaker.dev/v1/docs/features/developer-tools.html) ## API Logging URL: https://apimaker.dev/logging-api Every call you choose, with its request, response, console output and timing, searchable in the admin panel. A log profile lists what to log: generated, custom and system APIs, events, schedulers and WebSocket events, and whether to keep their response. Logs are written after the response, in batches, to a log database of their own. The log explorer finds any call and exports the list. ### At a glance - What is logged: Chosen in the log profile - When: After the response, in batches - Where: A MongoDB log database - Export: CSV or JSON ### How it works 1. Choose what is logged. The default log profile lists the APIs to log: generated, custom and system APIs, events, schedulers. For each one, save the response or not. 2. Logging never delays a call. The response goes out first, then the log is queued in memory. A call of an API that is not in the profile is not logged at all. 3. Written in batches, never a burden. Up to 1000 logs per write, or every 5 seconds. If the log database is slow, at most 10 000 logs wait in memory and a failed batch is dropped: the server keeps serving. 4. Everything about a call. Request body, query, params and headers, the response if the profile saves it, console output, errors, status and time. The time to get a sandbox is not counted. 5. Find any call, export the list. Filter by time, text, fields and duration, sort by the slowest, follow new logs live, and export the list as CSV or JSON. Old logs are removed past your limit. ### What you get - Profile based: Pick the APIs, events, schedulers and WebSocket events to log, and for each one whether its response is saved. - Never in the way: The response goes out first. Logs are written in batches of up to 1000, or every 5 seconds, and at most 10,000 wait in memory. - Everything about a call: Body, query, parameters and headers, the response if kept, console output of your code, errors, status and execution time. - Find it fast: Filter by time, text, fields and duration, sort by the slowest calls, and follow new logs as they arrive. - Export: Export the filtered list as CSV or JSON for reports or other tools. - Cleans itself: A daily job removes the oldest logs beyond maxLogsCount, so the log database does not grow without end. ### A real life example Why was this order rejected?: Support searches the logs for the order id, opens the call, and sees the body the app sent, the validation error it got, and the console output of the pre hook that refused it. ### Good to know - When the log database falls behind and 10,000 logs are waiting, the oldest waiting logs are dropped instead of slowing down the APIs. - Keeping responses in logs uses space: turn it on only for the APIs where you need it. ### Questions **Can I switch logging off?** Yes. logs.enableLogs turns it off for the server. The monitoring dashboards keep working, they do not read the logs. **Is the time of the sandbox counted?** No. The time spent waiting for a sandbox is not counted in the execution time of a call. ### Documentation - [Log profile](https://docs.apimaker.dev/v1/docs/logs/log-profile.html) - [Log table](https://docs.apimaker.dev/v1/docs/logs/log-table.html) - [API Maker configurations](https://docs.apimaker.dev/v1/docs/am-resources/api-maker-configurations.html) ## Monitoring URL: https://apimaker.dev/monitoring Calls, errors and latency of every API, the health of every server, and alerts when something drifts. API Maker counts every call after its response and writes one rollup a minute per process, with the vitals of the process. The analytics dashboard shows calls, errors and p95 per API and per server, built-in rules turn the health verdict to degraded or critical, and more dashboards cover the servers, Redis and your data model. ### At a glance - Metrics: Calls, errors, latency histograms - Resolution: 1 minute rollups, kept 30 days - Alerts: Built-in rules, one health verdict - Needs logs?: No, works with logging off ### How it works 1. Every call counted, after its response. The worker answers first, then adds the call to counters in memory: calls, errors and a latency histogram per API. No database, no Redis on the way. 2. One write a minute per process. Every 60 seconds the counters go out in a single bulk write as 1 minute rollups, kept 30 days, with memory, heap, CPU and event loop vitals. 3. Calls, errors and p95, per API and server. Histograms merge across servers, so the p95 of the whole cluster is right. Logs can be off: the dashboards do not depend on them. 4. Built-in rules, one health verdict. Error rate over 5 %, p95 over 2000 ms, memory, disk, heap, event loop lag, DB pool, sandbox queue, nodes down: each firing rule turns the verdict to DEGRADED or CRITICAL. 5. Servers, Redis and your data model. The nodes dashboard shows workers, memory, platform and DB connections. The Redis dashboard edits keys and TTLs. The ER diagram draws your instances and tables. ### What you get - Per API, per server, per cluster: Calls, errors and p95 latency for every API and every server. Histograms merge across servers, so the p95 of the cluster is right. - Vitals of every process: Memory, heap, CPU and event loop lag of each process, stored with the minute rollups. - Rules you do not have to write: Error rate over 5 %, p95 over 2000 ms, memory or disk over 85 %, heap over 90 %, event loop lag over 200 ms, a full DB pool, a sandbox queue or a node down turn the verdict to degraded or critical. - Server nodes: The nodes dashboard lists the servers and workers of the cluster with their memory, platform and database connections. - Redis dashboard: Browse the keys of Redis, read and edit values, and change their TTL. - ER diagram: See your instances, tables and the relations of their schemas as a diagram. ### A real life example A slow API after a release: After a deployment, the verdict turns degraded: the p95 of one API went past 2000 ms. The dashboard shows which server and since when, and the log explorer, sorted by duration, shows the slow calls themselves. ### Good to know - Counters are kept in memory and written once a minute, so the last minute of a process that crashes can be missing. ### Questions **Does monitoring slow down the APIs?** No. A call is counted after its response, in memory, and each process writes one bulk update a minute. **Where are the metrics stored?** In the log database of API Maker (or its main database when there is no log database), as minute rollups and vitals that expire after 30 days. ### Documentation - [Node dashboard](https://docs.apimaker.dev/v1/docs/dashboard/node-dashboard.html) - [Redis dashboard](https://docs.apimaker.dev/v1/docs/dashboard/redis-dashboard.html) - [Diagram dashboard](https://docs.apimaker.dev/v1/docs/dashboard/diagram.html) ## Index Maker URL: https://apimaker.dev/index-maker Part of API Maker + Extensions, installed with a license key. Records the queries your APIs run, finds the ones that use no index, and creates the index: on its own, on every environment. Index Maker is part of API Maker + Extensions. It records the database queries of your APIs, grouped by the shape of their condition, and asks your database with explain whether each one uses an index. Switch on the automatic mode and it starts its own scanning periods, gives an index to the slow or frequent queries of big tables, and writes every index into the migration script am_index_maker, which every environment runs on deploy. ### At a glance - Edition: API Maker + Extensions, license key - Automatic: Records, decides, creates, ships - Checks with: explain, on your database - Databases: All six API Maker supports ### How it works 1. A period records on its own. Every six hours the loop starts a scanning period of thirty minutes. The queries of your APIs are grouped by the shape of their condition, with their calls and their time. 2. Your database says which ones use no index. When the period ends, every shape is run through explain with a recorded sample. The orders query reads the whole table. 3. The picker weighs each shape against your thresholds. Slow or frequent, on a table big enough, not covered by an existing index, under the limit of indexes per table. Every decision is kept with its reason. 4. The index goes into am_index_maker as JSON. A migration script that always exists and can not be deleted. Index Maker rewrites only the list between the markers; the code applies it. 5. Created here, created everywhere. The index is created on this server at once. Every other environment runs the script on its next deploy: an index which exists already is left alone, so the script can run again and again. 6. The query is fast, the loop keeps watching. The orders query drops from 310 ms to 4 ms. One process of the cluster ticks once a minute; nothing is added to your requests. ### What you get - Automatic mode: Thresholds for table size, average time and number of calls, fields per index, indexes per table, excluded collections, how often and how long to record: settings of the account, kept in git. - One script for every environment: am_index_maker holds every index as a JSON entry with who added it and why. Deploy the repository and the environment creates the same indexes. Add your own entries too. - Grouped by shape: status = "paid" and status = "new" are the same query. Paging, sorting and selected fields do not split a group either. - Your database decides: The check is the explain of your own database, run with a recorded sample of the query. Nothing is guessed. - Every decision explained: WRITTEN, SMALL_TABLE, NOT_WORTH_IT, COVERED_BY_EXISTING_INDEX, TOO_MANY_INDEXES, NO_INDEXABLE_FIELD... each shape of a period shows what was decided and why. - Safe on every version: Plain CREATE INDEX statements, the existence check made in code, quoted identifiers, names within the limit of each database, and fields no plain index can hold are skipped with the reason. - Index system APIs: g.sys.system.createIndexes, dropIndexes and getIndexes work the same on all six databases, from custom APIs, schedulers and your own migration scripts. - Suggestions and the index form: Select the shapes without an index and add them to the script in one click. The index form shows every field with whether an index can hold it, the existing indexes and the size of the table. - A log of index changes: Every index created or removed is logged with its table, fields, date and source: the form, a suggestion, the automatic mode, custom code or the script on deploy. ### A real life example The orders list got slow, and nobody had to notice: Orders grew to millions of rows. The automatic period of the night records the orders query on customer_id and status: 1,240 calls, 310 ms on average, no index. It passes the thresholds, so the index is created on the production database and written into am_index_maker. The staging and the development environments get it on their next deploy, and the query takes 4 ms in the morning. ### Example: One entry of am_index_maker ```ts // const INDEXES: T.IIndexMakerScriptEntry[] = [ { "instance": "mysql8", "database": "inventory", "collection": "orders", "name": "am_ix_orders_customer_id_status_3f9a1c", "fields": [{ "name": "customer_id" }, { "name": "status" }], "source": "AUTOMATIC", "reason": "1,240 calls, 310 ms on average, orders has 2,400,000 rows." } ]; // ``` Index Maker rewrites only the list between the markers. The code of the script calls g.sys.system.createIndexes with ifNotExists, so it runs on every deploy without harm. ### Example: The same from your code ```ts const results = await g.sys.system.createIndexes({ indexes: [{ instance: 'mysql8', database: 'inventory', table: 'orders', fields: [{ name: 'customer_id' }, { name: 'status', order: 'DESC' }], }], }); // [{ status: 'CREATED', name: 'am_ix_orders_customer_id_status_3f9a1c', ... }] // the second time : [{ status: 'EXISTS', ... }] ``` CREATED, EXISTS, SKIPPED (a field no plain index can hold, with the reason) or FAILED (the message of the database). ### Good to know - The Index Maker screen comes with API Maker + Extensions: installing it needs a license key. - Only successful calls that did not come from the cache are recorded. - Each watched call adds a few writes to the database of API Maker after its response, while a period records. The automatic mode records for a window every few hours, not all the time. - The automatic mode creates indexes, it never drops one. A DROP entry can be added to the script by hand. - Shapes no single index serves ($or, $nor, $where, $text, $expr) are reported, not indexed. ### Questions **Which queries get an index automatically?** The shapes that use no index (part of one, if you ask), on a table with at least minTableRows rows, when the shape is slower than minAvgQueryTimeMS on average or was called at least minQueryCount times during the period. The fields are the equalities first, then the ranges, up to maxFieldsPerIndex. The defaults are 10,000 rows, 100 ms and 50 calls. **How do the other environments get the index?** Through the migration script am_index_maker, kept in your repository like every other migration script. Every change marks it "not executed" on the server that made it, so every environment runs it on its next deploy. An index which exists already is reported as EXISTS and left alone. **Can I edit the script?** Yes. Index Maker only rewrites the JSON list between its two markers. Edit the code outside them, or add your own entries to the list, including DROP entries. The script can not be deleted while Index Maker uses it; remove its entries from the Index Maker page instead. **What does it cost at run time?** Nothing per request. A scanning period costs a few writes to the database of API Maker after each watched call, while it records. The loop ticks once a minute in one process of the cluster, evaluates a period once when it ends, and waits for a quieter minute when the event loop is busy. Table sizes come from the catalog and a count bounded by the threshold. **Which index options are there?** MongoDB: unique, ascending, descending, hashed and text. MySQL and MariaDB: FULLTEXT, SPATIAL and UNIQUE, with BTREE, HASH or RTREE. SQL Server: clustered, nonclustered, XML and spatial, unique or not. PostgreSQL: btree, hash, gist, gin, spgist and brin, with NULLS FIRST or LAST. Oracle: unique, ASC and DESC. **Are multi tenant APIs covered?** Yes. Queries are kept per tenant, and an index goes to the database of that tenant: the entry names it as crm::acme. **Can I create indexes without the extension?** Yes. The index list of a table, on the instances page, shows its indexes and creates and removes them, and the system APIs createIndexes, dropIndexes and getIndexes are part of API Maker. ### Documentation - [Index Maker](https://docs.apimaker.dev/v1/extensions/index_maker/introduction.html) - [Automatic indexes](https://docs.apimaker.dev/v1/extensions/index_maker/automatic_indexes.html) - [g.sys.system.createIndexes](https://docs.apimaker.dev/v1/examples/sys/system/createIndexes.html) - [v3.3.0 release notes](https://docs.apimaker.dev/v1/docs/blog/release-notes/v3.3.0.html) # Deployment architectures ## Architecture: All-in-one server URL: https://apimaker.dev/architectures/single-server API Maker, its MongoDB, Redis and your database on a single VPS. The fastest way to production: one Ubuntu VPS from any provider runs everything. One install script sets it all up, with Caddy for HTTPS and PM2 to keep API Maker running: every feature of API Maker, at the lowest possible cost. - Servers: 1 VPS - Capacity: ~2,759 req/s on 4 cores - Best for: MVPs and prototypes; Internal tools; DEV and QA environments; Small production apps ### Benefits - Lowest cost: One VPS from any provider. In our benchmark a 4 vCPU server answered ~2,759 requests per second. - Live in minutes: The install script sets up Docker, MongoDB, Redis, API Maker under PM2 and Caddy with free HTTPS certificates. - Every feature included: Generated and custom APIs, schedulers, WebSockets, caching, Git deployment and logs: nothing is left out. - Fast by default: No network between API Maker and its databases: every query and cache read stays inside the machine. - Ready for Git: Perfect for DEV and QA: the same Git branches later deploy to the bigger architectures with a pull. - Easy to outgrow: Moving the databases out or adding servers later only changes connection strings in .env. Your APIs stay the same. ### Servers - All-in-one VPS ×1: 2 – 4 vCPU, 4 – 8 GB, 40 – 80 GB SSD. Runs Caddy: HTTPS, WSS and the admin panel, certificates from Let's Encrypt, API Maker backend, run by PM2, cpuCount AUTO, Docker: sandbox containers of custom code, MongoDB 6 replica set: api_maker_db, api_maker_logs, Redis 7: internal and cache, Your database: PostgreSQL, MySQL, MongoDB…. Minimum: 1 vCPU, 1 GB RAM with 2 GB swap, 20 GB storage, Ubuntu 22.04 LTS. ### Trade-offs - A single point of failure: when the VPS stops, every app stops. Keep snapshots of the VPS and dumps of the databases on another machine. - API Maker, the sandboxes and the databases share the same CPU, memory and disk. - It grows up only: a bigger VPS means a short restart while it is resized. ### Questions **Is a single server enough for production?** For many apps, yes. In our benchmark (Linode, MongoDB 6, 10 rows of 10 columns) a 4-core VPS answered about 2,759 requests per second and an 8-core one about 4,529. Keep backups on another machine, since one server is also one point of failure. **Does API Maker serve HTTPS and WSS itself?** No. API Maker speaks plain HTTP on port 38246 and WebSocket on port 38245. Caddy, set up by the install script, receives every request first: it holds the certificates, from Let's Encrypt and renewed by itself, answers the apps over HTTPS and WSS, and passes the traffic to API Maker on the same server. **What happens when the API Maker process stops, or the server reboots?** PM2 runs API Maker as the process api_maker_be. It starts it again within a second if it stops, and the install script registers it with pm2 startup and pm2 save, so it also comes back after every reboot. **What is the smallest server API Maker runs on?** 1 CPU core, 1 GB of RAM with 2 GB of swap and 20 GB of storage, on Ubuntu 22.04 LTS. Give it 4 GB or more when your database runs on the same server. **Can I move to more servers later?** Yes. API Maker keeps your APIs, schemas and settings in its own MongoDB. Move the databases out, or add servers that point to the same MongoDB and Redis: only the connection strings in .env change. ## Architecture: Separate database server URL: https://apimaker.dev/architectures/separate-database API Maker on one VPS, your databases on servers of their own. The first split most teams make: each database gets its own CPU, memory and disk, stays off the internet and grows on its own. API Maker reaches it over the private network, along with any database you already run. - Servers: 2+ VPS - Capacity: ~2,759 req/s on 4 cores - Best for: Growing production apps; Data-heavy apps; APIs over existing databases; Private databases ### Benefits - Resources of its own: The database gets its own CPU, memory and NVMe disk. Heavy queries never slow down the workers of API Maker. - Off the internet: Database ports open to the API server only. Every request passes the auth, roles and field level access of API Maker first. - Grow each tier on its own: Resize the database server without touching the API server, and the other way around. - Many databases, one API: PostgreSQL, MongoDB, SQL Server and 5 more types, on any servers, joined in one call with deep populate. - Keep what you have: Generate APIs for databases you already run, even in your own data center, without migrating them. - Simpler operations: Back up, restore and patch each database on its own schedule, with the tools made for it. ### Servers - API server ×1: 2 – 4 vCPU, 4 – 8 GB, 40 GB SSD. Runs Caddy: HTTPS, WSS and the admin panel, API Maker backend, run by PM2, cpuCount AUTO, Docker: sandbox containers of custom code, MongoDB 6 replica set: data and logs of API Maker, Redis 7: internal and cache. - Database server 1 per database: 4 – 8 vCPU, 16 – 32 GB, 200 GB+ NVMe. Runs One of your databases: PostgreSQL, MySQL, MariaDB, SQL Server, Oracle, MongoDB, TiDB or Percona, Its own backups and monitoring. Give it enough memory to keep the indexes you query most in RAM. ### Trade-offs - Still one API server: when it stops, the apps stop. The next architectures add more of them. - A network hop between API Maker and each database: keep them in the same data center, on the private network. - More servers to patch, monitor and back up. ### Questions **Do my APIs change when the database moves?** No. Only the connection string of the instance changes. Generated APIs, custom APIs, schemas and permissions stay exactly the same. **Can API Maker connect to a database I already have?** Yes. Point an instance at any of the 8 supported database types, on any server API Maker can reach, and it generates the APIs of its tables without changing the database. **How does custom code reach a database on another server?** Custom code runs in Docker sandbox containers. When the database address is different from inside a container, set the sandbox connection string of the instance: it is used from the sandbox only. ## Architecture: Dedicated data tier URL: https://apimaker.dev/architectures/dedicated-data-tier A stateless API server, with the MongoDB, logs and Redis of API Maker on servers of their own. Everything API Maker keeps moves off the API server: definitions to its MongoDB, cache and events to Redis, logs to their own database. The API server becomes disposable, ready to be rebuilt in minutes or joined by others. - Servers: 4 VPS - Capacity: ~2,759 req/s on 4 cores - Best for: Production apps; Heavy logging; Servers rebuilt often; The step before scaling out ### Benefits - A disposable API server: Nothing lives on it but code: rebuild, resize or move it in minutes with the same .env. Caddy gets its certificates again by itself. - Each store sized for its job: MongoDB gets fast disks, Redis gets memory, logs get room to grow: every server is sized for what it does. - Logs out of the way: Every call is logged to a database of its own and trimmed daily, without touching your data. - Cache apart from events: Responses live in their own Redis, so evicting cache never touches locks, events or WebSocket subscriptions. - Ready to scale out: A second API server with the same .env serves the same APIs at once. The next architectures do exactly that. - Less exposed: The apps only reach Caddy, on the API server. The data servers accept its private IP and nothing else. ### Servers - API server ×1: 4 vCPU, 8 GB, 40 GB SSD. Runs Caddy: HTTPS, WSS and the files of the admin panel, API Maker backend, run by PM2, cpuCount AUTO, Docker: sandbox containers of custom code. - MongoDB server ×1: 2 – 4 vCPU, 8 GB, 100 GB NVMe. Runs MongoDB 6, replica set rs0, api_maker_db: APIs, schemas, settings, users, api_maker_logs: a log of every call. Three members make it highly available: see the High availability architecture. - Redis server ×1: 2 vCPU, 4 – 8 GB, 20 GB SSD. Runs Redis 7 internal: events, locks, WebSocket subscriptions, auto increments, Redis 7 cache: responses of your APIs. - Database server 1 per database: 4 – 8 vCPU, 16 – 32 GB, 200 GB+ NVMe. Runs Your databases: any of the 8 supported types. ### Trade-offs - Still one API server: when it stops, the apps wait for a new one. The next architecture runs several. - MongoDB and Redis are single servers here: back them up, or make them clusters as the Resilient architectures show. - Four servers to run instead of one or two. ### Questions **What exactly is stored in the MongoDB of API Maker?** Everything you build in the admin panel: instances and their encrypted connection strings, schemas, custom APIs, hooks, schedulers, WebSocket events, users, roles and settings. That is why any API Maker server that points to it serves the same APIs. **Why two Redis?** redisInternal carries what API Maker needs to work: events between workers and servers, locks of the schedulers, WebSocket subscriptions and auto increment values. redisExternal holds the cached responses of your APIs, which can be evicted at any time. On small setups both can be the same Redis. **Can I turn logs off or keep fewer of them?** Yes. logs.enableLogs turns them off, logs.maxLogsCount sets how many of the newest logs are kept, and logs.logRemoveSchedulerInterval sets when older ones are removed, every day by default. ## Architecture: Environments with Git URL: https://apimaker.dev/architectures/environments DEV, QA, UAT and PROD, each with its own servers and data. Git pull is the deployment. Developers build on API Maker Local Run or on DEV and push to Git. Each environment pulls its branch and is live in about 15 seconds, with its own secrets and data. PROD pulls once, and every PROD server follows. - Servers: 4+ environments - Capacity: ~5,518 req/s on PROD, 2 × 4 cores - Best for: Teams of developers; Regulated releases; Agencies delivering to clients; Any serious production ### Benefits - Live in ~15 seconds: A git pull is the whole deployment: no build, no pipeline, no restart. It applies completely or not at all. - Reviewed like code: Every change goes through a pull request on GitHub, GitLab, Bitbucket or your own Git server before it moves on. - Each environment its own data: Secrets point the same APIs to the DEV, QA, UAT or production databases. Developers have secrets of their own. - Every PROD server at once: Pull on one PROD server and a cluster event through Redis reloads every worker of every other server. - Rollback in one click: Every item has its Git history. Revert, commit and pull: the previous version is back in seconds. - Develop anywhere: API Maker Local Run brings the whole platform to the computer of each developer, on macOS, Windows or Linux. ### Servers - DEV and QA servers ×1 each: 2 vCPU, 4 GB, 40 GB SSD. Runs Caddy, API Maker under PM2, its MongoDB and Redis, Test data of the environment. DEV can also be API Maker Local Run on each computer. - UAT server ×1: 2 – 4 vCPU, 8 GB, 80 GB SSD. Runs Caddy, API Maker under PM2, its MongoDB and Redis, A copy of production data, masked if needed. - PROD 2+ servers: 4 – 8 vCPU each, 8 – 16 GB each, 40 GB SSD each. Runs Caddy in front: HTTPS, WSS and load balancing, API Maker on every server, same .env, run by PM2, Shared MongoDB, Redis and your databases. See the Load-balanced API servers architecture for the details of PROD. ### Trade-offs - More servers to run: at least one per environment. Small DEV and QA servers keep the cost low. - UAT is only as useful as its data: refresh it from production, masked where needed. - PROD pulls should be deliberate: protect the PROD branch and allow pulls to a few people only. ### Questions **Do I need a CI/CD pipeline?** No. A git pull on the server of an environment is the deployment. API Maker applies the pulled changes in a MongoDB transaction, so they are applied completely or not at all, and the new version is live in about 15 seconds. A deployment hook URL can also trigger the pull, for example from a webhook of your Git host. **How does each environment reach its own database?** Connection strings live in secrets. Each environment has its own secrets, and each developer too, so the same instance and the same APIs read the DEV database on DEV and the production database on PROD. **Which Git hosts are supported?** Any Git repository: GitHub, GitLab, Bitbucket or your own Git server. The admin panel pulls, pushes, compares and reverts through it, and one repository can serve several admin users. ## Architecture: Load-balanced API servers URL: https://apimaker.dev/architectures/load-balanced Several identical API Maker servers behind Caddy, which balances the load, sharing one data tier. Add capacity by adding servers. Every server runs API Maker with the same .env, so any of them answers any request. Schedulers run once, cache and events are shared through Redis, and one git pull updates them all. - Servers: 7+ VPS - Capacity: ~13,587 req/s on 3 × 8 cores - Best for: High traffic; Growing SaaS products; Restarts without downtime; Seasonal peaks ### Benefits - Capacity that adds up: Each 8-core server adds about 4,500 requests per second in our benchmark. Add or remove one in minutes. - No sticky sessions: Tokens, the shared Redis and the shared API Maker DB let any server answer any user and any request. - Schedulers run once: A lock in Redis gives each scheduler to one server of the fleet, and another one takes over if it stops. - One cache for all: Responses cached through one server are served by all of them, and a write resets them everywhere. - Deploy once: A git pull on any server reaches every worker of every server through a cluster event. - One view of the fleet: The Analytics dashboard shows traffic, latency and health of every server and worker, and restarts them remotely. ### Servers - Caddy load balancer ×1: 2 vCPU, 2 GB, 20 GB SSD. Runs Caddy: HTTPS and WSS, certificates from Let's Encrypt, Load balancing by least connections, Health checks of the API servers on GET /ping. - API servers ×3 or more: 8 vCPU, 16 GB, 40 GB SSD. Runs API Maker, run by PM2, cpuCount AUTO: 8 workers, Docker: a sandbox for every worker, The admin panel on one of them, or all. - MongoDB server ×1: 4 vCPU, 8 GB, 100 GB NVMe. Runs API Maker DB and logs, replica set rs0. - Redis server ×1: 2 vCPU, 8 GB, 20 GB SSD. Runs Redis internal: events, locks, WebSocket subscriptions, Redis cache: responses of your APIs. - Database servers 1 per database: 8 vCPU, 32 GB, 200 GB+ NVMe. Runs Your databases: any of the 8 supported types. ### Trade-offs - Caddy and the data servers are still single: the High availability architecture doubles them. - Every server needs the same .env. Give each one its own serverName so it is easy to spot in the dashboards. - A restarted server drops the requests it was answering: restart one server at a time. ### Questions **Do I need sticky sessions?** No. Authentication uses tokens, the cache and the WebSocket subscriptions live in the shared Redis, and the definitions in the shared API Maker DB, so any server can answer any request. **How do WebSockets work behind Caddy?** Caddy takes the secure WebSocket and passes it, as a plain WebSocket, to port 38245 of one server, where it stays. Events raised on any other server reach it through Redis, and API Maker keeps quiet connections alive with pings. **Why Caddy as the load balancer?** It is the same Caddy the install script puts in front of API Maker, which has no TLS of its own. It gets and renews the certificates by itself, passes WebSockets through without extra settings, balances by least connections and takes a failed server out of the pool with health checks on GET /ping. **How many servers can I add?** As many as your data tier can serve. Each server opens its own connections to MongoDB, Redis and your databases, so size those for the whole fleet, or make them clusters as the Resilient architectures show. ## Architecture: Real-time events across servers URL: https://apimaker.dev/architectures/realtime-websockets WebSocket clients on any server, changes made on any server: Redis connects them. Dashboards, apps and devices keep a WebSocket open to whichever server they reached. An API call answered by any server is published in Redis and pushed to every matching subscriber, on every server, within milliseconds. - Servers: 5+ VPS - Capacity: ~8,277 req/s on 3 × 4 cores - Best for: Live dashboards; Delivery and logistics; Chat and notifications; IoT and screens ### Benefits - Every subscriber, every server: An order written through server 1 reaches dashboards connected to servers 2 and 3 within milliseconds. - Only what each client asked for: Conditions on the response decide who gets an event, and select sends only the fields a client needs. - Checked like any API: Clients subscribe with a token of the right auth provider, and your code can accept or refuse each one. - Events of your own: Custom code emits custom WebSocket events, delivered with the same conditions and checks. - Survives a server loss: Clients of a stopped server reconnect to another one and subscribe again. Nothing to configure. - No extra service: The WebSocket server is part of API Maker, built on uWebSockets.js, and Redis is the one you already run. ### Servers - Caddy load balancer ×1: 2 vCPU, 2 GB, 20 GB SSD. Runs Caddy: HTTPS in, plain HTTP to port 38246 of the servers, Secure WebSockets (wss) in, plain WebSockets to port 38245, Certificates of api. and ws.example.com, renewed by itself. - API servers ×3 or more: 4 – 8 vCPU, 8 – 16 GB, 40 GB SSD. Runs API Maker, run by PM2: HTTP APIs and the WebSocket server, Docker: sandboxes of custom code. Many long-lived connections: raise the limit of open files (ulimit -n) of Caddy and of the API Maker process. - Redis server ×1: 2 – 4 vCPU, 8 GB, 20 GB SSD. Runs Redis internal: a key per subscription, pub/sub of the events, Redis cache: responses of your APIs. - Data servers 2+: 4 – 8 vCPU, 16 – 32 GB, 200 GB+ NVMe. Runs API Maker DB, MongoDB replica set, Your databases. ### Trade-offs - Every WebSocket is a long-lived connection: size the servers and Caddy for their number, not only for requests. - The internal Redis carries every event: keep it close to the servers, with enough memory for the subscriptions. - Clients must reconnect on their own when a connection closes: every WebSocket client library can. ### Questions **Do WebSockets need sticky sessions?** No. A WebSocket is one long connection, so it stays on the server it reached by itself. Events raised on any other server reach it through Redis. **Does Caddy need special settings for WebSockets?** No. Its reverse_proxy passes WebSocket upgrades through as they are. Caddy ends the TLS of wss://ws.example.com, and each connection continues as a plain WebSocket to port 38245 of one server. API Maker has no TLS of its own. **What can a client subscribe to?** Calls of the generated APIs of your tables, like the insert or the update of orders, with conditions on the response and the fields to receive. Also calls of custom APIs and system APIs, and custom WebSocket events emitted by your code. **How are subscriptions secured?** A client connects with the token of an auth provider and each WebSocket event accepts the tokens of the provider chosen for it. Custom code can also decide, for each client, whether it may subscribe. ## Architecture: Many projects, one platform URL: https://apimaker.dev/architectures/multi-project One API Maker fleet hosts many projects, each with its own APIs, databases, sandboxes and Git. Every project is an admin account of the same platform: its own API path, databases, users, secrets and sandbox containers. The servers, API Maker DB and Redis are shared and run once. New projects come from a single API call. - Servers: 5+ VPS - Capacity: ~7,288 req/s on 2 × 6 cores, shared - Best for: Agencies and software houses; Internal platforms; Products with many apps; Hosting clients ### Benefits - Spaces that never mix: Each project has its own API path, instances, schemas, users, roles, secrets and developers. - Code in its own sandboxes: Custom code of each project runs in sandbox containers of its own, with the packages it chose. - One platform to pay for: Quiet projects leave room for busy ones on shared servers, instead of a half-idle server per project. - Projects on demand: One call to the management API creates an admin, saves its secrets and pulls its APIs from Git. - Pause without losing anything: An inactive project frees its sandboxes and keeps its APIs, settings and data until it comes back. - The right view for each: The root user sees the whole fleet in the dashboards, each admin and developer sees only their own traffic. ### Servers - Caddy load balancer ×1: 2 vCPU, 2 GB, 20 GB SSD. Runs Caddy: HTTPS and WSS for every project, HTTP to 38246, WebSockets to 38245 of the servers. - API servers ×2 or more: 6 – 8 vCPU, 12 – 16 GB, 60 GB SSD. Runs API Maker, run by PM2, cpuCount AUTO, Docker: sandboxes of every project. Sandboxes are per project: give the servers memory for the projects that run custom code at the same time. - Platform data ×1: 4 vCPU, 16 GB, 150 GB NVMe. Runs API Maker DB: definitions of every project, Redis: cache and events of every project. - Project databases 1+ per project: as needed, as needed, as needed. Runs Any of the 8 database types, chosen by each project, Small projects can share a database server. ### Trade-offs - Projects share the fleet: one very busy project uses capacity the others would like. Watch the traffic per project. - One platform to update: every project moves to a new version of API Maker together. - Projects that need strict physical separation, like regulated customers, are better served by the multi-tenant architecture. ### Questions **What is shared between projects, and what is not?** Shared: Caddy, the servers, API Maker DB and Redis. Separate: API paths, database instances, schemas, custom APIs, secrets, auth providers, users, roles, developers, sandbox containers and Git repositories. **Can each project have a domain of its own?** Yes. Add a site to the Caddyfile for each domain, like shop.example.com and crm.example.com, all passed to the same servers. Caddy gets and renews a certificate for each of them, and the path of the project stays in the URL. **Can a user log in to several projects?** By default a token works only in the admin account that issued it. A root setting lets the same token be used in other admin accounts where a user with the same username and password exists, with the permissions of that account. **How is this different from multi-tenant?** Here each project is a different product with its own APIs. In the multi-tenant architecture one product and one set of APIs serve many customers, each with a database of their own. ## Architecture: Multi-tenant: a database per customer URL: https://apimaker.dev/architectures/multi-tenant One product and one set of APIs, with the data of every customer in a database of its own. True multi-tenancy, supported out of the box: each request names its tenant, and API Maker runs it on that customer's own database, found in a tenants table. Customers share the servers, never the data. - Servers: 5+ VPS - Capacity: ~5,518 req/s on 2 × 4 cores, shared - Best for: B2B SaaS products; Regulated customers; Per-customer backups; Data residency ### Benefits - Isolation by design: Each customer has its own database. No tenant_id to forget in a query, no way to read another tenant's data. - One product to maintain: The same APIs, schemas, permissions and custom code serve every customer. Fix once, every tenant gets it. - Each tenant where it fits: Small tenants share a server, large ones get their own, regulated ones keep their database at home. - Onboarding is a row: Create the database, insert one row in the tenants table, and the new customer is served at once. - Encrypted connection strings: Connection strings in the tenants table can be encrypted by API Maker and are decrypted only when a pool opens. - Per-customer operations: Back up, restore, export, move or delete one customer without touching the others. ### Servers - Caddy load balancer ×1: 2 vCPU, 2 GB, 20 GB SSD. Runs Caddy: HTTPS and WSS for app.example.com, Optional: certificates for the domains of your customers. - API servers ×2 or more: 4 – 8 vCPU, 8 – 16 GB, 40 GB SSD. Runs API Maker, run by PM2, behind Caddy, A connection pool per tenant, opened on demand. Each server keeps a pool for every active tenant: plan the connection limits of the database servers for it. - Shared tenant DB servers as needed: 8 vCPU, 32 GB, 500 GB NVMe. Runs One database per small tenant, The same engine as the multi-tenant instance. - Dedicated tenant DB servers 1 per large tenant: 16 vCPU, 64 GB, 1 TB+ NVMe. Runs The database of one large customer. - Platform ×1: 4 vCPU, 8 GB, 100 GB NVMe. Runs The tenants table, in one of your databases, API Maker DB and Redis. ### Trade-offs - Every schema change must run on every tenant database: automate your migrations for all of them. - Pools add up: API servers × active tenants. Watch the connection limits of the database servers. - All tenant databases of a multi-tenant instance use the same database engine. ### Questions **How does a request choose its tenant?** The tenant is written after the instance name, separated by two colons: /api/gen/saas/crm::acme/…, or sent in the x-am-tenant-username header. Your frontend takes it from the account the user belongs to, and a user token made for a tenant works for that tenant only. **Where are the connection strings stored?** In a table of your own, the tenants table, with a column for the username of the tenant and one for its connection string. Turn on the encryption conversion of that column and API Maker stores it encrypted. **Can each customer use a domain of its own?** Yes, with the on-demand TLS of Caddy: it gets a certificate for a domain the first time the users of a customer reach it. Its ask setting points to an endpoint that answers 200 for the domains you allow, for example a custom API that looks the domain up in the tenants table. **Can tenants use different database types?** Not within one multi-tenant instance: its tenants share its engine, for example PostgreSQL. A product can have several multi-tenant instances, each with its own engine and tenants table. ## Architecture: Clustered Redis and databases URL: https://apimaker.dev/architectures/clustered-data Redis Cluster, a MongoDB replica set and sharded data: a data tier with no single node to lose. The data tier becomes clusters. Redis runs as masters with replicas, API Maker DB and its logs live on a replica set of three, and your own data can be sharded. When a node fails, a copy takes over and API Maker follows by itself. - Servers: 13+ VPS - Capacity: ~8,277 req/s on 3 × 4 cores - Best for: Large caches; Many WebSocket clients; Growing data sets; Data that must survive a node ### Benefits - No single data node: Every Redis slot has a replica and API Maker DB lives on three machines: one failed node stops no API. - Automatic failover: Redis promotes a replica, MongoDB elects a new primary, and API Maker follows both without a restart. - Memory that adds up: Cached responses, locks and WebSocket subscriptions are spread over the masters. Add one when memory runs short. - A replica set that pays twice: The replica set API Maker needs for its transactions also keeps live copies of every definition and log. - Your data scales too: Point an instance at the mongos routers of a sharded MongoDB, or at any cluster your database offers. - Only settings change: A list of nodes for Redis, a list of hosts for MongoDB. No code, no plugin, the same APIs. ### Servers - Caddy load balancer ×1: 2 vCPU, 2 GB, 20 GB SSD. Runs Caddy: HTTPS and WSS, certificates from Let's Encrypt, Health checks of the API servers on GET /ping. - API servers ×3 or more: 4 – 8 vCPU, 8 – 16 GB, 40 GB SSD. Runs API Maker, run by PM2, the same .env on every server, Connected to every node of the clusters. - Redis nodes ×6: 3 masters, 3 replicas: 2 vCPU, 4 – 16 GB, 20 GB SSD. Runs Redis 7, cluster-enabled yes, Cache, locks, events, auto increments, WebSocket subscriptions. Never put a master and its own replica on the same VPS. - MongoDB members ×3: 4 vCPU, 8 – 16 GB, 100 GB+ NVMe. Runs MongoDB, replica set rs0, api_maker_db and api_maker_logs. Keep an odd number of members, three or five, so a majority remains after a failure. - Your data as needed: 8+ vCPU, 32 GB+, 500 GB+ NVMe. Runs Sharded MongoDB: mongos, config servers and shards, Or a TiDB, Percona XtraDB or MariaDB Galera cluster. ### Trade-offs - More machines to run and watch: six Redis nodes and three MongoDB members instead of one of each. - Redis copies to its replicas asynchronously: a failover can lose the last writes of the failed master. - Spread the copies well: a master and its replica, or two members of the replica set, on one VPS fail together. - Caddy, the load balancer, is still a single machine here. The High availability architecture doubles it. ### Questions **How do I make Redis fail over automatically?** Run a Redis Cluster and list its nodes in redisInternal and redisExternal. API Maker connects to a single node when one is listed, and to the cluster when there are several. Even three masters with one replica each give every key a copy ready to take over. **Should redisInternal and redisExternal share a cluster?** They can. On busy setups, give each its own cluster: cached responses in redisExternal may be evicted when memory runs short, while redisInternal holds locks, events, auto increments and WebSocket subscriptions that must stay. **Which clusters can hold my own data?** Whatever your database offers behind a connection string: MongoDB replica sets and sharded clusters through mongos, TiDB, Percona XtraDB and MariaDB Galera clusters, or a primary with standbys behind a virtual IP for PostgreSQL, MySQL, SQL Server and Oracle. API Maker generates the same APIs on each of them. **What do the apps notice during a failover?** Requests that need the keys of the failed Redis master, and writes during a MongoDB election, wait or fail for the seconds the switch takes. Everything else is served as usual, and API Maker needs no restart before or after. ## Architecture: High availability URL: https://apimaker.dev/architectures/high-availability A spare for every tier: two Caddy load balancers on a floating IP, three API servers and replicated databases. Any single VPS can stop and the apps keep working. The public IP floats between two Caddy load balancers, health checks steer around failed servers, and every database has a copy ready to take over. Upgrades roll through the servers one at a time. - Servers: 10+ VPS - Capacity: ~8,277 req/s on 3 × 4 cores - Best for: Business-critical APIs; Uptime SLAs; Upgrades in office hours; Payments and bookings ### Benefits - Survives any one VPS: A load balancer, an API server, a database member: any single machine can stop while the apps keep working. - One address, always: The floating IP moves to the standby load balancer within seconds. Apps and DNS never see a change. - Health checks built in: Every API Maker server answers GET /ping, which tells Caddy on the load balancers where to send traffic, and where not to. - Schedulers never stop: The server holding the scheduler lock can fail: another one takes the lock and the jobs run on time. - Upgrades in office hours: Roll a new API Maker version through the servers one at a time, without a maintenance window. - Every database has a copy: API Maker DB elects a new primary by itself, and your database fails over to its standby. ### Servers - Caddy load balancers ×2: active, standby: 2 vCPU, 2 GB, 20 GB SSD. Runs Caddy: HTTPS, WSS and health checks, the same certificates on both, keepalived: the floating IP. - API servers ×3 or more: 4 – 8 vCPU, 8 – 16 GB, 40 GB SSD. Runs API Maker, run by PM2, the same .env on every server, The admin panel on each of them. - Data servers ×3: 4 vCPU, 16 GB, 200 GB NVMe. Runs A MongoDB member of rs0: API Maker DB and logs, A Redis master, and the replica of another master. Three servers hold both clusters: when one stops, its Redis master has a replica on another server. - Database servers ×2: primary, standby: 8 vCPU, 32 GB, 500 GB NVMe. Runs PostgreSQL with streaming replication, Patroni or repmgr, and a virtual IP. ### Trade-offs - Twice the machines of a simple setup, to patch and to watch. - A floating IP moves within one location of one provider. To survive the loss of a data center, see Multi-cloud. - Requests running on a server or database at the moment it fails return an error: let clients retry safe requests. - Test your failovers on purpose, before a real failure tests them for you. ### Questions **Why a floating IP and not DNS with two addresses?** A floating IP moves within seconds and every client follows it at once. DNS answers are cached by resolvers and devices, often for minutes, so users keep reaching a failed address. DNS is the right tool between regions, as the Geo-routing architecture shows, with a setup like this one in each region. **Does my VPS provider support floating IPs?** Most do, under names like floating, reserved, failover or additional IP. keepalived decides which load balancer holds it, and a small script calls the API of the provider when the IP has to be routed to the other one. **What happens to WebSocket clients when a server fails?** Their connections close. The clients reconnect through Caddy, land on another server and subscribe again. Plan for events missed while reconnecting, for example by reloading the data after a reconnect. ## Architecture: Multi-cloud URL: https://apimaker.dev/architectures/multi-cloud The same API Maker on VPS of several providers, joined by encrypted tunnels. No provider can take you down. Run API Maker on plain VPS of two or three providers at once. DNS spreads the users over the sites, WireGuard joins their private networks, and the databases keep a copy on every site. Lose a provider, or leave one, without downtime. - Servers: 12+ VPS - Capacity: ~11,036 req/s on 4 × 4 cores - Best for: No vendor lock-in; Provider outages; Hybrid and on-premises; Changing providers ### Benefits - No lock-in: Plain VPS and open source software on every site: any provider that rents a Linux server can host one. - Survives a provider: A whole provider can go dark: the other sites keep the majority, elect a primary and carry the traffic. - Private by design: Sites talk only through WireGuard tunnels. No database port is ever open to the internet. - Move at your pace: Add a site at a new provider, let it copy the data, then retire the old one. No export, no import. - Hybrid too: A site can be your own data center: keep databases on premises and serve them through the same APIs. - Each provider for its strength: Cheap compute from one, fast disks from another. Mix them by what they do best and what they cost. ### Servers - Caddy load balancers ×1 per serving site: 2 vCPU, 2 GB, 20 GB SSD. Runs Caddy: HTTPS and WSS, for the API servers of its site. - API servers ×2+ per serving site: 4 – 8 vCPU, 8 – 16 GB, 40 GB SSD. Runs API Maker, run by PM2, the same .env on every site, Reads from the nearest MongoDB member. - Data servers ×1 per site, 3 sites: 4 vCPU, 16 GB, 200 GB NVMe. Runs A MongoDB member of rs0, A Redis master, and the replica of another site's master. Three sites, not two: with two, losing either one also loses the majority. - WireGuard gateways ×1 per site: 1 vCPU, 1 GB, 10 GB SSD. Runs WireGuard, routing its site to the others, Can share a VPS with the load balancer. ### Trade-offs - Writes cross providers to reach the primary: keep the sites a few milliseconds apart. - Most providers bill the traffic leaving their network, and replication adds to it. - Three sites are needed for a majority: two sites alone cannot tell a lost provider from a lost tunnel. - More moving parts: tunnels, DNS health checks and several providers to keep patched and paid. ### Questions **Can one of the sites be my own data center?** Yes. A site is any set of Linux machines: VPS of a provider or servers of your own. Join it with the same WireGuard tunnels and it works like any other site. **How far apart can the sites be?** MongoDB and Redis work best with a few milliseconds between sites, such as neighbouring cities or countries. Farther apart, every write waits longer: for users on several continents, see the Geo-routing architecture. **How does the Caddy of each site get the certificate of api.example.com?** DNS answers with both sites, so either Caddy may be asked to prove the domain. Give them a shared certificate storage, reached through the tunnels, or the DNS challenge of your DNS provider: each one then holds a valid certificate. **Do I need a DNS provider with health checks?** It removes a lost site from the answers by itself. Without it, a script on a surviving site can update the record through the API of your DNS provider, or you switch it by hand. ## Architecture: Geo-routing: the nearest region URL: https://apimaker.dev/architectures/geo-routing HTTP calls and WebSockets answered by the region nearest to each user, behind one address for the whole world. Start with one address answered directly by one place, then add regions: GeoDNS sends every user to the nearest one, for HTTP calls and WebSockets alike. Reads stay in the region, while writes and events cross the world in the background. - Servers: 10+ VPS - Capacity: ~8,277 req/s on 3 regions × 4 cores - Best for: Users on several continents; Real-time apps worldwide; Mobile apps; Global SaaS ### Benefits - Fast everywhere: Users talk to servers a few milliseconds away instead of across an ocean, and the TLS handshake with Caddy happens nearby. - WebSockets nearby: Each client keeps its WebSocket in its own region, and events raised anywhere reach it through Redis. - One address: api.example.com stays the same for every user and every app. DNS does the routing. - Local reads: With readPreference=nearest, the servers of each region read from the database member next to them. - Regions back each other up: When a region fails, DNS sends its users to the next one and they keep working. - The same APIs everywhere: Every region reads the same API Maker DB: one git pull and every server in the world reloads. ### Servers - GeoDNS ×2, or your DNS provider: 1 vCPU, 1 GB, 10 GB SSD. Runs PowerDNS with the GeoIP backend, Health checks of every region. - Caddy load balancer ×1 per region: 2 vCPU, 2 GB, 20 GB SSD. Runs Caddy: HTTPS and WSS for api. and ws.example.com, Load balancing over the servers of its region. - API servers ×1+ per region: 4 – 8 vCPU, 8 – 16 GB, 40 GB SSD. Runs API Maker, run by PM2, the same .env in every region, Two per region to survive a server. - MongoDB members ×1 per region: 4 vCPU, 16 GB, 200 GB NVMe. Runs One replica set: your data and API Maker DB, The primary in the home region. - Redis Cluster ×3 – 6, home region: 2 vCPU, 8 GB, 20 GB SSD. Runs Events, locks and cache for every region. ### Trade-offs - Writes travel to the home region: from Singapore, each write waits a round trip to Frankfurt. - Nearest reads can lag the primary by a moment. Read from the primary where users must see their own writes at once. - Redis lives in the home region: for far regions a cache hit costs a round trip there, often more than a local read. - Every region adds servers, tunnels and monitoring. Start with two regions where most of your users are. ### Questions **Can one server take HTTP calls and WebSockets directly?** Yes. Caddy takes the HTTPS calls and secure WebSockets and passes them to API Maker, which answers HTTP on port 38246 and WebSockets on port 38245, on the same server or on the servers behind it. With one address, one server or one region takes everything; with GeoDNS, each region takes the users nearest to it. **Does a WebSocket client stay in one region?** Yes. A WebSocket stays on the server it connected to, in the region DNS gave it, and events raised in any region reach it through Redis. If its region fails, the client reconnects and DNS gives it the next region. **How does the Caddy of every region get its certificates?** GeoDNS gives each place of the world a different region, so the checks of Let's Encrypt can reach any of them. Give the Caddy of every region a shared certificate storage, or the DNS challenge of your DNS provider. **Do I need a paid GeoDNS service?** No. PowerDNS with its GeoIP backend runs on two small VPS and picks the closest region that answers. A DNS provider with geo routing works the same way. **Why not a Redis in every region?** API Maker clears cached responses and publishes events in the Redis shared by every server. With a Redis per region, the other regions would keep stale responses and never hear the events. ## Architecture: Global enterprise platform URL: https://apimaker.dev/architectures/global-enterprise Every piece together: regions on several providers, customer data kept at home, shared state and one release flow. The architecture large SaaS products grow into. Three regions on three providers serve users nearby, tenant databases stay in the region of their customers, and one API Maker DB and one Redis keep every region in step. Releases and events reach the whole world within seconds. - Servers: 25+ VPS - Capacity: ~16,554 req/s on 6 × 4 cores - Best for: Global SaaS; Regulated industries; Data residency; Enterprise customers ### Benefits - Near every user: Users reach the closest region for their HTTP calls and WebSockets, wherever they are. - Data residency: Each tenant database runs in the region of its customer, with the same APIs for every customer. - No provider can stop you: One provider per region: an outage, a price change or a new country is never a crisis. - One release flow: DEV, QA and UAT first, then one git pull reaches every server of every region. - Real time across continents: Events raised in any region reach WebSocket clients in every region through Redis. - Many products, one fleet: Several projects share the servers, each with its own APIs, users, secrets and databases. ### Servers - GeoDNS ×2: 1 vCPU, 1 GB, 10 GB SSD. Runs PowerDNS with the GeoIP backend, Or your DNS provider with geo routing. - Caddy load balancers ×2 per region: 2 vCPU, 2 GB, 20 GB SSD. Runs Caddy and keepalived: an active and a standby, HTTPS and WSS, certificates shared or from the DNS challenge. - API servers ×2+ per region: 8 vCPU, 16 GB, 40 GB SSD. Runs API Maker, run by PM2, cpuCount AUTO, The same .env in every region. - MongoDB members ×1 per region, ×2 at home: 8 vCPU, 32 GB, 500 GB NVMe. Runs One replica set: API Maker DB and shared data, The primary in the home region. - Redis Cluster ×6, home region: 4 vCPU, 16 GB, 20 GB SSD. Runs Events, locks and cache of every region. - Tenant database servers as needed, per region: 8 – 16 vCPU, 32 – 64 GB, 1 TB NVMe. Runs The databases of the customers of the region, A standby for the largest ones. ### Trade-offs - The most servers and moving parts: automate provisioning, patching and monitoring from the first day. - Writes and shared state live in the home region: the other regions pay a round trip for them. - A tenant database is as available as its region: give the largest customers a standby next to it. - Costs add up over providers and regions. Grow into this one region, and one customer, at a time. ### Questions **Do I need all of this?** Rarely at the start. Each piece comes from an architecture before it and can be added on its own: a second region, a tenant database in a new country, a third provider. **Can a customer move to another region?** Yes. Copy its database to the new region, update its row in the tenants table and call the multi-tenant-instance-updated system API: every server resets its pool for that tenant. **How are several products kept apart on the same servers?** As projects: each one has its own APIs, databases, secrets, users and roles on the same fleet, as the Multi-project architecture shows.