Test your API as every role before someone else does

Every test passed. The pipeline was green. Then a logged-in user opened their order at /orders/41, got curious, and changed the URL to /orders/42.

That is not a story about one company. It is a generic scenario, and it is boring on purpose, because that is how these bugs usually look. The tests were fine. They just never asked the question that mattered.

This is post 2 in my CalmAPI 2 series. Post 1 was about rewriting the framework. This one is about the habit I care about most, whatever framework you use: testing your API as every kind of user, not just the happy one.

Why happy-path tests never catch access bugs

Most API test suites log in once, as a normal user, and check that things work. Create an order, read it back, update it, delete it. Green.

Those tests prove what a user can do. Access bugs live in what a user cannot do, and nobody writes those tests because nothing visibly breaks when they are missing. The feature works. It just works for the wrong people too.

OWASP puts this at the very top of its API Security Top 10. On Broken Object Level Authorization they say it plainly: “The violation happens at the object level, by manipulating the ID.” The user is allowed to call the endpoint. They are just not allowed to see that record. Its cousin, Broken Object Property Level Authorization, covers the field version of the same mistake: returning properties a user should not read, or letting them write properties they should not control.

AI-written code makes this worse in a quiet way. An agent will happily generate a clean findById(req.params.id) that passes every test you ask for. It does not know who should own that record unless you tell it.

The persona matrix

The fix starts on paper. List your personas, list your routes, and write down what each persona should get. Four personas cover most apps: anonymous, the owner of the record, another signed-in user, and an admin.

Writing this table is half the value. If you cannot fill a cell without opening the code, that cell is where your next bug lives.

Doing it by hand with node:test

You do not need any framework for this. Here is the matrix turned into tests with Node’s built-in test runner and fetch. It works against any HTTP API.

import { test } from 'node:test';
import assert from 'node:assert/strict';

const BASE = process.env.API_URL ?? 'http://localhost:3000';
const tokens = {
  anonymous: null,
  owner: process.env.OWNER_TOKEN,
  otherUser: process.env.OTHER_TOKEN,
  admin: process.env.ADMIN_TOKEN,
};
const expected = {
  'GET /orders/41':    { anonymous: 401, owner: 200, otherUser: 404, admin: 200 },
  'DELETE /orders/41': { anonymous: 401, owner: 403, otherUser: 404, admin: 204 },
};

for (const [route, byPersona] of Object.entries(expected)) {
  for (const [persona, status] of Object.entries(byPersona)) {
    test(`${route} as ${persona} -> ${status}`, async () => {
      const [method, path] = route.split(' ');
      const auth = tokens[persona] ? { authorization: `Bearer ${tokens[persona]}` } : {};
      const res = await fetch(BASE + path, { method, headers: auth });
      assert.equal(res.status, status);
    });
  }
}

Seed the tokens and order 41 in a setup step against a test database, and you are done. Every new route gets a row. Every new role gets a column. When someone changes an access rule, a test fails, and that is exactly what you want.

The probes people forget

Status codes per persona are the core. But the bugs I find most often sit around the edges, in requests nobody tries because a normal client would never send them.

# mass assignment: must be rejected, never saved
POST /orders   {"items": [...], "role": "admin", "userId": "someone-else"}

# operator injection in the query string
GET /orders?status[$ne]=x

# oversized page
GET /orders?limit=100000

# forged token, and a real token in the URL
Authorization: Bearer forged.token.value
GET /orders?access_token=REAL_TOKEN

# malformed id: plain 404, no stack trace, no database error
GET /orders/not-an-id

Mass assignment is the one that bites hardest. If your create handler does Order.create(req.body), a user can set any field the model has, including the owner id. Same story for operator injection: a query string like status[$ne]=x turns into an object in Express, and if it reaches MongoDB untouched you are no longer filtering what you think you are filtering. If you have inherited a vibe-coded app, add these probes to the first ten things I check in a vibe-coded app.

Also check login. A wrong password and an unknown email should get the same status and the same message, or your login form becomes a tool for checking who has an account.

Return 404, not 403, for records you may not see

Look at the “Another user” column in the table again. It says 404, not 403. That is on purpose.

A 403 says “this exists, but not for you”. That alone leaks information. Someone can walk ids and learn which orders exist, how many there are, and roughly how fast you are growing. A 404 says nothing. From the outside, a record you cannot see and a record that does not exist look identical.

The easiest way to get this right is to scope the query itself: look up the order by id and owner in one query, instead of fetching it and checking ownership afterwards. If it is not yours, the query finds nothing, and 404 falls out naturally. Keep 403 for the cases where the user can clearly see the thing but cannot do the action, like the owner trying to delete in our table.

What I ended up building

After writing that loop by hand for the tenth project, I built it into CalmAPI, the Node.js framework I maintain. Version 2.0 is in beta (2.0.0-beta.2 on npm under the next tag as I write this), so treat it as a beta.

According to the audit docs, calm audit boots your app on a throwaway database, signs up one user per persona (the owner, another user, an admin, plus one for each custom role your policies use), seeds a valid record for each resource, and calls list, read, create, update and delete on every resource as every persona. It compares each answer with your access rules. On top of that it runs probes: mass assignment, operator injection, the limit cap, unknown filters, error hygiene, a forged bearer token, a token in the query string, and account enumeration. The database is dropped afterwards, and it does need a running MongoDB.

npm create calmapi@latest      # needs Node.js 22.12+ and MongoDB
npx calm audit
MONGODB_TEST_URI=mongodb://127.0.0.1:27017 npx calm audit --ci
npx calm audit --accept --reason "why access widened"

--ci exits with code 1 when a check fails or when access is wider than your committed security.snapshot.json. That snapshot is the exposure budget: once it exists, any widening of access fails the build until a person accepts it with a reason. AI agent shells are refused at that step. I like that part a lot. An agent can change a policy, but a human has to sign off on giving more people more access.

A checklist you can print

• Write the persona matrix: anonymous, owner, another user, admin, plus your custom roles.
• Test every route as every persona, and assert the status code.
• Return 404 for records a user may not see. Scope the query, do not check afterwards.
• Lists: every returned record must belong to the caller when the rule is owner-only.
• Responses: no field the persona should not read.
• Creates: reject role, owner id and read-only fields from the body.
• Query strings: reject operators like $ne, unknown filters and huge limits.
• Tokens: forged and expired get 401. Tokens in the URL get 401.
• Errors: malformed ids give a plain 404, no stack trace.
• Login: same answer for wrong password and unknown email.

Run it in CI, not once a quarter. Access rules change in small pull requests, and that is where they quietly widen.

Use whatever you like, the habit matters more than the tool. A hand-written loop that runs on every pull request beats a fancy scanner nobody opens. If you want a second pair of eyes on the access rules of an API you already shipped, our open source work is a good place to start, or write to contact@broadifitech.com.