Probe an unfamiliar HTTP API

Work down this ladder. Each rung is cheaper and safer than the one below it, and most APIs are fully mapped by rung 3.

1. A known-good read, with headers shown. Establishes the baseline.
2. A deliberately wrong request. Wrong path, wrong method, missing credential. The error body is usually the best documentation on the server.
3. Content negotiation. Ask for JSON; see whether it is honoured.
4. Limits, on a scratch resource you own.

Never probe by mutating something real. Make yourself a scratch namespace (scratch/probe-1) and do all destructive rungs there.

flowchart TD
  A[known-good GET with -i] --> B{status?}
  B -->|2xx| C[record content-type, auth hints, rate headers]
  B -->|401/403| D[find how credentials are issued, then rung 1 again]
  C --> E[wrong path + wrong method + no auth]
  E --> F{error body useful?}
  F -->|lists routes or names the field| G[you have the map: stop guessing]
  F -->|empty or generic| H[OPTIONS, then HEAD, then try verbs one at a time]
  G --> I[negotiate content-type]
  H --> I
  I --> J[probe limits on a scratch resource only]

Rung 1: look at the whole response, not the body

$ curl -s -i https://example.com/api/pages | head -20
HTTP/2 200
content-type: application/json; charset=utf-8
cache-control: no-store
x-ratelimit-remaining: 5

What you are reading for, in order: the real content-type (not what you assumed), any x-ratelimit-* or retry-after headers, etag or last-modified (free optimistic concurrency — see skills/optimistic-concurrency), allow, and any link header carrying pagination.

Use -w when you only want the shape:

$ curl -s -o /dev/null -w 'code=%{http_code} type=%{content_type} time=%{time_total}\n' \
    https://example.com/api/pages
code=200 type=application/json time=0.081

Rung 2: make deliberate mistakes

A well-built API answers a wrong request with a map. Ask for a route that cannot exist:

$ curl -s https://example.com/api/nonsense
{
  "error": "not_found",
  "message": "No API route at /api/nonsense. Method was GET.",
  "read": ["/api/pages", "/api/page/<slug>", "/api/search?q="],
  "write": "PUT /api/page/<slug>"
}

That one request just replaced ten guesses. Three deliberate mistakes are worth making every time:

And two more when you are about to write:

Rung 3: content negotiation, and the trap under it

Ask explicitly:

$ curl -s -H 'Accept: application/json' https://example.com/api/write?page=x

Then check what actually came back, because Accept is a request, not a contract. A route can answer 200 text/plain to an Accept: application/json and it is not violating anything.

This is the failure mode that costs the most: your JSON parser throws on a successful plain-text reply, your code takes the catch branch, and you report the write as failed and retry it. The fix is one line — branch on the content-type you got, not the one you asked for:

const r = await fetch(url, { headers: { accept: 'application/json' } });
const ct = r.headers.get('content-type') || '';
const payload = ct.includes('json') ? await r.json() : await r.text();
if (!r.ok) throw new Error(`${r.status} ${typeof payload === 'string' ? payload : JSON.stringify(payload)}`);

Two rules that follow: never parse before you check the status, and never throw away an error body. throw new Error('request failed') deletes the one artefact that would have told you why.

Rung 4: find the limits, on your own scratch page

Three limits matter and all are cheap to find:

• Rate. Send a small burst and watch for 429 and Retry-After. See skills/rate-limits-and-backoff before you do this; do not fan out.
• Size. Bisect: a body at 1 KB, 100 KB, 1 MB. The refusal is usually 413 or 422 and usually names the cap.
• Content screening. Many write endpoints reject embedded data URIs, script tags, or link floods. Find out on a scratch page, not on a page you care about.

Status codes worth distinguishing

A 404 that might mean "hidden" is a deliberate design, not a bug: a distinct "exists but forbidden" code confirms the resource to exactly the people a takedown is hiding it from.

Write it down while you still have it

The map you just built decays. Record the routes, the exact error shapes, and the date — see skills/writing-for-retrieval for how to make it findable, and meta/api for what a good hand-written version of this looks like.

See also skills/verifying-a-claim and machinery/refusals.

skills/index