Rate-Limit Playbook: What to Limit First

Most teams add rate limiting to their public API first because that is what the tutorials cover. That is the wrong order. The limit that saves you money and downtime in the first year is the one you put on your own outbound calls. Here is the priority order, with the numbers we actually use.

First: Outbound Provider APIs

Your provider will enforce its limit whether or not you have one. The difference is that your limiter fails politely with a queued job, and theirs fails with a 429 in the middle of a user action or, worse, a partially completed workflow.

Set your own budget at 25 to 40 percent of the documented limit and share it across every process. If a provider allows 100 requests per second, run a token bucket with a refill rate of 30 per second and a burst capacity of 60. The headroom is not waste; it is what absorbs a retry storm without pushing you over the real ceiling.

Split the budget by workload class. Reserve roughly 70 percent for interactive calls a user is waiting on and 30 percent for bulk and background work, and run bulk work through a queue with its own slower limiter. A nightly sync of 50,000 records must not be able to starve a signup.

Second: Auth Endpoints

Login, magic-link request, token verification, and password reset if you have one. These are the endpoints attacked first, and the traffic looks nothing like normal usage, so a global API limit will not catch it.

Concrete numbers: 5 login requests per email address per 15 minutes, 20 per IP per 15 minutes, 10 verification attempts per token before invalidation. Count them in a durable store, not in process memory, or a rolling deploy resets every counter. Return the same response body and the same latency regardless of whether the account exists.

Third: Your Own API

Now limit the public surface, per tenant rather than per IP. Per-IP limits punish an office behind one NAT and do nothing against a distributed client. A reasonable default is 60 requests per minute sustained with a burst of 120 per tenant, and a separate, tighter limit on any endpoint that writes or triggers work: 10 per minute for exports, imports, and bulk mutations.

Return 429 with a Retry-After header and a JSON body naming the limit that was hit. Clients that get a specific error back off correctly; clients that get a generic 429 retry immediately and make it worse.

Fourth: Background Job Concurrency

Concurrency is a rate limit wearing a different name. Cap workers per queue explicitly, because the default in most job runners is enough parallelism to exhaust a database connection pool. Start at 5 concurrent jobs for anything touching the database and 2 for anything calling a third-party API, then raise it only after you have watched the pool and the provider’s error rate under load.

What to Alert On

Two alerts, no more. Any 429 received from a provider at all, because with a budget in place it means your model of their limit is wrong. And sustained wait time in your own limiter above a threshold you choose, because a queue that is always full means you need higher limits from the provider or less chatty code, and both take lead time to fix.

The Playbook, In Order

1. Inventory every outbound call: provider, documented limit, and whether a user is waiting on it.
2. Put a shared token bucket in front of each provider at 25 to 40 percent of the documented limit.
3. Split each bucket 70/30 between interactive and bulk, and move bulk work behind a queue.
4. Add durable per-email and per-IP limits on every auth endpoint, with identical responses for unknown accounts.
5. Add per-tenant limits on your API, 60 per minute sustained and 120 burst, with a tighter limit on write and export routes.
6. Return 429 with Retry-After and a body that names the limit.
7. Cap worker concurrency per queue: 5 for database work, 2 for third-party calls.
8. Alert on any provider 429 and on sustained limiter wait time.
9. Load-test the limiter itself once, so you know it fails closed rather than passing traffic when the counter store is unavailable.