Use local queues and header-driven throttling to keep outbound calls inside provider limits without dropped work or cascading failures.
Read the headers on every response
Most providers return X-RateLimit-Remaining, X-RateLimit-Reset, and Retry-After on both successful and 429 responses. Parse these values first and store them alongside the response body rather than treating the status code alone as the signal. Ignoring the numeric remaining count leads to the common failure where a client continues to fire requests until it receives a hard 429, at which point the reset window has already started.
When the remaining count drops below a small safety margin, pause the next request until the reset timestamp. This single check prevents the majority of rate-limit errors without any additional backoff logic. Log the header values on every call so you can see drift between your local clock and the provider’s reset window.
Buffer outbound calls in a bounded queue
Place every API call into a local queue that only releases work when the current remaining quota permits it. The queue holds the request payload, a promise resolver, and a timestamp so that callers can still receive results once the slot opens. Keep the queue size small enough that memory pressure appears before the provider’s limit is reached; when the queue fills, the producer must slow down or drop low-priority work.
Implement the release logic as a simple loop that checks the stored remaining value and the current time against the reset header. If the value is zero, sleep until reset plus a small jitter interval. This approach turns a hard rate limit into a controlled delay rather than an error path that triggers retries.
Propagate backpressure to your own producers
Expose the queue’s current depth and the time until the next available slot through a simple status endpoint or callback. Upstream batch jobs or webhook handlers can then decide whether to enqueue more work or defer it. Without this signal, a sudden spike in sign-ups or events will fill the queue and cause either memory exhaustion or silent drops.
When the queue depth exceeds a threshold, return a 503 with a Retry-After header to the internal caller. This mirrors the provider’s behavior and keeps the pressure contained inside your own system instead of leaking it to the external API.
Combine header data with adaptive retry windows
On a 429, read the Retry-After value if present and use it as the next release time for the queue. When the header is absent, fall back to an exponential delay that starts from the last known reset interval. Record the actual delay applied on each retry so you can detect when your local backoff is consistently longer than the provider’s window.
Avoid fixed retry counts. Instead, let the queue size and the observed remaining quota determine whether a request is retried at all. Requests that have waited past the reset window should be dequeued first; newer requests stay behind them. This ordering preserves the original arrival sequence while still respecting the limit.