Every payment provider retries webhooks. Stripe retries a failed event for up to three days, and it will also deliver the same event twice under normal conditions. If your handler is not idempotent, that behavior eventually shows up as a double charge, a double provision, or a duplicate email to a customer.
Why Duplicates Are Guaranteed
Delivery is at-least-once by design. A handler that does its work, then times out before returning 200, has already committed the side effect — and the provider has no way to know that. It retries. Your database now has two subscription records for one payment.
The same pattern appears with any queue, any inbound integration, and any client that retries on network error. Assuming exactly-once delivery is the single most expensive assumption in early billing code.
The Five-Line Habit
Create a table with one column that matters: a unique constraint on the provider’s event ID. Before doing any work, insert the event ID. If the insert violates the unique constraint, return 200 immediately and do nothing else.
That is the whole mechanism. The unique index — not application logic — is what enforces correctness, which is why it survives concurrent delivery of the same event to two application instances. A check-then-act pattern in code does not; two workers can both read “not processed” in the same millisecond.
Where the Transaction Boundary Goes
The insert and the side effect belong in the same database transaction whenever the side effect is a database write. Insert the event ID, apply the subscription change, commit. If anything fails, the whole thing rolls back and the retry does clean work.
When the side effect is external — charging a card, sending mail, calling a partner API — you cannot get that atomicity. Split it: commit the event record with a status of pending, perform the external call, then update to done. A row stuck in pending for more than a few minutes is your signal for manual review, and it is far cheaper to investigate one ambiguous row than to reconcile a duplicate charge.
Handling Out-of-Order Events
Idempotency stops duplicates but not reordering. A subscription cancellation can arrive before the update that preceded it, and blindly applying both leaves the account in the wrong state.
Store the event’s timestamp or version alongside the ID, and refuse to apply an event older than the state you already hold. For Stripe subscription objects, compare against the object’s own version rather than the delivery time. This turns reordering from a data corruption bug into a no-op.
Retention and Verification
Keep processed event IDs for at least twice the provider’s maximum retry window. For Stripe, three days of retries means a seven-day minimum; 30 days is a safer default and costs almost nothing at early volume. Purge on a schedule so the table does not become the largest thing in your database by year two.
Test it the cheap way: replay the same webhook payload three times against staging and assert that exactly one subscription row exists and exactly one email was queued. That test takes ten minutes to write and catches the class of bug that otherwise surfaces as a refund request from your first enterprise customer.
Get articles like this by email
Production patterns, checklists, and failure stories for people shipping SaaS. No spam.