Idempotency in payment systems

A webhook can genuinely arrive twice. A retry can genuinely be sent twice. A payment system has to treat "process the same event again" as a normal, expected event, not a rare edge case.

Intermediate

3 min read

Why duplicates are guaranteed, not just possible

Most payment gateways promise at-least-once delivery for webhooks — if a merchant's endpoint doesn't respond fast enough, times out, or briefly errors, the gateway will retry the same event later. That's a deliberate reliability feature, not a bug: it means a webhook genuinely can, and eventually will, arrive more than once for the exact same underlying event. A payment system that assumes "one event, one delivery" will, sooner or later, double-process a real event — crediting an account twice, sending two confirmation emails, or worse.

The fix: make "processed twice" produce the same result as "processed once"

Idempotency means an operation can be safely repeated without changing the outcome beyond the first successful application. For payment webhooks specifically, this means: receiving the same event twice should result in exactly the same system state as receiving it once — not double the effect.

Pattern 1: a dedicated event-audit table with a natural unique key

WebhookEvent
  id
  externalEventId   -- built from the event's own identity, e.g.
                        "TRANSACTION:482913" or "SUBSCRIPTION_RENEWED:77"
  processed         -- boolean, false until fully handled
  receivedAt

The externalEventId is derived from fields the gateway itself considers identifying (a transaction ID, a subscription ID plus event type) — not a value the merchant invents. Before doing any real work, the handler does an upsert keyed on that column (enforced by a database unique constraint, not just an application-level check) and inspects the processed flag:

The unique constraint matters more than it looks — an application-level "check then insert" has a race window where two near-simultaneous deliveries could both pass the check before either finishes inserting. A database-enforced uniqueness constraint closes that race; an application-level check alone does not.

Pattern 2: a second, independent check at the domain-record level

Even with the audit table above, real systems often add a second layer: before creating a payment/invoice record, look it up by the gateway's own transaction ID (a unique column on that domain table too) and skip creating a duplicate if one's already there. This isn't redundant paranoia — it protects against a scenario the first layer alone doesn't fully cover: a bug or a manual replay that re-triggers the handling logic for an event already marked processed, without going through the same webhook-receipt code path. Two independent checks, at two different layers, catch two different classes of duplicate.

Idempotency keys for outbound requests too

The same concept applies in the other direction: when a merchant's own backend calls a gateway's API to create a charge, a slow or dropped response leaves the caller unsure whether the charge actually went through. Retrying blindly risks a double charge. The standard fix is an idempotency key — a unique value the caller generates once per logical operation and sends with the request; the gateway recognizes a repeated key and returns the original result instead of creating a second charge, even if the original response was never received.

POST /charges
Idempotency-Key: order-4821-attempt-1
{ "amount_cents": 5000, ... }

The general principle underneath both patterns

Anything that moves money, or represents that money moved, needs to be safe to attempt twice. That includes inbound webhook handlers and outbound API calls alike. Idempotency isn't an optimization to add later — it's a correctness requirement for the exact moment a payment system meets an unreliable network, which is to say: always.