Idempotency

An operation is idempotent if performing it more than once has the same effect as performing it once. In distributed systems this matters because repeats are guaranteed to happen: clients retry after timeouts, and message brokers redeliver messages.

Without idempotency, a retried payment request can charge a customer twice.

Why repeats happen

• A client times out and retries, but the first request actually succeeded.
• A message broker delivers a message again because the acknowledgement was lost.
• A user double-clicks a button.
• A batch job is re-run after a partial failure.

Techniques

Idempotency keys. The client generates a unique key per logical operation and sends it with every attempt. The server stores the key with the result. If the key is seen again, it returns the stored result instead of repeating the work.

POST /payments
Idempotency-Key: 7f3c1a2e-5b8d-4c1f-9e6a-2d4b8f0c1a37

Naturally idempotent operations. Prefer “set the status to SHIPPED” over “advance the status by one step”. Setting a value is safe to repeat; incrementing is not.

Deduplication on the consumer. Record the IDs of processed messages and skip any already seen. Store the record in the same transaction as the business change, so the two can’t drift apart.

Conditional updates. Use version numbers or “update only if current state is X” so a replayed change has no effect.

Pitfalls

• Storing the idempotency key after doing the work leaves a window where a retry slips through. Reserve the key first.
• Keys must cover the whole operation’s input. Reusing a key with a different request body should be rejected, not silently accepted.
• Decide how long keys are kept, and make sure retries can’t arrive after that.

Where it matters most

Payments and money movement, order placement, and any event-driven consumer.