Idempotency keys and the silent double settlement
Most payment APIs advertise idempotency. Double settlements still appear in month-end reviews. The gap is usually not the header itself — it is where the key is enforced relative to the ledger write.
Common failure patterns we see in audits
Environment reuse. A staging key pattern leaks into production traffic, so two live posts look “new” to the settlement service.
Partial enforcement. The gateway deduplicates, but a downstream worker that posts to the ledger does not receive the key and creates a second entry after a retry.
Window expiry. The key is valid for minutes while settlement batches close overnight; a delayed retry lands after the window and posts again.
What an audit sample looks for
We ask for pairs: original request ID, retry request ID, and the ledger lines within the settlement window. When those three cannot be lined up, the control is weaker than the policy document suggests.
A practical fix path
Teams that recover fastest assign a single owner for the idempotency contract across gateway and ledger, then add a reconciliation rule that flags same-amount, same-account posts within a defined interval even when keys differ. That rule is imperfect — and that honesty belongs in the control description.