What is an idempotency key?
An idempotency key is a unique value you send with a request so the server can recognise a retry. If the same request arrives twice with the same key, the server performs the action once and returns the original result the second time. It is usually sent as an Idempotency-Key HTTP header, and it is how payment APIs, job queues and crawl APIs make retries safe.
The underlying idea is idempotency: an operation is idempotent when doing it several times has the same effect as doing it once. Reading a record is naturally idempotent. Charging a card or starting a job is not, and the key is what makes it behave as if it were.
Why idempotency keys exist
Networks fail in an awkward way. A request can reach the server and succeed while the response is lost on the way back. The client sees a timeout and cannot tell whether the action happened, so it retries. Without a key, that retry charges the card twice, places two orders, or starts a second crawl that fetches every page again.
With a key, the retry is harmless. The server looks the key up, sees it has already handled that request, and returns the stored response.
How it works
- The client generates a unique value per logical operation, typically a UUID.
- It sends the value in the
Idempotency-Keyheader with the request. - The server stores the key with the request body and the response.
- A retry with the same key and body gets the stored response. No new action runs.
- Keys expire after a period the API defines, after which the value can be reused.
Two edge cases every implementation handles: the same key with a different body (usually rejected, because it signals a client bug), and a retry that arrives while the first request is still processing (usually answered with a conflict, telling the client to wait).
Which HTTP methods need one
| Method | Idempotent by definition? | Needs a key? |
|---|---|---|
| GET, HEAD | Yes | No |
| PUT, DELETE | Yes | Rarely |
| POST | No | Yes, when it creates something or costs money |
| PATCH | Not necessarily | Sometimes |
Best practices
- One key per operation, not per attempt. Generate the key once, before the first try, and reuse it for every retry of that same operation.
- Use random values. UUIDs avoid collisions between clients. A timestamp alone is not unique enough.
- Store the key with your own job record, so a restarted process retries with the same key instead of inventing a new one.
- Combine it with backoff. The key makes the retry safe; exponential backoff makes it polite. The blog covers retry patterns for Python Requests and Axios.
Idempotency keys in web crawling
Starting a crawl is exactly the kind of request that should not run twice by accident. A crawl is a long-running job that fetches many pages, so a duplicate start doubles the work and the load on the target site.
Zenrows Crawl accepts an Idempotency-Key header when you start a crawl. A retry with the same key and the same body returns the same crawl and starts no second one, so a client can retry a timed-out start without thinking about it. The details are in the Crawl endpoint reference.
Frequently asked questions
What is idempotency in simple terms?
Doing something twice has the same result as doing it once. Pressing a lift button again does not call a second lift.
Is an idempotency key the same as a request ID?
No. A request ID identifies one attempt for logging and tracing. An idempotency key identifies one logical operation across all its attempts, and the server uses it to decide whether to act.
Should the client or the server generate the key?
The client. It has to exist before the first request is sent, so that a retry after a lost response can carry the same value.
How long are idempotency keys kept?
It depends on the API, commonly between a day and a few weeks. Check the provider's documentation before relying on a key for a long-delayed retry.
Go deeper on the blog
In the docs
Last updated: Oct 9, 2026