Every request/response pattern this domain has covered quietly assumes the work finishes fast enough for the caller to just wait. Generating a large report, processing a video, or training a model doesn't fit that assumption at all — and forcing it to fit produces exactly the kind of timeout errors that make an API feel broken.
3 min read
A request that takes 30 seconds, 5 minutes, or an hour to complete can't simply hold the HTTP connection open until it's done — client timeouts, proxy timeouts, and load balancer timeouts all impose real limits, and a connection held open that long is also a real resource cost on the server regardless. An API needs a different pattern entirely for operations that genuinely don't finish quickly.
Instead of blocking until the work finishes, the server immediately
responds 202 Accepted (not 200 OK — a deliberate signal that the
request was accepted but not yet completed) along with a Location header
or resource ID pointing to a status endpoint. The client then checks that
endpoint — by polling, or via a webhook (lesson 10) when the operation
completes — instead of waiting on the original connection at all.
A well-designed status endpoint returns enough for a client to build a
reasonable experience around waiting: a status field (queued,
processing, complete, failed), and once complete, either the result
directly or a link to fetch it. Modeling this as an actual, addressable
resource (GET /reports/42) rather than an ephemeral in-memory flag means
it survives a client disconnecting and reconnecting later, or a completely
different client (a different device, a different tab) checking the same
operation's status independently.
Polling is simpler to implement on the client side and needs no publicly reachable endpoint from the client — a reasonable default, especially for a browser client that often can't receive inbound webhooks at all. Webhooks (lesson 10) avoid the wasted "still processing" checks and deliver the result with less delay, at the cost of the complexity a webhook consumer has to handle (signature verification, idempotent processing, handling missed deliveries). Many real APIs offer both: a status endpoint for polling, and an optional webhook for consumers set up to receive one — neither is strictly better, they fit different kinds of consumers.
Submitting the same long-running job request twice — a client retrying
after a network blip, unsure whether the first request actually landed —
should not start the expensive operation twice. This is System Design's
idempotency lesson applied directly to job submission: an idempotency key
on the initial POST lets the server recognize a retried submission and
return the existing job's status instead of starting a duplicate one.
This pattern is really a synthesis of nearly everything covered across
this domain: a resource-oriented status endpoint (lesson 1), a
deliberately-chosen status code (202, distinct from 200), consistent
error shapes if the job fails (lesson 8), an optional webhook for
completion (lesson 10), and idempotent submission (System Design). None of
these pieces is new — designing for operations that don't finish instantly
is what happens when all of them are combined to solve one real, common
problem that a naive request/response API design simply can't handle
correctly.
Check your understanding
A quick comprehension check — not tracked, not graded, just for you.
1. Why can't a long-running operation simply keep the client's HTTP connection open until the work finishes?
2. Why does the standard long-running-operation pattern return 202 Accepted instead of 200 OK for the initial request?
3. What role does an idempotency key play when submitting a long-running job?
API Design