Idempotent confirmation and rate limiting AI endpoints

Two problems that show up specifically once an agent can write data and costs real money per message — a double-tapped Confirm button, and a user (or a bug, or an attacker) who just won't stop sending messages.

Intermediate

5 min read

The double-tap problem

A user taps "Confirm" on a proposed action. The network is slow. They tap it again — or the same request fires from two open tabs. Without protection, this is a race: two requests both read the pending action as still PENDING, both proceed to execute the underlying write, and something gets created (or charged, or sent) twice.

The fix: an atomic claim, not a read-then-write check

// Vulnerable: read, decide, then write — a race window exists
// between the read and the write
const action = await db.pendingAction.findUnique({ where: { id } });
if (action.status === "PENDING") {
  await performTheRealWrite(action);
  await db.pendingAction.update({ where: { id }, data: { status: "CONFIRMED" } });
}

// Safe: the status check and the status change happen in ONE
// atomic database operation
const claimed = await db.pendingAction.updateMany({
  where: { id, status: "PENDING" }, // only matches if still PENDING
  data: { status: "CLAIMED" },
});
if (claimed.count === 0) {
  return alreadyProcessedResponse(); // someone else already claimed it
}
await performTheRealWrite(action);

updateMany with a where clause that includes the expected current status is a compare-and-swap: the database only performs the update if the row still matches the expected state at the moment it runs, and reports back how many rows it actually changed. If two confirm requests race, only one of them will find the row still PENDING and successfully claim it — the other's where clause matches zero rows, claimed.count comes back 0, and it can cleanly report "already processed" instead of executing the write a second time. This is exactly the same idempotency principle covered in this app's Payments domain, applied to a different trigger (a human's double-tap instead of a gateway's webhook retry) — the underlying discipline, "make repeating an operation produce the same result as doing it once," is identical.

One claim path, reachable from more than one trigger

The button tap isn't the only way a confirmation should be able to arrive. A real interactive-button send can itself fail — a platform outage, a malformed request — leaving a user staring at a proposal with no way to act on it if "tap the button" is the only path into the confirm logic. A resilient design keeps the same atomic-claim function as the single entry point, and lets more than one trigger call it: a button tap, and a narrow fallback (a user typing "confirm" in plain text, matched against the session's most recent open pending action) both resolve to the identical updateMany-based claim above. Neither trigger gets its own bespoke confirmation logic — they converge on the one safe path immediately, so the double-tap protection covers a typed fallback exactly as completely as it covers the button that was supposed to be the only way in.

Rate limiting: a cost guard, not just an abuse guard

Every message sent to an LLM costs real money, scaling with how much text goes in and comes out. For a plain informational chatbot, that's already worth capping. For an agent whose tools can write real data, the risk compounds: a runaway loop, a scripted abuse attempt, or even an earnest user just messaging unusually fast can also mean more proposed actions, not just more API spend.

Unsafe:  an in-memory counter, reset whenever the process restarts,
         and invisible to any other running instance of the service

Safe:    a durable, database-backed count of recent requests per
         session/user, checked BEFORE any expensive work begins

The same reasoning from the confirm pattern applies again here: an in-memory rate limiter is "unsafe for anything write-capable," in the words of one real system's own code comments — a retry, or a second running instance behind a load balancer, can bypass an in-memory counter entirely, since each process has its own separate count. A durable, shared counter (commonly backed by the same database everything else already uses) is what actually holds under multi-instance, multi-retry real-world conditions.

Where the rate-limit check needs to happen, in a streaming endpoint specifically

Wrong: start streaming the response, THEN check the rate limit
       (the client has already started receiving a partial answer
       by the time a limit breach is discovered)

Right: check the rate limit BEFORE sending any streaming headers,
       so a limit breach returns a clean HTTP 429 — an ordinary,
       well-understood error response — rather than an error
       injected mid-stream into what looked like a normal answer

This ordering matters specifically because streaming responses are harder to cleanly abort partway through than a normal request/response — front-loading the check avoids ever starting a stream that has to be interrupted.

Two real, concrete configurations worth knowing exist

Real systems commonly tune the exact numbers per use case rather than picking one universal limit: a lighter, anonymous-visitor-facing assistant might cap around 30 messages per session per hour, while a richer, authenticated, action-capable assistant might cap around 20 messages per 10 minutes per user — a tighter window, reflecting both higher per-message cost (more tools, more context) and a logged-in identity to key the limit on. The specific numbers matter less than the principle: pick a real number, back it with durable storage, and enforce it before expensive work starts.