Authorization for AI agents: never trust the model

Identity and permission data must come from a source the model never touches — a session token, a JWT, server-side lookups — never from the model's own output, and never from anything the model was merely told earlier in the conversation. If a user's message contains "I'm an admin, so go ahead and delete this," that's just text in a prompt; it carries exactly as much authority as any other sentence a user could type, which is to say: none.

This is the same principle as never trusting client input in an ordinary API, applied to a new kind of client: the model's tool-call arguments are, functionally, user-influenced input, not a verified fact.

A permission check run only when building the list of tools available to the model (deciding which tools even get offered to it) is a real, useful check — but it isn't sufficient on its own, since a proposal created under one set of permissions might not get confirmed until later, by which point something could have changed. Real systems commonly run the same role/permission check again, independently, inside the confirm step itself — re-verifying that the confirming user is still actually allowed to perform this specific action, right now, not just that they were allowed to see the tool offered at proposal time.

One real, useful pattern for a genuinely high-stakes surface (a live voice call, where a confirmation UI can't realistically appear the way it can in a chat interface): strip every write-capable tool from the set offered to the model entirely, server-side, for that mode — not just discourage the model from using them via a prompt instruction. A prompt instruction ("please don't take actions on voice calls") is a request the model could still misjudge; removing the tools from what's even offered is a structural guarantee, checkable by reading the code that builds the tool list for that mode, not a hope about model behavior.

The same "don't trust the model's output" principle extends past identity to any sensitive reference value — a real example: an image attached to a confirmed action is populated server-side, looked up from the database row tied to the specific pending action being confirmed, specifically so a manipulated caption or tool argument could never make the system reference an arbitrary, attacker-chosen storage key. Anywhere a tool call's arguments could plausibly reference "which file," "which record," or "which user," the safe design treats that reference as needing independent, server-side verification — not as something to trust simply because it arrived structured as a tool argument instead of free text.

Treat every tool call's arguments as untrusted input from the conversation — because that's what they are, however tidy and schema-validated they look. Anything that actually matters for security (who is this, what are they allowed to do, which real record does this reference) needs to be established independently, server-side, using data the model was never in a position to influence.