Skip to main content

AI security model

The principle

The assistant listing the capabilities and the gaps of the account asking Asked what it can do, the assistant reports the caller's own effective permissions — including what is missing. Here it names the absent codes and concludes the account is a steward who can clean, dedupe and merge, but cannot configure the schema or delete records.

An agent acts as the signed-in user. There is no service account, no broker credential, and no privileged identity the AI layer falls back on.

This is the whole security model, and it is deliberately boring. An agent's reach is exactly its operator's reach, so reasoning about what an agent can do reduces to reasoning about what that person can do — a question you already know how to answer.

What an agent cannot do

CannotBecause
See data its operator cannot seeEvery enforcement layer re-evaluates per request
Reconstruct a masked valueIt receives the mask; masking is applied when the response is generated
Escalate its permissionsIt holds no identity of its own to escalate
Act untraceablyEvery action is audited, attributed to the user
Mutate in one unconsidered stepMutations are gated behind an explicit preview-then-commit

Untrusted data

Master data contains free text supplied by other people — names, comments, descriptions, request bodies. Any of it can contain text that reads like an instruction.

The system prompt instructs the model to treat tool output as untrusted data and to fence it when surfacing it.

This is a model-side instruction, not a server-side control

Tool results are returned as raw API responses; the server does not currently wrap them in an untrusted-data envelope. So this mitigation depends on the model following instruction, which is exactly the assumption prompt injection attacks.

The load-bearing defence is the permission model, not the prompt. Even a fully subverted agent carries only the operating user's permissions, so the worst it can reach is what that person could have reached anyway — and every action it takes is audited under their identity.

Treat the fencing instruction as defence in depth. If your threat model requires server-side enforcement, raise it as a gap rather than assuming it is present.

Masked values

An agent receives masked values as masks and can only repeat them. It has nothing to reconstruct from.

Instructing it to unmask is not refused by policy — it is not possible. The value is not present in what the agent received.

Autonomy is not authorization

An autonomy mode governs whether a person is consulted before an action. It never governs what the underlying request is permitted to do.

An agent in the most permissive mode, operating for a read-only user, still cannot write. The request is refused exactly where any other unauthorized request is refused.

Mutations

Mutating operations require an explicit two-step: a preview showing what would change, then a commit. An agent cannot self-confirm — the confirmation is not something it can produce on its own.

Every exposed mutation is reversible, which is what makes the more permissive autonomy modes defensible.

Auditing

Agent actions land in the same append-only audit log as any other action, attributed to the user whose identity was used. There is no separate, less-traceable path.

Next


Last verified against commit f7daa483 (2026-08-03)