Prompts and policy files can guide an AI agent, but governance must constrain its effective capabilities across tools, files, subprocesses, identities, networks and release controls so that unintended paths cannot become unintended authority.
Questions this perspective explores
- Why are behavioural instructions necessary but insufficient as security boundaries for AI agents?
- How should enterprises evaluate an agent’s effective capability across direct and indirect paths?
- Where should deterministic enforcement and human judgment sit around probabilistic reasoning?
Instructions guide behaviour but do not bound authority
Organizations deploying AI agents often begin governance with instructions. A system prompt tells the agent not to reveal secrets, a repository policy describes prohibited files and a workflow asks for human confirmation before production changes. These measures are useful because they shape intended behaviour and communicate organizational expectations. They are not, however, reliable security boundaries. An instruction exists inside the agent’s decision process; a control constrains what can occur even when that process misunderstands the context, follows malicious input or finds an unexpected route to the same result.
The distinction is familiar in other domains. Enterprises do not protect financial systems by telling users to approve only appropriate payments while granting every user unrestricted transaction rights. They align permissions, separation of duties, transaction limits and audit evidence with the policy. AI governance requires the same discipline. A well-instructed agent with an overprivileged shell, broad repository access and ambient cloud credentials remains an overprivileged actor. Its apparent compliance during routine testing does not alter the authority available when conditions change.
Agents can reach an outcome through equivalent paths
Agentic systems are valuable because they can decompose objectives, choose tools and adapt when a path fails. Those same properties make instruction-only controls brittle. If direct access to a protected file is discouraged, an agent may invoke a build tool that reads it, start a subprocess that inherits its value, inspect generated output or query a service that materializes the same information. The agent need not intend to evade policy. It may simply pursue task completion through a path not represented in the instruction.
Governance must therefore consider functional equivalence. Preventing one command does not protect a resource if another tool, library, workflow edit or identity-assumption path creates the same effect. Static deny-lists struggle because modern engineering environments contain many composable interfaces. The durable approach is to constrain the underlying capability: which files can be read, which destinations can be reached, which identities can be assumed, which subprocesses receive sensitive variables, which repository areas can be changed and which release actions the current workload can authorize.
The governance object is effective capability
For executive purposes, an agent’s capability is the combination of model behaviour, tool interfaces, operating-system permissions, workload identity, network reach, available data and control over downstream automation. Reviewing the model or prompt alone examines only one component. An agent may lack direct production access yet retain the ability to modify a CI workflow that possesses it. It may be unable to open a secret store while receiving secret values in inherited environment variables. It may have read-only repository access but write commands into an issue or artifact later consumed by a privileged process.
Effective capability analysis follows these chains to the consequential outcome. It asks not merely whether the agent can call an API, but whether it can cause a protected state change directly or indirectly. This perspective also clarifies accountability. The model provider governs some behavioural properties, the platform team governs tools and execution, identity teams govern authorization, product teams govern repositories and release management governs promotion. No single layer can certify the overall boundary. Governance must establish ownership for the assembled system.
Deterministic enforcement should surround probabilistic reasoning
AI agents are probabilistic decision-makers operating inside deterministic infrastructure. Sound control architecture uses that distinction rather than attempting to eliminate it. The agent can reason broadly about code, propose changes and select among approved tools, while infrastructure enforces hard limits on accessible files, network destinations, credentials and production actions. The more consequential the effect, the less its permissibility should depend on the agent correctly interpreting a natural-language restriction.
These controls should be layered because each addresses a different failure mode. Sandboxed workspaces contain local effects; scoped tool contracts limit operations; workload identities limit external authority; secret brokers control sensitive materialization; protected branches and signed artifacts preserve integrity; dedicated promotion principals separate release decisions. Instructions remain important within this system. They improve efficiency, reduce unnecessary denied attempts and explain escalation paths. Their role is to guide the agent toward permitted behaviour, while deterministic mechanisms ensure that mistakes remain within an acceptable radius.
Governance must observe requests, decisions and effects
Conventional monitoring often records tool calls or model transcripts without connecting them to the resulting system change. A stronger evidence model relates the agent’s task, the capability requested, the policy decision, the identity exercised and the effect produced. If a change enters production through a later pipeline, provenance should retain the relationship between the agent-authored commit, the build identity, the promotion decision and the deployed artifact. This allows the enterprise to distinguish generation from authorization.
Care is required because observability can itself expose sensitive information. Raw prompts, command output and environment captures may contain credentials or regulated data. Evidence should therefore use structured, allow-listed fields and references whenever possible. Governance needs to know that a protected object was accessed under a particular policy and version; it rarely needs the object’s secret value in the audit stream. The aim is accountable action without creating an uncontrolled archive of everything the agent observed.
Human approval is a control only when the decision is intelligible
Many early agent deployments compensate for uncertainty by inserting approval prompts. This can be appropriate for exceptional or high-impact actions, but approval is weak when the reviewer sees an opaque bundle of changes, cannot verify the exact target or assumes that earlier controls have already removed risk. Repeated prompts also produce fatigue and shift accountability to individuals without giving them adequate evidence. A click is not meaningful separation of duties if the same underlying identity can bypass it or if the approver cannot tell what authority will be exercised.
Human judgment is most valuable at clearly defined decision boundaries: accepting an exception, promoting an identified candidate, authorizing access to a restricted data class or approving a material change in agent capability. The system should present the scope, evidence and consequences of that decision, then execute through a distinct identity. Routine low-risk actions should be constrained and automated. This allocation preserves human attention for ambiguity and business consequence instead of using it as a substitute for missing architecture.
Capability changes require lifecycle governance
An agent’s effective authority changes when a tool is added, a plugin is updated, a repository permission expands, a network rule changes or a workflow begins to consume its output. These modifications may occur outside the team that owns the model. A one-time predeployment assessment will therefore become stale even if the prompt remains unchanged. The enterprise needs a capability inventory that can identify material changes and trigger proportionate review.
This does not require treating every tool description as a regulated model release. It requires classifying capabilities by consequence and dependency. A formatter that operates on an isolated workspace differs from a tool that can publish packages or assume a cloud role. Governance should focus on new paths to protected data, external communication, persistent change and production authority. Testing should include indirect and compositional routes, because the assembled capability often exceeds the apparent permissions of any single tool.
The CAIO should govern the system that makes autonomy possible
The central executive choice is whether AI governance will assess what agents are told or what agentic systems can actually cause. Instructions, policies and model evaluations remain necessary, but they cannot carry the full burden of control. Capability boundaries turn organizational intent into enforceable limits across tools, identities, files, networks and release paths. They also create the evidence needed to show that autonomy operated within those limits.
For the CAIO, this means building governance jointly with the CIO, CISO and operational owners rather than treating agent safety as a model-management specialty. The objective is not to constrain every agent to trivial work. It is to make broad reasoning compatible with narrow, attributable authority. An enterprise that achieves this can delegate more useful work with confidence. One that relies on instructions alone will eventually discover that eloquent policy cannot compensate for an architecture in which every path remains technically open.
