Most institutions already know how to govern a model. A scorecard produces a number, and a person decides what happens next. An AI agent changes that arrangement, because it can carry out the next step itself. The governance has to be in place before the agent acts, not after.

A model advises. An agent does.

An AI agent is software that uses a language model to pursue a goal over several steps. It chooses what to do next, and it reaches into other systems through tools: it searches records, reads documents, drafts messages, updates a file or prepares a payment instruction.

Credit scorecards, fraud rules and pricing models already sit inside most governance frameworks. Agents put pressure on those frameworks in five places:

  • They carry out actions. The output is not a score for someone to weigh. It can be the change itself.
  • Their route is not fixed. Give an agent the same task twice and it may take different steps. Approving a procedure once does not mean it will be followed.
  • They take in unchecked text. Emails, uploads and web pages all reach the model, and any of them may contain words written to redirect it.
  • They borrow authority. Every tool call runs under some account. If that account is a staff member's login, the agent can do whatever that person can.
  • Their mistakes compound. A wrong fact early in a task becomes the starting point for every step after it. Unless the steps are logged, nobody sees where it went wrong.

How much is it allowed to do?

A useful way to frame any agent is a ladder of agency with five rungs. At the bottom, the agent only assists. At the top, it acts with no routine human review, which is rarely justified in a regulated institution.

A five-rung ladder of agency for AI agents: assist, recommend, act with approval, act within limits, and act freely, with autonomy and the need for controls rising at each rung. THE LADDER OF AGENCY 1 Assistfinds · summarises 2 Recommenda person decides 3 Act with approvaleach action signed off 4 Act within limitssamples · exceptions 5 Act freelyno routine review RATE EACH ACTION, NOT THE WHOLE AGENT, AND CHOOSE THE LOWEST RUNG THAT WORKS
Each step up the ladder hands the agent more of the decision. Moving up should be a recorded choice, not something that happens by drift.

Two habits make the ladder work in practice.

Rate the action, not the agent. One customer chat agent might answer routine questions on its own, from approved information, while only proposing anything that involves money. Those are two rungs, and each needs its own controls.

Watch the rung more than the model. The same model carries modest risk when it only recommends, and serious risk when it acts alone. What an agent may do, and what is at stake when it does, matter more than how capable the model underneath is. So the discipline is simple to state: use the least agency that does the job. If one rule or one form would do, use that instead.

Six places the risk comes from

It helps to ask where trouble could enter, rather than starting from a long list of threats.

  1. What it reads. Any text can carry instructions. A customer email that tells the system to refund an account in full is written for the model, not for a person.
  2. What it can do. Every tool is a power. A broad tool, or an agent running under a manager's login, turns a small slip into a large one.
  3. What it knows, and where that goes. Agents pass data to models, and those models are often hosted by a third party, sometimes in another country.
  4. How errors travel. In multi-step and multi-agent set-ups, one bad output becomes the next step's input.
  5. Who trusts it. People approve fluent, confident text too readily. This is automation bias, and attackers can exploit it too.
  6. What it is built from. Models, connectors and platforms change underneath you, often without notice, and any component can be compromised.

Security teams have a shared reference for these. The OWASP Top 10 for Agentic Applications for 2026, published in December 2025 by the OWASP Gen AI Security Project, names risks such as agent goal hijack, tool misuse, identity and privilege abuse, cascading failures and human-agent trust exploitation. A governance team does not need to master each one. It needs to know which apply to a given agent, and to make sure each has an owner.

The register and the identity

You cannot govern agents you cannot list. Many institutions keep a model inventory, and the US National Institute of Standards and Technology's voluntary AI Risk Management Framework expects one: its GOVERN 1.6 reads, "Mechanisms are in place to inventory AI systems and are resourced according to organizational risk priorities".

An agent, though, is more than a model. It is a bundle: a model, its standing instructions, its tools and their permissions, the identity those tools run under, the data it reads and any memory it keeps. Change any part and the behaviour changes, even if nobody touched the model. A new tool or a wider permission can lift an agent up the ladder without anyone deciding it should. A register that records the whole bundle makes those changes visible. A well-controlled spreadsheet is enough to begin with.

Identity deserves particular care. The working principles are plain:

  • One agent, one account, never a staff login and never shared with another system.
  • Least privilege: for each permission, name the approved action that needs it, or remove it.
  • Short-lived credentials, measured in minutes or hours, not days.
  • A tested way to stop it. Disabling an account may not end tokens already issued. A stop plan should also revoke active sessions and switch off the feature where the product allows.

Scope also has to be enforced by the system, not by the instructions. An agent told to look only at the current customer's account can be talked out of it by a well-crafted message. An agent that can only reach that account cannot.

When approval stops being oversight

Putting a person in the loop is not the same as having oversight. Approval only counts if the reviewer has the file, the time and the authority to say no.

One measure tells you a lot: the override rate, meaning how often people reject or change what the agent proposes. A rate that falls close to zero looks like success. It is more often a warning that people have stopped checking.

An approval step that is always clicked through is not a control. It is a signature on someone else's decision.

Questions for a risk committee

Before an agent goes live, or moves up a rung, five questions are worth settling:

  1. What rung is each action on, and who decided that it should be?
  2. What is the worst thing this agent could do in one day, and who would notice?
  3. Which identity do its tools run under, and what can that identity change?
  4. Where is the model hosted, and what does the contract say about our data?
  5. How do we stop it now, and when did we last test that?

In Kenya, the Central Bank's survey on AI in the banking sector, published in July 2025 with data as at 31 December 2024, found that half of respondents had adopted AI; of those, 59 percent had no AI policy and 68 percent were using generative AI tools. On 10 September 2026 the Central Bank published draft revised Risk Management Guidelines, with comments due by 7 November 2026. The draft names AI systems as a source of reputational risk and expects human oversight, model validation, bias testing, explainability and continuous monitoring, along with an inventory of all models. As at October 2026 it is still a draft, so check the final text. The Data Protection Act, 2019 already requires a data protection impact assessment before high-risk processing. The Artificial Intelligence Bill, 2026 was read a first time in the Senate in April 2026 and had not become law by September 2026.

For why the final call in a regulated decision should rest with rules and people rather than a model, see Why AI advises and rules decide.

Our course Governing AI Agents works through screening, the agent register, risk tiers, impact assessment, controls, buying, monitoring and board reporting in depth, with a template for each step.

Sources