A maturity model for human oversight of automated decisions
Most organisations describe their oversight as a process and cannot say what artefact it produces. A maturity model that asks about artefacts rather than intentions puts them somewhere specific.
How do you assess AI oversight maturity honestly?
By the artefact, not the policy. Pick three automated decisions, find the oversight point, ask what record exists, then ask who that record would convince: you, your auditor, a regulator, or an opposing party. The answer is your level, and it is usually lower than the policy suggests.
- Each level is defined by the artefact the oversight produces, not by the policy describing it.
- Most enterprises sit at level two: oversight exists in the interface and produces an application log.
- Level five is not a universal target; the right level depends on the consequence of the decisions being overseen.
Part of Research, datasets and methods
The five levels
| Level | Oversight mechanism | Artefact produced |
|---|---|---|
| 1 — Instructional | Instructions in a prompt or a policy document | None |
| 2 — Interface | A confirmation step in the application | An application log entry |
| 3 — Attributed | Confirmation tied to an authenticated individual | A log entry naming a person |
| 4 — Bound | Confirmation bound to the rendered content, signed | A verifiable receipt with a content digest |
| 5 — Independently verifiable | As level four, verifiable by third parties offline | A receipt anyone can check without you |
The progression is about who can be convinced. At level one, nobody. At level three, an internal auditor who trusts your systems. At level five, a regulator, a court or a counterparty who does not.
Level 1 — Instructional
The constraint is expressed as an instruction: a system prompt telling a model to seek confirmation, or a policy telling staff to review outputs.
There is no enforcement and no artefact. This is where most agent deployments begin, and it is not a control by any definition.
Level 2 — Interface
The application requires a confirmation step. A user clicks, the system records that it happened.
This is genuine — the action does not proceed without the click — and it is the modal state of enterprise deployments. Its weaknesses: the record is an application assertion, it does not identify a person beyond a session, and it does not capture what was displayed.
Level 3 — Attributed
The confirmation is tied to an authenticated individual rather than a session, and the record names them.
This closes the provenance question for internal purposes. It remains an application-generated record, so it establishes attribution to anyone who trusts the application.
Level 4 — Bound
The confirmation carries a signature over a digest of what was rendered. Two properties appear that did not exist below.
- The approval covers a specific object, so the object cannot change afterwards without detection
- What the person was shown is part of the record, so the interpretation question is answerable
This is the level at which the record answers a regulator's question about what a natural person actually reviewed.
Level 5 — Independently verifiable
The receipt verifies against published keys with no call to your systems, and remains verifiable after the vendor relationship or the system ends.
The difference from level four is who can be convinced. A level-four record persuades someone who trusts your infrastructure; a level-five record persuades someone who does not, which is the position an examiner or an opposing party occupies.
Choosing a target
Not everything should be at level five. The cost rises and the friction rises with it.
| Decision consequence | Appropriate level |
|---|---|
| Reversible, low value, high frequency | 1–2 |
| Internal, moderate consequence | 3 |
| Regulated decisions affecting individuals | 4 |
| Irreversible financial effects | 4–5 |
| Decisions likely to be litigated or examined | 5 |
An organisation at level two everywhere has a gap at the bottom rows. One at level five everywhere has built friction it did not need and will find its approvals becoming reflexive.
How to assess yourself
- Pick three consequential decisions your systems make automatically.
- For each, find the oversight point and ask what artefact exists.
- Ask who that artefact would convince — you, your auditor, a regulator, a court.
- That answer is your level, and it is usually lower than the policy suggests.
Step three is the honest test. Most organisations discover they are at level two while describing themselves at level four, because the policy describes intent and the artefact describes reality.
Choosing a target level
Not everything should reach level five. Cost and friction rise together, and an organisation at level five everywhere has built friction it did not need — which produces reflexive approval, the failure the model exists to prevent.
| Decision consequence | Level |
|---|---|
| Reversible, low value, high frequency | 1–2 |
| Internal, moderate consequence | 3 |
| Regulated decisions affecting individuals | 4 |
| Irreversible financial effects | 4–5 |
| Likely to be litigated or examined | 5 |
An organisation at level two everywhere has a gap at the bottom rows. That gap is the finding, and it is usually invisible until someone asks for a specific instance.
Objections and honest limits
“We are at level four.” Test it. Ask for the record of a specific override from three months ago and see whether it contains what was displayed. Most organisations describing themselves at four are at two, because policy describes intent and the artefact describes reality.
“This is another maturity model.” It is, with one difference: each level is defined by an artefact you can produce or cannot, rather than by a process description. That makes self-assessment falsifiable.
The scoring rubric
A level is only useful if two people assessing the same system land in the same place, so the model needs a rubric rather than descriptions. Score each dimension from 1 to 5 against the statements below, take the lowest rather than the average, and record the evidence you used.
| Dimension | 1 | 3 | 5 |
|---|---|---|---|
| Identity | A shared or service account acted | A named user account acted | A key bound to a person at enrolment signed |
| Rendering | The person saw a request or a prompt | The person saw a summary | The person saw the effect that will occur |
| Binding | Nothing links approval to action | A foreign key links them | The effect hash is inside the signature |
| Evidence | A screenshot or a chat message | An application log | A signed record, tamper-evident |
| Verifiability | Only we can interpret it | A third party can read it with our help | Anyone can verify it offline |
| Revocation | Undocumented | Documented, untested | Enforced at the endpoint, tested, fails closed |
Taking the lowest rather than the average is deliberate. A control with a perfect signature over a rendering nobody could understand is not a strong control with one weak dimension; it is a weak control.
A worked self-assessment
| Dimension | Score | Evidence used |
|---|---|---|
| Identity | 4 | SSO with a phishing-resistant second factor |
| Rendering | 2 | Approver sees payee and amount, not the full batch effect |
| Binding | 2 | Approval row references a payment id |
| Evidence | 3 | Application audit log, retained 400 days |
| Verifiability | 1 | Nobody outside the company can check it |
| Revocation | 3 | Documented, never tested |
| Overall | 1 | Lowest dimension governs |
That organisation would describe itself as mature, and on four dimensions it is. The verifiability score is what an auditor, a regulator or an opposing party will actually test, and it is a one.
Most organisations should target four rather than five. Five is worth the cost where an external party will test the evidence — regulated decisions, high-value transactions, anything likely to be litigated. Below that, four is proportionate and honest.
Assessing yourself in an afternoon
- Pick three consequential automated decisions. Real ones, with real identifiers.
- Find the oversight point in each. Where a human can override, halt or confirm.
- Ask what artefact exists. Retrieve it. Do not describe it.
- Ask who it would convince. You, your auditor, a regulator, an opposing party. That is your level.
Terms used here
- Oversight point
- A place in a system where a human can confirm, override or halt an automated output.
- Bound approval
- An approval carrying a digest of the rendered content, so what the person saw is part of the record and the object cannot change afterwards.
- Independent verifiability
- The property that a party who does not trust your infrastructure can still check the record.
Frequently asked questions
What defines each level? The artefact the oversight produces and who that artefact would convince — not the policy describing the process.
Where do most enterprises sit? Level two. Oversight exists as a confirmation step in the interface and produces an application log entry.
Should everything reach level five? No. Cost and friction rise with the level. Reversible high-frequency decisions belong at one or two; litigation-prone ones at five.
How do you assess honestly? Pick three automated decisions, find the oversight point, ask what artefact exists and who it would convince. That answer is your level.
Why take the lowest score rather than the average? Because the dimensions are not independent strengths. A signature over an unintelligible rendering is a weak control, not a strong one with a gap.
Should everyone target level five? No. Four is proportionate for most decisions. Five earns its cost where an external party will test the evidence.
Where this fits in Manav
Manav is the mechanism for levels four and five: a signature over a digest of exactly what was rendered, countersigned by the issuer, verifiable against a published key by a party who does not trust your systems.