{
 "slug": "always-allow-most-dangerous-button-enterprise-ai",
 "topic_id": "TOPIC-019",
 "cluster": "Agentic Commerce & MCP Tool-Call Gating",
 "tier": "Tier A",
 "title": "Why 'always allow' is the most dangerous button in enterprise AI software",
 "summary": "Approval fatigue converts a human-in-the-loop system into an unmonitored autonomous executor, and the conversion happens through a button the user was given specifically to escape the prompts.",
 "lede": "Every agent product ships a checkbox that says some version of <em>do not ask me again</em>. It exists because the prompts are unbearable. It also silently deletes the only control the architecture had.",
 "date": "2025-12-26",
 "category": "Developer",
 "author_id": "elias-vanterpool-osei",
 "tags": [
  "approval fatigue",
  "human in the loop",
  "agent UX",
  "always allow",
  "friction design",
  "AI safety"
 ],
 "image_title": "Always Allow Dangerous Button",
 "schema": "Article",
 "key_takeaways": [
  "Prompt volume, not user carelessness, produces the click. A system that asks about everything trains people to dismiss everything.",
  "The friction ladder assigns interaction cost by consequence: silent for reads, visible for reversible writes, signed for irreversible effects.",
  "If a persistent grant must exist, bound it — scope, ceiling, expiry — rather than making it permanent and unconditional."
 ],
 "body": [
  {
   "type": "h2",
   "text": "Watching someone use it"
  },
  {
   "type": "diagram",
   "kind": "chain",
   "alt": "How approval fatigue becomes autonomy",
   "caption": "Each step is a rational response to the previous one.",
   "nodes": [
    {
     "label": "Prompts fire constantly",
     "sub": "per tool call",
     "note": "unusable"
    },
    {
     "label": "User clicks Always allow",
     "sub": "rationally",
     "note": "the escape",
     "bad": true
    },
    {
     "label": "System becomes autonomous",
     "sub": "silently",
     "note": "no announcement",
     "bad": true
    },
    {
     "label": "Record shows approval",
     "sub": "indistinguishable",
     "note": "",
     "bad": true
    }
   ]
  },
  {
   "type": "p",
   "html": "The developer had been at it for forty minutes. The agent was refactoring a service, and every file write produced a dialog: allow this tool to write to this path?"
  },
  {
   "type": "p",
   "html": "She read the first four. By the twelfth she was clicking through them while looking at the terminal. At the nineteenth she found the checkbox, ticked it, and did not see another dialog that afternoon."
  },
  {
   "type": "p",
   "html": "She is a careful engineer. Nothing about her judgement failed. The interface presented her with a decision nineteen times in forty minutes and gave her a mechanism to stop, so she used it."
  },
  {
   "type": "h2",
   "text": "What the click actually does"
  },
  {
   "type": "p",
   "html": "The semantics vary and are rarely stated clearly. Depending on the product, one click can mean any of:"
  },
  {
   "type": "table",
   "head": [
    "Apparent meaning",
    "Common actual meaning"
   ],
   "rows": [
    [
     "Allow this action again",
     "Allow this tool, any arguments, this session"
    ],
    [
     "Allow this tool",
     "Allow this server, all its tools, indefinitely"
    ],
    [
     "Allow for now",
     "Persist across restarts, stored in config"
    ],
    [
     "Allow in this project",
     "Allow globally, because scoping was not implemented"
    ]
   ]
  },
  {
   "type": "p",
   "html": "A user cannot distinguish these from the dialog. The grant they think they are giving and the grant they are giving differ, and the difference is invisible until something goes wrong."
  },
  {
   "type": "h2",
   "text": "The fatigue is arithmetic, not character"
  },
  {
   "type": "p",
   "html": "Approval fatigue is well documented in security contexts — push notification fatigue, certificate warning fatigue, alert fatigue in operations centres. The pattern is consistent: as prompt frequency rises, considered response rate falls, and it falls faster than linearly."
  },
  {
   "type": "p",
   "html": "An agent performing a coding task may execute dozens of tool calls per minute. A system that prompts on each of them is not implementing human oversight; it is implementing a rate limiter that the human will remove."
  },
  {
   "type": "h2",
   "text": "The friction ladder"
  },
  {
   "type": "p",
   "html": "The design answer is to stop treating all tool calls as equivalent. Four rungs, and most calls belong on the first."
  },
  {
   "type": "table",
   "caption": "Interaction cost should track consequence. The great majority of tool calls belong on rung 1.",
   "head": [
    "Rung",
    "Action class",
    "Interaction",
    "Example"
   ],
   "rows": [
    [
     "1",
     "Read-only, no side effects",
     "<strong style=\"font-weight:600\">Silent</strong>",
     "Search files, read a record, query a schema"
    ],
    [
     "2",
     "Reversible write, local scope",
     "Visible, not blocking",
     "Edit a file in a working tree, draft a document"
    ],
    [
     "3",
     "Reversible write, shared scope",
     "Blocking confirm",
     "Commit to a branch, update a shared record"
    ],
    [
     "4",
     "Irreversible or external effect",
     "<strong style=\"font-weight:600\">Signed</strong>",
     "Payment, deletion, send external message, production change"
    ]
   ]
  },
  {
   "type": "p",
   "html": "The critical design move is rung 1. A system that does not prompt on reads reduces prompt volume by an order of magnitude, which restores the meaning of the prompts that remain."
  },
  {
   "type": "h2",
   "text": "Classifying reliably"
  },
  {
   "type": "p",
   "html": "The ladder only works if classification is trustworthy, and it cannot be inferred from a tool's name or description — both are attacker-controlled when the tool server is third-party."
  },
  {
   "type": "p",
   "html": "Classify at the registry level, by review, and bind the classification into the signed registry entry. A tool server that changes its schema after approval no longer matches, and the runtime can detect that."
  },
  {
   "type": "h2",
   "text": "If a persistent grant must exist"
  },
  {
   "type": "p",
   "html": "Sometimes it genuinely must — a long-running agent, an overnight batch, a workflow nobody will be present for. The answer is not to forbid it but to bound it."
  },
  {
   "type": "code",
   "text": "grant:\n  tool:      write_file\n  scope:     path_prefix = /workspace/service-a/\n  excludes:  **/*.env, **/secrets/**\n  max_calls: 500\n  notAfter:  2026-06-25T18:00:00Z\n  signed_by: [human credential]"
  },
  {
   "type": "p",
   "html": "Four bounds — scope, exclusions, count, expiry — and a signature. The user has still escaped the prompts, and the escape has edges."
  },
  {
   "type": "h2",
   "text": "What to measure in your own product"
  },
  {
   "type": "ol",
   "items": [
    "Prompts per user per session. If the median exceeds single digits, users will find the escape hatch.",
    "Time-to-decision distribution. A median under two seconds means the prompts are being dismissed rather than read.",
    "Persistent grant rate: share of users who have enabled one, and how broad it is.",
    "Grant breadth: for each persistent grant, what could be done under it. Most products have never computed this."
   ]
  },
  {
   "type": "p",
   "html": "That fourth number is usually the one that changes the roadmap, because it reveals that a single click three weeks ago authorises everything the agent has done since."
  },
  {
   "type": "h2",
   "text": "Why the button exists and what to do instead"
  },
  {
   "type": "table",
   "caption": "The design problem, honestly",
   "head": [
    "Cause",
    "Consequence"
   ],
   "rows": [
    [
     "Prompts fire per tool call",
     "Dozens per task; slower than doing it manually"
    ],
    [
     "Prompts describe the command",
     "Not the effect — nothing to react to"
    ],
    [
     "Approval is a single binary",
     "No scope, no bound, no expiry"
    ],
    [
     "<strong style=\"font-weight:600\">Always-allow is the only escape offered</strong>",
     "<strong style=\"font-weight:600\">Users take it, rationally</strong>"
    ]
   ]
  },
  {
   "type": "p",
   "html": "The fix is not to remove the button but to make the prompts rare enough that nobody wants it: gate the irreversible subset, render the effect, and offer a bounded scope grant instead of an unbounded one."
  },
  {
   "type": "h2",
   "text": "Objections and honest limits"
  },
  {
   "type": "p",
   "html": "<strong style=\"font-weight:600\">“Users should just be careful.”</strong> They are being careful about the fortieth prompt in a session, which is a different cognitive task from the first. Designing around that expectation is how the button got clicked."
  },
  {
   "type": "p",
   "html": "<strong style=\"font-weight:600\">“Record which actions were auto-approved.”</strong> Worth doing and it is telemetry rather than control. It tells you afterwards that nobody looked; it does not stop the action."
  }
 ],
 "faq": [
  {
   "q": "Should the button be removed?",
   "a": "No. Removing it without reducing prompt volume produces users who abandon the product or route around it. Fix the volume first, then bound the grant."
  },
  {
   "q": "Why not prompt on everything to be safe?",
   "a": "Because it is not safe. High prompt volume produces reflexive dismissal, which is worse than a well-designed ladder with genuine attention on rung 4."
  },
  {
   "q": "How do we classify third-party tools?",
   "a": "By review at registry level, bound into a signed registry entry with a schema hash, so a post-approval change is detectable."
  },
  {
   "q": "What about long-running autonomous workflows?",
   "a": "A bounded persistent grant with scope, exclusions, a call ceiling and an expiry, signed by a human. The human signs the envelope rather than each action."
  },
  {
   "q": "Why do users click always-allow?",
   "a": "Because per-command confirmation is slower than doing the work manually. It is a rational response to a badly posed choice."
  },
  {
   "q": "What should the prompt show?",
   "a": "The effect — rows affected, recoverability, environment — not the command string."
  }
 ],
 "sources": [
  {
   "t": "Published research on alert, certificate and multi-factor approval fatigue."
  },
  {
   "t": "Model Context Protocol specification",
   "u": "https://modelcontextprotocol.io/specification"
  },
  {
   "t": "2 CFR Part 200 — Uniform Administrative Requirements for Federal Awards",
   "u": "https://www.ecfr.gov/current/title-2/subtitle-A/chapter-II/part-200"
  },
  {
   "t": "Reported incidents involving broadly scoped agent tool approvals."
  },
  {
   "t": "Anthropic — Claude Code security and permissions",
   "u": "https://docs.claude.com/en/docs/claude-code/security"
  },
  {
   "t": "OWASP Top 10 for Large Language Model Applications",
   "u": "https://owasp.org/www-project-top-10-for-large-language-model-applications/"
  }
 ],
 "related": [
  {
   "slug": "intelligence-cannot-mint-permission-mcp-security",
   "title": "Intelligence cannot mint permission",
   "category": "Developer"
  },
  {
   "slug": "cursor-mcpoison-cve-2025-54135-scope-blind-tool",
   "title": "Scope-blind tool approvals",
   "category": "Developer"
  },
  {
   "slug": "signed-tool-registry",
   "title": "The enterprise MCP registry",
   "category": "Developer"
  },
  {
   "slug": "langgraph-interrupts-human-loop-hitl-database-pause-states",
   "title": "Why database pause states fail as human-in-the-loop evidence",
   "category": "Developer"
  }
 ],
 "image": "https://cdn.twc.sh/images/igcache/Always%20Allow%20Dangerous%20Button/1500_900/blog.jpg",
 "wordcount": 982,
 "url": "/blog/always-allow-most-dangerous-button-enterprise-ai.html",
 "reading_time": "4 min read",
 "seo_title": "Why 'always allow' is enterprise AI's worst button",
 "meta_description": "Approval fatigue turns a human-in-the-loop system into an unmonitored executor, through a button the user was given specifically to click.",
 "hub": {
  "slug": "topics/agent-tool-call-gating",
  "title": "Agent tool-call gating"
 },
 "answer": "It converts a human-in-the-loop system into an unmonitored autonomous executor, through a button the user was given specifically to escape the prompts. The conversion is invisible afterwards, because nothing in the record distinguishes an approved action from an auto-approved one.",
 "answer_q": "What does 'always allow' actually do?",
 "glossary": [
  {
   "term": "Approval fatigue",
   "def": "The degradation of attention caused by frequent prompts, converting review into reflex."
  },
  {
   "term": "Bounded grant",
   "def": "A scoped, expiring authorisation covering a class of actions — the alternative to an unbounded always-allow."
  },
  {
   "term": "Endpoint enforcement",
   "def": "Gating at the service that performs the effect rather than in the client the user configures."
  }
 ],
 "checklist": {
  "title": "Making always-allow unnecessary",
  "id": "unnecessary",
  "desc": "Four steps.",
  "steps": [
   {
    "name": "Gate only irreversible effects.",
    "text": "So the prompt is rare."
   },
   {
    "name": "Render the effect, not the command.",
    "text": "Row counts, recoverability, environment."
   },
   {
    "name": "Offer a bounded grant instead of always.",
    "text": "Scope, ceiling and expiry, signed once."
   },
   {
    "name": "Enforce at the endpoint.",
    "text": "So a client setting cannot disable the control."
   }
  ]
 },
 "cta": {
  "title": "Where this fits in Manav",
  "html": "Manav binds a named human to an agent's consequential actions through a signed delegation with scope and expiry, and a per-action receipt where the effect is irreversible.",
  "href": "../docs.html",
  "label": "See delegation chains"
 }
}