{
"slug": "unintended-consequence-phishing-simulation-training-click-rates",
"topic_id": "TOPIC-069",
"cluster": "Cyber Insurance & Risk Transfer",
"tier": "Tier B",
"title": "A 1.2% click rate and a seven-figure wire out the door",
"summary": "Phishing simulation measures whether people click suspicious links. Executive impersonation does not involve a link, and the metric does not extend to it.",
"lede": "The security awareness programme is working by its own measure. The failure rate is low, completion is high, and the organisation still wires money to a criminal because a voice on a call sounded like the CFO.",
"date": "2024-08-31",
"category": "Vertical",
"author_id": "desmond-okafor-hale",
"tags": [
"security awareness",
"phishing simulation",
"metrics",
"deepfake",
"training",
"budget"
],
"image_title": "Click Rate Versus Wire Fraud",
"schema": "Article",
"key_takeaways": [
"Click rate measures one behaviour against one attack format. It does not generalise to voice, video or authority-based pressure.",
"The employees targeted in high-value fraud are often outside the population the simulation represents.",
"Training and deterministic controls are not substitutes; the budget question is whether the ratio between them reflects where losses occur."
],
"body": [
{
"type": "h2",
"text": "What the metric measures"
},
{
"type": "diagram",
"kind": "compare",
"alt": "What the metric covers and where the losses are",
"caption": [],
"nodes": "The measured behaviour and the losing attacks do not overlap.",
"left": {
"title": "Click rate predicts",
"items": [
"Bulk credential phishing",
"Malicious attachments, partly",
"Email artefact inspection",
"Commodity volume attacks"
]
},
"right": {
"title": "It does not predict",
"items": [
"Voice impersonation",
"Synthetic video calls",
"Compromised genuine threads",
"Vendor invoices with changed details"
]
}
},
{
"type": "p",
"html": "A simulation sends an email with a link or attachment and records who interacts. The click rate is the proportion who did."
},
{
"type": "p",
"html": "This is a real measurement of a real behaviour, and driving it down has genuine value against commodity phishing, which remains high-volume. The problem is generalisation."
},
{
"type": "table",
"head": [
"Attack",
"Does click rate predict resistance?"
],
"rows": [
[
"Bulk credential phishing email",
"Yes — this is what it measures"
],
[
"Malicious attachment",
"Partly"
],
[
"Voice call impersonating an executive",
"No"
],
[
"Video call with synthetic participants",
"No"
],
[
"Compromised genuine account in a real thread",
"No — there is nothing to spot"
],
[
"Vendor invoice with changed bank details",
"No"
]
]
},
{
"type": "p",
"html": "The last three are where the large losses are, and none of them involves the behaviour being measured."
},
{
"type": "h2",
"text": "Why the training does not transfer"
},
{
"type": "p",
"html": "Simulation training teaches pattern recognition on email artefacts: sender mismatch, urgency language, hovering over links, domain inspection."
},
{
"type": "p",
"html": "An impersonation call has none of those artefacts. The number may be spoofed, the voice is familiar, the request is plausible, and the person is senior. The trained behaviour — examine the message for signs of forgery — has nothing to operate on."
},
{
"type": "p",
"html": "There is also a worse effect. An employee who has passed every simulation may be more confident, not less, that they can tell the difference."
},
{
"type": "h2",
"text": "The population mismatch"
},
{
"type": "p",
"html": "Simulations are typically sent broadly. High-value fraud targets a narrow set: treasury, accounts payable, executive assistants, finance leadership."
},
{
"type": "p",
"html": "An organisation-wide 1.2% failure rate says little about how a specific AP clerk responds to a call from someone claiming to be the CFO, at 4:45pm on a Friday, about a payment that must go today."
},
{
"type": "h2",
"text": "What actually helps that clerk"
},
{
"type": "p",
"html": "Not better judgement under pressure. A rule they can apply without judgement."
},
{
"type": "code",
"text": "# Training approach\n \"Be alert to urgent payment requests from executives.\"\n → requires the employee to assess authenticity in the moment,\n against a professional who chose the moment.\n\n# Control approach\n Payments above threshold, or with changed beneficiary details,\n require a signed authorisation from the approver's enrolled\n device. No signature → no payment.\n → the employee's task is to check for a receipt, not to judge\n whether a caller is genuine."
},
{
"type": "p",
"html": "The second gives the employee something to say: the payment needs an authorisation and I cannot process it without one. That is a defensible position under pressure in a way that \"I was not sure it was really you\" is not."
},
{
"type": "h2",
"text": "The budget question, framed fairly"
},
{
"type": "p",
"html": "This is not an argument to stop awareness training. It reduces commodity phishing exposure, it is often required by insurers and frameworks, and it is inexpensive per head."
},
{
"type": "p",
"html": "It is an argument about proportion. If the awareness budget is large and the deterministic control budget for the same risk is zero, the allocation does not reflect where the losses are."
},
{
"type": "table",
"head": [
"Risk",
"Best addressed by"
],
"rows": [
[
"Bulk credential phishing",
"Awareness plus phishing-resistant authentication"
],
[
"Malware delivery",
"Endpoint controls plus awareness"
],
[
"Executive impersonation for payment",
"Deterministic control on the payment"
],
[
"Vendor bank detail change",
"Deterministic control on the change"
],
[
"Compromised thread continuation",
"Deterministic control on the action"
]
]
},
{
"type": "h2",
"text": "A better metric to report"
},
{
"type": "p",
"html": "Click rate is reported because it is easy to produce, not because it predicts loss. Two figures are more informative and only slightly harder."
},
{
"type": "ol",
"items": [
"Control coverage: what proportion of payments above the threshold carried an authorisation, over the period.",
"Exception rate and shape: where the control was not applied, and why."
]
},
{
"type": "p",
"html": "Both are directly connected to whether a loss can occur, both are auditable, and neither depends on how an employee felt about an email."
},
{
"type": "h2",
"text": "The population mismatch"
},
{
"type": "p",
"html": "Simulations are sent broadly; high-value fraud targets treasury, accounts payable, executive assistants and finance leadership. An organisation-wide 1.2% failure rate says little about how one AP clerk responds to a call from someone claiming to be the CFO at 4:45pm on a Friday."
},
{
"type": "table",
"caption": "A better pair of metrics",
"head": [
"Metric",
"Why it predicts loss"
],
"rows": [
[
"Control coverage on payments above threshold",
"Directly connected to whether a loss can occur"
],
[
"Exception rate and shape",
"Shows where the residual risk is"
]
]
},
{
"type": "h2",
"text": "Objections and honest limits"
},
{
"type": "p",
"html": "“Awareness training is required by our insurer.” It is, frequently, and it should continue. The argument is about proportion — a large awareness budget alongside zero deterministic control budget for the same risk."
},
{
"type": "p",
"html": "“Training makes people more careful generally.” Possibly, and it can also make them more confident that they can tell the difference — against an attack format the simulation never covered."
}
],
"faq": [
{
"q": "Should we stop phishing simulations?",
"a": "No. They address commodity phishing, are often required by insurers and frameworks, and are inexpensive. The issue is whether budget proportion matches where losses occur."
},
{
"q": "Why doesn't the training transfer to voice attacks?",
"a": "It teaches pattern recognition on email artefacts. A call has none of them, so the trained behaviour has nothing to operate on."
},
{
"q": "Could training make things worse?",
"a": "Possibly at the margin. An employee who passes every simulation may be more confident they can tell the difference, against an attack format the simulation never covered."
},
{
"q": "What metric is better than click rate?",
"a": "Control coverage on payments above threshold, and the exception rate with its reasons. Both connect directly to whether a loss can occur."
},
{
"q": "Should phishing simulations stop?",
"a": "No. They address commodity phishing, are often required, and are inexpensive. The issue is budget proportion relative to where losses occur."
},
{
"q": "Why doesn't the training transfer?",
"a": "It teaches inspection of email artefacts. A phone call has none, so there is nothing for the trained behaviour to act on."
},
{
"q": "What should be reported instead?",
"a": "Control coverage on payments above threshold, and the exception rate with reasons."
}
],
"sources": [
{
"t": "NIST — human-centered cybersecurity research on phishing",
"u": "https://csrc.nist.gov/projects/human-centered-cybersecurity/research-areas/phishing"
},
{
"t": "FBI IC3 2025 Internet Crime Report",
"u": "https://www.ic3.gov/AnnualReport/Reports/2025_IC3Report.pdf"
},
{
"t": "CISA — known exploited vulnerabilities and incident reporting",
"u": "https://www.cisa.gov/known-exploited-vulnerabilities-catalog"
},
{
"t": "NAIC — cyber risk resources",
"u": "https://content.naic.org/cipr-topics/cyber-risk"
}
],
"related": [
{
"slug": "mfa-questionnaires-fail-inside-silent-cyber-liability",
"title": "Signing the MFA question",
"category": "Vertical"
},
{
"slug": "social-engineering-sub-limit-trap-10m-cyber",
"title": "The social engineering sub-limit",
"category": "Vertical"
},
{
"slug": "account-masking-ap-systems-directly-enables-wire",
"title": "Account masking in AP systems",
"category": "Vertical"
}
],
"image": "https://cdn.twc.sh/images/igcache/Click%20Rate%20Versus%20Wire%20Fraud/1200_630/blog.jpg",
"wordcount": 933,
"url": "/blog/unintended-consequence-phishing-simulation-training-click-rates.html",
"reading_time": "4 min read",
"hub": {
"slug": "topics/cyber-insurance",
"title": "Cyber insurance and risk transfer"
},
"answer": "No. Click rate measures one behaviour against one attack format — an email with a link. Voice and video impersonation involve no link, no sender to inspect and no domain to check, so the trained behaviour has nothing to operate on.",
"answer_q": "Does a low phishing click rate predict resistance to executive fraud?",
"glossary": [
{
"term": "Click rate",
"def": "The proportion of simulated phishing recipients who interact — a real measurement of one behaviour."
},
{
"term": "Control coverage",
"def": "The proportion of in-scope transactions that carried a deterministic control."
},
{
"term": "Format transfer",
"def": "Whether a trained behaviour generalises to a different attack format. Here, it does not."
}
],
"checklist": {
"title": "Reporting something that predicts loss",
"id": "metrics",
"desc": "Four steps.",
"steps": [
{
"name": "Keep the simulations.",
"text": "They address commodity phishing and are cheap per head."
},
{
"name": "Stop reporting click rate as the security metric.",
"text": "It measures one format."
},
{
"name": "Report control coverage on payments above threshold.",
"text": "A percentage over twelve months."
},
{
"name": "Report the exception shape.",
"text": "Where the control was not applied, and why."
}
]
},
"cta": {
"title": "Where this fits in Manav",
"html": "Manav produces the artefact underwriting, claims and forensics all lack: a per-action receipt verifiable without the insured's cooperation, and a measurable coverage rate.",
"href": "../docs.html",
"label": "See the evidence"
}
}