A 1.2% click rate and a seven-figure wire out the door
The security awareness programme is working by its own measure. The failure rate is low, completion is high, and the organisation still wires money to a criminal because a voice on a call sounded like the CFO.
Does a low phishing click rate predict resistance to executive fraud?
No. Click rate measures one behaviour against one attack format — an email with a link. Voice and video impersonation involve no link, no sender to inspect and no domain to check, so the trained behaviour has nothing to operate on.
- Click rate measures one behaviour against one attack format. It does not generalise to voice, video or authority-based pressure.
- The employees targeted in high-value fraud are often outside the population the simulation represents.
- Training and deterministic controls are not substitutes; the budget question is whether the ratio between them reflects where losses occur.
Part of Cyber insurance and risk transfer
What the metric measures
A simulation sends an email with a link or attachment and records who interacts. The click rate is the proportion who did.
This is a real measurement of a real behaviour, and driving it down has genuine value against commodity phishing, which remains high-volume. The problem is generalisation.
| Attack | Does click rate predict resistance? |
|---|---|
| Bulk credential phishing email | Yes — this is what it measures |
| Malicious attachment | Partly |
| Voice call impersonating an executive | No |
| Video call with synthetic participants | No |
| Compromised genuine account in a real thread | No — there is nothing to spot |
| Vendor invoice with changed bank details | No |
The last three are where the large losses are, and none of them involves the behaviour being measured.
Why the training does not transfer
Simulation training teaches pattern recognition on email artefacts: sender mismatch, urgency language, hovering over links, domain inspection.
An impersonation call has none of those artefacts. The number may be spoofed, the voice is familiar, the request is plausible, and the person is senior. The trained behaviour — examine the message for signs of forgery — has nothing to operate on.
There is also a worse effect. An employee who has passed every simulation may be more confident, not less, that they can tell the difference.
The population mismatch
Simulations are typically sent broadly. High-value fraud targets a narrow set: treasury, accounts payable, executive assistants, finance leadership.
An organisation-wide 1.2% failure rate says little about how a specific AP clerk responds to a call from someone claiming to be the CFO, at 4:45pm on a Friday, about a payment that must go today.
What actually helps that clerk
Not better judgement under pressure. A rule they can apply without judgement.
# Training approach
"Be alert to urgent payment requests from executives."
→ requires the employee to assess authenticity in the moment,
against a professional who chose the moment.
# Control approach
Payments above threshold, or with changed beneficiary details,
require a signed authorisation from the approver's enrolled
device. No signature → no payment.
→ the employee's task is to check for a receipt, not to judge
whether a caller is genuine.The second gives the employee something to say: the payment needs an authorisation and I cannot process it without one. That is a defensible position under pressure in a way that "I was not sure it was really you" is not.
The budget question, framed fairly
This is not an argument to stop awareness training. It reduces commodity phishing exposure, it is often required by insurers and frameworks, and it is inexpensive per head.
It is an argument about proportion. If the awareness budget is large and the deterministic control budget for the same risk is zero, the allocation does not reflect where the losses are.
| Risk | Best addressed by |
|---|---|
| Bulk credential phishing | Awareness plus phishing-resistant authentication |
| Malware delivery | Endpoint controls plus awareness |
| Executive impersonation for payment | Deterministic control on the payment |
| Vendor bank detail change | Deterministic control on the change |
| Compromised thread continuation | Deterministic control on the action |
A better metric to report
Click rate is reported because it is easy to produce, not because it predicts loss. Two figures are more informative and only slightly harder.
- Control coverage: what proportion of payments above the threshold carried an authorisation, over the period.
- Exception rate and shape: where the control was not applied, and why.
Both are directly connected to whether a loss can occur, both are auditable, and neither depends on how an employee felt about an email.
The population mismatch
Simulations are sent broadly; high-value fraud targets treasury, accounts payable, executive assistants and finance leadership. An organisation-wide 1.2% failure rate says little about how one AP clerk responds to a call from someone claiming to be the CFO at 4:45pm on a Friday.
| Metric | Why it predicts loss |
|---|---|
| Control coverage on payments above threshold | Directly connected to whether a loss can occur |
| Exception rate and shape | Shows where the residual risk is |
Objections and honest limits
“Awareness training is required by our insurer.” It is, frequently, and it should continue. The argument is about proportion — a large awareness budget alongside zero deterministic control budget for the same risk.
“Training makes people more careful generally.” Possibly, and it can also make them more confident that they can tell the difference — against an attack format the simulation never covered.
Reporting something that predicts loss
- Keep the simulations. They address commodity phishing and are cheap per head.
- Stop reporting click rate as the security metric. It measures one format.
- Report control coverage on payments above threshold. A percentage over twelve months.
- Report the exception shape. Where the control was not applied, and why.
Terms used here
- Click rate
- The proportion of simulated phishing recipients who interact — a real measurement of one behaviour.
- Control coverage
- The proportion of in-scope transactions that carried a deterministic control.
- Format transfer
- Whether a trained behaviour generalises to a different attack format. Here, it does not.
Frequently asked questions
Should we stop phishing simulations? No. They address commodity phishing, are often required by insurers and frameworks, and are inexpensive. The issue is whether budget proportion matches where losses occur.
Why doesn't the training transfer to voice attacks? It teaches pattern recognition on email artefacts. A call has none of them, so the trained behaviour has nothing to operate on.
Could training make things worse? Possibly at the margin. An employee who passes every simulation may be more confident they can tell the difference, against an attack format the simulation never covered.
What metric is better than click rate? Control coverage on payments above threshold, and the exception rate with its reasons. Both connect directly to whether a loss can occur.
Should phishing simulations stop? No. They address commodity phishing, are often required, and are inexpensive. The issue is budget proportion relative to where losses occur.
Why doesn't the training transfer? It teaches inspection of email artefacts. A phone call has none, so there is nothing for the trained behaviour to act on.
What should be reported instead? Control coverage on payments above threshold, and the exception rate with reasons.
Where this fits in Manav
Manav produces the artefact underwriting, claims and forensics all lack: a per-action receipt verifiable without the insured's cooperation, and a measurable coverage rate.