A hospital administrator once showed me a binder. Inside were signed attendance sheets from four awareness sessions, photographs of staff at a projector, and a certificate from the vendor who ran them. "We have done the training," he said. So I asked him one question. When was the last time a member of staff reported a suspicious email to your IT desk? He did not know. Nobody had ever logged one. The binder proved attendance. It said nothing about whether a single nurse, clerk or pharmacist would actually behave differently the next time a payroll lure landed in their inbox.
That gap is the whole problem. Awareness training is easy to run and hard to measure, so most hospitals measure the easy thing, completion, and quietly assume the hard thing, behaviour, followed along. It often did not. The good news is that you do not need a fancy platform to find out. The signals are already sitting in your helpdesk tickets, your email gateway and your EHR audit logs. You just have to decide to read them.
Attendance is an input, not a result
Counting how many people attended a session tells you what you spent, not what you got. It is the cybersecurity equivalent of judging a vaccination campaign by how many syringes were opened rather than how many people gained immunity. The questions that matter come after the room empties. Do people now recognise a phish? Do they report it? Do they stop sharing the ward login? A behaviour you cannot observe is a behaviour you cannot claim to have changed.
A quiz score taken an hour after the workshop measures short-term memory. What you actually want to know is what a tired records officer does at 4pm on a Friday, three months later, when a convincing email asks her to confirm her login. That answer is not on any attendance sheet. It is in the way she behaves, and behaviour leaves a trail you can read.
Five behavioural signals you already have
None of the following requires a new tool. Each comes from systems a Nigerian hospital running an EHR and a corporate mailbox already operates.
Phishing report rate. Of the suspicious emails reaching staff, what share gets reported to IT rather than ignored or, worse, clicked? You can baseline this honestly only by running the occasional internal simulation, but even organic reports of real lures tell you whether the reporting habit exists at all. A rate that climbs over two or three quarters is the single clearest sign that training landed.
Mean time to report. Speed is its own metric. An email reported in four minutes gives your team a chance to pull it from every other inbox before the day shift opens it. The same email reported in two days is a post-mortem. Track the median, not the average, because one weekend straggler will distort the mean.
Staff-surfaced versus IT-found incidents. Who catches problems first? In a hospital where awareness is real, a growing share of incidents arrives from the floor: a records clerk flags a colleague's account behaving oddly, a matron queries an unfamiliar "engineer" at the nurses' station. When IT or an external tester finds everything and staff find nothing, the culture is still asleep.
Repeat-clicker concentration. In most organisations a small group of people accounts for a disproportionate share of risky clicks. Find them. Not to punish them, but because targeted coaching of twelve people beats re-lecturing twelve hundred. If the same names recur quarter after quarter, your general training is not reaching the people who most need it.
Shared-account trend in the EHR. This one is uniquely clinical and uniquely revealing. Pull the audit logs and look at how often a single login is in use on two wards at once, or how many staff are still typing the casualty department's communal password at 2am. Awareness that is working shows up as that number falling as people request and use their own credentials. Awareness that is theatre leaves the shimmering shared monitor at the nurses' station logged in exactly as it always was.
The reporting-rate paradox
Here is the trap that catches well-meaning leaders. You run good training, reporting climbs, and suddenly your IT desk is logging far more suspicious-email tickets than it did last year. A board member reads the dashboard and concludes the hospital is under heavier attack, or that staff have become careless. The opposite is true. The volume of threats did not rise. Your visibility did.
The dangerous state is silence. A hospital logging zero phishing reports is almost never a hospital with no phishing; it is a hospital where staff click, panic, and say nothing. So when you present these numbers, frame the rise correctly before someone misreads it. More reports, faster, is the goal. The day to worry is the day reporting flatlines while everyone knows the lures are still arriving.
A one-page quarterly scorecard
Metrics drift into noise without an owner and a baseline. Put the five signals on a single page, measure each one before you run any intervention so you have a true starting point, then review them every quarter. Give every line a named owner, because a metric nobody owns is a metric nobody fixes.
- Phishing report rate and mean time to report: owned by the IT security lead, sourced from the email gateway and helpdesk.
- Staff-surfaced versus IT-found incidents: owned jointly by IT and the relevant department head, so the floor gets credit for what it catches.
- Repeat-clicker concentration: owned by whoever runs the awareness programme, with coaching, not blame, as the response.
- Shared-account trend: owned by the clinical lead for the ward in question alongside IT, since the fix is partly workflow and partly credentials.
This scorecard is not only a management tool. Under the Nigeria Data Protection Act, accountability is an obligation, not a courtesy: a data controller has to be able to demonstrate that its measures are appropriate and that they work. "We held four sessions" is an input. "Our report rate rose and our shared-account usage on the medical ward fell over three quarters" is evidence. If the NDPC ever asks what your awareness programme achieved, the scorecard is the answer, and it is the same answer that tells your own board whether the money was well spent.
- Attendance is spend, not result. Completion sheets prove people sat down. They say nothing about whether behaviour changed.
- Read the evidence you already have. Phishing report rate, time to report, staff-surfaced incidents, repeat-clicker concentration and EHR shared-account trends need no new tooling.
- Understand the reporting paradox. More reported phish is a sign training worked, not a sign of more attacks. Silence is the real warning.
- Baseline before you intervene. Measure each signal first, or you cannot tell whether anything moved.
- Put it on one page with owners. A quarterly scorecard tied to NDPA accountability turns training from a cost into demonstrable evidence.
If you cannot say what your last training changed, you did not measure it, you witnessed it.
