Can I tell you about a phishing simulation I ran early in my career that I am not particularly proud of (I wasn’t perfect, nor am I now). We had just rolled out a new security awareness training program. The vendor was enthusiastic, the platform had dashboards and the dashboards had charts. The charts went up and to the right, which is the universal signal that a security team is doing their job. We launched a phishing simulation, caught about 30% of the organization clicking a fake credential harvesting link, automatically enrolled every one of them in a mandated training module, and then sent a report to the executive team showing our “click rate” and “training completion percentage.”

Everyone was satisfied and the numbers looked good but the incidents where people fell for. Yet, I spent the next six months wondering why the behavior had not actually changed.  The answer (when I finally got honest with myself) was that I had not built a training program, what I had built was a compliance ritual with a user interface.

Let me be Crystal Pepsi clear about something… phishing simulations are not inherently bad, they are needed. They serve a real diagnostic purpose, but if you want to understand where your organization’s weakest links are, sending a controlled fake phish and watching what happens is a perfectly reasonable thing to do. I am not here to tell you (nor do I recommend) to stop running simulations.

What I am here to tell you is that the simulation is not the program, and somewhere along the way (for a lot of organizations, and some I helped), the simulation became the entire program. Run the sim, catch the clickers, enroll them in training, report the numbers, repeat quarterly. That is not a security awareness program… that is a treadmill with metrics.

The reason this matters is that behavior change, actual, durable behavior change in how people think about and respond to threats, does not happen because someone clicked a fake UPS tracking link and felt bad about it for ten minutes before their remedial module auto-completed in another browser tab. Behavior change is slow. It requires repetition, relevance, and some degree of genuine engagement…and most security awareness training programs, if we are being honest, are optimized for none of those things. They are optimized for completion rates.

Here is the uncomfortable truth about the security awareness training market… it grew up inside the compliance ecosystem, not the learning science ecosystem. The reason most programs look the way they do…annual mandatory modules, click-through acknowledgments, phish-and-shame simulations…is because that is what auditors wanted to see, not because anyone did a serious study of how adults change their behavior.

Auditors want evidence. They want a record that Employee X completed Training Module Y on Date Z. That is a tractable problem. You build a platform, you track completions, you export a report. Done. Box checked.

What (some) auditors do not particularly care about, because it is genuinely hard to measure, is whether Employee X now makes better security decisions than they did before the training. That is a harder problem to prove on a spreadsheet. It requires actually thinking about pedagogy (word of the day calendar for the win…it means the art, science, and theory of teaching) , about how people learn, about what makes security relevant to a person who spends their day thinking about accounts payable or patient intake or quarterly sales targets.

Most vendors do not want to solve that problem. It is expensive, it is hard to package, and frankly the market has not demanded it. As long as organizations are buying security awareness training platforms based on completion dashboards and phish click rates, vendors will keep building completion dashboards and phish click rates.

I think that’s on us as security leaders, not on the vendors. We set the requirements. We sign the contracts. We define what “good” looks like. If we keep telling ourselves that a 12% click rate on our Q3 phishing simulation means our program is working, we are going to keep getting programs that produce 12% click rates and very little else.

I want to talk about what a program designed for behavior change looks like, because I think some of this gets lost in the vendor noise.

First, it is continuous, not episodic. Annual training is better than nothing (only barely). The research on how people retain information is unambiguous on this: spaced repetition, where you revisit concepts at regular intervals over time, this produces significantly better retention than a single annual event. This does not have to mean drowning your employees in monthly modules. It can mean short, targeted reminders. A two-minute video while onboarding. A relevant tip in the company newsletter when a high-profile breach is in the news. A brief all-hands moment when your business or industry gets hit with a targeted campaign. Little and often beats big and annual, almost every time.

Second, it is relevant to the person receiving it. This is where a lot of programs fail. Security awareness content written for a generic employee at a generic company will not land. Content that acknowledges who your people actually are will. Your finance team is a high-value target for business email compromise and wire fraud. Your HR team handles sensitive personal data and is regularly targeted with credential phishing disguised as employee inquiries. Your developers are going to make different threat-related decisions than your administrative staff. Training that acknowledges those differences, even modestly, is dramatically more effective than training that treats everyone as if they have the same threat profile.

Third, the program is psychologically safe. This one is worth dwelling on, because the phish-and-shame model actively works against it. When employees feel like the security team is out to catch them making mistakes and then publicly (or semi-publicly) embarrassing them, they do not become more security conscious. They become more resentful of the security team. They might stop reporting suspicious things because they are worried about getting in trouble. They will find workarounds to avoid the processes that feel punitive. I have seen this play out enough times that I am no longer surprised by it, but it still frustrates me. Security awareness programs that generate fear and resentment are not just useless; they are actively counterproductive. They erode exactly the kind of human sensor network you are trying to build.

What you want, what actually makes an organization more secure, is an environment where people feel comfortable saying “I got a weird email, I’m not sure what to do” or “I clicked something and I’m worried about it.” That kind of culture does not emerge from a program that punishes mistakes. It emerges from one that treats mistakes as learning opportunities and makes it genuinely easy to ask for help.  I often say, I would prefer by users to report 1000 spam emails I need to look at, then not tell me the one real phishing email they see.

Let me spend a minute on metrics, because this is where a lot of well-intentioned security leaders get stuck. Click rates are easy to measure. Training completion rates are easy to measure. These numbers go in the board reports, they satisfy the auditors, and they give you something to point out to when someone asks how the awareness program is doing. I understand why we use them. I have used and still do use them.

The problem is that they measure activity, not outcome. A 7% click rate on a phishing simulation tells you that 7% of your employees clicked a fake phishing link during a controlled test. It does not tell you whether your employees are making better security decisions. It does not tell you whether your incident reporting rate has improved. It does not tell you whether the number of credential-related incidents is trending down. It tells you about a simulation.

The metrics that matter are harder to get and require more work. How often are employees voluntarily reporting suspicious emails? Is that metric going up over time? When security incidents did occur, how many of them involve a human error that could of been prevented by better security habits? Are those numbers changing? How quickly does your organization respond when a real phishing campaign hits, compared to six months ago?

Those things are hard to measure cleanly, and they are influenced by a lot of factors besides your training program. I am not pretending this is easy, but if the only data you have on your awareness program is completion rates and simulated click rates, you do not actually know if your program is working. You know if people are finishing modules and whether they can spot a fake UPS email in a controlled environment. Those are not the same thing.

Here is the thing that rarely comes up in conference sessions or vendor demos…a lot of security awareness training is genuinely bad. Not bad in a “could be optimized” way. Bad in a “I would be embarrassed to show this to a competent adult” way.

The content is often condescending; the scenarios are often absurdly obvious. The phishing simulations frequently use templates that nobody with more than three months of professional experience would ever fall for, which produces artificially low click rates that make the program look effective without actually testing anything meaningful. The completion quiz at the end of the module has answers so obvious that you could pass it without watching the video. (I know this because I have done it…and I assume you have too.)

When your employees, who are good at their actual jobs (well some of them), have to sit through training that feels like it was designed for someone who has never used email before, you are not just wasting their time…you are sending them a message: the security team does not think much of you. That message is hard to walk back.

The bar for what constitutes “acceptable” security awareness content needs to be higher. Not because the auditors or regulators require it, but because your employees deserve training that respects their intelligence and actually gives them something useful. If the training you are running would embarrass you, if a skeptical employee asked you to defend it in a meeting, that is probably a sign it needs to change.

I want to be careful here not to make this sound easier than it is. Building a security awareness program that actually changes behavior, respects employees, and produces meaningful outcomes is genuinely hard work…It takes time, it takes iteration, and it takes the willingness to admit when something is not working.

That said, I think there are some concrete things that separate programs that work from programs that just check boxes.

The first is treating your employees as partners rather than liabilities (I’m guilty of this thought process). The framing does matters…a program built on the premise that employees are the weakest link and need to be protected from themselves produces different outcomes than one built on the premise that employees are a critical part of your security posture and you want to give them the tools to play that role well.

The second is making it genuinely easy to do the right thing. Reporting a suspicious email should take one click, not navigating a ticketing system. Asking the security team a question should feel like reaching out to a helpful colleague, not filing a formal inquiry. The friction in your security processes is a direct predictor of how often people will skip them.

The third is closing the feedback loop. When an employee reports a phishing email and it turns out to be real, tell them. When a simulated phish goes out and someone reports it instead of clicking it, acknowledge that. People are more likely to repeat behaviors that get positive reinforcement, and most security awareness programs provide essentially zero positive reinforcement. They only surface the mistakes.

The fourth is measuring what matters and being honest about what you do not know. Track your reporting rates. Track your incident trends. Be willing to say “I think this program is helping but I cannot prove it conclusively yet” rather than waving click rates around as if they tell the whole story.

I want to close with something that I think is worth sitting with…

Security awareness training exists, fundamentally, because we have not figured out how to build systems that do not require humans to make perfect security decisions under pressure. Every email that asks an employee to evaluate whether a link is legitimate is, on some level, a failure of technical controls. We are asking people who have a hundred other things on their minds to be the last line of defense against attackers who are specifically trained to exploit human psychology.

That does not mean training is pointless. A more security-aware workforce is genuinely harder to compromise than one that has never thought about these things. The margin matters. But I think it is worth being honest with yourself about what training can and cannot accomplish.

Training can raise the floor. It can make obvious attacks less effective. It can build a culture where people feel comfortable reporting problems. It can turn your employees into an early warning system that your technical tools cannot replicate.

Training cannot make humans infallible. It cannot compensate for an email security stack that lets sophisticated phishing through. It cannot replace MFA, or proper access controls, or an incident response capability that can move fast when something goes wrong.

The organizations who gets security awareness right are the ones that truly understand that training is just one layer in a defense-in-depth model (a genuinely useful layer, worth investing in properly) rather than either the whole program or a box to check.

I need you to ask yourself (and answer honestly) Are you building a program that works in reality, or something that only looks good on a dashboard?