Walk through any office right now and you will find people experimenting with AI. They are watching demos, forwarding links to colleagues, signing up for free trials, and genuinely trying to figure out how this technology can make their work better. That energy is a good sign. It is what healthy, forward-thinking organizations look like — and attackers have noticed.
On June 8, Microsoft's Security Blog published a report titled "AI Brands as Bait: How Threat Actors Are Using the AI Hype in Social Engineering." Researchers documented a fast-growing wave of phishing campaigns built entirely around AI branding: fake ChatGPT login pages, imitation Microsoft Copilot portals, Claude imposters. None of these campaigns are particularly sophisticated. They work because they arrive at exactly the moment employees are most receptive, when curiosity about AI is already running high and something that looks like a useful new tool feels worth clicking.
For CISOs, this is a signal. The attack works on human motivation. Sebastian Schuetz, an assistant professor at the University of Colorado Boulder who researches human behavior and cybersecurity outcomes, put it plainly in a 2025 interview: "Awareness alone isn't enough to avoid phishing emails. Spotting scams depends not just on knowing what to look for, but on paying attention to subtle cues." With generative AI removing the formatting errors and generic greetings that once gave phishing away, those cues are harder to catch than ever. That means the response has to meet the problem where it actually lives.
The Trust Problem Is Significant, and It's Growing
According to SlashNext's 2024 State of Phishing Report, phishing attacks have grown by more than 4,000% since the launch of ChatGPT. Attackers are pulling open-source intelligence (OSINT) on organizations and employees, then using large language models to build lures that feel personally relevant. A fake invitation to try a new AI productivity tool lands differently in someone's inbox than a generic wire transfer request. It fits into a conversation employees are already having with themselves, which is precisely what makes it hard to dismiss.
In a late 2024 survey of 181 CISOs conducted by Adaptive Security, AI-powered social engineering ranked as the top concern, scoring an average of 8.6 out of 10. The responses were direct. A CISO at a major semiconductor company: "Deepfakes are happening now, and it's dangerous." A security leader at one of the largest entertainment venues in the country said the AI-generated attacks reaching their employees are "much higher in sophistication and harder to detect."
Most organizations already have solid technical controls in place. Email filtering, endpoint protection, MFA: these are table stakes. What they do not address is the scenario where a motivated, well-intentioned employee navigates to a convincing fake login page on their own, outside the reach of any filter. A controlled study by Fredrik Heiding and colleagues at Harvard Kennedy School's Belfer Center found that AI-generated phishing emails achieved click-through rates of 30 to 44 percent, compared to 19 to 28 percent for human-written emails. The gap exists because AI removes exactly the signals employees were trained to catch: awkward phrasing, generic greetings, formatting errors. Those scenarios are where the most damaging attacks originate, and they require a human solution.
The Skill You Want Your Team to Have
Think about what a well-prepared employee actually does in this situation. They notice that they were not expecting this link. They check before entering credentials. That instinct is what stops the attack. Dr. Lorrie Cranor, director of Carnegie Mellon's CyLab Security & Privacy Institute and a founder of phishing simulation as a discipline, has been making this argument for years. Speaking at the RSA Conference in March 2026, she said: "Humans make errors, but they make errors doing things they shouldn't have to be doing in the first place." For Cranor, the answer is to design security so the right action is the easy one, and to train people specifically on the scenarios where human judgment is unavoidable and cannot be automated away.
That instinct is a learnable skill, and the research on how to build it is fairly clear. Organizations running monthly simulations reduce employee susceptibility by 50% or more within a year, compared to those running annual programs. The metrics worth tracking are click rate on simulated phishing attempts and mean time to report, measured at the role and team level, updated on a rolling basis.
Role specificity matters here. An employee in finance faces a different threat profile than someone in HR or engineering. The finance employee needs to recognize a deepfake voice call fabricated to sound like a CFO. The HR employee needs to spot a fake onboarding tool that harvests credentials on the first click. Generic training covers the concept. Targeted simulations, built around the scenarios each role is likely to face, build the reflex.
Where to Start: Four Actions CISOs Can Take Now
- Before layering in new simulations, CISOs should understand exactly what attackers can already see. An OSINT audit of your own organization covers executive names, headshots, voice recordings, org charts, and publicly available contact information. It reveals the raw material attackers use to build personalized lures, and often surfaces exposures that no technical control is currently addressing.
- Second, establish a verbal verification protocol for high-risk transactions. A shared safe word or a required callback procedure for finance, HR, and anyone handling wire transfers costs almost nothing to implement and closes one of the most exploited gaps in AI-powered voice attacks. Most organizations do not have one.
- Third, audit which communication channels your defenses actually cover. Email are well established, but voice calls, SMS, and video conferencing are largely unmanaged. If your current program does not include simulations across those channels, the coverage gap is significant.
- Fourth, update your board briefing language. Budget conversations with leadership rarely turn on technical detail. Lead with business risk: the cost of a successful deepfake transfer fraud, the reputational exposure from a credential breach. Frame it that way and human readiness becomes a line item that competes differently than security awareness training.
Your Existing Program Is the Foundation. This Extends It.
CISOs who have built mature awareness programs have done real work. Employees know what phishing is. They know to be cautious with unexpected attachments and suspicious links. That baseline is genuinely valuable and worth protecting.
The question AI-branded social engineering raises is a specific one: were those programs built for today's threat environment? Traditional awareness training was designed for a world where phishing emails were recognizable and voice-based attacks were rare. Attackers today are generating personalized deepfake audio, convincing AI-branded login pages, and SMS lures at scale. The behavioral signals employees learned to watch for remain valid. The volume, the personalization, and the channels have shifted substantially since those programs were designed.
The programs delivering results right now layer AI-specific simulations on top of existing foundations. Dr. Frédéric Lemieux, Faculty Director of Cybersecurity Risk Management at Georgetown University, advocates for exactly this approach: "I advocate for wargaming exercises and the integration of cyber threat simulations, enabling teams to anticipate, analyze, and counter evolving adversarial tactics." In practice, that means running employees through scenarios that feel identical to actual AI tool adoption: an invitation styled like a genuine product launch from a known vendor, a voice call that sounds exactly like a familiar colleague, an SMS that references a real internal project name. Encountering those scenarios in a controlled environment first is what builds recognition that holds when an actual attack arrives.
And when that recognition shows up, it compounds. Someone who flags a suspicious message, reports a simulation attempt, or pauses before clicking demonstrates a behavior that influences the people around them. Teams that reinforce those moments consistently see measurable shifts in reporting rates within a quarter. That change shows up in incident data and it closes the gaps technical controls cannot reach.
AI Curiosity Is an Asset. Treat It Like One.
The organizations that will be in the strongest position a year from now are the ones investing in human readiness alongside their technical stack. A workforce that adopts AI tools productively and has the reflexes to recognize when something is off reduces the incidents that reach your technical controls in the first place.
That combination is achievable. It requires simulations, role-specific training, and consistent reinforcement over time. As Hise O. Gibson, Senior Lecturer at Harvard Business School, wrote in April 2026: "The leaders who treat AI security as a fiduciary obligation, not an IT line item, will be the ones still standing when the next attack comes."
Adaptive Security helps security teams meet that obligation. To learn more, visit adaptivesecurity.com.
Contents
