The Risk You Can't See Yet

AI has already changed your security picture, in both directions. Most institutions have only updated one of them, and it is the exciting one.

For years, the advice for spotting a phishing email was reassuringly simple. Look for the bad grammar, the odd phrasing, the greeting that got your name wrong. If it read like it was written in a hurry by someone who did not speak the language, you deleted it and moved on.

That advice is now worthless. The phishing emails are well written. They are personalized, they reference real people and real deadlines, and they arrive in volume, faster than a stretched security team can flag them. The tell we trained everyone to look for has been quietly erased, and most security awareness training has not caught up. That is a small example of a larger problem, which is that AI changed the security picture on a campus and the security picture is often the last thing anyone updates.

The institutional question

Is AI in your incident response plan, or only in your innovation plan?

On most campuses there is an AI conversation happening in one room, full of energy and pilots and opportunity, and a security team in another room that has not formally been asked to think about AI as a threat. The same technology is a headline in the first room and a blind spot in the second. Readiness means putting it on both agendas, because AI is not only something the institution adopts. It is also something used against the institution, and something that introduces new ways for the institution’s own tools to fail.

What this looks like in practice

The risk runs in two directions, and it helps to separate them.

The first is AI pointed at you. Phishing and social engineering are the obvious cases, now cheaper, faster, and far more convincing than they were two years ago. An attacker no longer needs to be a good writer or to speak the language, and can produce a thousand tailored messages in the time it used to take to write one clumsy one. The volume and the polish both went up at the same moment, which is a hard combination for training built around spotting mistakes.

It is not only email. The same tools can clone a voice from a few seconds of audio, which turns the help desk into a target. A caller who sounds like a specific dean, stressed and in a hurry, asking for a password reset before a board meeting, is a much harder thing to refuse than a badly spelled message ever was. The attacks got more human at the same moment they got more scalable, and the people most exposed to them are often the front-line staff we ask to be helpful for a living.

The second is harder to see, which is that your own AI deployments are new surfaces that can be attacked or can misbehave. A student-facing chatbot that can be steered with a carefully worded prompt into ignoring its own rules is a real category of problem, not a hypothetical. A tool connected to institutional systems can become a path for data to leave through a door no one thought to lock. You do not have to understand the technical detail to grasp the governance point. Every AI system you turn on is something new to secure, and turning it on is not the same as securing it.

Underneath both is a problem this series keeps circling. You cannot secure what you have not catalogued. If the institution does not maintain an inventory of the AI tools actually in use, including the features embedded in systems it already owns and the shadow use from the last issue, then the security team is defending a footprint it cannot see the edges of. A workable inventory is not glamorous. It is a list, with an owner and a risk level for each entry, that the people responsible for risk can actually look at.

There is also a habit worth breaking, which is treating risk assessment as a one-time gate. A tool gets reviewed before it goes live, gets its approval, and is never looked at again. But AI systems are not static. Models change under you, accuracy drifts, new vulnerabilities surface, and a tool that was reasonable in March can behave differently by October. Approving something once is not the same as watching it over time, and only one of those is actually security.

None of this is a knock on security teams, who are usually stretched thin and were not consulted before the interesting pilots launched. It is understandable that the shiny opportunity got attention before the threat model did. But understandable is not the same as safe, and the gap does not close on its own.

The Atlas connection

In Atlas, our AI operating map, this is the security and risk domain, in the Defense group — Protect — and it sits close to the domains around it. It runs into data governance, because most of what can go wrong is ultimately a data exposure. It depends on governance, over in Foundation, because someone has to own the decision that an AI system is too risky to run as configured, and have the authority to say so. And it quietly depends on the inventory that shadow AI makes so hard to keep. Security is where the abstract risks of the whole map become concrete, usually at an inconvenient time.

Questions worth putting on the agenda

A useful move is to pull AI out of the innovation conversation for an hour and put it in front of the people whose job is to imagine what goes wrong.

• Is AI in our incident response plan, or does it live only in our innovation plan?
• Do we keep an inventory of the AI tools in use, including embedded features, each with an owner and a risk level?
• Has our security awareness training been updated for AI-written phishing, now that “spot the typo” no longer works?
• Do we assess an AI tool’s risk once at approval, or continuously as the tool and its threats change?
• If an AI system we deployed were manipulated or leaked data next week, who would notice, and how?

The bottom line

The risks that hurt institutions are rarely the ones already on the agenda. They are the ones nobody was assigned to watch, sitting in the space between the team that is excited about AI and the team that is supposed to worry about it.

You cannot defend against a threat you have only ever discussed as an opportunity.