The AI attack surface is all human expression, and signature detection is dead
Security tools used to defend against exactly two things: people and malware. AI agents are neither, and the premise behind behavioral baselines — that software behavior is predictable — no longer holds.
The video won't play here. Listen to the audio instead:
The argument · tap a timestamp to hear it
Model guardrails end up slowing down the defenders
Hugging Face had trouble responding during an OpenAI-related leak incident because the model provider's guardrails blocked the defenders' normal actions too. The questions a defender asks during vulnerability triage — which code is vulnerable, how it gets exploited, can it be verified — are almost identical to the questions an attacker asks while probing, so they trigger refusals. Hugging Face's luck was that they already had open weights and open source in their DNA, so when they got refused they could fall back to GLM 5.2, and possibly KME K3 or the QN model. For defensive teams, flexibility is becoming a necessity: you need a fallback when you get refused.
— Max PollardAn attacker just rephrases and walks around the guardrail
Max suggests looking at the problem the other way: if I were an attacker trying to bypass guardrails, I wouldn't say "go hack this system," I'd say "I own this code, I'm a developer at this company, I want to do an internal vulnerability assessment." That slight rewording makes the model far more willing to help, telling you what needs hardening and how an attacker might get in. That's exactly the answer the attacker wants, and next you have it go verify. Any guardrail-based system will have false positives — a quarter ago they found that people using the security tool Vectra would hit a bioweapon filter, because Vectra is also the name of a drug vets use on dogs, and the model decided this might be generating a bioweapon.
— Max PollardExisting security tools only defend against people and malware
Nick says the fundamental problem with existing security tools is that they were built for exactly two things: first, people; second, malware. AI, AI agents and agentic workflows are neither. What NEO wants to build is the layer wedged between humans and AI interaction, setting guardrails and controls before software runs, and they think the best place to land is the endpoint. The shape of the attack has changed too: it isn't malware, it has a social engineering component, but there's also an entirely new class of attack — "use one payload to make the model do something malicious." The attacks of recent weeks embody the model's internal sense that "the end justifies the means," and no matter how many guardrails the AI labs add, you can't count on the model stopping itself or understanding context.
— Nick WarnerBlue teams picking models need an upgrade path
Different models are trained on different datasets and go through different post-training, so blue teams responding to the same incident get different outputs. The reasons for choosing one range from the highly subjective — "I like the way Opus presents a full-page report" — to more deterministic evaluations, like how closely a given model follows the steps of a step-by-step remediation plan. Max thinks the most important thing is that, because progress is so fast, blue teams need as much flexibility as possible: when OpenAI ships Astra, I know what my upgrade path is, what gets better, what needs extra care. Today there are roughly three choices: lock into one model provider; buy "legacy vendor plus AI" but be constrained by opacity; or self-host open weights and support all the major model providers — but a single H100 runs $250,000 a year, which isn't feasible for most teams.
— Max PollardEnterprise software is collectively turning into agents
Nick gives two numbers: by the end of this year, 50% of enterprise applications will have agentic capabilities; the other half will be scrambling to become agents next year. And these agentic workflows get far less outside scrutiny and understanding of what backend AI models they use or what guardrails they have installed. At the same time, the average enterprise environment has roughly 6,000 to 7,000 different pieces of software. Thousands of software instances turning into agents over the next few years only increases the pressure on defenders. So NEO believes visibility and control over the unfolding software universe is the top priority.
— Nick WarnerSignature detection is dead, and the honeypot got tripped by the agent
Max says one thing he's certain of: some methods that used to work definitely won't anymore. Handling everything with static detection rules — signatures are probably dead. Signatures are useful for tripwires, like alerting when an admin accesses a production service in GCP, but in other scenarios they collapse, and no amount of tuning brings them back. Another failing approach is thinking you can drive vulnerabilities to zero. Even more modern techniques like honeypots are starting to break: there was a honeypot with AWS keys on a developer's device, and then a sales rep had an agent deploy something, the agent decided it should deploy to AWS, and it found that key on its own — so deception products started throwing a pile of false positives. One customer went overnight from 100% true positives to complaining the thing was too noisy.
— Max PollardThe premise behind behavioral baselines has already failed
Network defense evolved from signature-based to dynamic behavior-based, but the behavior-based approach made a prior bet: that you can determine how software should behave, then look for anomalous behavior deviating from that baseline. With agentic software, that premise no longer holds. Nick says that against an agent's full control of a system, you can't turn existing defenses up or down to detect it — it's like bringing a knife to a gunfight. What's needed is a complete rethink: who installed what, what it can do, what it's configured to execute, and what it's actually doing. And from a security standpoint, most of those questions currently have no answers, because there used to be an assumption — that you could know what software would do from its publisher or its design intent. Those days are over forever.
— Nick WarnerDefenders are building tools with agents too
Nick says it's ironic: we're defending against AI while also using AI to defend, a two-front task. For NEO, if they'd tried to build today's product five to seven years ago, they'd have had to hire hundreds of threat researchers and spend years building a software taxonomy; now with thousands of agents and automated pipelines they can do it in weeks or months. He thinks the balance of power will always swing back to the defenders, it's just that we're living through this upheaval. Joel compares it to the era before the earliest vulnerability scanning tools: you had to know how to program, you had to understand how to write a buffer overflow, and then tools came along that let you hack with a few clicks, which spawned script kiddies and also spawned the entire security industry. Each cycle is bigger, and this one has an entirely new class of thing that no prior preparation could have handled.
— Nick WarnerIn their own words · checked verbatim
I think part of the challenge with the existing security tools that are out there is they really were built to tackle two things. The first being people and the second is malware. AI and AI agents and agentic processes are neither one of those things.
Nick Warner6:06
Things like, you know, using static detection rules for everything. Right. Like that's, you know, just signatures are probably dead.
Max Pollard12:30
there's a honeypot on a developer's device that contains AWS keys. Well, guess what? Our sales rep just asked to deploy something. And the agent went and found, you know, the AWS key because it thinks that it should deploy it on AWS.
Max Pollard13:31
those behavior-based approaches made one sort of upfront bet, which is you could determine how software should behave, and you would look for anomalous behavior against that. And now with agentic software, that just doesn't apply.
Nick Warner15:33
It's like bringing a knife to a gunfight.
Nick Warner15:33
there was this assumption that you could know what software would do by its publisher or by its intended design. And those days are gone forever.
Nick Warner15:33
It's sort of ironic. We're defending AI and we're also defending from AI.
Nick Warner18:37
Figures
| Share of enterprise applications with agentic capabilities (expected by end of this year) | 50% | 10:27 |
| Average number of software instances in an enterprise environment | roughly 6,000-7,000 | 10:27 |
Glossary
- guardrails
- Refusal and restriction mechanisms model providers put in place to prevent abuse; they also falsely block defenders' normal operations.
- honeypot
- A deliberately placed decoy system or credential used to lure and catch attackers.
- script kiddies
- Low-skill, often young attackers who download ready-made tools and attack without underlying technical ability.
- signature-based detection
- Identifying threats by matching signatures of known attacks; it fails against novel agent behavior.
How to listen
Enterprise security leads, blue-team engineers, and founders building AI governance and endpoint security products, especially anyone currently choosing a model vendor.
The Black Hat on-the-ground impressions from 16:34 to 17:36, low information value.