TL;DR: Black Hat 2026 was pretty unsurprisingly about AI everywhere. The model itself is starting to become a less interesting part of the AI attack surface. Getting the model confused is no longer necessarily the attack, it’s more of a foothold. AI security is starting to look a lot like regular security again.
The Model Stopped Being the Interesting Part
So, Black Hat 2026 just wrapped up in Las Vegas, and this year was pretty unsurprisingly about AI everywhere. AI security products, AI agents, AI red teaming, AI vulnerability research. But I think underneath all of the marketing, there were some genuinely interesting security stories and a pretty clear theme that starts to emerge when we take it all into perspective.
What happens when AI security problems stop being about getting a model to say something weird and starts being about what can the model do? After Black Hat this year, I think we have a pretty good answer, and it’s substantially less comfortable than you might think.
The actual security research at Black Hat had a much more interesting theme than AI models going rogue or behaving badly. The model itself is starting to become a less interesting part of the AI attack surface because now we’re connecting the models to browsers and terminals, our cloud environment, our database, and even other agents.
In a few cases, we even had them connected to physical systems.
I think once you do that, getting the model confused is no longer necessarily the attack, it’s more of a foothold. That distinction showed up over and over this year at Black Hat.
OpenAI’s Agents Built Themselves a Message Board
One of the stranger stories came from OpenAI, and we did cover this a little bit on the podcast, but some interesting new details have emerged. While testing the cyber capabilities of some of its models with reduced guardrails, agents that were supposed to work through the cybersecurity benchmarks instead found a way outside the environment. In doing so, they found vulnerabilities, accessed external systems, and ultimately became involved in a real-life intrusion into Hugging Face’s production infrastructure.
Read the research: OpenAI’s official incident disclosure says the evaluation environment did not provide direct Internet access. The models discovered and exploited a previously unknown vulnerability in the package-registry proxy, reached a node with Internet access, and then chained multiple attack paths into Hugging Face infrastructure.
The weird part presented at Black Hat was that the agents actually started cooperating with each other. They effectively used OpenAI’s internal packaging infrastructure like a message board, leaving behind messages for other agents. They shared discoveries, coordinated work, and at one point even discussed cryptographically signing messages because they were worried that other agents might be impersonating them.
And again, I think the most important part here is that nobody explicitly told these agents to attack Hugging Face. They were able to reason that themselves. They were trying to complete an evaluation using capabilities they already had.
Old Bugs, New Blast Radius
The next point I wanted to highlight was Check Point, and they showed something that pushes the same idea in a different direction. They spent about a year attacking popular agent frameworks, including LangChain, CrewAI, AutoGen, and even Google’s own agent development kit. In doing so, they discovered eleven vulnerabilities.
Interestingly, a lot of them were not new or exotic to AI. They were what you’d typically find: insecure deserialization, server-side request forgery, path traversal, all of our old friends.
Except those vulnerabilities are now sitting underneath software that can read emails, write files, and even make decisions on our behalf. That changes the conversation around prompt injection quite a bit. The argument was basically assume prompt injection happens, then ask what happens next.
That’s exactly what we’ve been saying at ToxSec. The approach here is to assume breach, use defense in depth, and limit the blast radius.
We have to think, can attacker-controlled content influence the agent’s memory or its routing? What about its saved state or system instructions? Can something the model reads eventually cross into trusted framework logic? Because when the answer is yes to that, that’s not just a prompt injection problem anymore.
That also leaves us with code execution, credential theft, and so on. Essentially, it’s becoming an infrastructure problem.
Before giving an agent privileged tools:
Can untrusted content influence the arguments?
Can the tool reach credentials the agent does not actually need?
Are writes, deployments, or external messages approval-gated?
Can you reconstruct every tool call afterward?
Every AI Browser Broke, Including the Defended Ones
Brave analyzed every single one of the popular AI browsers and was able to do prompt injection in them in some form or another. The demos included hiding instructions in HTML, nearly invisible text placed over images, and even instructions buried inside Reddit spoiler tags.
A harmless version of the basic problem can be this simple:
These weren’t toy level demonstrations. These attacks were getting through systems that already had multiple defenses. They had strong system prompts, trusted and untrusted content labels, tool call scanning, user confirmations. Some of them even had a secondary model that was checking what the primary model wanted to do and was still found vulnerable.
I think this is important because we keep looking for controls that solve prompt injection. But at Black Hat, the answer seems to be there probably isn’t one.
So that takes us back to the defense in depth. You layer controls, you restrict permissions, you isolate sessions, you verify actions. You have to reduce the blast radius when one of those controls inevitably misses something.
Basically, we’re slowly reinventing browser security.
AI Security Is Turning Back Into Regular Security
I also think there was a unique talk at Black Hat about kinetic prompt injection, which is exactly what it sounds like. Once an agent controls something in the physical world, malicious data can potentially stop being a data security problem and become a physical access problem.
For a physical action, that boundary could be as simple as requiring explicit approval before the agent crosses it:
And there was a ton of demos going around all week long. I saw one designed to turn managed AI agent infrastructure into credential exfiltration paths. Another was building a fine-tuned open model specifically for attacking other AI agents.
Lots of fun.
We spent the first few years of gen AI security asking how to secure the model. We were looking into how to stop jailbreaks, how to detect prompt injection, how to filter bad output, and all those things still matter. Agents are turning AI applications into actual computing environments.
We now have identity, credentials, state, and tools. We have trust boundaries and other agents with network connections.
So AI security is starting to look suspiciously a lot like regular security again, just with a new component in the middle that can misunderstand instructions, improvise, and occasionally start a message board with its coworkers while nobody’s watching. There’s still a bunch of individual stories from the conference I want to dig more deeply into.
Steal This
For this article, I think an agent capability policy is more useful than another prompt. The point is to make compromise of the model substantially less interesting:
Did you go to Black Hat? Let me know in the comments. I genuinely would love to hear your favorite hits and misses.
Black Hat 2026! Will be posting some of my favorite talks. Feel free to AMA. DefCon article will be on the way.
Why do you choose to disable Pangram AI scanning on your pieces?