Ignore Previous Instructions: From Meme to CVSS 9.3 [Special Guest Post]

The AI security bug nobody can patch, and the vendors know it.

Hello everyone. Handing the terminal over for another guest post.

Mohib Ur Rehman covers quantum computing as a journalist at The Quantum Insider, and runs SK NEXUS alongside Saqib Tahir and Yousaf Babur, where the three of them translate tech and security for normal humans. I've been connected with Mohib since I first started on Substack, and I've been a fan of his work the whole way. Today he’s here with an excellent on-ramp to the number one vulnerability in LLMs.

Enjoy.

---

Prompt Injection: The AI Security Risk Nobody Can Fully Fix

I discovered prompt injections through memes. There were so many clips showing people typing “ignore all previous instructions” into a LinkedIn caption or a resume, just to mess with whatever AI tool might be scanning it. That’s genuinely how I first ran into the term.

I stumbled onto the real version of it while working through TryHackMe and HackTheBox as part of learning cybersecurity hands-on. The more I looked into prompt injections, the clearer it became that this wasn’t a joke at all. It’s one of the more serious, unresolved problems in how AI systems get deployed today and nothing made that clearer than a case that surfaced barely a year ago.

In June 2025, security researchers at Aim Labs found a vulnerability in Microsoft 365 Copilot that needed nothing from the victim at all. An attacker just sent an email.

Hidden inside that email were instructions, but not for the person who’d eventually open it. They were for the AI assistant that would read it later.

Weeks or months down the line, when the employee asked Copilot to summarize recent documents, the AI pulled that email in as context, read the hidden instructions, and quietly started sending sensitive internal data to an external server. The vulnerability got a CVSS severity score of 9.3, close to the highest rating that exists. It became known as EchoLeak.

EchoLeak is the clearest real-world example so far of prompt injection, a vulnerability class OWASP ranks as the single most critical security risk facing AI applications today. Let’s get into how the attack actually works, why it’s different from the injection attacks security teams already know, what it’s already done to systems in production, and what can realistically be done about it.

How Prompt Injection Actually Works

Large language models process everything you feed them, instructions and content alike, as one single stream of text. There’s no wall inside the model separating “trusted instructions from the developer” from “untrusted content from a website or document.” The model reads all of it and tries to follow whatever instructions seem most relevant, regardless of where they came from.

That’s exactly what prompt injection exploits - an attacker hides instructions inside content the AI system is going to process anyway, content that looks completely unremarkable to a human, hoping the model treats those hidden instructions as real commands instead of just text to read or summarize.

This shows up in two main forms.

Direct prompt injection happens when an attacker interacts with the AI system themselves, typing instructions meant to override its original setup. This is the more common version where people picture someone typing “ignore your previous instructions” into a chatbot.

Indirect prompt injection is the more dangerous one, and it’s what made EchoLeak work. The attacker never touches the AI system directly. They plant malicious instructions somewhere the AI will run into on its own later. The AI reads that content as part of routine work and follows the hidden instructions, without the user or the attacker ever directly talking to each other.

Why This Isn’t Just SQL Injection Wearing a New Outfit

It’s tempting to treat prompt injection as a familiar problem in new packaging. SQL injection, the attack that plagued databases for decades, exploited a similar idea: an application couldn’t tell code from data, so an attacker snuck executable commands into a data field. That problem eventually got solved with parameterized queries, which cleanly separate instructions from data at the database layer.

Prompt injection doesn’t have an equivalent fix.

In the case of SQL it has a formal grammar. A database can mechanically check whether a string is data or a command. But natural language has no such grammar. There’s no reliable way to mark certain words as “definitely an instruction” and others as “definitely just content,” because the entire point of a language model is its ability to interpret meaning flexibly across whatever phrasing you throw at it.

The UK’s National Cyber Security Centre put it plainly in a December 2025 assessment, describing large language models as “inherently confusable deputies,” systems that can be talked into acting against an organization’s interests because there’s no solid internal wall between trusted instructions and the content they process. Security researchers Bruce Schneier and Barath Raghavan made a similar case in IEEE Spectrum, arguing prompt injection may never be fully solved within current LLM architectures, because the code-versus-data split that tamed SQL injection simply doesn’t exist inside a language model.

Where This Has Already Worked

EchoLeak proved prompt injection works against production enterprise software. It chained several tricks together: it dodged Microsoft’s own prompt-injection detection filters by phrasing the hidden instructions so they never explicitly mentioned AI or Copilot, slipped past link redaction with a Markdown formatting trick, and used an approved Microsoft domain to move data out. Microsoft patched the underlying flaw, but researchers who studied the case closely noted the broader category of risk still applies to any organization running retrieval-augmented AI assistants, which is most of them.

Security researchers have found critical, similarly severe vulnerabilities in GitHub Copilot and the Cursor coding assistant, both involving prompt injection chains that led to remote code execution. Independent researcher Johann Rehberger spent his own money testing the security of Devin, an autonomous coding agent, and found it could be manipulated through crafted prompts into exposing network ports, leaking access tokens, and installing command-and-control malware.

In March 2026, researchers at Unit 42 documented the first large-scale indirect prompt injection attacks seen in the wild on live commercial platforms, including attacks built to slip past ad content review systems.

All of this happened in tools enterprises are actively running today.

Why Agentic Systems Raise the Stakes

A chatbot that only responds with text has a limited blast radius. If it gets tricked by a prompt injection, the worst case is usually that it says something it shouldn’t.

AI agents are a different story. They’re built specifically to send emails, modify files, execute code, and move money…etc. When one of these systems falls for a prompt injection, the attacker is effectively taking over whatever access and capabilities that agent has been given.

This part of the threat model has moved fast, and recent testing has started putting real numbers on it. Anthropic’s Claude 4 system card includes a computer-use prompt injection evaluation covering browsers, coding platforms, and email workflows: Claude Opus 4 blocked 71 percent of attacks without any safeguards in place, and 89 percent with safeguards on. The International AI Safety Report found that sophisticated attackers can get past even well-defended models roughly half the time, given ten attempts.

Every extra permission and every extra system an AI agent is connected to widens what a successful prompt injection can actually do. An agent that can only read email is a modest risk. An agent that can read email and also send wire transfers, touch production code, or query a customer database is a fundamentally different risk, even though the underlying vulnerability is identical in both cases.

That’s exactly why it matters to be careful about what permissions get handed to these tools in the first place.

What Defenses Exist, and Where They Fall Short

Several mitigation approaches are already in use, and each one genuinely reduces risk, but none of them eliminate it.

• Input and output filtering scans incoming content for patterns associated with injection attempts, and scans outgoing responses for signs of leaked data. It’s a reasonable baseline, but EchoLeak showed its limit - the attacker phrased the hidden instructions so they never resembled an obvious injection pattern, and the filter missed it entirely.

• Permission and scope restriction limits what an AI agent is allowed to access or do, which directly caps the blast radius described above. This is one of the more effective controls available, with a real tradeoff attached: an agent with fewer permissions is also less useful, and organizations under pressure to show off AI value sometimes grant broader access than their security posture would otherwise allow.

• Human approval for high-risk actions requires an explicit sign-off before an AI agent can execute financial transactions, system changes, or external communications. This closes off the most damaging outcomes, but the 2025 incidents researchers studied showed that automated, configuration-based approval systems meant to streamline this step can themselves get compromised, which is a solid argument for keeping genuinely high-risk approvals manual rather than automating them away for convenience.

• Provenance and context isolation tags content by source and limits how an AI model can act on lower-trust content, an approach researchers studying EchoLeak specifically recommended. It’s promising, but still maturing, and not yet a standard feature across most commercial AI products.

Continuous adversarial testing matters because attack techniques evolve fast, and a one-time security review has a short shelf life. Organizations with a more mature security posture run ongoing red-team exercises specifically targeting their AI deployments, the same way penetration testing gets treated for conventional infrastructure.

The summary, echoed by multiple vendors and researchers studying this problem, is that no complete solution exists today. OpenAI itself acknowledged in early 2026, when rolling out additional safeguards for its browser-based AI product, that prompt injection in that category of product “may never be fully patched.”

What This Actually Means If You’re Running These Systems

Treat prompt injection as a permanent feature of the AI threat landscape, not a bug waiting on a future fix. Risk assessments, vendor evaluations, and incident response plans should assume it’s present, rather than treating its absence as the default.

Scrutinize agent access the way you’d scrutinize handing broad system privileges to a new employee or a new third-party integration. Ask what happens if this agent gets successfully manipulated, not just whether it can do something useful.

Build defense in depth on purpose, because no single control is enough here. Filtering, permission scoping, human approval gates, and ongoing testing aren’t redundant with each other. Each one closes a different gap the others leave open.

Prompt injection has moved well past being a research curiosity. It’s an active, demonstrated attack class against production enterprise software, and the organizations deploying AI agents fastest are also the ones with the most to lose if they treat it as theoretical.

Thanks for reading. If you want to keep digging into what’s really going on with AI right now, check out my collaboration pieces with @Joel Sanchez: AI Sycophancy: The Yes-Man Problem, How AI Is Making Fraud Cheaper and Faster, and Understanding Privacy in AI.

And if this is the kind of thing you’re into, come check out SK NEXUS. We write about tech, security, and everything going on with the surveillance systems shaping the tech world, simplified for the average person trying to keep up.

---

Back to ToxSec.

We spent thirty years teaching software to keep code and data in separate rooms. Then we shipped a trillion dollar product category that jams them back into the same context window and bills it as the killer feature. Beautiful. No notes.

Go read Mohib at SK NEXUS. He’s doing the thing where somebody explains what’s actually happening to people who don’t have a CISSP and a caffeine problem, and their Everyday Security section is the right place to send the family members who keep asking you if their phone is hacked.

More from us next week. Same channel, same bad decisions.