Strong framing on the context leak problem. Most Shadow IT discussions focus on data exfiltration, but the relational dimension gets overlooked. When someone pastes a customer email thread into ChatGPT to draft a reply, they're not just exposing names and addresses they're exposing negotiation history, pain points, internal escalation patterns. That cumulative context is way harder to audit or claw back than a staic file download.
Shadow AI may be on of the strongest arguments for augmenting employees with AI. They are discovering these tools naturally, the momentum is increasing, so instead of worrying about risk, take action and implement approved tools for people to use. The slight initial dip (implementation cost and trading) of the J-curve is quickly offset by the boost in productivity and reduced risk of shadow AI.
fully agree. imo, you give them the tools you need, with ironclad ToS with your LLM SaaS provider, or host a solution on prem. you need to make the friction low, or you can hand your data to OpenAI.
I agree with the core point here. Shadow AI is one of the clearest signals that employees want AI augmentation and are already discovering value on their own.
That’s exactly why focusing only on risk misses the opportunity. When organizations provide approved, usable AI tools, they don’t just reduce Shadow AI, they align productivity and security instead of forcing a trade-off!
Hey guys there is something I’ve been curious about for a while and it’s kind of an extension of this issue. A law firm using a public Ai could conceivably have their threads subpoenaed by opposing counsel, do you think using an on prem or completely walled off system ( like azure local ) could get the threads protected under attorney client privilege ? If the data isn’t being stored anywhere other than a local server then wouldn’t it be considered the same as a filing cabinet in the firm’s storage room ? I think this is gonna wind up being a big issue in the not too distant future.
Hey Josh! I did some digging because this was actually a very good question.
Public AI = basically fucked.
Sam Altman publicly stated that conversations on ChatGPT aren’t protected by privilege or confidentiality, and they’d legally be required to produce those communications during a lawsuit.
For the on-prem question, it’s actually unsettled law, but most likely to be defensible.
Since private AI systems function under direct policies and supervision, the system effectively acts on the firm’s behalf rather than independently. A closed system could operate more like an expert witness or interpreter, qualifying as an “agent” of the firm rather than a third party.
Also i found a related case that might set a precedent: Tremblay v. OpenAI (N.D. Cal. 2024) is the first case where courts really grappled with AI and attorney work product.
Oh yeah now that is interesting... I read the case summary, ' This decision is significant as one of the first judicial clarifications on AI-generated content and attorney work product privilege, establishing that strategically framed prompts by counsel may be protected as opinion work product ', that line about strategically framed prompts being protectable opinion work product really stuck with me.
I’ve handled a lot of raw ground-entitlement and development work over the years, and it meant spending a ton of time with our attorneys. As a client, I’d make damn sure the firm had this kind of setup figured out.
Haha from where I’m sitting, building on-prem / legally protected AI systems for law firms sounds like one hell of a business idea, and the tech to do it is basically here.
absolutely! and this will be a huge year for on prem. one of the big advancements right now is token efficiency, some of these local models are starting to tear up their work with smaller and smaller computer. i think OpenClaw is a nice example, but we are going to see some major products built around running on prem local ai.
Great post, Erich and ToxSec, pot on about Shadow AI making Shadow IT look tame by comparison, especially how prompts leak not just data but business context that trains external models on your operations, amplifying risks in ad campaigns where bots already siphon insights. Recent reports highlight 47% of gen AI users sticking to personal accounts, doubling sensitive data incidents monthly and outpacing governance efforts. The fix lies in vetted platforms offering seamless, monitored AI access that detect anomalies in real-time while keeping workflows frictionless, channeling employee ingenuity securely.
absolutely! great point mentioning the additional business context leakage. it’s pretty tough to really get employees to understand how serious the issue is when they grow accustomed to their favorite genai tool and don’t have a replacement
I'm wondering, how companies are communicating their AI usage policies to their clients? I'm sure, clients are concerned how their data will be used by vendors or service providers... do you guys know of any best practices?🩷🦩
it’s really something they need to communicate. 4th party data sharing is one of the easiest ways to lose track of your data sharing, and 3rd party data sharing contracts can get complicated fast. the last stat i hear was only 40% of companies even had an ai policy!
"They’re training, prompting, and explaining the relationship between individual data and an external entity." exactly! Great post, thanks for sharing 👌
Is it possible to train an LLM to reject the prompt based on keywords? There are already solutions rolling out that are meant to discourage Self-Harm. We can't put the genie back into the bottle, but upon receiving info that violates a company's ip rights or some such, can the LLM return an output warning the user not to upload that kind of information? I imagine something like this would result in a downturn of sensitive info being shared.
in post training, you can increase the likelihood of the model rejecting it, but never absolute. you can add application level controls, regex and guardrails, etc.
in theory saas companies could do this, but they can both increase user friction through false positives, and are technically incentivized to collect this data for training, though it would be unethical.
the companies can structure devices and vpns to ban access to the url and domains as well. i just don’t see it often.
Brilliant collaboration team! Do you think that training against shadow AI will go the way of phishing attacks, i.e. employees get box-ticking exercises on how to spot it to relinquish institutional blame?
I doubt it will work. As long as the tool is more convenient than the "safe" tools that the company offers to employees, people will always find a way..
Thank you for showing us the details of this problem. For the last three decades employers have not been training employees who are expected to sit down and perform. But, two days of training solves this idea. But, employers only want speed at the lowest cost. And they're pricing jobs out of existence. Now, enter AI which is seen as the genie to solve all problems. But, the employers themselves need to understand the risks as well as the rewards of AI.
Feel free to ask Erich or myself any questions!
Feel free to ask me or Toxsex any questions! We’ll be happy to answer them!
I loved this read. I'd only add, don't forget to tag in whoever's responsible for AI in the organization - they tend to know how to influence people.
I call it soothsaying.
that’s a fantastic point. hopefully in 2026 we get better policy and training in place. engaged policy makers can affect the culture really well!
Absolutely it’s a team effort!
Thank you, we really appreciate it!
And that is a great point!
Thank YOU!
Strong framing on the context leak problem. Most Shadow IT discussions focus on data exfiltration, but the relational dimension gets overlooked. When someone pastes a customer email thread into ChatGPT to draft a reply, they're not just exposing names and addresses they're exposing negotiation history, pain points, internal escalation patterns. That cumulative context is way harder to audit or claw back than a staic file download.
Erich really nailed it here!
Absolutely! That was the point I was trying to make here! :)
🔥🔥
Shadow AI may be on of the strongest arguments for augmenting employees with AI. They are discovering these tools naturally, the momentum is increasing, so instead of worrying about risk, take action and implement approved tools for people to use. The slight initial dip (implementation cost and trading) of the J-curve is quickly offset by the boost in productivity and reduced risk of shadow AI.
fully agree. imo, you give them the tools you need, with ironclad ToS with your LLM SaaS provider, or host a solution on prem. you need to make the friction low, or you can hand your data to OpenAI.
That was exactly the point I was trying to make here! :)
I agree with the core point here. Shadow AI is one of the clearest signals that employees want AI augmentation and are already discovering value on their own.
That’s exactly why focusing only on risk misses the opportunity. When organizations provide approved, usable AI tools, they don’t just reduce Shadow AI, they align productivity and security instead of forcing a trade-off!
💯
Hey guys there is something I’ve been curious about for a while and it’s kind of an extension of this issue. A law firm using a public Ai could conceivably have their threads subpoenaed by opposing counsel, do you think using an on prem or completely walled off system ( like azure local ) could get the threads protected under attorney client privilege ? If the data isn’t being stored anywhere other than a local server then wouldn’t it be considered the same as a filing cabinet in the firm’s storage room ? I think this is gonna wind up being a big issue in the not too distant future.
Hey Josh! I did some digging because this was actually a very good question.
Public AI = basically fucked.
Sam Altman publicly stated that conversations on ChatGPT aren’t protected by privilege or confidentiality, and they’d legally be required to produce those communications during a lawsuit.
https://theformtool.com/legal-privilege-cloud-ai-and-the-ethics-gap-in-document-automation/
For the on-prem question, it’s actually unsettled law, but most likely to be defensible.
Since private AI systems function under direct policies and supervision, the system effectively acts on the firm’s behalf rather than independently. A closed system could operate more like an expert witness or interpreter, qualifying as an “agent” of the firm rather than a third party.
https://www.ibanet.org/Digital-strangers-in-litigation
Also i found a related case that might set a precedent: Tremblay v. OpenAI (N.D. Cal. 2024) is the first case where courts really grappled with AI and attorney work product.
Oh yeah now that is interesting... I read the case summary, ' This decision is significant as one of the first judicial clarifications on AI-generated content and attorney work product privilege, establishing that strategically framed prompts by counsel may be protected as opinion work product ', that line about strategically framed prompts being protectable opinion work product really stuck with me.
I’ve handled a lot of raw ground-entitlement and development work over the years, and it meant spending a ton of time with our attorneys. As a client, I’d make damn sure the firm had this kind of setup figured out.
Haha from where I’m sitting, building on-prem / legally protected AI systems for law firms sounds like one hell of a business idea, and the tech to do it is basically here.
absolutely! and this will be a huge year for on prem. one of the big advancements right now is token efficiency, some of these local models are starting to tear up their work with smaller and smaller computer. i think OpenClaw is a nice example, but we are going to see some major products built around running on prem local ai.
Great question, John! I have to admit I wasn’t sure what the right answer was myself.
I’m sure ChatGPT conversations aren’t protected by privilege or confidentiality. I think we can all agree on that.
On-prem solutions are a different story. Looks like I still have some reading to do!
Great post, Erich and ToxSec, pot on about Shadow AI making Shadow IT look tame by comparison, especially how prompts leak not just data but business context that trains external models on your operations, amplifying risks in ad campaigns where bots already siphon insights. Recent reports highlight 47% of gen AI users sticking to personal accounts, doubling sensitive data incidents monthly and outpacing governance efforts. The fix lies in vetted platforms offering seamless, monitored AI access that detect anomalies in real-time while keeping workflows frictionless, channeling employee ingenuity securely.
absolutely! great point mentioning the additional business context leakage. it’s pretty tough to really get employees to understand how serious the issue is when they grow accustomed to their favorite genai tool and don’t have a replacement
Thank you, we really appreciate it! That is also a great point! It isn’t easy to convince employees about how serious this problem is!
Very well written piece!
I'm wondering, how companies are communicating their AI usage policies to their clients? I'm sure, clients are concerned how their data will be used by vendors or service providers... do you guys know of any best practices?🩷🦩
it’s really something they need to communicate. 4th party data sharing is one of the easiest ways to lose track of your data sharing, and 3rd party data sharing contracts can get complicated fast. the last stat i hear was only 40% of companies even had an ai policy!
Thank you! We appreciate it!
This is really tricky. It is something they need to communicate, but how are you going to enforce it?
Especially when many companies still don’t have an AI policy.
"They’re training, prompting, and explaining the relationship between individual data and an external entity." exactly! Great post, thanks for sharing 👌
really appreciate it Yuri. You can tell Erich did his research!
Thank you, Yuri! We appreciate it! It is always a good idea to write about something that you experience first hand :)
Is it possible to train an LLM to reject the prompt based on keywords? There are already solutions rolling out that are meant to discourage Self-Harm. We can't put the genie back into the bottle, but upon receiving info that violates a company's ip rights or some such, can the LLM return an output warning the user not to upload that kind of information? I imagine something like this would result in a downturn of sensitive info being shared.
in post training, you can increase the likelihood of the model rejecting it, but never absolute. you can add application level controls, regex and guardrails, etc.
in theory saas companies could do this, but they can both increase user friction through false positives, and are technically incentivized to collect this data for training, though it would be unethical.
the companies can structure devices and vpns to ban access to the url and domains as well. i just don’t see it often.
Thanks for breaking it down.
thanks for engaging! we both appreciate the questions!
Really great content here!
Thank you! We appreciate it!
Brilliant collaboration team! Do you think that training against shadow AI will go the way of phishing attacks, i.e. employees get box-ticking exercises on how to spot it to relinquish institutional blame?
would be a great idea. the amount of people i’ve seen getting tickets for using competitor ai is silly… even with policy!
Thank you, Sam! We appreciate it!
I doubt it will work. As long as the tool is more convenient than the "safe" tools that the company offers to employees, people will always find a way..
2 of my favorite creators collabing!
Amazing work, guys.
thanks a ton Mohib!! 🔥
Thanks! We appreciate it!
Thank you for showing us the details of this problem. For the last three decades employers have not been training employees who are expected to sit down and perform. But, two days of training solves this idea. But, employers only want speed at the lowest cost. And they're pricing jobs out of existence. Now, enter AI which is seen as the genie to solve all problems. But, the employers themselves need to understand the risks as well as the rewards of AI.
absolutely! training is one of those things that has a high roi, it’s just hard to see because the end product is “not” getting compromised!
i’ve seen it happen at many companies too. up to 60% of companies don’t even have policy established on it yet!
And another 30% of companies have an existing policy that is only a piece of paper that nobody understands..
Plenty of data, I can tell you that! This really isn’t easy to handle.