What connects seahorses and American Football? Answer: they both are capable of bamboozling ChatGPT.

As Chief Security Strategist at Cato Networks Etay Maor showed SDxCentral during a virtual interview, if you ask the OpenAI chatbot whether a seahorse emoji exists, the AI assistant du jour essentially goes off the rails. Said emoticon does not exist, but instead of relaying this fact, the AI instead begins spitting out a stream of text and emoji including mermaids, dolphins, and unicorns.

A similar meltdown happens if you ask ChatGPT for a list of all NFL teams whose names end with the letter "s."

Why does this phenomenon happen exactly? Nobody quite knows, and this is Maor’s main point: even with famous models like ChatGPT, understanding the underlying reasoning of AI systems can’t always be taken for granted.

This poses a problem when employees bring in unknown or unvetted AI tools into an organization. This conundrum has a name – shadow AI – with Gartner predicting that the issue will befoul more than 40% of enterprises by 2030.

Cybernews research, meanwhile, claimed 59% of employees use workload-reducing AI apps unapproved by their workplace – and 75% of these shared possibly sensitive information while using them.

The damage has already been done to some extent, with Tesla employees sharing proprietary manufacturing data with ChatGPT, and health care clinicians exposing patient records using unapproved diagnostic tools on personal devices.

According to Cato Networks CEO Shlomo Kramer, this pressing threat is one of the reasons his firm snapped up AI cybersecurity firm Aim Security this year, in the secure access service edge (SASE) specialist’s first acquisition to date.

“Adding Aim has added significant capabilities in this area,” Maor said. “In the strategic framework we need to work with when we're thinking about shadow AI, the first step is to observe. You need to have complete visibility into everything.”

Maor views this as the natural extension of SASE, in which the entire network is made visible, with a policy on what is and isn’t allowed into that framework, as well as the next stage of shadow IT.

“This is crucial for shadow AI, because by its nature, you may not know where it is,” Maor explained. "It may be somebody bringing their own device, somebody working from home, somebody bringing it into the organization.

“Everything that we did with shadow IT in the past for application security applies, but we have additional components now that we have to understand,” Maor added.

The major problem Maor has noted is that some CISOs see the black box nature of AI applications – seahorse mysteries and all – and assume they can’t investigate any further, while misunderstanding what their priorities should be on the permission front.

Examples he has noted are CISOs worried about hallucinations and incorrect, maybe even offensive output due to mistakes or deliberate tinkering with an application.

“I agree, but you have to go deeper than that," Maor said. "You also have to understand the model, the training data, the infrastructure behind every application.”

It’s a network problem, actually

This means understanding that shadow AI is more than white-collar employees feeding confidential data into a chatbot panel. In fact, according to Žilvinas Girėnas, product and engineering manager for Nexos, it’s network teams and network operations we should be worried about.

“They're often the biggest adopters, creating a paradox where security staff introduce the risks they're meant to prevent,” Girėnas commented. “When network engineers use unauthorized coding assistants, they risk exposing infrastructure configurations, implementing vulnerable automation scripts with hard-coded credentials, or leaking network architecture that maps your entire environment.

“Unlike everyday employees sharing documents, technical teams have privileged access that turns individual shadow AI use into organization-wide infrastructure compromise."

This includes security teams, with Next DLP research claiming 73% of cybersecurity professionals have used unsanctioned apps including AI in the past year, while one in 10 admitted that use of shadow AI and software-as-a-service (SaaS) applications had led to a data breach in their firms.

Leading Resolutions COO Jon Bance told SDxCentral he had witnessed cases where network engineers installed unsanctioned monitoring tools to collect data, unknowingly providing unapproved information to third parties.

“There are examples of where a bot running on a production system caused outages by overwhelming resources and deleting critical files,” Bance claimed.

Sachin Dharmapurikar, head of client technology at The Modern Data Company, meanwhile, explained that network teams can mistakenly expose critical infrastructure when integrating AI tools or scripts without proper review for root cause analysis (RCA).

“A simple example of using log snippets or error messages or certain firewall rules to use with online AI-integrated tools to debug or provide RCA could result in the leakage of sensitive information,” Dharmapurikar noted. “A bad actor who has access to this information can significantly exploit this.”

Shadow automation

According to Nexos, threats arise from developers using unapproved cloud models, thus risking exposing proprietary code and training on poisoned datasets. This can mean retrieval-augmented generation (RAG) pipelines then process data that appears truthful but is actually malicious.

Queries can also leak through large language model (LLM) logs and bypass access controls by retrieving documents users shouldn't see.

“Unvetted AI applications expand endpoint attack surfaces by creating invisible access points with unmanaged API tokens and unpredictable data pathways that bypass traditional controls,” Girėnas said.

In addition, shadow automation and scripts create security oversights when employees develop unauthorized workflows that operate with elevated privileges beyond IT oversight.

Girėnas explained that shadow automation introduces a control-level telemetry threat to organizations. Attackers exploit compromised shadow scripts through exfiltration, which both regularly and stealthily transmits data outside a user’s network, operating as “continuous background noise” that evades detection within conventional security frameworks.

“These scripts often contain hard-coded credentials, and establish persistent access channels that survive employee departures,” Girėnas added, who gave the recent example of a British Airways breach where attackers compromised marketing tool scripts to steal customer payment data directly from checkout pages.

The problem is compounded by fragmented security policies that occur as employees adopt AI tools faster than enterprises can govern them. Shadow APIs create compliance gaps, and third-party SaaS tools often have conflicting retention policies.

“We need to give people approved options that actually work better than what they're sneaking around to use,” Girėnas noted. “The risk amplifies with free-tier services, where prompts become retained training data, turning your proprietary information into third-party datasets without your knowledge.”

“People often create their own automations to save time or bypass slow processes, but these scripts can easily introduce vulnerabilities or disrupt operations,” Bance added. "The takeaway is that shadow automation isn’t just a technical nuisance, but also a business risk that can lead to lost revenue, reputational damage, and regulatory trouble."

Bad vibes and shadowy browsers

The threat can be exacerbated by use of model context protocol (MCP), the burgeoning standard for AI applications to "communicate" with another.

As Dharmapurikar explained, desktop AI clients such as Claude and ChatGPT now support sophisticated app integrations through MCP, meaning employees routinely use MCP-enabled applications in their daily workflows.

“While MCP creates powerful integration possibilities, it also introduces significant security risks," Dharmapurikar noted. "Consider this scenario: a company provides ChatGPT Enterprise access with contractual guarantees that employee data won't be used for telemetry or training purposes. However, third-party MCP servers have no obligation to honor these agreements.

“These external MCP servers might provide services like Excel analytics, data visualization, or custom reporting, and they operate under their own data policies.”

Dharmapurikar added the risk increases further when employees create custom MCP integrations using development tools such as Claude or Cursor to build their own scripts and semi-automated workflows without going through IT approval.

A double threat is when MCP is used as part of so-called vibe coding, where coding is practised by workers without formal oversight. Dharmapurikar said vibe coding on its own can lead to charts or insights generated with incomplete lineage or metadata, and then acted upon without verification.

“Unlike traditional shadow IT, vibe coding can embed inaccuracies deep into decision-making workflows. These AI-generated applications may appear to work correctly but contain significant flaws or vulnerabilities that non-technical users cannot identify,” Dharmapurikar explained.

Dharmapurikar also pointed to the growing prevalence of AI-driven browsers such as Perplexity’s Comet and Atlas from OpenAI. In the same way browser extensions can already intercept data before it leaves the device, AI browsers function as autonomous agents or plugins that provide API access to various LLM endpoints.

“Each endpoint has different data logging practices and training policies, making it increasingly difficult to maintain secure, locked-down systems for employees,” Dharmapurikar noted, and advised that educating employees is the first step in combating the shadow AI threat.

Shadowboxing shadow AI

For Bance, the next stage in the shadow fight is blocking or restricting access to unapproved APIs and browser extensions at the network and endpoint level.

“We need to enforce strict access controls and regularly audit what tools are being used across our teams," Bance said. “If we don’t know what’s running, we cannot protect our data.”

Nexos’ Girėnas recommended implementing data loss prevention (DLP) to block sensitive data transmissions in real-time, as well as network monitoring to identify shadow API usage by tracking unknown endpoints.

“For browser extensions, establish centralized whitelisting policies," Girėnas noted. “You should also adopt zero-trust architecture, treating all AI API calls as suspicious until verified, with continuous monitoring for anomalous data patterns. Research shows that blocking alone is less effective than making approved tools easier to use.”

Girėnas also said organizations need unified policy management that works consistently across APIs, browser extensions, and third-party services in place of fragmented point solutions for individual tools.

“Instead of playing whack-a-mole with shadow AI detection, security teams need comprehensive visibility into the complete AI workflow with policies that prevent data exfiltration by design, not after the fact,” Girėnas said, before also pointing out that some sanctioned AI tools can significantly reduce shadow usage.

With fire fighting fire, therefore, organizations may able to shine a strong light on the blight of shadow AI.