For most of the history of enterprise software, security teams could rely on a basic distinction: code is instructions, and data is data. A customer record in a database does not tell the application what to do. Large language models erase that distinction. A model reads instructions and content in the same stream of text, and it can be persuaded by either. An email, a web page or a contract clause can carry instructions that the model follows.
That single property explains why AI security for enterprise systems is a different discipline from application security, even though it borrows many of its tools. It is why prompt injection exists at all, why AI assistants leak data in ways that firewalls do not see, and why AI agents with access to business systems create a new class of insider risk.
This piece sets out what the evidence says about AI security incidents, explains the three threats that dominate enterprise exposure (prompt injection, data leakage and agent abuse), and offers a threat matrix that maps each entry in the OWASP Top 10 for LLM Applications 2025 to enterprise scenarios, control layers and owners. The aim is to give security leaders and risk committees a common map, not a catalogue of fears.
Why AI security for enterprise teams is a different problem
The clearest statement of the core problem comes from a 2023 paper by Kai Greshake and colleagues, "Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection." The authors argued that LLM-integrated applications "blur the line between data and instructions", and showed that attackers could compromise such applications remotely, without any direct interface, by "strategically injecting prompts into data likely to be retrieved." They demonstrated attacks on production systems, including a GPT-4 powered search chat, and catalogued impacts including data theft and contamination of information. Their abstract concluded that "effective mitigations of these emerging threats are currently lacking."
Three years on, defences have improved but the property remains. No model reliably distinguishes a legitimate instruction from a malicious one embedded in content. The OWASP guidance on prompt injection is explicit that no prevention method is fully reliable and that mitigations aim to reduce impact. That changes the security posture required. Instead of trying only to keep bad input out, enterprises must design systems so that a model which has been manipulated cannot do much harm.
What the evidence says about incidents
Systematic incident data on AI systems is still sparse, but IBM's Cost of a Data Breach Report 2025, based on research by the Ponemon Institute across 600 organisations globally between March 2024 and February 2025, gives the most cited picture.
According to IBM's July 2025 release, 13% of organisations reported breaches of AI models or applications, and a further 8% did not know whether they had been compromised in this way. Among the organisations that reported such breaches, 97% said they did not have AI access controls in place. Of the AI-related security incidents, 60% led to compromised data and 31% to operational disruption.
The report also quantified shadow AI, the use of AI tools not sanctioned by the organisation. One in five organisations reported a breach due to shadow AI, and organisations with high levels of shadow AI faced breach costs about USD 670,000 higher than those with low levels or none. Governance lagged: 63% of breached organisations either had no AI governance policy or were still developing one, and among those with policies, only 34% performed regular audits for unsanctioned AI. For context, the report put the global average cost of a data breach at USD 4.44 million.
These figures describe a global sample and self-reported incidents, so they should be read as indicative. But the pattern is consistent with the threat model: most AI incidents are failures of access control and governance, not exotic attacks on model weights.
The reference frameworks
Two frameworks give enterprises a shared vocabulary.
The OWASP Top 10 for LLM Applications 2025, published by the OWASP GenAI Security Project, lists the ten most important risks for applications built on language models: prompt injection (LLM01), sensitive information disclosure (LLM02), supply chain (LLM03), data and model poisoning (LLM04), improper output handling (LLM05), excessive agency (LLM06), system prompt leakage (LLM07), vector and embedding weaknesses (LLM08), misinformation (LLM09) and unbounded consumption (LLM10). It is written for builders and is the most practical starting point for application teams.
NIST AI 100-2 E2025, "Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations", published in March 2025, takes a broader view. It defines terminology for attacks on machine learning systems across types of methods, stages of the life cycle, and attacker goals, capabilities and knowledge, and describes mitigation approaches. It is more useful for security architects and for organisations building or fine-tuning their own models.
Neither framework tells an enterprise which risks matter most for its own systems or who should own each control. The threat matrix later in this piece is designed to fill that gap.
Prompt injection: direct and indirect
OWASP distinguishes two forms. Direct prompt injection occurs when a user's own input alters the model's behaviour, for example by instructing an assistant to ignore its rules. Indirect prompt injection occurs when the model accepts input from external sources, such as websites or files, that contain instructions.
For enterprises, indirect injection is the larger risk because it scales with integration. Every document an assistant can retrieve, every email an agent can read and every web page a research tool can browse is a potential carrier. A supplier invoice containing hidden text, a CV with white-on-white instructions, or a shared document edited by an outsider can redirect a model that processes it.
OWASP's mitigations reflect the fact that prevention is partial. They include constraining model behaviour through clearly defined roles, validating output formats with deterministic code, filtering inputs and outputs, enforcing least privilege so that the model's access is restricted "to the minimum necessary for its intended operations", requiring human approval for privileged actions, segregating and clearly marking untrusted content, and running regular adversarial testing that treats the model as an untrusted user.
The last of these is the most important mindset shift. A model that processes untrusted content should be treated, for security purposes, as if it were an untrusted user.
Data leakage: four routes out
Data leakage in AI systems rarely looks like a classic breach. It follows four main routes.
Into external models. Employees paste confidential material into public AI tools, or applications send prompts containing personal data to external model providers without the controls the data requires. IBM's shadow AI findings describe this route.
Out through outputs. A model reveals information it should not, whether from its training data, from the context it was given or from its system prompt. OWASP lists sensitive information disclosure (LLM02) and system prompt leakage (LLM07) as separate risks.
Through retrieval. Retrieval-augmented systems can surface documents a user is not entitled to see. OWASP's entry on vector and embedding weaknesses (LLM08) warns that "inadequate or misaligned access controls can lead to unauthorized access to embeddings containing sensitive information" and recommends permission-aware vector stores. A separate PanScience Research piece on why enterprise RAG fails in production covers this route in detail.
Through logs and caches. Prompts and responses captured for debugging, evaluation or audit are copies of sensitive data, often held with weaker access controls than the source systems.
Agent abuse: when the model can act
The risk profile changes again when a model can take actions: send emails, update records, approve transactions or run code. OWASP's entry on excessive agency (LLM06) identifies three root causes: excessive functionality, where an agent has tools with more capabilities than its task needs; excessive permissions, where those tools reach downstream systems with more access than required; and excessive autonomy, where high-impact actions proceed without independent verification or approval.
Its recommended controls are those of good access management applied to a new kind of actor. Limit the tools an agent can call to the minimum necessary. Prefer narrow tools over open-ended ones such as general shell access. Limit each tool's permissions so that a read-only task cannot modify data. Have agents act in the context of the specific user, with least-privilege scopes. Require human approval before high-impact actions. And, critically, "implement authorization in downstream systems rather than relying on an LLM to decide if an action is allowed or not."
McKinsey's State of AI 2025 survey found that 23% of organisations were scaling an agentic AI system somewhere in the enterprise and 39% were experimenting. As agents move into finance, procurement and customer operations, the agent becomes a privileged identity, and its permissions need the same review as any service account. A separate PanScience Research piece covers AI agents in the enterprise.
A threat matrix mapped to the OWASP Top 10 for LLM applications
The matrix below is PanScience Research's mapping of each OWASP entry to a representative enterprise scenario, the system types most exposed, the primary control, the layer where that control sits and the function that should own it. Layers are: the gateway (the shared layer between applications and models), the application, the data layer (indexes, documents, training data) and downstream systems the AI can act on. Exposure ratings are qualitative judgements for a typical enterprise, not measured frequencies.
A second view shows how exposure concentrates by system type. Ratings are High (H), Medium (M) or Low (L), again as qualitative judgements.
Three conclusions follow from the matrix. First, exposure is driven by integration, not by the model. An internal chat assistant with no data access is a modest risk; the same model connected to documents and tools is not. Second, agents concentrate risk: they rate high on more entries than any other system type, which argues for treating agent deployment as a separate approval decision. Third, most primary controls are familiar security disciplines, including access control, input validation, data classification and quota management, applied at a new layer. Very few require exotic AI-specific technology.
Controls by layer, and the role of the gateway
The matrix shows that no single layer can carry AI security. The gateway, the shared layer through which model calls pass, is the natural home for controls that must be consistent across applications: personal data detection before prompts leave the organisation, output filtering, logging and quotas. A separate PanScience Research piece on the LLM gateway sets out what that layer must do. PanScience's portfolio includes Proteus, which describes itself as an LLM gateway that routes, secures and monitors model traffic (company-reported).
But the gateway cannot see whether an agent should be allowed to approve a payment, or whether a document in an index carries the right access labels. Those controls belong in downstream systems and in the data layer. The practical design principle is defence in depth: assume that the model will at some point be manipulated, and make sure that the gateway, the application, the data layer and downstream authorisation each limit what a manipulated model can do.
The Indian regulatory context
For Indian enterprises, AI security incidents that involve personal data fall within the Digital Personal Data Protection Act, 2023. DLA Piper's guide to Indian data protection law notes that a personal data breach must be reported to the Data Protection Board within 72 hours of the fiduciary becoming aware of it, unless the Board allows longer, and that civil penalties can reach ₹250 crore. A RAG assistant that exposes personal data to unauthorised employees, or an agent that sends customer data to the wrong recipient, may be a reportable breach.
In financial services, the Reserve Bank of India's FREE-AI committee report of August 2025 sets out seven principles, including Resilience and Accountability, and 26 recommendations under six pillars that include Protection and Assurance. The report's recommendations are not yet binding regulation, but they indicate the direction of supervisory expectations. A separate PanScience Research piece covers AI governance for boards.
Where to start
For a security leader beginning an AI security programme, a short sequence follows from the evidence and the matrix:
1. Inventory. List AI systems in use, sanctioned and unsanctioned, with their data access and tool access. IBM's shadow AI findings suggest the unsanctioned list will be longer than expected. 2. Classify by integration. Use the system-type view of the matrix to rank systems by exposure. Agents and enterprise RAG come first. 3. Fix access control. Apply least privilege to agents and permission-aware retrieval to RAG. These address the control gap IBM's data points to most directly. 4. Centralise the shared controls. Route model traffic through a gateway that applies data detection, logging and quotas consistently. 5. Test adversarially. Run prompt injection and permission tests against each high-exposure system before launch and on a regular schedule, treating the model as an untrusted user.
Conclusion
AI security for enterprise systems begins with accepting a property of language models that will not be engineered away soon: they cannot reliably tell data from instructions. Prompt injection, data leakage and agent abuse all follow from that property, amplified by how deeply models are integrated into documents, workflows and business systems.
The response is less exotic than the threat sounds. The evidence shows that most incidents involve missing access controls and weak governance. The OWASP Top 10 for LLM Applications gives application teams a shared list; the threat matrix in this piece maps that list to the systems, layers and owners that enterprises actually manage. Organisations that treat models as untrusted users, give agents only the permissions their tasks require, and enforce authorisation outside the model will be secure enough to keep deploying.
Methodology and sources
Risk names and descriptions are from the OWASP Top 10 for LLM Applications 2025 (genai.owasp.org/llm-top-10), with detailed guidance taken from the entries for prompt injection (genai.owasp.org/llmrisk/llm01-prompt-injection), excessive agency (genai.owasp.org/llmrisk/llm062025-excessive-agency) and vector and embedding weaknesses (genai.owasp.org/llmrisk/llm082025-vector-and-embedding-weaknesses). The NIST taxonomy is NIST AI 100-2 E2025 (csrc.nist.gov/pubs/ai/100/2/e2025/final, March 2025). The indirect prompt injection research is Greshake and colleagues (arxiv.org/abs/2302.12173). Incident and cost figures are from IBM's Cost of a Data Breach Report 2025 press release (newsroom.ibm.com, 30 July 2025), based on Ponemon Institute research across 600 organisations. Agent adoption figures are from McKinsey, The state of AI in 2025 (mckinsey.com/capabilities/tech-and-ai/our-insights/the-state-of-ai-2025). DPDP breach notification and penalty provisions are from DLA Piper's Data Protection Laws of the World guide for India (dlapiperdataprotection.com/index.html?c=IN). The RBI FREE-AI summary is as verified by the editor from rbi.org.in and KPMG India. The scenarios, control layers, owners and exposure ratings in the threat matrix are PanScience Research's qualitative judgements for a typical enterprise and are illustrative, not measured frequencies. The Proteus description is company-reported from panscience.xyz.