AI Security: What Often Goes Wrong

Many organizations have deployed some form of AI in recent years. However, relatively few have fully considered the implications for their security posture.

This is not about exotic attack scenarios. It is about fundamental questions that often remain unanswered: What data does the system have access to? Who authorized that access? What happens if the model behaves unexpectedly?

These questions are not inherently complex, but they are frequently overlooked when the primary focus is on getting a pilot into production.

Here is something worth sitting with. An AI system inherits the access rights of the user or service account it runs under. If that account can read the HR database, the AI can too. If it can write to production, so can the AI. At the speed of an API call, not at human speed.

That is not a flaw in the technology. It is how it works. The risk is that most deployments do not start from that fact and work outward. They start from the use case and add security later if at all.

A similar pattern was observed during early cloud adoption, where rapid implementation often relied on default configurations that later proved insufficient from a security perspective.

AI introduces additional layers of exposure beyond traditional threat vectors, including prompts and responses, plug-ins and functions, orchestration layers, retrieval-augmented data sources, and the models themselves. Each represents a potential entry point that did not previously exist. As a result, the key question shifts from “Is the network secure?” to “What happens if AI systems are directly targeted?”

OWASP published its Top 10 for LLM Applications in 2025. If you work with AI systems and have not read it, you should. The list is specific: prompt injection, sensitive information disclosure, system prompt leakage, excessive agency, data model poisoning.

Excessive agency is particularly notable. It describes situations where an AI system takes actions beyond its intended scope due to excessive permissions and insufficient constraints. This is primarily an architectural challenge that must be addressed during system design rather than after deployment.

A recurring pattern can be observed in AI implementations: a pilot succeeds, scaling decisions are made, and during the transition to production, previously identified security requirements are deferred.

The reasons are predictable: timeline pressure, unclear ownership, the assumption that security can be retrofitted once the system is stable. It rarely can be. The issues that get deferred – guardrails, vulnerability assessment, monitoring and logging, integration with core systems, regulatory compliance – are exactly the issues that create exposure once the system is live.

AI security cannot be separated from AI governance. Both are required, and they address different dimensions of the same risk.

Governance covers the legal and organizational layer: legal requirements, risk management, local data protection obligations, ethics and integrity, transparency and accountability. Security covers the technical layer: identity and access protection, data permissions, application protection, environment safety, device protection, secure processes and communication.

Neither is sufficient without the other. A technically secure system that operates without governance is still a liability. A well-governed system with security gaps is still vulnerable.

The EU AI Act has been in force since August 2024. Most organizations have not yet classified their AI use cases under it.

That matters because the numbers are not small. Violations of prohibited practices can reach €35 million or 7% of global annual turnover. Breaches of obligations for high-risk systems up to €15 million. Even misleading documentation alone up to €7.5 million.

For comparison: these exceed GDPR fines. And unlike GDPR, most organizations are starting from zero on this one.

High-risk systems under the Act include AI used in HR decisions, credit assessment, and critical infrastructure. A lot of companies are running exactly these use cases without realizing the compliance obligations that come with them. The requirements for high-risk systems apply fully from August 2027. That sounds distant. Building the documentation, processes and governance structures required does not happen quickly.

The starting point is an inventory. What AI systems are running in your organization, including the ones IT did not officially approve? What data do they access? What is their risk classification under the AI Act?

That is not glamorous work. But it is the foundation. You cannot govern what you have not mapped.

From there, security and governance need to be part of the architecture from the beginning – not reviewed at the end of a project. In practice this means involving security and legal early, defining data access boundaries before deployment, and building monitoring in from day one.

AI security is not an IT problem that security can solve alone. It requires alignment across C-level, domain experts, IT specialists, security, compliance and key stakeholders. Security and compliance at first and by design – not as an afterthought.

The organizations that get this right are not necessarily the most advanced technically. They are the ones that treat AI security as a cross-functional problem with clear ownership.

That is where most of the gaps are. Not in the technology.

Scale responsibly. If your organization is running AI in production and has not yet done this assessment, now is the right time.