Data Poisoning & Model Vulnerabilities: OWASP Top 10 for Large Language Applications

Overview #

A cybersecurity analysis of the OWASP Top 10 vulnerabilities for LLMs, including training data poisoning, insecure output handling, excessive agency, and denial-of-wallet attacks.

The OWASP Top 10 for LLM Applications #

The Open Worldwide Application Security Project (OWASP) identified the critical security threats unique to generative AI:

1. Prompt Injection: Manipulating instructions via user input.
2. Insecure Output Handling: Trusting LLM output without sanitization, leading to XSS or SQL injection.
3. Training Data Poisoning: Tampering with pre-training or fine-tuning datasets.
4. Model Denial of Service: Exhausting context windows to cause high bills and latency.
5. Excessive Agency: Granting autonomous agents dangerous permissions without human confirmation.

Insecure Output Handling: Never Trust LLM Generations #

A dangerous misconception is that LLM output is inherently safe. If an LLM generates a markdown response containing and your React app renders it using dangerouslySetInnerHTML, your users are vulnerable to Cross-Site Scripting (XSS). Always sanitize AI markdown with DOMPurify.

Denial-of-Wallet (DoW) & Resource Exhaustion #

Attackers submit prompts with 200,000 repetitive tokens or craft queries that trigger infinite agentic loops, draining enterprise API balances rapidly. Defense requires strict per-user rate limits and maximum prompt length validation at the gateway.

Code Example: Sanitizing AI Markdown Outputs Against XSS with DOMPurify #

import DOMPurify from "isomorphic-dompurify";
import { marked } from "marked";

function renderSafeAIMarkdown(rawLLMOutput: string): string {
  // 1. Convert markdown to HTML string
  const rawHtml = marked.parse(rawLLMOutput) as string;

  // 2. Sanitize HTML strictly before rendering
  const safeHtml = DOMPurify.sanitize(rawHtml, {
    ALLOWED_TAGS: ["p", "b", "i", "em", "strong", "a", "code", "pre", "ul", "ol", "li", "h1", "h2", "h3"],
    ALLOWED_ATTR: ["href", "target", "class"],
  });

  return safeHtml;
}

Frequently Asked Questions #

Q: What is Denial-of-Wallet in AI?

Denial-of-Wallet is a cyberattack where an adversary floods an AI API with computationally expensive prompts to bankrupt the victim organization through utility bills.

Q: How do I prevent Excessive Agency in AI agents?

Enforce human-in-the-loop approvals for destructive operations (e.g. sending emails, deleting records, executing payments).

Q: Can an LLM generate SQL injection attacks?

Yes, if an AI writes SQL queries that are directly concatenated into database drivers rather than using parameterized prepared statements.