Data Poisoning in RAG Systems

Large Language Models learn how to behave from the data they are trained on and the data they continue to consume over time. Every pattern, association, and assumption the model uses originates from this data. If the data is manipulated, the model's behaviour changes, even if no one ever interacts with it directly. This type of attack is known as data poisoning or model poisoning. Instead of targeting prompts or users, the attacker targets the information the model learns from. These attacks are categorised under OWASP LLM04 and focus on influencing the system before it is queried.

Poisoning is fundamentally different from prompt injection or excessive agency. Prompt-based attacks manipulate instructions at inference time. Poisoning attacks work upstream, shaping how the model understands information long before any prompt is processed.

This room will take you through training data poisoning, embedding and corpus poisoning, ingestion pipeline attacks, how these manipulations change model behaviour, and the layered detection strategies used to counter them.

Learning Objectives

By completing this room, you will be able to:

• Clearly understand what poisoning attacks are
• Recognise poisoning as an attack class, not a misconfiguration
• Understand why poisoning targets data, embeddings, and models instead of prompts
• Prepare to explore specific poisoning techniques in later tasks

Prerequisites

Before starting this room, you should:

• Have basic familiarity with LLMs and how they generate responses
• Understand that some LLM systems retrieve and learn from external data
• Have completed the RAG Security Fundamentals room for broader AI security context (recommended, not required)

No machine learning or data science background is required.