Data Integrity & Model Poisoning

Modern AI systems depend heavily on the quality and trustworthiness of their data and model components. When attackers compromise training data or model parameters, they can inject hidden vulnerabilities, manipulate predictions, or bias outputs. In this room, you'll explore how these attacks work and how to detect and mitigate them using practical techniques.

Learning Objectives

• Understand how compromised datasets or model components can lead to security risks.
• Examine common ways adversaries use to introduce malicious inputs during training or fine-tuning. 
• Assess vulnerabilities in externally sourced datasets, pre-trained models, and third-party libraries.
• Practice model poisoning through the eyes of an attacker.

Prerequisites

Data integrity and model poisoning are specialised threats within the broader field of machine learning security. To get the most out of this room, you should have a foundational understanding of how machine learning models are trained and deployed, as well as the basics of data preprocessing and model evaluation. Additionally, you should be familiar with general security principles related to supply chain and input validation.

• AI/ML Security Threats
• Detecting Adversarial Attacks

Set up your virtual environment

To successfully complete this room, you'll need to set up your virtual environment. This involves starting the Target Machine, ensuring you're equipped with the necessary tools and access to tackle the challenges ahead.

Target machine

Status:Off

Start Machine

Please click the Start Machine button to boot up the VM. It will take approximately 3-4 minutes to load and warm up all the models. You will need the VM in Task 4, so by the time you reach that task, the LLM Lab page will be fully ready for use.