LLM Pentesting

You are on day three of an authorised penetration test for Hartwell, a B2B SaaS company whose employee portal includes an internal AI assistant. Your scope explicitly covers all application components, including AI services. Standard web testing found the expected surfaces. Then your port scanner returned a different result.

LLMs do not behave like the web targets you already know. A SQLi payload will not touch them. A directory brute-force will not reveal their attack surface. Their vulnerability surface sits one layer deeper, in the natural language they process and the configuration they carry.

The attack surface is not just the chat window. Every layer of a deployed LLM is a separate entry point.

Researchers at Wiz discovered more than 1,000 Ollama instances (opens in new tab) exposed on the public internet with no authentication. The same default deployments that make LLMs easy to run also make them easy to target. In December 2023, a Chevrolet dealership in California deployed a customer-facing AI chatbot (opens in new tab)that, via prompt injection, was redirected to agree to sell a new vehicle for $1 and to promise to help users commit crimes. These are not theoretical risks.

This room covers the three stages of attacking an LLM on a live engagement: finding and fingerprinting the service, extracting its hidden configuration, and exploiting it through injection and jailbreak techniques. By the end of this room, you will have run the full LLM attack chain against a real target, on an authorised engagement.

Learning Objectives

• Find and fingerprint LLM services on a target machine
• Extract a deployed LLM's system prompt via direct and indirect techniques
• Execute direct and indirect prompt injection attacks
• Apply jailbreaking techniques to bypass safety training
• Use automated LLM scanning tools on an engagement
• Document LLM findings against the OWASP LLM Top 10 (2025)

Prerequisites

• HTTP and web fundamentals: Web Application Basics
• API concepts and basic web pentesting: Web Application Pentesting
• No prior AI or machine learning knowledge required