AI Red-Teaming

Coverage includes adversarial prompt engineering and prompt injection, LLM robustness evaluation, testing for bias and hallucination, data exfiltration via model outputs, and building repeatable red-team frameworks for AI deployments. Compliance with emerging AI security standards is also addressed.

Experience with cybersecurity testing methodologies is required. You do not need a machine learning or data science background. The course teaches the AI-specific layer (LLM failure modes, adversarial ML, model evaluation) on top of the security testing skills you already have.

The course uses Python and Jupyter for scripting attacks and analyzing model responses, running in a Centaur VM lab environment. Students work through adversarial scenarios hands-on rather than just reading about techniques.

Testing involves probing whether the model reveals information from its training data, system prompt, or connected data sources through targeted prompting. This includes direct extraction attempts, indirect approaches that build context across multiple conversation turns, and evaluating whether safety controls hold under sustained adversarial pressure. Automated scripts help scale coverage across attack variants.

Alignment failures occur when the model behaves in ways that contradict its intended purpose even without adversarial input. Safety boundary violations involve deliberately pushing the model past its guardrails through jailbreaking or prompt injection. Both are in scope for a thorough red-team engagement but require different testing approaches and have different root causes.