AI red-teaming applies the traditional adversarial security mindset to AI systems. It covers prompt injection, jailbreaking, adversarial machine learning, model evasion, data poisoning, and the full range of ways AI systems fail under attack.
This is an active research area for GTK Cyber. Our work combines practitioner offensive security experience with deep ML understanding, which is the combination enterprises need when they deploy LLM-powered tools and AI agents.
Training
The AI Red-Teaming course covers adversarial testing of AI systems at a practitioner level. The AI Cyber Bootcamp includes AI red-teaming as part of a broader 4-day AI security curriculum.
Reading
Start here if you are new to AI red-teaming:
- What Is AI Red-Teaming? A Practical Introduction for Security Professionals
- AI Red-Teaming for Beginners: Where to Start and What to Test