A risk management framework and a Python scanner keep turning up in the same bullet list, as though NIST AI 100-1 and garak were alternatives to each other. They are not. One is something you cite in a board deck. The other is something you run on a Tuesday.
That flattening is what makes most AI red teaming resource roundups useless. Here is the split worth keeping, and where each layer stops helping.
The Taxonomies You Cite in a Finding
This is the layer that does real work, because it turns “the chatbot misbehaved” into something a defender can route and fix.
MITRE ATLAS is the adversarial AI counterpart to ATT&CK, and it is specific enough to carry a report. The techniques that come up on nearly every LLM engagement:
- AML.T0051 LLM Prompt Injection, split into
.000Direct,.001Indirect, and.002Triggered. The sub-technique is the interesting part, since indirect injection through retrieved content is a different fix from a user typing a jailbreak. - AML.T0054 LLM Jailbreak and AML.T0056 Extract LLM System Prompt.
- AML.T0057 LLM Data Leakage and AML.T0024 Exfiltration via AI Inference API.
- AML.T0053 AI Agent Tool Invocation, which is where the impact usually lives once an application can call tools.
If the target is a classifier rather than a language model, the relevant entries are different: AML.T0015 Evade AI Model, AML.T0043 Craft Adversarial Data with its white-box, black-box, transfer, and manual variants, and AML.T0020 Poison Training Data.
The OWASP Top 10 for LLM Applications covers similar ground from the application side and is the better reference when your audience is an application security team rather than a detection team. Use both. They are not competing standards, and quoting an OWASP category next to an ATLAS ID costs you nothing.
The Tools That Run the Tests
Three tools cover most of the practical surface, and they are not interchangeable.
garak is a scanner. It ships probe families that line up with the taxonomy above, and it is the correct first pass because it is cheap to run and produces a report you can diff:
python -m garak --model_type ollama --model_name llama3.2:3b \
--probes promptinject,dan,leakreplay --report_prefix baselineRun that against a local model first, using Ollama, so you learn the tool’s output format without burning API spend or tripping someone’s abuse detection.
PyRIT picks up where a scanner stops. It is an orchestration library, so it handles attacks that carry state: multi-turn conversations, an attacker model generating the next prompt from the last response, and scoring logic you define. Anything that depends on conversation history needs this rather than a probe list.
promptfoo is the one to put in CI, because assertions live in YAML next to the application code and fail a build like any other test. This is where a red team finding becomes a regression test instead of a PDF.
For classifiers, none of the above applies and you want the Adversarial Robustness Toolbox, which implements the evasion and poisoning attacks from the research literature against scikit-learn, PyTorch, and TensorFlow models directly.
The step teams skip is the boring one: pin versions and keep the raw output. Probe sets change between releases, so a scan that got cleaner may reflect a changed probe rather than a fixed application. Store the report, the tool version, and the model version together, or the second scan means nothing.
The Governance Documents, and What They Are For
These do not help you test anything. They help you show that testing is part of a program, which is a real requirement and a different job.
The NIST AI Risk Management Framework (AI 100-1) is voluntary and organizes work into Govern, Map, Measure, and Manage. Its Generative AI Profile (NIST AI 600-1) is the more useful companion, since it enumerates risks specific to generative systems rather than AI in general. ISO/IEC 42001 is the certifiable AI management system standard, which matters when a customer contract asks for a certificate rather than a policy.
Read them once, map your existing test plan onto them, and get back to work. A team that spends a quarter on framework alignment before running a single probe has the order backwards.
What None of It Covers
The gap in 2026 is severity. Application security has CVSS and a CVE identifier, so a finding arrives pre-anchored. Adversarial AI has neither, which is why an engineering team can wave off a jailbreak as a curiosity. A prompt injection that causes an agent to invoke a tool with the user’s credentials and a jailbreak that produces rude text land in the same bucket unless you write the impact in terms of what the application actually did.
Agentic behavior is where this bites hardest, and the tooling is furthest behind there. Scanners test a model endpoint. They do not test the loop where a model reads a document, decides to call a tool, and feeds the result back into its own context. Testing that surface is still mostly manual, and it is the surface that carries real consequences.
We teach AI red-teaming as a two-day advanced course, and it requires security testing experience while explicitly not requiring an ML background, because the skills transfer better in that direction than the other way around. Knowing how to build a payload set and write a reproducible finding is the harder half. The AI red-teaming course covers the tooling and the reporting discipline together, and the technique-level walkthrough is in how to red team an LLM-powered application.