# AI Model Security Training: What a Platform Must Teach

By Curtis Lambert · 2026-08-26

> Most AI model security training stops at prompt injection. A checkpoint file is executable code, and here is the curriculum that has to cover it.

A PyTorch checkpoint is not data. It is a program, and `torch.load` is the interpreter.

That sentence is the entire subject, and most training marketed as AI model security never gets to it. The syllabus goes prompt injection, jailbreaks, maybe a RAG poisoning lab, and stops. Those attacks target a model's behavior. None of them address the more basic question of whether the file you loaded onto a GPU box with cloud credentials attached was doing something other than defining tensors.

## Start With the Vulnerability Class

The failure mode is CWE-502, deserialization of untrusted data, applied to machine learning artifacts. A `.pt` or `.bin` checkpoint is a zip archive with a Python pickle inside, and unpickling executes opcodes that can import and call arbitrary functions.

[CVE-2025-24357](/cve/CVE-2025-24357) is the clean teaching example. vLLM's `hf_model_weights_iterator` in `weight_utils.py` loaded checkpoints downloaded from a model hub using `torch.load`, with `weights_only` left at its default of `False`. A malicious checkpoint got code execution on the inference host. CVSS 7.5, fixed in v0.7.0. It was not an exotic bug. It was one keyword argument.

```python
import torch

# Runs the pickle VM. Arbitrary code, on the box holding your model weights.
state = torch.load("downloaded.bin")

# Tensors only. No callable imports, no REDUCE opcode.
state = torch.load("downloaded.bin", weights_only=True)
```

PyTorch flipped that default to `True` in 2.6, which protects teams that upgraded and does nothing for the pinned 2.3 environment running in production. The same class reaches further up the stack: [CVE-2024-11393](/cve/CVE-2024-11393) is a deserialization RCE in Hugging Face Transformers reached through MaskFormer model file parsing, CVSS 8.8, reported through the Zero Day Initiative as ZDI-24-1514.

Training that teaches this well spends its time on the inspection step, not the vulnerability trivia:

```bash
# A .pt/.bin checkpoint is a zip archive. The pickle lives inside it.
unzip -o model.bin -d unpacked/
python -m pickletools unpacked/*/data.pkl | grep -E "GLOBAL|STACK_GLOBAL|REDUCE"
```

A checkpoint that only defines tensors has no reason to import `posix` or `builtins.eval`. `GLOBAL` paired with `REDUCE` is a callable being resolved and invoked during load, and seeing that output once teaches more than an hour of slides.

## Map Findings to a Taxonomy or Nobody Acts on Them

"We downloaded a sketchy model" is not a finding a security organization can route. The same observation expressed as [AML.T0010](/atlas/AML.T0010), AI Supply Chain Compromise, with the [Model](/atlas/AML.T0010.003) sub-technique, is initial access with an ID, an owner, and a place in a report. Malicious code inside the artifact is [AML.T0018.002](/atlas/AML.T0018.002), Embed Malware, under Manipulate AI Model.

We teach adversarial attacks against models inside [Applied Data Science and AI for Cybersecurity](/courses/applied-data-science-ai), and the taxonomy mapping travels with the technique for exactly this reason. A red team that reports in [MITRE ATLAS](https://atlas.mitre.org/) IDs gets remediation. A red team that reports in prose gets a thread nobody closes.

## The Four Blocks a Curriculum Needs

- **Artifact triage.** Which formats execute on load (pickle, `.pt`, `.bin`, joblib, Keras H5 with Lambda layers) and which do not ([safetensors](https://github.com/huggingface/safetensors), GGUF, ONNX with care). Hands-on inspection with `pickletools`, [fickling](https://github.com/trailofbits/fickling), and [modelscan](https://github.com/protectai/modelscan).
- **Provenance.** Hash and sign what you promote, mirror approved models into an internal registry, and pin by digest rather than by tag. A deploy step that pulls `latest` from a public hub is an unauthenticated code path into production. Defense contractors already run this program for software and can usually extend it to weights, which we wrote about in the context of [defense industrial base teams](/blog/ai-security-training-defense-industrial-base).
- **Containment.** Assume the load executes. Inference workers run non-root, without cloud instance credentials, with egress restricted to the endpoints they need. This is ordinary infrastructure hardening and it is the control that survives a scanner miss.
- **Detection.** What the load looks like in telemetry: a Python process spawning a shell or resolving an unexpected domain shortly after a model file lands on disk. Sysmon Event ID 1 with parent-child rarity scoring, and outbound connections on Event ID 3, both mapped to [T1059](https://attack.mitre.org/techniques/T1059/).

The detection block is the one platforms skip, and it is the one that matters most to a SOC. Attacking a model is a red-team skill. Noticing that someone attacked yours is a detection engineering skill, and they are taught by different people.

## What Scanning Will Not Fix

Pickle scanners are heuristic. Opcode allowlists get defeated by indirection, and a determined author can express a payload in ways a static pass does not flag. Anyone selling a scanner as the answer is selling the wrong thing. The durable fixes are format migration and provenance, both of which are engineering programs rather than course modules.

This training also does not help much if you consume models only through a hosted API. Then the artifact risk belongs to the provider, and your work is procurement: ask how they verify weights, and move on to the application layer where your actual exposure lives.

And it does not cover the behavioral attacks. Those are a separate discipline with separate labs, covered in [what AI red-teaming actually involves](/blog/what-is-ai-red-teaming) and [RAG poisoning and jailbreaking](/blog/rag-poisoning-llm-jailbreaking).

## Testing the Claim

Before buying, ask for one thing: a lab that hands you a malicious checkpoint and requires you to catch it before it loads. A platform that has built that lab has thought about this subject. A platform that offers a video module titled "Model Security" and a quiz has not.

Two follow-ups worth asking. Do the labs run with the network cable pulled, since an exercise that reaches a public model hub dies on a managed corporate laptop. And does the curriculum end at findings or continue into detections, because a team that can only attack leaves the SOC exactly where it started.

Our own take on evaluating this category is on the [AI-powered security training platforms](/lp/ai-powered-security-training-platforms) page, and the adversarial half of the work is the subject of the [AI Red-Teaming](/courses/ai-red-teaming) course.

## FAQ

### What should AI model security training cover?

Four blocks. Artifact triage: what a checkpoint file actually is, which formats can execute code on load, and how to inspect one before it reaches a GPU. Provenance and integrity: hashing, signing, and mirroring models into an internal registry instead of pulling from a public hub at deploy time. Runtime containment: treating every model load as untrusted code execution. And detection engineering: what a malicious load looks like in process telemetry. Prompt injection is a real subject, and it is a different layer.

### Are AI model files really executable code?

Some formats are. A PyTorch .pt or .bin checkpoint is a zip archive containing a Python pickle, and unpickling runs opcodes that can import and call arbitrary functions. That is CWE-502, deserialization of untrusted data. CVE-2025-24357 is a working example: vLLM called torch.load with weights_only defaulting to False, so a malicious checkpoint from a model hub achieved code execution. Safetensors and GGUF carry tensors without a code path, which is why format migration is the durable fix.

### How do you evaluate an AI model security training platform?

Ask for a lab where you are handed a deliberately malicious checkpoint and have to catch it before loading it. Ask whether the curriculum maps findings to MITRE ATLAS technique IDs, because that is what makes a finding reportable inside a security organization. Ask whether the labs run with no network access, since anything touching a public model hub will break on a locked-down enterprise laptop. If the syllabus is entirely prompt injection and jailbreak exercises, it teaches AI application security, not model security.

### What tools scan a model file for malicious code?

Fickling from Trail of Bits performs static analysis on pickle files and can decompile the embedded bytecode. ModelScan checks common serialization formats for unsafe operations. Python's own pickletools module disassembles opcodes with no third-party dependency, which is enough to spot GLOBAL and REDUCE pairs indicating a callable import. All three are heuristic, so treat them as triage rather than proof.

### Is prompt injection testing the same as AI model security?

No, and conflating them is the most common gap in AI security curricula. Prompt injection attacks the behavior of a deployed model through its inputs. Model security concerns the artifact and its supply chain: whether the weights you loaded are the weights the publisher built, and whether loading them executed something. A team can be excellent at one and blind to the other. Most training sold today covers the first.


---

Canonical: https://gtkcyber.com/blog/ai-model-security-training/