The question is usually asked as a model comparison, and that framing is the problem. Every current frontier assistant explains TLS session resumption, Kerberos delegation, or the difference between precision and recall well enough that the model is not what limits you. What limits you is whether the thing is producing answers or producing practice.
Those two modes feel almost identical while you are in them. Only one of them survives contact with an actual investigation.
What They Are Genuinely Good At
Four tasks where an LLM beats the alternatives for a learner:
- Decoding something you pasted. A base64 blob, an obfuscated PowerShell one-liner, a packet capture summary, a stack trace. The model reads it faster than you do and explains the parts you did not recognize. This is real value and it is low risk, because you have the artifact in front of you to check against.
- Turning a reference page into a lab plan. Give it a MITRE ATT&CK technique page and ask what telemetry would show that technique firing in a Windows environment, then go generate that telemetry yourself.
- Reviewing code you already wrote. Paste your detection logic and ask what input breaks it. Models are better critics than authors, and the critique keeps you as the person who wrote the thing.
- Generating practice data. Synthetic log lines are perfect for drilling parsing and feature engineering, because you control the ground truth and can check your work exactly.
The common thread: in all four, you supply the artifact and the model supplies the commentary. Reverse that and the value inverts too.
Where It Will Confidently Teach You Wrong Things
Language models generate the most plausible next token, and a well-formed identifier is extremely plausible. That is why the failure mode is not gibberish, it is a technique ID, CVE number, or function name that looks exactly right and does not exist.
Three that show up constantly:
Invented sub-technique IDs. T1059.003 is Windows Command Shell. The model will just as fluently produce T1059.009 for something it made up. Check it:
curl -s -o /dev/null -w '%{http_code}\n' https://attack.mitre.org/techniques/T1059/003/
# 200 means the technique exists. Anything else means you were handed fiction.Deprecated command syntax. Ask about memory forensics and you will often get Volatility 2 invocations (volatility -f mem.raw --profile=Win7SP1x64 pslist) because a decade of blog posts used that form. Volatility 3 dropped profiles entirely: vol.py -f mem.raw windows.pslist. The model is reproducing the corpus, not the current tool.
API calls that do not exist. scikit-learn has no precision_at_k, but it is such a natural name that models emit it regularly. One line settles it:
import sklearn.metrics as m
print([n for n in dir(m) if "precision" in n])Conceptual explanation is usually sound. Any specific string is unverified until you look it up. Hold that line and most of the risk disappears.
The Loop That Works
Invert the default. Instead of asking the model to explain, make it test you.
You are quizzing me on MITRE ATT&CK T1557, Adversary-in-the-Middle.
Ask one question at a time. Wait for my answer before responding.
Do not give me the answer, even if I ask. Tell me only whether I am
right, and if I am wrong, ask a narrower question that exposes the gap.That prompt is worth more than any model upgrade, because it forces retrieval instead of recognition. Recognition is the trap in AI-assisted study: a good explanation produces a strong feeling of understanding that does not survive being asked the same question cold, two weeks later, in a ticket.
Then build the thing. Pull real telemetry into a notebook, write the detection, and hand the model your code rather than your question. Run the model locally through Ollama if the data is sensitive:
ollama pull llama3.2:3b
ollama run llama3.2:3b "Critique this Sigma rule for false positives: ..."A 3B model is a weaker explainer than a hosted frontier assistant. It is also offline, free per token, and allowed to see logs you could never paste into a browser.
What It Cannot Give You
Reps under constraint. The lab that fails because the kernel version is unsupported, the parser that dies on a line the documentation swore could not exist, the detection that looked clean until it fired forty times on a backup window. The model describes the clean path, because the clean path is what got written down.
The other missing piece is somebody telling you your framing is wrong, not your answer. Models are agreeable by construction. A student who has quietly misunderstood what a base rate does to precision will get fluent, confident, useless agreement for months.
That gap is why roughly half the time in our classes is lab work, and why students leave with code that runs in their own environment rather than a transcript of a conversation. Use the model for the reading and the drilling. Come to a hands-on course for the part where things break.