The skill gap between running queries and building detections
Most SOC analysts can write a SIEM query. Fewer can take a raw log source, decide what in it actually separates malicious from benign, and turn that into a detection that holds up. That gap is data science, and it is the difference between a Tier 1 ticket-closer and a detection engineer.
You do not need a statistics degree. You need a specific, small set of skills you can practice against your own logs. Here is the checklist we teach analysts, roughly in the order they pay off.
The five skills, in order
1. Pandas for tabular data. Almost everything in a SOC is a table: auth logs, DNS records, process events, netflow. Pandas lets you load a few million rows, filter, group, and pivot without exporting to Excel and giving up at the row limit. If you can do a groupby on source IP and count failed logins, you can already spot password spraying that a dashboard buried. Start here.
2. Feature engineering. A model, or even a good heuristic, only sees what you give it. Raw fields rarely separate attacker from user. Derived fields do: logins per hour, entropy of a domain name, ratio of bytes sent to received, count of distinct destinations per host. This is where detection work lives, and it is the hardest skill to fake. Our feature engineering walkthrough goes deeper on the security-specific version.
3. Basic classification. You do not need deep learning. A decision tree or logistic regression trained on labeled phishing versus legitimate email will teach you more about your data than a black-box vendor model. The value is not the model. It is that building one forces you to define your labels, pick features, and confront how imbalanced your data is. See our ML phishing detection pipeline for a concrete build.
4. Anomaly detection. Not every problem has labels. When you do not know what the attack looks like, you model normal and flag deviation. Isolation Forest and simple statistical baselines cover most SOC cases. The trap is treating every outlier as an alert, so learn to tune and to expect noise. We cover the practical version in anomaly detection for security operations.
5. Reading a model’s output critically. This is the skill most analysts skip and most need. A model that flags 10,000 events at 99 percent accuracy on a base rate of one attack in a million is worthless, and you have to be able to say why. Precision, recall, and base rate are not academic. They decide whether your detection drowns the queue. Analysts who understand this cut noise instead of adding to it, which is the same discipline behind reducing false positives with machine learning.
Where analysts get stuck
The common failure is not math. It is starting with a model instead of the data. Analysts jump to scikit-learn before they can reliably load and clean their own logs, then blame the algorithm when the results are garbage. Spend your first month in Pandas, on real data from your environment, and the rest gets easier.
The second failure is working alone from blog posts. These skills compound faster with feedback, working through security-specific problems with people who have built detections that shipped. That is the whole point of hands-on training rather than a video course.
If you want the structured version of this checklist, worked against real security datasets, that is what the Applied Data Science & AI for Cybersecurity course is built around. We run it at Black Hat USA 2026, and it assumes no prior data science, just curiosity and a laptop.
The course runs as two 2-day sessions at Black Hat USA 2026, August 1 to 4 in Las Vegas: reserve a seat here.