How to Detect Ransomware with Machine Learning

By the time a classifier decides a binary is ransomware, the useful question is how many files you lost, not whether you caught it.

That framing is missing from most work on this problem. Papers and vendor datasheets report AUC and accuracy on static PE-file classification, which measures whether a model can tell a packed executable from Notepad. Operationally you are graded on a different number: files encrypted at the moment of detection. Optimizing the first number barely moves the second.

Set a detection budget before you pick a model

Splunk’s SURGe team benchmarked ten ransomware families encrypting the same file corpus and reported a median total encryption time of roughly 43 minutes, with the fastest family under six (the full comparison is worth reading). Against a six-minute family, a pipeline that batches telemetry every five minutes has already conceded most of the disk.

So write the budget down as an engineering requirement: telemetry ship time, plus aggregation window, plus inference, plus response action. Every architectural choice after this point is constrained by that sum. A model that needs ten minutes of behavior to reach confidence is not a detection, it is a post-incident report.

Features from behavior, not from the binary

Score processes over short windows, not individual events. A single FileCreate is meaningless; two hundred of them across ninety directories in forty seconds is not.

Sysmon gives you the raw material: event ID 1 (ProcessCreate), 11 (FileCreate), 23 (FileDelete archived), and 26 (FileDeleteDetected). The Sysmon documentation covers the config schema, and you will want to filter aggressively at the agent because event 11 is high volume by default.

import pandas as pd

fe = sysmon[sysmon['EventID'].isin([11, 23, 26])].copy()
fe['ts'] = pd.to_datetime(fe['UtcTime'])
fe['win'] = fe['ts'].dt.floor('60s')
fe['dir'] = fe['TargetFilename'].str.rsplit('\\', n=1).str[0]
fe['ext'] = fe['TargetFilename'].str.rsplit('.', n=1).str[-1].str.lower()

feat = fe.groupby(['ProcessGuid', 'win']).agg(
    ops=('TargetFilename', 'size'),
    files=('TargetFilename', 'nunique'),
    dirs=('dir', 'nunique'),
    exts=('ext', 'nunique'),
    deletes=('EventID', lambda s: s.isin([23, 26]).sum()))

feat['dirs_per_op'] = feat['dirs'] / feat['ops']
feat['delete_ratio'] = feat['deletes'] / feat['ops']

dirs_per_op is the feature that does the most work. A compiler writes thousands of files into a handful of build directories. A backup agent reads broadly and writes narrowly. Ransomware walks the tree, so its writes are spread thin across many directories, and that shape is hard for an operator to change without slowing the encryption down.

Entropy is the feature everyone reaches for first and it disappoints. Ciphertext sits near 8.0 bits per byte over a 4KB head sample, and so does every .zip, .jpg, and .docx on the endpoint. Absolute entropy flags your photo library. The delta on a given path is what carries signal:

import math
from collections import Counter

def head_entropy(path, nbytes=4096):
    with open(path, 'rb') as fh:
        buf = fh.read(nbytes)
    if not buf:
        return 0.0
    n = len(buf)
    return -sum((c / n) * math.log2(c / n) for c in Counter(buf).values())

Storing a prior entropy value per path is real infrastructure cost. Decide whether you are paying it before you write the feature into your design doc.

Two behaviors are worth pulling out of the model entirely. Shadow copy destruction (T1490) and service stops (T1489) precede the encryption stage (T1486) in most deployments, and vssadmin.exe delete shadows /all /quiet has close to zero legitimate use in a managed environment. That is a field-value rule with better precision than any model you will train, and it fires earlier. Build it first.

The negatives are the hard part

Once you have windowed features, the model is the least interesting decision. HistGradientBoostingClassifier from scikit-learn handles the mixed scales and missing values in this feature set without preprocessing, trains in seconds, and gives you something you can explain to a detection engineer.

Two disciplines matter more than the estimator:

• Split by host, not by row. Windows from the same machine share a software footprint. GroupKFold with the host ID as the group is the difference between an honest cross-validation score and a number that collapses on deployment.
• Curate the negatives deliberately. Your false positives are known in advance: backup agents, search indexers, antivirus full scans, robocopy, video transcoders, and CI build agents. Collect windows from each of those on purpose. A model trained on ransomware versus idle endpoints has learned to detect disk activity.

Pick the operating threshold from precision_recall_curve against an alert budget your analysts actually have, not from predict(). On this problem the argument for accepting a higher false positive rate is stronger than usual, because the miss is unrecoverable and the false positive is a killed process. Not free, but recoverable.

What this misses

The failure modes are specific enough to name.

Intermittent encryption breaks the volume and entropy signals by design. SentinelOne documented the shift in LockBit 2.0, BlackCat, and Play, where the payload encrypts only portions of each file to finish faster (their write-up covers the variants). A partially encrypted file reads at lower entropy and takes fewer operations. Both of your best features degrade at once.

Remote encryption over SMB defeats process-level features entirely. If an unmanaged host encrypts a file server’s shares across the network, the server sees one legitimate SMB service doing the writes and your per-process aggregation has nothing to key on. That detection lives in file server I/O or SMB audit telemetry, grouped by remote session.

And detection at the encryption stage is late by construction. T1486 is the last thing an operator does. Credential access, lateral movement, and exfiltration all happened first, over hours or days, and those stages leave signal in authentication and network data where you have far more time to work. If you are building one model, build it upstream.

Cheap complement while you build any of this: a directory of decoy files with a filesystem watcher on it. No model, no training data, near-zero false positive rate, and it fires the moment something walks the tree.

For behavioral data to train on, Atomic Red Team ships executable tests for T1486 and T1490 that produce real telemetry in a lab without running live samples. Start there, then detonate real families once your collection pipeline is proven.

We teach model tuning to reduce false positives as its own block in Threat Hunting with Data Science, and this is the problem that justifies the time: the negatives are where the work is, and they are different in every environment. The broader scoring mechanics are in how anomaly detection works in security ops, and the threshold tradeoff shows up again in reducing false positives with machine learning.