How to Apply Anomaly Detection to Authentication Logs

Point one anomaly detection model at a domain’s authentication events and it will spend its first week flagging your executives and your backup service account. Neither is compromised. Both look strange next to the average user, and the average user is a fiction: nobody authenticates like the mean of 12,000 accounts.

The unit of normal in auth data is the account. Almost every practical decision follows from that.

Pull the right fields first

You need 4624 (successful logon) and 4625 (failed logon) from endpoint and domain controller Security logs, plus 4768 and 4769 if you care about Kerberos. Minimum usable schema: timestamp, target account, host, logon type, source address, authentication package, event ID.

Logon type is the field that carries the meaning. Microsoft’s event 4624 documentation lists the values from 0 (System) through 13 (CachedUnlock). Three matter disproportionately: 3 (Network), 10 (RemoteInteractive, which is RDP), and 9 (NewCredentials, what runas /netonly produces). Drop that column and an RDP session and a scheduled task become the same row.

Why the global model floods your queue

Authentication counts per account span orders of magnitude. Fit anything across all of them and the model learns a mixture distribution whose tails are permanently occupied by service accounts on one end and low-activity humans on the other. The queue that comes out is a list of your most unusual accounts, and it is the same list tomorrow, because nothing about those accounts changed.

The mechanics of scoring outliers are covered in how anomaly detection works in security ops. This post picks up where that approach starts flooding the queue.

Two changes fix most of it: baseline each account against itself, and fall back to a peer group when an account has too little history to have a baseline.

Baseline per account with median and MAD

Mean and standard deviation are the wrong statistics here. Both are pulled by the events you are hunting, so one burst of activity raises the threshold and hides the next one. Median and median absolute deviation barely move.

import numpy as np
import pandas as pd

auth['ts'] = pd.to_datetime(auth['ts'])
auth['hour'] = auth['ts'].dt.floor('h')

hourly = (auth[auth['event_id'] == 4624]
          .groupby(['target_user', 'hour'])
          .size()
          .rename('logons')
          .reset_index())

prof = hourly.groupby('target_user')['logons'].agg(
    med='median',
    mad=lambda s: (s - s.median()).abs().median(),
    buckets='size')

hourly = hourly.join(prof, on='target_user')

# Iglewicz and Hoaglin modified z-score; the usual cutoff is 3.5
hourly['mz'] = 0.6745 * (hourly['logons'] - hourly['med']) / hourly['mad']
hourly.loc[hourly['mad'] == 0, 'mz'] = np.nan

Two things bite in production. Accounts with a MAD of zero, which is every service account that authenticates an identical number of times each hour, divide by zero and yield inf rather than an error, so pandas will happily rank them at the top of your queue forever. Set them to NaN and cover them with a field-value rule. Accounts with fewer than about 336 hourly buckets (two weeks) have no baseline worth using; score those against a peer group from the same organizational unit until they accumulate one.

Rarity carries more signal than magnitude

Most real findings in auth data are not “more logons than usual.” They are “this account has never done this before.”

Microsoft’s own monitoring recommendations on that same 4624 page read like a rarity model rather than a statistical one: watch for a logon type that does not match the account type, such as Batch or Service used by a member of a domain admin group, and for a service account authenticating from a source address outside its expected set. No distribution fitting involved, only a record of what each account has done before.

cut = auth['ts'].max() - pd.Timedelta(days=1)
hist = auth[auth['ts'] < cut]
recent = auth[auth['ts'] >= cut].copy()

known = hist.groupby('target_user').agg(
    hosts=('host', lambda s: set(s)),
    types=('logon_type', lambda s: set(s)),
    subnets=('src_ip', lambda s: set(s.str.rsplit('.', n=1).str[0])))

def novelty(row):
    if row['target_user'] not in known.index:
        return ['no_history']
    p = known.loc[row['target_user']]
    out = []
    if row['host'] not in p['hosts']:
        out.append('new_host')
    if row['logon_type'] not in p['types']:
        out.append('new_logon_type')
    if row['src_ip'].rsplit('.', 1)[0] not in p['subnets']:
        out.append('new_subnet')
    return out or ['known']

recent['flags'] = recent.apply(novelty, axis=1)

The /24 grouping on source address is crude and will misfire on any network that does not allocate by subnet the way you assume. Check that against your own IPAM before trusting it.

Stack the flags instead of alerting on each. One new host is a laptop refresh. A new host plus a first-seen logon type 9 plus a new subnet in the same hour is an investigation, and type 9 specifically is a common artifact of pass-the-hash (T1550.002).

What a per-account baseline cannot see

Password spraying is the clean example. In T1110.003 one source tries a couple of passwords against hundreds of accounts, which is one extra 4625 per account per hour. Every per-user model on earth ignores it. The detection is a different grouping key, not a better model:

spray = (auth[auth['event_id'] == 4625]
         .groupby(['src_ip', 'hour'])['target_user']
         .nunique())

Two more worth writing down before anyone trusts the output. Kerberoasting (T1558.003) lives in 4769 rather than 4624, and the signal is a run of service ticket requests with encryption type 0x17 (RC4-HMAC) in a domain that otherwise issues 0x12: a field-value rule, not an outlier score. And an attacker authenticating as a real user, from that user’s own workstation, during that user’s normal hours raises no novelty flag and no count anomaly. Valid accounts (T1078) is the technique this entire approach handles worst, and tuning does not fix it.

Where to practice

Open Threat Research publishes recorded Windows event data from simulated attacks in Security-Datasets, which is a better place to test the code above than production.

We teach this exact progression, global model to per-account baseline to rarity flags to the pivot in grouping key, in the anomaly detection block of Threat Hunting with Data Science. Organization-specific model creation is a topic in that course for a practical reason: a baseline built on somebody else’s domain does not transfer to yours, and the tuning work is where the detection actually gets built. The same material shows up in the machine learning for threat hunters track.