Why Pandas beats grep and Excel
grep is great for finding a string. It is bad at answering “which accounts had more than 20 failed logins in a five-minute window.” Excel can do that, until your log file has three million rows and the app gives up.
Pandas sits in the middle: it loads structured data into memory, lets you filter and aggregate with a few lines, and keeps every step as code you can rerun next week. For security data, where the same auth log question comes up constantly, that repeatability is the real win. Your analysis becomes a script, not a spreadsheet you cannot reproduce.
A few operations that cover most of the work
Loading a log is one line. Most security logs are CSV or JSON lines:
import pandas as pd
# Load an auth log exported as CSV
df = pd.read_csv("auth.log.csv")
# For big files, load only the columns you need
df = pd.read_csv("auth.log.csv", usecols=["timestamp", "user", "src_ip", "result"])Filtering is how you narrow to the interesting subset. Here, just the failed logins:
# Boolean filtering: keep only failures
failures = df[df["result"] == "failed"]The operation that earns its keep is groupby. Password spraying, brute force, and unusual access patterns all show up as counts per key:
# Count failed logins per source IP, sorted worst-first
spray = (
failures.groupby("src_ip")
.size()
.sort_values(ascending=False)
)That is three lines to surface the spraying source that a dashboard buried under noise. This is the same first move behind almost every threat hunting pipeline in Python and Jupyter: load, filter, group, look.
Time is where security data gets interesting, so parse timestamps early and build features from them:
# Parse the timestamp column into real datetimes
df["timestamp"] = pd.to_datetime(df["timestamp"])
# Derive hour-of-day: 3am logins are worth a second look
df["hour"] = df["timestamp"].dt.hour
# Failed logins per user per hour
per_hour = (
failures.set_index("timestamp")
.groupby("user")
.resample("1h")
.size()
)Hour-of-day and per-window counts are the raw material for detection. A human logging in is boring. The same account logging in at 3am from a new IP, 40 times an hour, is not.
Where analysts get stuck
Three things trip people up. First, timestamps: if you skip pd.to_datetime, every time-based operation silently treats your timestamps as strings and gives nonsense. Parse them first, always.
Second, chained filtering and the SettingWithCopyWarning. When you filter and then assign a new column, Pandas warns you may be editing a copy. Use .copy() when you intend to keep a filtered subset, and the warning goes away along with the subtle bugs.
Third, memory. A multi-gigabyte log will not fit at once. Use chunksize in read_csv to stream it, or filter columns with usecols so you load only what matters. Analysts abandon Pandas here when the fix is one parameter.
None of this is hard. It just needs reps on real data, which is exactly what we do in the Applied Data Science & AI for Cybersecurity course. We start in Pandas on security datasets before touching a single model, and we run the full course at Black Hat USA 2026.
The course runs as two 2-day sessions at Black Hat USA 2026, August 1 to 4 in Las Vegas: claim your seat.