How do I structure a POC for an AI security vendor?

Define pass/fail exit criteria before the vendor touches your environment, build a labeled test set from your own historical alert dispositions plus a batch of freshly generated attack telemetry, run the vendor's product and your incumbent tooling over the same events, and compare precision, recall, time-to-detect, and alerts per analyst per shift. Score against the baseline you already run, not against zero. Insist on access to raw per-detection output, not a dashboard summary. If the vendor will not run on your data or will not expose raw output, that refusal is your result.

How big should the test set be for an AI security POC?

Big enough that the difference you care about is not noise. If your incumbent tool catches 80% of a technique and you want to detect a 10-point improvement, you need on the order of a few hundred positive (attack) samples to see it reliably, and a much larger volume of benign traffic to measure false positive rate at production scale. A test set of a few thousand labeled events, with at least 200 to 500 true positives spread across the MITRE ATT&CK techniques you care about, is a reasonable floor. Compare the two tools on the same events and use McNemar's test to check whether the difference is statistically real rather than a lucky draw.

Should I give an AI security vendor my raw production logs during a POC?

No more than the evaluation requires. Redact or hash credentials, PII, and full payloads before the data leaves your environment, and prefer a POC that runs inside your own cloud tenant or on-premises over one that ships telemetry to the vendor. Require a written no-training, zero-retention data processing agreement covering anything you do send, and keep it on file for your auditors. Treat the POC data flow as a production data flow, because for the duration of the POC it is one.

How do I test whether an AI detection product can be evaded?

Generate attack telemetry with a known-good tool like Atomic Red Team, then run obfuscated and staged variants of the same techniques (encoded PowerShell, renamed binaries, slow beaconing, protocol tunneling) and measure how detection rate degrades. MITRE ATLAS tracks model evasion as AML.T0015. A product that catches the textbook version of a technique but misses every realistic variant is detecting signatures, not behavior, regardless of the AI branding. Robustness under adversarial input is part of the POC, not a follow-up.

What is the single most common mistake in an AI security POC?

Judging the tool on the vendor's demo data and dashboard instead of your data and raw output. The demo is tuned to succeed. Your environment has different baselines, different noise, and different attackers, and that is exactly where AI-based detection either earns its keep or falls apart. The second most common mistake is not defining exit criteria in advance, which lets a mediocre result get rationalized into a purchase after the fact.

How to Run a POC for an AI Security Vendor

The demo always works. That is what it is built to do. The proof of concept on your own data is where an AI security product either earns the purchase or exposes itself, and it is the step most teams run badly: no labels, no baseline, no exit criteria, and a verdict driven by whichever dashboard looked most convincing.

Knowing which questions to ask a vendor gets you into the room. Running a disciplined POC is how you get an answer. Here is the mechanics of doing that: the test set, the baseline, the metrics, and the exit criteria that turn a sales cycle into an experiment.

Set Exit Criteria Before the Vendor Touches Anything

Write down what pass looks like before the POC starts, in numbers, and get sign-off on it. Decide it after you see results and you will rationalize whatever you saw.

Concrete criteria look like this:

Detection lift: at least a 10-point improvement in recall on the techniques we care about, over our current tooling, on our data.
False positive budget: no more than X additional alerts per analyst per shift, measured on a representative slice of production traffic.
Time-to-detect: median detection latency under N minutes for the priority techniques.
Explainability: every detection exposes the features or evidence that drove it, not just a score.

If the product cannot clear the bar you would have set honestly in advance, it does not clear it because the demo was slick.

Build a Labeled Test Set From Your Own Telemetry

You cannot measure detection without ground truth. You have most of it already: every alert your analysts dispositioned is a label. Mine the close codes from your SIEM or SOAR for true positives (escalated or confirmed incidents) and true negatives (closed benign), and you have the benign-and-known-bad backbone of a test set.

Historical data will not cover the attacks you have never seen, so generate those. Atomic Red Team executes real techniques mapped to MITRE ATT&CK IDs, so you can produce labeled attack telemetry for exactly the techniques in your threat model: T1059.001 (PowerShell), T1055 (process injection), T1071 (application-layer C2), and so on. Run the atomics in a controlled environment, capture the resulting logs, and label them by technique.

The result is a frozen, labeled corpus you control. Both the vendor’s product and your incumbent tooling get scored against the same events, blind to the labels.

Score Against Your Baseline, Not Against Zero

A tool that catches 90% of something sounds excellent until you learn your current stack already catches 88%. The number that matters is lift over what you run today, so measure both tools on the identical test set and compare.

from sklearn.metrics import precision_recall_fscore_support, confusion_matrix

# y_true: 1 = attack, 0 = benign, from your labeled corpus
# y_vendor, y_incumbent: each tool's alert/no-alert decision per event

for name, y_pred in [("incumbent", y_incumbent), ("vendor", y_vendor)]:
    p, r, f1, _ = precision_recall_fscore_support(
        y_true, y_pred, average="binary", zero_division=0)
    tn, fp, fn, tp = confusion_matrix(y_true, y_pred).ravel()
    print(f"{name}: precision={p:.3f} recall={r:.3f} "
          f"false_positives={fp} missed_attacks={fn}")

Recall tells you how many real attacks each tool caught. The false_positives count, scaled to your production event volume, tells you what the tool will cost your analysts every shift. A product that lifts recall by 5 points while tripling false positives is not an upgrade, it is a staffing request.

When the two tools look close, check whether the difference is real or a lucky draw. McNemar’s test compares two classifiers on the same test set by looking only at the events where they disagree:

from statsmodels.stats.contingency_tables import mcnemar

# contingency of disagreements: rows = incumbent right/wrong, cols = vendor right/wrong
inc_correct = (y_incumbent == y_true)
ven_correct = (y_vendor == y_true)
b = int(((inc_correct) & (~ven_correct)).sum())   # incumbent right, vendor wrong
c = int(((~inc_correct) & (ven_correct)).sum())   # incumbent wrong, vendor right
result = mcnemar([[0, b], [c, 0]], exact=True)
print(f"vendor-only wins={c} incumbent-only wins={b} p={result.pvalue:.4f}")

If p is not below your threshold, the vendor did not beat your baseline, it tied it, and you should not pay a premium for a tie.

Try to Evade It

A detection product that catches the clean version of a technique and misses every realistic variant is matching signatures with an AI label on the box. Test for that directly. Take the same techniques you generated with Atomic Red Team and run obfuscated and staged variants: base64-encoded PowerShell, renamed and re-signed binaries, beaconing slowed to hourly jitter, C2 tunneled over DNS or HTTPS. Measure how much detection rate drops between the textbook version and the evasive one.

MITRE ATLAS tracks this as AML.T0015 (Evade ML Model), and robustness under adversarial input belongs inside the POC, not in a follow-up you never run. The same discipline that goes into evaluating ML model robustness applies to a vendor’s model you cannot see inside: probe the decision boundary from the outside and watch where it collapses.

Protect Your Data While You Do It

For the length of the POC, the evaluation data flow is a production data flow. Redact or hash credentials, PII, and full payloads before anything leaves your environment. Prefer a POC that runs inside your own cloud tenant or on-premises to one that ships telemetry to the vendor. Get a written no-training, zero-retention data processing agreement for anything you do send, and keep it where your auditors can find it. A vendor who resists these terms during the courtship phase will not improve after the contract is signed.

Read the Raw Output

Insist on per-detection raw output: the event, the score, and the evidence, not a rolled-up dashboard count. Dashboards are designed to summarize favorably. The raw stream is where you see the duplicate alerts, the near-misses, and the detections that fired on the right event for the wrong reason. If a vendor will only show you aggregate numbers, you cannot validate anything, and that is itself a finding.

A POC run this way stops being a demo you react to and becomes an experiment you designed. That shift, from consuming a vendor’s numbers to producing your own, is the core skill. GTK Cyber’s applied AI and data science training teaches security teams to build exactly these evaluations: labeled test sets, honest baselines, and the metrics that tell you whether a model earns its place in your stack.