What questions should I ask an AI security vendor about their training data?

Ask: what data was used, when it was collected, whether it includes data from your industry, how often the model is retrained, and what happens with inputs outside the training distribution. A vendor who cannot answer specifically either lacks visibility into their own model or is hiding something.

How do I structure a proof of concept for an AI-based detection product?

Run the POC on your own data or realistic synthetic data, define success criteria before starting, and require access to raw detection output, not just dashboard counts. Compare false positive rate, time-to-detect, and analyst workload against your existing tools. If the vendor refuses a POC on your data, that is the answer.

What does explainability mean in the context of an AI security tool?

Explainability means the model can identify the features or evidence that drove a specific verdict. Practical signals include SHAP values, feature attribution, retrieval citations for LLM outputs, or per-detection access to the raw inputs. A model that says 'malicious, confidence 0.92' with no supporting evidence cannot be tuned, validated, or used to build analyst understanding.

How can I tell if a vendor is using real machine learning or just calling a rule engine 'AI'?

Ask for the model architecture, training process, evaluation metrics on held-out data, and how performance is monitored in production. Real ML systems have versioned models, retraining cadences, and drift monitoring. Rule engines branded as AI usually cannot describe any of these.

What false positive rate should I expect from an ML-based detection tool?

Vendor benchmarks rarely match production. Expect higher false positive rates than the vendor advertises, and budget for tuning effort. A useful comparison is alert volume per analyst per shift before and after deployment. A meaningful tool reduces analyst workload net of any new tuning burden it creates.

How to Evaluate AI Security Vendors Without Getting Fooled

Every security vendor has an AI story now. Some of them are real. Many aren’t.

The challenge for security leaders is that the people doing the selling know more about the marketing than the technology, and the people doing the buying often lack the technical depth to probe the claims. The result is a lot of expensive tools that underdeliver.

Here’s a practical framework for cutting through it.

Start With the Claim

The first step is identifying exactly what the vendor is claiming AI does in their product. Be specific. “AI-powered” is not a claim. “Our ML model detects novel malware variants not in known signature databases by analyzing behavioral patterns in PE file execution” is a claim.

Press vendors to be specific:

What problem does the AI solve, specifically?
What does the AI do that a non-AI approach (rules, signatures, heuristics) cannot?
Where does the AI sit in the detection or response workflow?

If they can’t answer these questions specifically, the AI in their product is probably a marketing feature, not an operational one.

Ask About the Training Data

Machine learning models are only as good as the data they were trained on. The training data determines what the model knows, what it can generalize from, and where it will fail.

Questions to ask:

What data was the model trained on? How recent is it?
Was it trained on your industry’s data or general data?
How often is the model retrained?
What happens when the model encounters data outside its training distribution?

A vendor who can’t answer training data questions either doesn’t know (a problem) or doesn’t want to tell you (also a problem).

Understand the False Positive Rate

Every detection system generates false positives. The question is how many, under what conditions, and how that impacts your team’s workload. AI-based detections are not inherently better or worse than rule-based ones, but vendors often imply they are.

Ask for:

False positive rates in customer environments similar to yours
How alert volume changed after deployment
What tuning is required and who does it

A vendor who claims near-zero false positives either hasn’t been deployed at scale or is cherry-picking numbers.

Test It on Your Data

The strongest signal is a proof of concept on your actual environment. Generic demos on vendor-supplied data are not meaningful. Your environment has different baselines, different noise, different attack patterns.

Before any significant purchase, insist on:

A POC using your data (or realistic synthetic data matching your environment)
Clear success criteria defined in advance
Access to raw detection output, not just a dashboard

If the vendor won’t run a POC, ask why.

Look for Explainability

A model that tells you something is malicious without telling you why is a black box. In a security context, black boxes are dangerous. They fail silently, they can’t be tuned intelligently, and analysts can’t use them to build understanding.

Ask:

Can the model explain why it flagged a specific alert?
What features drove the detection?
Can analysts access the underlying evidence, not just the verdict?

Explainability isn’t just a nice-to-have. It’s what separates a useful detection tool from an expensive alert generator.

Don’t Buy AI to Buy AI

The most common mistake is acquiring AI capabilities because AI is expected, not because there’s a specific problem it solves better than alternatives.

Before any AI security purchase, define:

The specific problem you’re trying to solve
What you’re doing now and why it’s insufficient
What success looks like in measurable terms
What the non-AI alternative would cost

If the AI solution doesn’t clearly outperform the alternative on your specific problem, it probably doesn’t justify the premium.

GTK Cyber’s executive AI training is built around this kind of rigorous evaluation framework, not vendor presentations, but the technical literacy to ask the right questions and interpret the answers. If you’re making AI security decisions for your organization, it’s worth a day to develop that foundation.