The AI feature that sounds good---until you ask one question

AI vendors love the phrase:

"Our product gets better every time you use it."

In HR, "better" without a definition isn't a benefit, it’s a liability.

It sounds like a promise of built-in continuous improvement.

But a vendor makes this claim, ask one question:

better for whom, and using whose data?

Most AI systems that improve through use do so because they learn from your interactions: documents processed, decisions made, corrections applied.

That creates two problems worth understanding separately.

1. The Privacy Gap

We scrutinize inference-time risk (what happens while the tool is running). We pay less attention to training-time risk (what happens after the session ends). If your performance evaluations are "learned from," they are being incorporated into a system you don’t fully control.

2. The Quality Drift

If a model is constantly learning, it is constantly changing. The tool you used last month is not the tool you are using today. A decision the model handled correctly last month may be handled differently today, without any visible change.

We don’t accept silent updates in payroll software, so why do we accept them in AI that influences hiring and compensation?

Most vendors will not have clear answers to all of this.

That is useful information in itself.

Before signing, ask four questions:

1. Can our organisation's data be excluded from your training set?
2. What notification do we receive when the model changes?
3. How are model updates validated before reaching production?
4. How do you ensure the model does not scale our existing biases?

The answers will tell you more than the promise ever could.

07.04.2026