What is hallucination testing?

Hallucination testing measures how often an AI system states false or unsupported information. It uses test sets of questions with known correct answers, checks the system's responses for factual accuracy, and quantifies the rate and severity of hallucinations.

How do you measure AI hallucination rates?

By running the system against a curated set of questions with verifiable answers, then checking each response for factual correctness and whether claims are supported by provided sources. The result is a measured hallucination rate, not a vague impression.

Can hallucination testing be automated?

Partly. Source verification — checking whether claims are grounded in retrieved documents — can be automated, and AI-assisted fact-checking helps at scale. High-stakes domains still require human verification of a sample to confirm the automated checks are reliable.

Hallucination Testing for AI Systems

Quick answer

Hallucination testing measures how often an AI system states false or unsupported information. It works by running the system against a curated set of questions with verifiable correct answers, checking each response for factual accuracy and whether its claims are actually supported by the sources provided, and reporting a measured hallucination rate and severity. This matters because every language model hallucinates sometimes, and the only responsible way to deploy one in a setting where accuracy counts is to know — by measurement, not assumption — how often and how badly it does so. A hallucination rate you have measured is manageable; one you have only hoped is low is a liability.

What this means

A hallucination is a confident, plausible-sounding statement that is false or unsupported. Because language models generate fluent text regardless of whether they "know" an answer, hallucinations are not bugs to be fully patched but a property to be measured and contained.

Hallucination testing turns this property into a number. Rather than "the system seems accurate," it produces "on this test set, 3% of responses contained an unsupported factual claim, of which one in five was material." That precision is what lets leaders decide whether a system is fit for a given purpose.

Why it matters for business

The cost of a hallucination depends on where it lands. In a brainstorming aid it is harmless; in a client-facing report, a legal summary or a financial figure it can be damaging or actionable. For Australian organisations in regulated sectors, a hallucinated statement presented as fact can create compliance and liability exposure.

Knowing the rate lets the organisation match the system to the use case and design the right controls — grounding, source citation, human review — where the rate is too high for the stakes. Without measurement, organisations either avoid AI for accuracy-sensitive work out of unquantified fear, or use it there and absorb unquantified risk. Measurement replaces both with informed decisions.

How it works technically

Hallucination testing typically involves:

Curate a test set — questions with verifiable, known-correct answers representative of real use, including hard and edge cases.
Run the system — collect its responses under realistic conditions, including any retrieval grounding.
Check factual accuracy — compare each response against the known answer.
Check grounding — for RAG systems, verify that claims are actually supported by the retrieved sources, not invented around them.
Score severity — classify hallucinations by impact, since a trivial error and a material false claim are not equal.
Report rate and profile — produce an overall rate plus the distribution of severity and topic.

Grounding checks are especially powerful: a claim that cannot be traced to a source document is flagged regardless of whether it happens to be true, because unsupported claims are the risk.

Practical implementation considerations

The test set should emphasise the questions where errors would matter most, not just average cases. A system can have a low overall hallucination rate yet fail badly on a specific high-stakes category, and the testing must be designed to surface that.

Edison AI's AI readiness audit includes hallucination testing for accuracy-sensitive use cases, giving leaders a measured reliability figure and a clear view of where grounding or human review is required. This evidence is what supports a defensible decision to deploy — or not — in a sensitive context.

For high-stakes domains, automated checks should be backed by human verification of a sample, both to validate the automated method and to catch subtle errors automation may miss.

Common mistakes

Assuming a hallucination rate instead of measuring it. Unmeasured reliability is unmanaged risk.
Easy test sets. Average questions understate the rate on the hard cases that matter.
Ignoring severity. A low average rate can hide a small number of material, damaging errors.
Checking accuracy but not grounding. In RAG systems, unsupported-but-true claims still signal a system that invents rather than retrieves.
One-off testing. Hallucination behaviour changes with model and prompt updates; testing must be repeatable.

What leaders should do next

For any use case where accuracy matters, require a measured hallucination rate before deployment, produced against a test set weighted toward high-stakes questions. Classify hallucinations by severity, not just count them. Use grounding checks for retrieval-based systems. Where the measured rate exceeds what the use case can tolerate, add controls — stronger grounding, source citation, human review — and re-measure. Make hallucination a quantity you manage with evidence, not a fear you manage with avoidance.

Edison AI builds evaluation and human-review checkpoints into every AI implementation we ship.

Frequently asked

Questions, answered.

What is hallucination testing?
Hallucination testing measures how often an AI system states false or unsupported information. It uses test sets of questions with known correct answers, checks the system's responses for factual accuracy, and quantifies the rate and severity of hallucinations.
How do you measure AI hallucination rates?
By running the system against a curated set of questions with verifiable answers, then checking each response for factual correctness and whether claims are supported by provided sources. The result is a measured hallucination rate, not a vague impression.
Can hallucination testing be automated?
Partly. Source verification — checking whether claims are grounded in retrieved documents — can be automated, and AI-assisted fact-checking helps at scale. High-stakes domains still require human verification of a sample to confirm the automated checks are reliable.

Take the next step

Ready to put this into practice?

Edison AI helps Australian businesses move from AI curiosity to practical implementation, with workflow design, team training and measurable outcomes. Tell us about your setup and we'll come back with a sequenced plan grounded in the same thinking you just read.

Talk to our AI team