Are AI hallucination reported scores basically meaningless right now?

A lot of reports claim specific hallucination rates for models.

But the numbers don’t really line up across studies.

Some say low. Others show much higher rates.

Found an interesting report that tries to make sense of it - comparing results across OpenAI, Anthropic, and Google and shows how much the methodology changes the outcome.

Reason seems to be:

• No shared definition of “hallucination”
• Different benchmarks test completely different things
• Evaluation methods vary (automated vs human grading)
• Difficulty of tasks isn’t consistent

So “Model X has Y% hallucination rate” doesn’t actually translate across papers.

Worth looking at here if following model evals.