A new analysis by The New York Times, in collaboration with the AI development firm Oumi, reveals that while AI Overviews is right 90% of the time, it also gets things wrong one out of every ten times. This means millions of inaccuracies are disseminated daily.
The test involved running over 4,000 questions through Gemini, assessing its factual accuracy. Despite improvements from Gemini 2.5 to Gemini 3, the error rate remains significant, with tens of millions of lies per day being propagated by Google’s AI tool.
Examples abound: when asked about Bob Marley's former home becoming a museum, AI Overviews incorrectly cited non-relevant dates, and even got the classical music hall of fame wrong. These inaccuracies raise important questions about the reliability of information in an era dominated by AI.
The report highlights that while AI Overviews is getting better, its error rate is still unacceptably high for a tool designed to provide factual data. As humanity increasingly relies on these tools for everyday information, the implications are substantial and concerning.
