As state governments fret over large language models (LLMs) parroting dangerous propaganda, the Estonian Language Institute has developed a new benchmark. This test ranks various LLMs based on their ability to resist narratives from the Russian Federation.

The ELI, alongside Propastop—a volunteer-run defence collective—identified 14 categories of Russian influence operations. These range from the status of Crimea and justifications for the Ukraine war, to historical narratives surrounding NATO and Russia’s annexation of Baltic states during World War II. Each category was designed with neutral questions, those laden with false assumptions, and others aiming to elicit misinformation.

For evaluation, the LLMs were given English, Estonian, and Russian versions of each question. Judging was done by an AI calibrated against Propastop’s expertise. Among proprietary models, Anthropic's Claude models performed best, with Opus 4.7 achieving a top score of 94.9 out of 100.

The results suggest that while LLMs are improving at resisting foreign narratives, more work remains to ensure their responses are always accurate and unbiased.