Starting September 15, 2026, Cloudflare is cracking down on mixed-use web crawlers. Any site hosting ads will face default blocking unless owners adjust settings, impacting how AI models access content.

The move aims to protect intellectual property while offering publishers more control and commercial opportunities. However, it raises questions about the sustainability of free internet crawling and could lead to a premium for accessing content.

Matthew Prince, Cloudflare’s CEO, believes this shift is necessary as non-human traffic now dominates the web, marking a new milestone in digital evolution.

To adapt, Cloudflare has introduced “Pay Per Use,” allowing publishers to charge AI companies when their content creates value. This could help conserve bandwidth and resources, but also potentially limit access for those who can’t afford it.

Exciting times ahead as the tech giants battle over the future of internet content and AI training data.