Cloudflare splits AI crawler controls into search, agent and training categories

New defaults starting September 15 let all Cloudflare sites, including free users, granularly allow or block AI crawler types as publishers push back on scraping without traffic or payment.

Summary

verifying reliability

Terms & Concepts

No specialized terms available for this topic.