Cloudflare introduces continuous evaluation for Verified bots
🎧 Listen to this story
Cloudflare announced a system to continuously evaluate whether Verified Bots respect site preferences, including robots.txt instructions, requests to slow or stop crawling, and HTTP 429 (“Too Many Requests”) responses. The company plans to measure repeated behavior patterns and turn them into an explainable trust band, rather than a permanent judgment about a bot or its operator.
Cloudflare says the results will eventually be published on Radar, and that the system will also consider whether a bot responds to direct feedback. As an example, the company says it removed Perplexity’s bot from its Verified bot list in 2025 after it sought to access resources on sites that had opted out of scraping and attempted to circumvent controls after being blocked.
Why it matters · editorial interpretation
The proposal could give sites a more continuous and explainable way to assess whether bots respect their preferences, rather than relying only on permanent verification. Publishing the results on Radar could also increase visibility into bot behavior, although Cloudflare describes the system as planned rather than already available.
Sources
Security