G
GEO Toolbox

AI Crawlers

ClaudeBot

Also: Anthropic ClaudeBot, Claude-SearchBot, Claude-User

ClaudeBot is Anthropic's training crawler, which gathers publicly available content that may train future Claude models. It is one of three Anthropic bots: Claude-SearchBot indexes pages for Claude's web search, and Claude-User fetches pages live during a user's session. Each respects robots.txt and is controlled separately.

Updated

Training and citation are separate doors. Blocking ClaudeBot only opts your content out of training future models; per Anthropic's crawler documentation, it has no effect on whether Claude cites you. Citations ride the other two bots: if you want to appear in Claude's web-search answers, keep Claude-SearchBot (search index) and Claude-User (live fetches) able to reach you, whatever you decide about ClaudeBot.

As with other AI crawlers, the most common reason an Anthropic bot cannot reach a site is an unintended bot-management or firewall rule, not a deliberate robots.txt block.