A single overly broad rule, such as a catch-all User-agent: * disallow meant to stop scrapers, can silently block the AI crawlers you actually want. Check each AI user agent explicitly.
User-initiated fetchers are the exception, and behavior varies by vendor: OpenAI says robots.txt rules may not apply to ChatGPT-User, Perplexity says Perplexity-User generally ignores them, while Anthropic says blocking Claude-User in robots.txt does stop its fetches.