Guide
How to get cited by Claude
What Anthropic publishes: ClaudeBot, Claude-SearchBot, and Claude-User are separate robots.txt controls. A training opt-out is not a search opt-out.
14 October 2026 · CiteScore Team · 9 min read
Anthropic’s public note for site owners is a crawler page, dated 7 April 2026. It names three robots and says each one honors robots.txt. It does not publish a citation rate, a required schema type, or a word count. CiteScore does not log into Claude. The claims below are from Anthropic’s crawler article.
The same kind of split exists for the other engines: ChatGPT, Perplexity, Gemini, and Microsoft Copilot. The short cross-engine version is how to get cited by AI answer engines.
Three robots
| Token | What Anthropic says it does |
|---|---|
| ClaudeBot | Collects web content that could contribute to training generative models. Restricting it signals that the site’s future materials should be excluded from those training datasets. |
| Claude-SearchBot | Navigates the web to improve search result quality. Anthropic says disabling it prevents indexing for search optimization, which may reduce the site’s visibility and accuracy in user search results. |
| Claude-User | Fetches pages when someone asks Claude a question. Anthropic says disabling it prevents retrieval in response to a user query, which may reduce visibility for user-directed web search. Unlike OpenAI’s ChatGPT-User note, this article says Anthropic’s bots honor robots.txt. |
A training opt-out is not a search opt-out. Allow Claude-SearchBot and Claude-User, and disallow ClaudeBot, when the policy is “cite the page, do not train on new copies.” Anthropic says the settings are separate robots.
Allow the search robot
This is the group that matches the search job Anthropic describes. A token with no group of its own follows User-agent: *, because Anthropic says the bots honor standard robots.txt directives. A blanket Disallow: / under * covers Claude-SearchBot until it has its own group.
User-agent: Claude-SearchBot Allow: / User-agent: Claude-User Allow: /
Anthropic also supports the non-standard Crawl-delay extension, and the example they publish is on ClaudeBot. Repeat the file on every subdomain you want to control. Their article says an opt-out has to be in robots.txt, and that blocking the crawler’s IP addresses may not stick, because that block can stop the crawler from reading robots.txt.
How these tokens sit next to GPTBot and Google-Extended is on the robots.txt guide. The three tokens CiteScore’s audit records — which do not include Claude-SearchBot — are on AI crawlers.
What the crawler page leaves out
Anthropic’s article does not list a schema type or an llms.txt file as the way into a Claude answer. A page the robot cannot fetch cannot be indexed from that fetch. Put the name of the business and the fact you want quoted in the HTML. Anthropic does not publish that as a score.
The label for this work, and what it does not promise, is on answer engine optimization.
Check the public files
The free check reports GPTBot, PerplexityBot, and Google-Extended. It does not parse ClaudeBot, Claude-SearchBot, or Claude-User. A passing row on that check is not a Claude answer.
Run the free check on the host. It reads the homepage, robots.txt for GPTBot, PerplexityBot, and Google-Extended, /llms.txt, a sitemap, and JSON-LD on the homepage. It does not log into an answer engine. One URL continues at checkout.
Source
- Does Anthropic crawl data from the web, and how can site owners block the crawler? Claude Help Center, 7 April 2026. The three robots, what disabling each one does, robots.txt, Crawl-delay, subdomains, and the note on IP blocks.