Skip to content
Theyfetch

Bot catalog

We match the user agent, then verify the IP against the operator list when one exists. The IP is dropped after that. Coding-agent fetches often come from the user's own address, so they stay unverifiable.

Live-answer fetchers are the default citation. Search and training are separate. Blocking training while allowing ChatGPT-User is a valid default. We do not sell opening the site to every scraper.

BotProviderPurposeVerification
claude-codeAnthropicCoding agentsUnverifiable
Claude-SearchBotAnthropicSearch indexIP or rDNS
Claude-UserAnthropicLive answersIP or rDNS
ClaudeBotAnthropicTrainingIP or rDNS
ChatGPT-UserOpenAILive answersIP or rDNS
OAI-SearchBotOpenAISearch indexIP or rDNS
GPTBotOpenAITrainingIP or rDNS
Perplexity-UserPerplexityLive answersIP or rDNS
PerplexityBotPerplexitySearch indexIP or rDNS
Google-GeminiNotebookGoogleLive answersIP or rDNS
Google-AgentGoogleLive answersIP or rDNS
GooglebotGoogleSearch indexIP or rDNS
bingbotMicrosoftSearch indexIP or rDNS
meta-externalfetcherMetaLive answersUnverified
meta-webindexerMetaSearch indexUnverified
meta-externalagentMetaTrainingUnverified
MistralAI-UserMistralLive answersIP or rDNS
MistralAI-IndexMistralSearch indexIP or rDNS
MistralAI-TrainingMistralTrainingUnverified
Amzn-UserAmazonLive answersIP or rDNS
Amzn-SearchBotAmazonSearch indexIP or rDNS
AmazonbotAmazonTrainingIP or rDNS
DuckAssistBotDuckDuckGoLive answersIP or rDNS
ApplebotAppleSearch indexIP or rDNS
CCBotCommon CrawlTrainingIP or rDNS
BytespiderByteDanceTrainingUnverified
opencodeOpenCodeCoding agentsUnverifiable