GPTBot
GPTBot is OpenAI's training crawler, distinct from OAI-SearchBot which builds the ChatGPT search index.
Why it matters
Blocking GPTBot does not remove a site from ChatGPT's search results, because that index is fetched by a different agent.
How it is measured
Detect GPTBot by the token GPTBot in the User-Agent header, then verify the request IP against OpenAI's published ranges, because the user-agent string alone is trivially spoofed. In robots.txt, confirm GPTBot and OAI-SearchBot have separate, intentional rules.
Generate a robots.txt that blocks GPTBot, free and without an account.
Checks that involve gptbot
Related terms
Questions
What is gptbot?
GPTBot is OpenAI's training crawler, distinct from OAI-SearchBot which builds the ChatGPT search index.
Why does gptbot matter?
Blocking GPTBot does not remove a site from ChatGPT's search results, because that index is fetched by a different agent.
How is gptbot measured?
Detect GPTBot by the token GPTBot in the User-Agent header, then verify the request IP against OpenAI's published ranges, because the user-agent string alone is trivially spoofed. In robots.txt, confirm GPTBot and OAI-SearchBot have separate, intentional rules.