snuupDoes my website block ChatGPT?
Updated 24 July 2026
A file on your website called robots.txt tells crawlers what they can read. One wrong line in it hides your whole site from ChatGPT. Website builders and firewalls sometimes add that line without you knowing. Open yoursite.com/robots.txt and look for the name GPTBot. The fix is usually one line.
Why AI reads your website at all
The crawlers, and who they feed
GPTBotChatGPT
ClaudeBotClaude
Google-ExtendedGemini
PerplexityBotPerplexity
ChatGPT learns about your business from two places: pages other people wrote, and your own website. It reads your website with a crawler, a program that visits pages and saves what they say. OpenAI's crawler is called GPTBot. The other AIs send their own.
A blocked crawler cannot read your site. ChatGPT then knows only what other people say about you. If they say little, you fade out of the answers.
Check your robots.txt
yoursite.com/robots.txt
User-agent: GPTBot
Disallow: /
These two lines lock ChatGPT out of every page on the site.
robots.txt is a plain text file. Crawlers read it before they read anything else on your site. Open yoursite.com/robots.txt in your browser, with your own domain in place of yoursite.com.
No file at all is fine. An empty file is fine. The problem is the word Disallow under an AI crawler's name. Look for GPTBot, ClaudeBot, PerplexityBot and Google-Extended.
Our free AI crawler check reads the file for you. Type your website there. It shows who is blocked.
How to read what you see
✕ Blocked
User-agent: GPTBot
Disallow: /
The slash means the whole site.
✓Open
User-agent: GPTBot
Disallow:
Nothing after the colon means nothing is blocked.
Disallow with a slash blocks the whole site. Disallow with nothing after the colon blocks nothing. One character is the whole difference. That is why this mistake survives for months.
One more line to know: User-agent: * means every crawler. A Disallow: / under it blocks them all, AI and Google alike. That one hides you from search too.
Check your firewall and your website builder
robots.txt is not the only gate. Cloudflare has a one-click setting that blocks AI crawlers. Some website builders and security plugins block them by default. The owner never sees a warning.
Check it: open your firewall or builder dashboard. Look for a setting with AI, bots or crawlers in the name. Read what it blocks before you trust the default. On WordPress, a free AI visibility plugin checks whether your robots.txt turns AI away.
How to unblock
Blocking AI on purpose is a fair choice for a site that sells its writing. A business that wants customers from AI wants the door open. Four steps:
- 1Find where your site edits robots.txt. Website builders keep it under SEO or crawler settings. Ask their support if you can't find it.
- 2Delete the Disallow lines under GPTBot, ClaudeBot, PerplexityBot and Google-Extended.
- 3Turn off firewall rules that block AI crawlers.
- 4Keep Googlebot and Bingbot open too. ChatGPT's live search leans on Bing, so blocking Bing hides you twice.
Unblocked is the floor, not the win
An open site lets AI read you. It does not make AI recommend you. Recommendations come from pages AI trusts that praise you. The full list of causes and fixes is in Why doesn't ChatGPT recommend my business? Unblock first. It is the fastest fix on the list.
