If your website is missing from ChatGPT's answers, the cause is almost always one of four things, in this order of likelihood: your robots.txt blocks OAI-SearchBot, your pages need JavaScript to show their content, the URLs ChatGPT tries are broken, or nothing on the page answers the question being asked. The first three are technical and fixable in days. Here is how to tell them apart.
Work through the causes in order and stop at the first one that is true for your site.
Key takeaways
- Opting out of OAI-SearchBot takes you out of ChatGPT's search answers, and OpenAI documents that explicitly.
- Blocking GPTBot is a training decision, not a search decision: the two bots need separate rules.
- No major AI crawler executes JavaScript, so a client-side page is an empty document to ChatGPT.
- One missing answer is an anecdote. Measure a mention rate over repeated runs before you call yourself invisible.
What does "not appearing in ChatGPT" actually mean?
Two problems hide behind the same complaint, and the fixes differ.
- ChatGPT never shows your page. You ask a question your page answers almost word for word, ChatGPT searches the web, and it cites three other sites. This is a retrieval problem, and it is technical.
- ChatGPT knows your category but not your brand. You ask "what is the best X for Y" and the answer names competitors. This is a visibility problem, and it moves on a slower clock.
Fix the first, because it blocks the second: a page ChatGPT cannot fetch or read is never cited, however good it is. The mechanism behind both is in How AI models choose which brands to cite.
Cause 1: is your robots.txt blocking the wrong bot?
The most common cause, and the fastest to fix. OpenAI runs several crawlers with different jobs, and blocking them costs you different things:
| User agent | What it does | What blocking it costs |
|---|---|---|
| OAI-SearchBot | Surfaces websites in ChatGPT's search features | Your site is not shown in ChatGPT's search answers |
| ChatGPT-User | Fetches a page for an action a person asked ChatGPT to perform | A page a user explicitly asked for may not be read |
| GPTBot | Collects content for OpenAI's generative models | Your content is not used for model training |
The middle column is OpenAI's own description of each bot, and the first row is the one people get wrong: OpenAI states that sites opted out of OAI-SearchBot are not shown in ChatGPT's search answers (OpenAI). Two mistakes cause most of the damage:
- A copied "block the AI bots" snippet. Many snippets in circulation disallow every AI user agent at once. Refusing training is a legitimate choice, but if OAI-SearchBot is on that list you have also opted out of ChatGPT's answers while your competitors stay in them.
- A staging rule that shipped.
User-agent: *withDisallow: /blocks OAI-SearchBot too. Read the live/robots.txton the production domain, not the copy in your repository.
User-initiated fetches are a separate category: OpenAI notes that robots.txt rules may not apply to ChatGPT-User, because the request follows an action taken by a person, while Anthropic states that ClaudeBot for training, Claude-SearchBot for search quality and Claude-User for user requests all honor robots.txt directives (Anthropic). There is no single AI bot switch, and being visible in one assistant does not carry over to another. Bot by bot, the full map is in AI crawlers explained.
Cause 2: does your page need JavaScript to show its content?
If your content arrives through client-side rendering, ChatGPT's crawler receives an empty shell. In a December 2024 study of the AI crawler traffic across their network, Vercel and MERJ found that none of the major AI crawlers from OpenAI, Anthropic and Perplexity rendered JavaScript, over roughly 960 million fetches in a month from GPTBot, Claude's crawlers and PerplexityBot; ChatGPT's crawler downloaded script files in 11.5% of its requests and Claude's in 23.8%, without executing them (Vercel and MERJ). In the same study Gemini, which runs on Googlebot's infrastructure, does render the page.
Google behaves differently, which is why this cause stays hidden for months: Google states that it can process content inside JavaScript as long as the scripts are not blocked (Google Search Central). A site can rank respectably on Google and still be a blank page for ChatGPT.
The test takes one command: fetch the page the way a crawler would, with curl -s https://yourdomain.com/your-page, and search the raw HTML for the sentence you expect an AI to quote. If it is not in that output, it is not in ChatGPT's copy of your page either. The fix is server-side rendering or prerendering of every public page.
Cause 3: can ChatGPT reach the URL at all?
Pages that are allowed and readable still get lost. In the same measurement, ChatGPT's crawler spent about 35% of its fetches on 404 pages and another 14% following redirects, so a large share of its crawling never reaches live content. Housekeeping matters more than it sounds:
- keep URLs stable, and when one has to change, use a single permanent redirect instead of a chain;
- fix internal links that point to dead pages, because crawlers discover URLs from your own HTML;
- keep the sitemap current, with every URL in it returning 200 and its real content;
- return a real 404 for missing pages instead of a soft 404 that answers 200 with an empty template;
- check that consent banners, country redirects and bot challenges do not sit between the crawler and the text.
Cause 4: does anything on the page answer the question?
If the first three are clean and you are still absent, the problem is the content. ChatGPT composes an answer and quotes what is easy to lift. The pages that win carry a self-contained answer of two or three sentences at the top of each section, headings shaped like the questions people actually type, and specifics (definitions, numbers, dates) where most sites put adjectives.
This is measurable rather than folklore. The research paper that named GEO rewrote pages with this kind of change and measured up to 40% more visibility in generative engine answers across a benchmark of 10,000 queries from nine sources, with results that varied by domain (Aggarwal et al., KDD 2024). The patterns are collected in What is GEO? A practical guide.
Does llms.txt get you into ChatGPT?
Nothing in the providers' documentation says it does. llms.txt is an optional convention, a plain-text map of your site for AI systems. Google clarified in its June 2026 Search Central changelog that these files are not needed for Google Search and neither help nor harm visibility or rankings, and its AI optimization guide adds that no machine-readable file and no special schema are required for its generative features, which rest on the same ranking and quality systems as ordinary Search. OpenAI and Anthropic both document robots.txt, not llms.txt, as the way to manage their crawlers. Publish one if a tool you use consumes it, once the four causes above are closed. The longer version of this answer is in Does llms.txt improve Google rankings?.
Are you invisible, or were you unlucky?
Check how you measured before you conclude anything. ChatGPT's answers are not deterministic: the same prompt, same model, same day can name you once and skip you the next time. What you need is a mention rate, from a fixed set of realistic prompts, run several times in clean sessions and repeated over weeks. The method is in How to check if ChatGPT mentions your brand, and the shift from a stable rank to a sampled rate is the heart of GEO vs SEO.
On paid plans, Huntair SEO runs the prompts you choose across ChatGPT, Gemini, Claude and Perplexity on a schedule and tracks your mention rate next to your Google keyword rankings, so a fix you ship in week one has a number attached to it in week four.
A 30-minute diagnostic
- Open
/robots.txton the production domain and look for OAI-SearchBot, by name and inside anyUser-agent: *block. Decide about GPTBot as a separate question. - Fetch an important page without a browser and search the HTML for the sentence you expect to be quoted. If it is missing, prerender the page.
- Confirm that the page returns 200, is in your sitemap, and is not behind a redirect chain or a consent wall.
- Ask ChatGPT five questions your customers ask, in fresh sessions, without your brand name, note who gets cited instead of you, and repeat four weeks after the fixes.
FAQ
I blocked GPTBot. Does that keep me out of ChatGPT's answers?
No. GPTBot governs whether your content may be used to train OpenAI's models. Appearing in ChatGPT's search answers depends on a separate user agent, OAI-SearchBot, and OpenAI states that sites opted out of it are not shown in those answers. You can refuse training and stay visible in search.
My site ranks well on Google. Why is ChatGPT still not citing it?
Because the two crawlers do different amounts of work. Google states it can process content rendered by JavaScript when the scripts are reachable, while the major AI crawlers from OpenAI, Anthropic and Perplexity do not execute JavaScript at all. A site built as a client-side app can hold a strong Google ranking and still hand ChatGPT an empty document.
How long does it take to appear once I have fixed this?
Weeks rather than days, and it depends on the cause you fixed. Unblocking a search crawler and publishing readable HTML only pay off once those pages are fetched again, and the model still has to prefer your page to the ones it already uses. Re-measure four weeks later instead of checking the next morning.
Does adding FAQ or Article schema get me cited by AI models?
It helps machines read your page, but it is not an entry ticket. Google states that structured data is not required for its generative AI features and that there is no special markup to add for them. Structured data still earns its place in the rest of your SEO, so keep it accurate and identical to the text a reader sees.