The short version
- Assistants that search the web can only cite pages their crawlers may fetch, so check robots.txt first.
- Google says a page must be indexed and eligible for a snippet to appear in AI Overviews or AI Mode, with no extra requirements.
- Write one clear page that says what the product is and who it’s for, in text a crawler can read.
- Assistants also show what other sites say about you, so real reviews, comparisons and community threads matter.
- Nobody outside these companies can guarantee a mention, and most “GEO” advice comes from people selling it.
How assistants find pages
An assistant can answer from what its model learned in training, or it can search the web and cite what it finds. The second path is the one each company documents, and the one you can act on.
- ChatGPT search. OpenAI says its
OAI-SearchBotcrawler is used to surface websites in ChatGPT’s search features, and that sites which opt out of it won’t be shown in ChatGPT search answers, though they can still appear as navigational links (OpenAI’s crawler docs). - Perplexity. Perplexity says it searches the internet in real time and that each answer includes numbered citations linking to the sources (help centre). Its
PerplexityBotcrawler is described as designed to surface and link websites in Perplexity’s search results (Perplexity’s crawler docs). - Google AI Overviews and AI Mode. Both draw on Google’s Search index. Google says they may use a “query fan-out” technique, issuing multiple related searches across subtopics, and that a page must be indexed and eligible to be shown with a snippet to appear as a supporting link (AI features and your website).
- Claude. Anthropic says Claude invokes a search tool when a question benefits from current information and that responses include citations (help centre). It lists a
Claude-SearchBotcrawler used to improve search results (crawler article). - Microsoft Copilot. Bing Webmaster Tools has an AI Performance report that shows how often your pages are cited in Copilot and in AI-generated summaries in Bing (Bing’s announcement).
None of these pages explains how sources are ranked or chosen once a page is eligible. That part is not public.
Let the right crawlers in
A page that a search crawler can’t fetch can’t be cited from a search. Open yoursite.com/robots.txt and make sure none of these are disallowed, either by name or by a blanket User-agent: * rule:
OAI-SearchBotfor ChatGPT search. OpenAI recommends allowing it and notes that a robots.txt change can take about 24 hours to be picked up.PerplexityBotfor Perplexity. Perplexity gives the same advice and the same 24-hour delay.Googlebotfor Google’s AI features. Google says robots.txt rules for Googlebot are the control for access to AI Overviews and AI Mode.Bingbotfor Bing and Copilot, andClaude-SearchBotfor Claude.
Check the layers in front of your site too. A firewall rule, a bot-protection setting or a host default can block these user agents even when robots.txt allows them. OpenAI and Perplexity both publish the IP ranges their crawlers use, so you can allow those specifically.
Training crawlers such as GPTBot are a separate decision. You can block them and still appear in search answers. Should you block AI crawlers? covers each token and the trade-off.
Get indexed in Google and Bing first
For Google’s AI features, indexing is the documented entry requirement. Google states there are no additional technical requirements and no special optimisations: you don’t need new machine-readable files, AI text files or special schema.org markup (source). Its newer guide adds that a site must be included in Search generative AI features in Search Console to be eligible, so look for that setting when you verify your site (AI optimisation guide).
Bing matters for the same reason: Copilot’s citations are reported in Bing Webmaster Tools alongside your search data. So the base work is ordinary indexing:
- Verify the site in Google Search Console and Bing Webmaster Tools.
- Submit a sitemap to both.
- Confirm your key pages are indexed, with no stray
noindexor wrong canonical. The indexability checker tests a page for these.
How to get indexed by Google and Bing walks through each step. Whether ChatGPT or Perplexity lean on third-party search indexes for a given answer isn’t something their crawler docs state, so treat being indexed in Google and Bing as necessary groundwork, not as proof you’ll appear elsewhere.
Say plainly what the product is
An assistant that fetches your homepage has to work out what you make from the text on it. Google’s guide says the same content advice applies to AI features as to search: write from what you know, organise pages into sections with headings, and don’t chop content into small pieces for AI, because there is no such requirement (source). In practice that means:
- One sentence near the top that names the product, the category and the user. “Acme is an invoicing app for freelance designers” can be quoted. “Work, reimagined” can’t.
- The facts people ask assistants about, written out as text: what it does, who it’s for, platforms, pricing, and what makes it different from the obvious alternative.
- Content in the HTML your server sends. Google asks you to follow its usual JavaScript practices, and you shouldn’t assume every assistant’s fetcher runs scripts.
- A title and meta description that match the page. Check them with the meta tag checker.
Use the same product name everywhere: site, app stores, social profiles, listings. If the name is a common word, pair it with the category in titles and descriptions (“Acme invoicing”) so a page about you can’t be mistaken for a page about something else. This is common sense about ambiguity, not a documented ranking factor.
Get described on other sites
When someone asks for “the best tool for X”, an assistant that searches will read whatever pages rank for that kind of query, and those are usually not your homepage. Google says so directly: its generative AI features can show what’s being said about products and services across the web. In the same passage it warns that seeking inauthentic mentions isn’t as helpful as it might seem (source).
So the work is to be described, accurately, in places that already get found:
- Reviews and comparisons. Category round-ups, review sites and “alternatives to” pages. You can also publish honest comparison pages of your own; see the guide to alternatives pages.
- Community threads. Answers in forums and communities where your users ask for recommendations. Disclose that you’re the maker.
- Launch boards and directories. A listing with its own page, your description and a link gives crawlers a second, consistent statement of what the product is. Where to launch covers how to choose them.
Give each of these the same one-sentence description you use on your site. Paid placements dressed up as independent reviews, and mass-posted mentions, are the inauthentic kind Google is describing.
What nobody can promise
Be clear about the limits before you spend money or weeks on this:
- No company publishes how its assistant picks sources. The docs cover access and eligibility. They don’t offer a way to submit a product, request a mention or influence an answer.
- Nobody outside these companies can guarantee a mention. Google notes that no third-party tool has access to its internal ranking or AI systems, and suggests checking any “AEO” or “GEO” advice or service against its guidance on evaluating third-party SEO advice (source).
- Most of the advice in circulation comes from agencies and tool vendors. They sell the service or the tracking software, so they have a commercial reason to say the problem is large and solvable. Tactics such as special file formats, “AI-friendly” rewrites and content chunking are not in any of the primary docs cited here, and Google’s guide lists several of them as unnecessary.
- What a model remembers from training is not documented either. Whether a small product shows up in an answer given without a search depends on training data nobody outside can inspect.
What you can do is measure. Ask the assistants the questions your users would ask and write down what comes back; How to test whether AI assistants know your product gives a repeatable method. Google’s Search Console and Bing Webmaster Tools both report on AI features now, which gives you first-party numbers for two of the systems.
More guides
- llms.txt: what it does and what it doesn’tWhat the llms.txt proposal is, who has said they use it, where it helps (documentation and coding agents), and what not to expect from it.
- Should you block AI crawlers?The documented AI crawlers from OpenAI, Anthropic, Perplexity, Google, Apple, Common Crawl and Bing, what each does, and a robots.txt that blocks training only.
- How to test whether AI assistants know your productA repeatable manual test for what AI assistants say about your product: the questions, clean sessions, a results sheet, server-log checks and what to fix.