Back to BlogSEO

Best AI Search Engines: How Each One Picks Sources

SimranOctober 31, 20269 min read
Best AI Search Engines: How Each One Picks Sources

Google says its AI Overviews reach more than two billion people a month, and they now sit above the results for a large share of commercial searches. The engines that matter to a commerce brand are Google AI Overviews and AI Mode, ChatGPT, Gemini and Claude, and each one finds your pages through a different crawl, so a page can be quoted in one and invisible in another.

What AI search actually is

Strip the branding and three products are in play.

AI Overviews are a generated summary above Google's organic results, with a handful of citation links. AI Mode is Google's conversational layer: one question becomes several sub-queries, Google runs them in parallel, then writes an answer. Assistants such as ChatGPT, Gemini, Copilot and Claude hold the conversation and fetch pages to support it.

Every one of them is two systems stacked. Retrieval is a crawl, an index and a query. Generation is a model reading the passages retrieval handed it and writing something shorter. You cannot influence the second system directly. You can absolutely influence which passages reach it.

Comparison of ChatGPT, Google AI Overviews, Gemini, Claude and Copilot showing which search index each one answers from
The right-hand column is the part you can act on: each engine answers from an index you can either be in or out of.

Different front doors, one building. The content that wins in all five is the same content, formatted so a machine can lift a paragraph without guessing. Two consequences follow. Classic SEO is not replaced by any of this. And the formatting work AI engines reward is the same work that makes a product page easier for a human to skim.

The five engines, and what each one really does

ChatGPT is the largest assistant surface and the one buyers use for "what should I buy" questions. It pulls from its own crawl and from Bing, and OpenAI runs separate agents for search (OAI-SearchBot), training (GPTBot) and user-triggered fetches (ChatGPT-User). Its shopping features opened with Shopify merchants and Etsy sellers, so product data now has a second distribution channel.

Google AI Overviews and AI Mode share the Google index, so the entry ticket is a normal ranking. Studies of AI Overview citations keep finding the same pattern: most cited pages were already in the top ten for the query. AI Mode is looser, because fan-out pulls in pages that answer sub-questions the original query never mentioned. Narrow pages with specific answers do well there.

Gemini runs on the same index with a longer leash on answer length, and its output gets reused across Workspace and Android surfaces. It rewards a clean hierarchy and a real table.

Claude is smaller in raw traffic and over-indexed with technical and B2B buyers, which is the audience for a store selling components, equipment or software-adjacent products. It reads long passages and quotes them closely, so precision beats persuasion.

Copilot is Bing with a chat window. If Bing Webmaster Tools shows no impressions for a category, Copilot has nothing to say about it. Perplexity is worth a look for research-heavy categories, but it is not where a Shopify brand's volume sits.

How an engine decides what to cite

There is no single ranking to win. Each system runs three steps, and each step drops you for a different reason.

Retrieval decides whether you are in the candidate set at all. For Google that means the index and a ranking. For ChatGPT and Copilot it means the Bing index plus whatever OpenAI has crawled. A page that is blocked, orphaned or rendered only in the browser never enters the room.

Passage selection decides which part of the page gets read. Models work in chunks. A heading followed by a tight paragraph is one clean chunk. A 900-word brand story that mentions the answer once is a chunk that says nothing.

Entity resolution decides who you are. The model wants to know that the thing it is quoting is a real company selling a real product in a defined category. Consistent naming, a stated category, schema markup and third-party mentions all feed that judgement.

You can watch this happen. Ask an engine a question your page answers and see which sentence it repeats back. If it paraphrases something from paragraph nine, your opening is decoration. If it cites a competitor for a fact you also publish, the passage was better shaped on their side.

What actually gets a page cited

Five factors that drive AI citations weighted from high to low, led by extractable answers and third-party corroboration
Weights are our read after auditing citation patterns, not a published algorithm. Treat them as priorities, not physics.

Extractable answers. Every important heading should be followed by a direct answer in the first 40 to 60 words. Not a wind-up, not a restatement of the question. "A Shopify review app costs between $0 and $299 a month, and free tiers cover most stores under 500 orders" is quotable. A paragraph that circles that sentence is not.

Third-party corroboration. Models sample consensus. When four review sites, a supplier page and a forum thread describe your product the same way, that description becomes the answer. This is the hardest factor to fake and the one most brands skip.

Entity clarity. Use the same company name everywhere. Use the same category label on your homepage, About page, Google Business Profile and schema. Put Organization markup on the home page with the same logo and the same profile links you use elsewhere. Ambiguity gets resolved in favour of the other guy.

Crawlable rendering. Server-rendered HTML is the baseline, and it is the same fight as Shopify page speed with a different judge: a slow page costs you a ranking, an unrendered page costs you the citation entirely.

Freshness. A visible date and current prices help. So does updating specifications when a product changes. Stale facts get corrected by the model, which usually means it cites whoever is current.

The Shopify work: making pages extractable

Table showing which parts of a Shopify product page an AI crawler can read, including specifications, reviews and accordion copy
Accordion copy is usually safe because it is real text behind CSS. App widgets that arrive after load are not.

Product pages carry most of the money and most of the mess. Three fixes cover the bulk of it.

Write specifications as text, not as a spec image. Materials, dimensions, compatibility, what is in the box, all in a table or a definition list under a heading called "Specifications". If a value changes a buying decision, it should exist as characters on the page.

Get the reviews into the DOM. The better review apps ship review text and ratings in crawlable markup, and the cheap ones inject everything after load. If your schema claims 4.8 stars and the page body contains no review copy, you have handed an engine a number with nothing behind it.

Make the variant content real. Size charts, compatibility notes and "which model do I need" guidance are the questions buyers actually ask, and on too many stores they live inside an app widget. Put the shared answers in the product description where everything can read them.

Answer format by page type on a Shopify store, showing the opening move and the proof to add for product, collection, comparison and guide pages
Four page types, one rule: the answer comes before the persuasion, and a fact follows it.

Collection pages need a definition, not a grid. Forty words at the top that say what the category is, who it is for and how the range is organised. That paragraph is what gets quoted when someone asks an engine to explain two product types. Pick the collections by commercial intent, not by catalogue order.

Then the plumbing: product structured data for price, availability, ratings and variants, Organization markup on the home page, and a heading hierarchy that mirrors the questions in your prompt set. Our Shopify SEO checklist covers the order of operations.

Crawler access, and the blocks nobody notices

This is the cheapest fix here and the one most stores fail. Check robots.txt and your firewall rules against six tokens.

CrawlerTokenWhat blocking it actually costs you
Google SearchGooglebotEverything. AI Overviews draw from this index.
Gemini trainingGoogle-ExtendedNothing in Search or AI Overviews. It governs training and grounding only.
ChatGPT searchOAI-SearchBotYour place in the index ChatGPT answers from, which is the one that sends visitors.
ChatGPT trainingGPTBotOpts you out of model training. No effect on citations today.
ClaudeClaudeBot, Claude-SearchBotClaude's crawl and its search index.
Bing and CopilotbingbotThe source Copilot answers from, and half of ChatGPT's retrieval.

Submit your sitemap in Bing Webmaster Tools as well. One submission covers Copilot and a large share of ChatGPT's retrieval, which makes it the highest-return twenty minutes in this section.

Then check the layer above robots.txt. Cloudflare's bot controls, a few Shopify apps and most "security" configurations will challenge AI user agents by default. If someone switched on an AI-blocking toggle last year and wondered why citations never arrived, that is usually the answer. Most AI crawlers do not execute JavaScript either, so anything that only exists after hydration does not exist.

Why llms.txt is mostly theatre

In late 2024 a proposal appeared for a file called llms.txt: a markdown index at the domain root listing your important pages with short descriptions, so a language model can find the good material quickly. It is a reasonable idea. Almost nobody uses it.

Google has said it does not consume the file. OpenAI has not adopted it for ChatGPT's answers. Anthropic has not either. Ask the engineers who build the retrieval systems and the answer is consistent: they crawl HTML, they index HTML, and there is no pipeline reading your markdown wishlist. The file costs an afternoon and produces a pleasant feeling of having done something.

It caught on because it resembles robots.txt, and robots.txt demonstrably works. But robots.txt works because crawlers agreed to read it decades ago. llms.txt has no such agreement, and a file nobody fetches changes nothing.

What to do instead, in order of effect:

  1. Serve real HTML, with one H1 and headings that match the questions buyers ask.
  2. Allow the crawlers listed above, and re-check quarterly, because a theme update or a firewall rule can undo it.
  3. Make facts machine-readable with Product, Offer, AggregateRating, Organization and BreadcrumbList markup.
  4. Get named elsewhere: reviews, directories, supplier pages, forum threads, industry listicles. Citations are won in public.
  5. Keep one source of truth for prices and specifications, and update it. Nothing loses a citation faster than a stale number.

If a stakeholder insists on an llms.txt, publish it as documentation. Just do not count it as a channel, and do not let it delay the work that moves citations. The AI SEO service we run does not bill for it.

How to measure AI visibility without fooling yourself

Five step monthly AI visibility audit starting with crawler logs and ending with earning a third-party mention
Step one is free and takes twenty minutes. It also explains most of the misses you will find in step three.

Start with traffic, but read it correctly. GA4 groups assistant referrals into an "AI Assistant" channel, and the raw referrers show up as chatgpt.com, gemini.google.com, copilot.microsoft.com and claude.ai. Judge it on conversion rate, which runs high, and on which pages get cited.

Expect undercounting. A lot of assistant sessions arrive as direct traffic because the referrer gets stripped, the same reason direct traffic in GA4 is always inflated. Pair the referral report with a checkout survey question and you get closer to the truth.

Server logs are the leading indicator. Count hits from OAI-SearchBot, Googlebot, ClaudeBot and bingbot by path, once a month. Crawl frequency moves weeks before citations do, and a sudden drop usually means a robots change or a firewall rule rather than a content problem.

Then the prompt audit, which is the only direct measurement. Write 25 questions a buyer would ask before choosing a category, not questions containing your brand name. Run them logged out in ChatGPT, Google AI Mode, Gemini and Claude. Record which URLs get named, which get cited, and which competitors appear. Do it on the first of the month and keep the sheet.

Citation share is the number to report: the percentage of tracked prompts where your domain appears. It moves slowly, in steps, and usually after a third-party mention lands rather than after a rewrite. Pair it with branded search volume and revenue per session and you have an argument a finance team will accept. Our SEO ROI framework is the version of that we use in reporting.

What to do this month

Pick one collection, not the whole catalogue.

Week one, fix the page. Write the 40-word definition at the top. Make sure the products under it have text specifications and review copy in the HTML. Confirm the crawlers can reach it, then ask an engine the question the collection is meant to answer and read what comes back.

Week two, write the prompts. Twenty-five questions a buyer would ask before choosing a category. Run them logged out and log the citations, including the competitors you keep seeing. That sheet becomes the baseline you argue from in three months.

Week three, earn one mention. A supplier page, a review site, a forum thread. Pick the placement your competitors already occupy.

Week four, do nothing and let the numbers arrive. Citation share moves on a quarterly clock, not a weekly one, and a store that rewrites everything at once cannot tell which change worked.

That loop, run across a catalogue over a few quarters, is the work. Our Shopify SEO team runs it alongside the technical fixes, and the stores that win the answer layer are almost always the ones that had already done the boring structural work.

Written by

Simran

Editorial Team

The strategy team behind AAA Digital's Shopify growth work.

AAA Digital brand mascot

WANT RESULTS LIKE THESE?

We implement these exact strategies for brands that want to scale. Book a free strategy call and let us audit your current setup.

Book a Free Strategy Call