preferly

Blog

We asked ChatGPT, Claude and Gemini the same 300 questions. For local businesses, they rarely agreed

By the Preferly team · Published

Short answer: for local businesses, mostly not. We asked ChatGPT, Claude and Gemini the same 300 questions, three times each, for 2,700 answers in all. On a local question, the three put three different businesses first in 74.3% of cases, and all three gave the same top pick in just 3.6%. For software, they agreed far more often.

This post is the summary. The full tables, method and data are in the study, Do ChatGPT, Claude and Gemini recommend the same businesses?

How we asked

We wrote 250 questions about local businesses, in 10 industries across 25 US cities, and 50 about software for a small business, one per category. A typical question was "Who is the best family law attorney in Houston, TX?" No question named a business. Each assistant answered every question three times between November 9 and 11, 2026, with web search on, and after each answer it listed every business it named and its top pick.

Six numbers from the study

A question-run is one question in one run, answered by all three assistants: 750 for local questions and 150 for software.

What we measured Result
Local question-runs with three different top picks 557 of 750 (74.3%)
Local question-runs with the same top pick from all three 27 of 750 (3.6%)
Local businesses named by one assistant only 87.4% of names
Software question-runs with a name all three share 125 of 150 (83.3%)
Local questions where ChatGPT gave the same top pick in all three runs 208 of 250 (83.2%)
ChatGPT's cited addresses, local, that were a named business's own site 66.5%

They rarely agree on local businesses

In about three of every four local question-runs, each assistant put a different business first. The share moved only between 71% and 77% from one run to the next, so it isn't the result of one odd day. Even a looser test, whether any single business appeared in all three answers, passed in only 91 of 750 question-runs (12.1%). Most names came from one assistant alone: 87.4% of the businesses named for a local question in a run were named by just one of the three.

Software is the opposite

For the 50 software questions, all three named at least one of the same products in 83.3% of question-runs. They gave three different top picks in 14.0% and the same top pick in 37.3%.

They also disagree with themselves

Asked the same local question in all three runs, ChatGPT gave the same top pick every time for 83.2% of questions. Gemini did for 39.6% and Claude for 39.2%. For software it was ChatGPT 76%, Gemini 84% and Claude 68%. Each of these findings gets its own post over the next few weeks.

They read different sources

Of the addresses each assistant cited for local questions, 66.5% of ChatGPT's were a named business's own site, against 33.2% of Gemini's and 7.4% of Claude's. Claude's most cited sites were directories, review and booking sites. They also answered in different shapes: on a local question, Claude named 6.5 businesses per answer on average, Gemini 4.4 and ChatGPT 1.5.

What it means for a business owner

  • One assistant naming you says little about the other two. For most local questions, the three named different businesses first.
  • A single check is one draw. Claude and Gemini changed their top pick between runs for most local questions. ChatGPT was steadier, but not fixed.
  • The sources differ, so the work differs. ChatGPT cited businesses' own sites most. Claude and Gemini leaned more on third-party sites.

To see what AI says when your customers ask, run the free check. It asks 3 AI models the questions your customers ask, without naming you. To find out who AI names instead of you and whether what it says matches your website, get the full $99 AI visibility audit, paid once. It asks 10 AI models.

Limits

  • No right answer. "Best" has no ground truth. This study measures agreement, not accuracy, so it can't say which assistant is right.
  • No cause. We measured that the sources differ. We didn't test whether the sources cause the different picks.
  • These channels, these dates. ChatGPT was run in its app, Claude as agents with web search and Gemini through its API, in US cities, in English, over three days in November 2026. A small follow-up test on 48 questions found that the channel changes the answers, though that test changed two things at once for ChatGPT and Claude: the channel, and asking one question per request instead of 20 per file. We didn't run the consumer Claude or Gemini apps.
  • Automatic name matching. A hand check of 120 questions disagreed with the matcher on 6 of 360 top-pick pairs (1.7%).

Data: Preferly, "Do ChatGPT, Claude and Gemini recommend the same businesses?" (2026), https://www.preferly.ai/research/who-ai-recommends. 2,700 answers from ChatGPT, Claude and Gemini to 300 questions, November 9 to 11, 2026. The per-question data is free to reuse under CC BY 4.0 with that credit.

Questions

Which assistant gives the best recommendations?

The study can't say. A question about the best business has no single right answer, so we measured whether the assistants agree with each other and with themselves, not whether any of them is accurate.

Did the questions name any business?

No. Every question asked for a recommendation, for example "Who is the best family law attorney in Houston, TX?", without naming a business.

Can I use the data?

Yes. The per-question data is free to reuse under CC BY 4.0. Credit Preferly and link to the study page.

Sources

Start free check

3 AI models, 5 customer questions, about a minute. Or check whether AI crawlers can reach your site.

Check your site free

No email. It reads your robots.txt and homepage, the way AI crawlers do.