preferly

You saw PreferlyBot in your logs. Here’s what it was doing.

Someone asked Preferly how AI assistants see a business, and your site was part of that check. We read some of your pages the way an AI agent would, so we can compare what AI says with what your site says.

What it fetches

  • Your robots.txt, to see which AI crawlers you allow.
  • Your homepage and some of the pages it links to, such as about, contact and pricing. These are plain page requests: no JavaScript runs.
  • Up to 25 of your site’s own links, to find any that are broken.
  • For a full report, one more test in a real browser: it starts at your homepage and tries to reach your booking page, contact form or similar, the way an AI agent acting for a customer would. This browser runs JavaScript, follows links and clicks buttons that don’t submit anything.

The test agent and the link checks follow your robots.txt rules for PreferlyBot.

It only runs when someone requests a check, so it visits rarely.

One exception: once, in October 2026, we ran it for a public research study of 4,111 small business websites. It made the free AI crawler check’s two requests to each site (robots.txt, then the homepage), fetched the homepage only where robots.txt allowed PreferlyBot, and sent one request at a time to each site. Sites that returned an error or a bot check were checked once more a few minutes later.

The free AI crawler check, on our website or in ChatGPT, reads only robots.txt, the homepage and /llms.txt, when someone asks for it.

What it never does

  • It never types, fills in fields, submits forms, books, buys or signs in.
  • It never clicks to accept cookies or terms, and never solves a CAPTCHA.
  • Its browser only lets requests that read a page through (GET, HEAD and OPTIONS) and blocks everything else, so it can’t send anything to your site.
  • It never disguises itself or tries to get past bot protection. If your site challenges it, we report that and stop.

How it identifies itself

Every request carries this User-Agent: PreferlyBot/1.0 (+https://preferly.ai/bot)

Its requests are also signed with Web Bot Auth, so your site or CDN can verify they really come from PreferlyBot. Our public keys are in our key directory.