How Presence Scout Checks the AI Engines, and How Accurate It Is

A visibility score is only as good as the check behind it. So here is exactly what Presence Scout does when it checks whether AI engines mention your brand, and the limits every tracker in this category lives with, ours included.

Full disclosure: I make Presence Scout, so this is my own product. Everything below describes how it works today.

What happens in one check

A check is one buyer question, asked to one engine. For each brand you set the questions buyers ask, like "best CRM for a small agency", and the country you sell in. Then:

1. The question goes to the engine through its official API, with web search switched on, so the answer draws on today's web like the real product does.
2. The answer is stored in full: the text, the sources it cited and, for ChatGPT, the web searches it ran to write the answer.
3. A second model reads the answer and lists every brand it named, in order. That order is your position.
4. Your brand is matched against your name and the variants you confirm, so "HubSpot" and "HubSpot CRM" count as one company.

Every number in the dashboard links back to that stored answer, so you can read what the engine actually said.

How each engine is asked

ChatGPT, Gemini and Perplexity

• ChatGPT: OpenAI's API with its web search tool, on the same model ChatGPT serves its free users. I chose it on purpose: a more expensive model would answer differently from what most people see.
• Gemini: Google's Gemini API with Google Search grounding switched on.
• Perplexity: Perplexity's API, which always searches the web before answering.

Each engine is told the person asking is in your country, so it recommends brands that are actually available there.

Google AI Overviews and AI Mode

These live inside Google, so a chatbot can't stand in for them. Presence Scout reads real Google results for the question, from your country, and keeps the AI Overview and the AI Mode answer with the pages they cited. When Google shows no Overview for a question, that check names no brand at all, yours or any competitor's, so it never tilts the comparison.

Your Google rankings

Keyword rankings come from live Google results in your country, on desktop and mobile, down to position 50, including the local map pack where Google shows one.

When I moved desktop rankings to a new data source this month, I first compared 20 keywords from both sources side by side. Positions agreed within one or two places, and only the new source returned map pack results, so it was an easy switch.

How accurate is it?

Accurate about what it measures, and honest about what nobody can measure. Four things to know:

• No tracker sees a person's own chat. A logged-in user brings history and memory that change the answer. Trackers, ours included, see what a new visitor sees, which is the fair baseline for comparing brands.
• Answers vary from run to run. Ask the same question twice and the wording changes; sometimes the list does too. One answer proves little. Trends across many questions and weeks are what to trust.
• Location is set by country, not by street. City-level differences in AI answers are real, and a country-level check won't show them.
• Reading the answer is a model too. The same model reads every answer, so a ranking doesn't move because the reader changed, and each check records which model read it. When the main provider failed in September, I added a second one behind it and an hourly job that re-reads any answer that failed.

Where Presence Scout can't know something, the cell shows a plain dash and the reason sits behind it, rather than a guessed number.

Check us yourself

Don't take my word for any of this. Open any stored answer in Presence Scout, ask the same engine the same question, and compare which brands are named and in what order. I wrote a 5-minute test for exactly that, and it works on any tracker.

Checks run weekly, by design, so the allowance covers more of your questions and keywords instead of re-asking the same few daily. You can try it on the free plan, or see the whole picture on the AI search visibility page.

Google publishes how AI features in Search choose and show sources on its AI features documentation page, which is worth reading alongside any tracker's numbers.