How is an AI visibility score calculated, and how can I tell if a tool's metrics are trustworthy?
How AI visibility scores are calculated, what share of voice and position add to them, and a checklist for judging whether a tool's metrics can be trusted.
There is no standard formula for an AI visibility score. Each vendor builds its own, from a prompt set, a set of engines and a few counted outcomes, and then names the result something friendly. That is not a scandal, but it does mean two tools can give the same brand very different numbers on the same day. Before you trust one, you need to know what went into it.
What usually goes into a visibility score
Most scores combine some mix of these inputs:
- Mention rate. Of the prompts you track, in what share of answers is your brand named?
- Position. When you are named, are you first, or buried at the end of a list?
- Share of voice. Of all brand mentions across your tracked prompts, how many are yours versus competitors'?
- Sentiment. Is the mention positive, neutral or a warning?
- Citations. Is your site linked as a source, and how often?
Promptwatch reports these as separate figures rather than one opaque blend: Visibility Score, Share of Voice, Sentiment (a per-response score from 1 to 100, with a weekly AI summary) and Position in Answer. The numbers come from prompt panels run daily per model, country and language. Having the components visible matters, because a score that moves can then be explained.
Where scores go wrong
The score is only as good as the sample behind it. These are the places we would look.
The prompt set. If the prompts are generated by the vendor, a high score may only mean the questions were easy. If you write them, the score reflects your priorities. Look for a tool that lets you control prompts, tag them by topic and intent, and separate branded from non-branded questions.
Run-to-run variation. An AI answer is not a fixed page. The same prompt can return a different list tomorrow. One check is a sample, not a measurement. A daily panel, compared over weeks, gives you a trend you can lean on.
Real interface versus API. Consumer products like ChatGPT Search can answer differently from a raw API call. Promptwatch monitors the live interfaces for ChatGPT (Search), Google AI Overviews, Google AI Mode, Copilot, Gemini, Perplexity, Alexa and Meta AI, and tracks Claude, DeepSeek, Mistral and Grok via API. Ask every vendor which of their engines are which, and be wary of answers that blur it.
What counts as a mention. If your brand name is a common word, or customers use a short form, a naive text match misses or over-counts. Promptwatch handles this with brand aliases and a Brand Book whose instructions decide what counts as a mention.
Credit math. Some platforms meter usage in credits. AthenaHQ, for example, counts one credit as one AI response, and G2 reviewers report burning through credits early. Scrunch AI consumes prompt credits per engine. That affects how many prompts you can afford to run, and therefore how reliable your sample is.
A checklist for judging trust
Ask each vendor, in writing, and compare.
- Which engines are measured through the real interface, and which through an API?
- How often does each prompt run, and is that daily or weekly? Scrunch AI's own G2 reviewers flag weekly refresh as a weakness, so cadence is worth asking about directly.
- Can I see the underlying responses, not only the score?
- Can I define what counts as a mention, and add aliases?
- Can I export raw data? Promptwatch offers Excel and CSV exports with a column picker.
- Does the score reconcile with a manual check? One G2 theme for Profound is citation counts that do not match manual ChatGPT checks, so test this yourself on five prompts before signing.
- Is there a second source of truth outside the tool?
That last question is the most important and the one most buyers skip.
Corroborate with data you own
A visibility score tells you what the engines said. Your own logs and analytics tell you what happened on your site. Promptwatch pairs the two: Agent Analytics shows IP-verified AI crawler visits with a crawl-to-citation path and a citation rate per page, and visitor analytics shows AI-referred sessions and conversions. If the score climbs and AI-referred visits stay flat, that gap is worth investigating. If both rise, you have independent confirmation.
Tools that report only a score give you nothing to check against. Most of the trackers we compare, including AthenaHQ, Scrunch AI and Profound, do not publish a crawler-log product or conversion-level attribution at that depth.
Who should pick which
If you want a score you can interrogate, with components, prompt control, raw exports and an owned-data cross-check, Promptwatch is the one we would start with. If you want a GEO score with a built-in action queue and can budget for credit metering, look at AthenaHQ. If you need weekly reporting and crawler-side delivery, Scrunch AI is the option. If you are buying at enterprise scale, Profound is a different category, priced custom, and the same checklist applies.
The honest limit
No vendor can promise that a higher score means more revenue. The score is a proxy. Treat it as a trend line built from a defined sample, keep the prompt set stable, and check it against traffic you can verify. If a tool cannot explain its own number, that is your answer.