AI search metrics are not one score
A single “AI visibility score” is easy to put on a dashboard and hard to defend. It usually hides the sample size, the question set, the date and the answer behind it. A useful measurement has five numbers, each with a denominator and a raw answer.
The unit is the answer. Ten buying questions, three logged-out runs each, is thirty answers. Percentages on three answers are misleading; report counts with the denominator until the sample is large enough.
The five numbers
- Mention rate: answers where the brand appears ÷ total answers. It answers “are we in the room?”, not “are we recommended?”.
- Recommendation rate: answers where the brand is put forward as a choice ÷ total answers. This is a reading judgement, so keep the sentence.
- First-mention position: the average position of the brand’s first appearance relative to named competitors. Report the distribution, not only the mean.
- Citation mix: the share of citations by source type — directories, community, editorial, first-party, video. It answers “which pages are doing the work?”.
- Variance: the share of questions whose mention status changed across runs. It answers “is this stable enough to act on?”.
What each number can and cannot tell you
| Metric | What it answers | What it cannot |
|---|---|---|
| Mention rate | Are you in the answer at all? | Whether the mention is positive or recommended. |
| Recommendation rate | Are you put forward as a choice? | Whether the buyer acts on it. |
| Position | Where you first appear against competitors. | Whether the order is stable across sessions. |
| Citation mix | Which pages the answer used. | Whether the cited page supports every claim. |
| Variance | How much the answer changes between runs. | Which change will persist. |
Share of voice, done honestly
Share of voice needs a denominator. Define it as the brand’s mentions divided by all mentions of the brand plus the three named competitors, across the same answer set. Say whether it is answer-level (one count per answer) or mention-level (every occurrence counts). Do not compare a brand’s answer-level share with a competitor’s mention-level share.
If a vendor will not state the denominator, the number is not comparable to anything.
What not to use as the primary metric
- “AI rank”: no stable ranking exists across logged-out sessions. A position without a date and a sample is a screenshot.
- Sentiment without reading: a naive sentiment score on a product answer is noise. Read the sentence or do not report it.
- AI referral traffic alone: useful, but undercounts. Many AI referrals arrive as direct or unknown. Use it as a lagging signal, not the baseline.
- One “visibility score”: unless the formula and the sample are public, it cannot support a decision.
A scorecard that survives a meeting
- State the question set, the engine, the location, the date and the number of runs.
- Report counts first, percentages second. “2 of 30” is more honest than “7%”.
- Keep the raw answer and the citations behind every count.
- Show variance; do not average it away.
- Re-run the identical set at day 60. Do not add questions after seeing the result.
Questions
What is a good mention rate?
There is no universal benchmark. It depends on the category, the question set and the competitors. Compare your own baseline over time, not a number from another brand.
How many answers do I need?
Ten questions × three runs is thirty answers. That is enough to see a pattern; it is not a statistical estimate of all buyers. Report it as a dated sample.
Should I track AI referral traffic?
Yes, as a lagging signal. It undercounts because many AI referrals arrive as direct or unknown. Do not use it as the only measure of visibility.
Run the free three-question check · See the $850 diagnostic · Read the method