The seven questions
- What is the question set? A tool should show the exact buyer questions it asked. “Your visibility score: 62” without the questions is not checkable.
- How many runs per question? One run is a sample. Answers change between sessions — in our 45-answer study, the brand set was identical in only 6 of 15 repeated questions. A tool that cannot show repeated runs cannot separate movement from variance.
- Are the answers dated? An AI answer is a dated artifact. Undated rows cannot be compared, and any “change” report without dates is unfalsifiable.
- Does it trace the citation owner? Being mentioned is the easy half. The tool should show which domain was cited for each answer — and whether it was yours.
- Is the competitor set fixed? A fixed set of three named competitors makes trends comparable week to week. An open “who else appeared” list is useful context, not a baseline.
- What is the output format? A dashboard is not a deliverable. Ask for a document with the answers, the sources and the dates — something a client can read without the tool open.
- Is there a day-60 re-run? Without a dated re-check after your content changes, “improvement” is an assumption.
Red flags worth walking away from
- A single number with no question set or dates.
- “We monitor ChatGPT” with no citation list — mention without citation ownership hides the actual risk.
- Rank-like precision (“you are #4 in ChatGPT”) with no sampling plan.
- Reports that cannot be reproduced on the same day.
What a defensible measurement looks like
Ten buyer questions, three runs per question across three days, every cited source traced against three named competitors, delivered as a dated document, with a day-60 re-run. That is the format we sell; the free check here is the one-run sample of the same method.
Evidence behind the claim: the 45-answer variance pilot · the 37-brand pilot index · method note.
How to try the method for free
Run the free one-run check with one category question. If the answer names you but cites a third-party page, you have found the gap that a paid diagnostic maps across ten questions. If it does not name you at all, you have a content decision before you have a measurement problem.
FAQ
How do I choose an AI visibility tool?
Ask for the exact buyer questions, the number of runs per question, the date and engine of each answer, the citation owner for every answer, and the deliverable format. If any of those is missing, the number cannot be checked.
What is a good AI visibility score?
A score is only comparable within the same question set and competitor set. Across our pilot, 74% of answers named the brand while 30% cited the brand’s own site — the gap matters more than the score.
How many runs per question do I need?
At least three, on separate days. In our 45-answer study the brand set was identical in only 6 of 15 repeated questions, so a single run cannot separate movement from variance.
Is a dashboard enough as a deliverable?
No. Ask for a dated document with the answers, the cited sources and the question list. A dashboard is not evidence a client can read or a team can act on.
Related
AI SEO audit: the 8 checks · ChatGPT brand tracker: what to record · ChatGPT brand monitoring: 5 numbers · For media