What one measurement is
- Prompts. Questions a buyer would ask an AI assistant. A free check generates 3 from the brand name and industry; monitoring uses the prompts you set.
- Engines. Each prompt is sent, with web search on, to 4 engines (table below). The verbatim answer and any citations the engine returns are stored.
- Analysis. A model reads each answer into structured fields: mentioned or not, rank, sentiment, role, a context snippet, cited URLs and the competitors named.
- Scores. The fields are aggregated into the four scores defined below and, for monitored brands, into the visibility ladder.
The engines
| Engine | Model called | Web search |
|---|---|---|
| ChatGPT | gpt-4o-mini (OpenAI) | OpenAI web-search tool, medium search context |
| Claude | claude-haiku-4-5 (Anthropic) | Anthropic web-search tool, up to 3 searches per answer |
| Gemini | gemini-2.5-flash (Google) | Google Search grounding; redirect URLs resolved to the real source |
| Perplexity | sonar (Perplexity) | Built-in search; citations returned by the API |
All calls go through Vercel AI Gateway. Model versions are pinned in the code and changed only with a note here.
The analysis
Each answer is passed to gpt-4o-mini with the brand name and asked for a JSON object validated against a fixed schema: mentioned, rank, sentiment (−1 to 1), role (primary recommendation, alternative, neutral comparison, negative example, background mention or not mentioned), a context snippet, citations and competitorMentions.
A brand is never counted as its own competitor. If the analysis model is unavailable, a rule-based matcher is used instead and the result is stored with analysisMethod=fallback so it can be told apart.
The scores
- Visibility score
- answers that mention the brand ÷ all answers × 100
- Recall prompts ("What is <brand>?") are excluded — they name the brand by construction.
- Average rank
- mean position of the brand among answers that mention it
- 1 = first brand named. Empty when never mentioned.
- Sentiment
- mean of per-answer sentiment (−1 to 1) over answers that mention the brand
- 0 when never mentioned; it is not a rating of the brand, only of how the answer talks about it.
- Share of voice
- brand mentions ÷ (brand mentions + competitor mentions) × 100
- Competitors are the other brands the engine named in the same answers.
Scores are rounded to one decimal. A free check is 3 prompts × 4 engines = 12 answers, one run. Monitoring runs every prompt daily and keeps the history.
The visibility ladder
For monitored brands the same answers are also read as a ladder, so that 0% is a rung, not a verdict. The highest rung with evidence wins.
- 0AI doesn't know you yet — No engine mentions the brand on any prompt, including when asked by name.
- 1AI knows you exist — At least one engine describes the brand when asked by name (the recall prompt).
- 2AI mentions you for niche questions — Mentioned in at least one niche or local answer.
- 3AI recommends you for your category — Recommended in at least one category answer (discovery, comparison, use-case or recommendation prompts).
- 4AI cites your website — The brand's own website appears among the sources an engine cited.
Known limitations
- Engines are non-deterministic: the same prompt can get a different answer a minute later. One run is a snapshot; monitoring over days is the measurement.
- We call the models through their APIs with web search on. Consumer apps (chatgpt.com, the Gemini app) may use different models, settings and personalisation, so what a specific user sees can differ.
- The analysis step is itself a model (gpt-4o-mini) reading the answer into structured fields. It can misread a mention; when the model is unavailable we fall back to rule-based matching and mark the result analysisMethod=fallback.
- A free check uses three generated prompts; a small prompt set means each answer moves the score by a lot. Monitoring lets you set the prompts that matter.
- Aggregate statistics come from a self-selected sample — brands whose owners chose to run a check — and are not a census of any industry.
- Google AI Overviews are not part of the score today.
Reproduce it
scripts/evals/measure-evidence.ts— the read-only query behind the aggregate statistics (2,611 free checks as of the date on the evidence page).- The free check — the same measurement on any brand, no account needed.