What AI Visibility Tools Disclose About How They Measure
An AI visibility score is a sample, and a sample can only be checked if its method is known.
Key Findings
- 01No vendor's public documentation answers all five questions a buyer needs to reproduce its numbers. Across 16 vendors and five questions, 20 of 80 answers were stated in full, 28 in part and 32 not at all on the pages we read. No vendor stated more than three of the five in full.
- 02Collection channel is the one thing most vendors explain. Ten of 16 say which channel each engine is collected through, and only one says nothing about it.
- 03Vendors split three ways on the channel. One collects only through model APIs, five only through the consumer interface, and seven mix the two by engine.
- 04Only one vendor publishes a run count above one. Evertune states 100 runs per prompt per model. Four state one run per cycle, and nine give a schedule or say runs are repeated, without a count.
- 05Language is the least controlled input. Only four vendors describe how both country and language are set. One states plainly that it does not force the answer language; another writes the country into the prompt instead of sending the prompt from that country.
- 06Only one vendor documents which model version produced each answer. Peec AI records the model behind every response. Profound marks platform model changes on customer charts automatically. Ten vendors say nothing about the model.
- 07No vendor says what happens to past data when its own method changes. Four cover related cases, Profound most fully for changes to a customer's configuration. Two vendors that publicly switched or rebuilt their collection say nothing about the data collected before.
- 08Documentation sometimes contradicts itself. One vendor's pages disagree with each other on channel, and another vendor's single data page gives two cadences. Separately, AuthorityTech found that one vendor redefined its headline metric on a page that shows only a relative update date.
Why We Ran This Audit
An AI visibility score is a sample, and a sample can only be checked if its method is known. Published research shows how much the method moves the result. Brand overlap between API answers and interface answers for the same prompt ranged from 6% to 12% in one study (Brandlight) and from 21% to 32% in another, after brand-name variants were merged (Surfer). Repeated runs of one prompt rarely return the same list (Search Engine Journal on SparkToro). Markets diverge too: in one five-country study, the share of ChatGPT answers with no citation reached 85% in Germany (seoClarity).
In September 2026, AuthorityTech audited the public documentation of nine platforms for how they define and calculate their headline metric (AuthorityTech). That audit covered formulas, denominators and run counts. It did not audit collection channel, geography or model version. This audit covers those three, plus run counts and how each vendor treats past data when its method changes.
The question is narrow: from public pages alone, can a buyer tell how a vendor's numbers are produced? We did not test any tool.
How We Chose the Vendors and Read Their Documentation
Selection. We included any AI visibility vendor that publishes a page describing how it collects answers: a help article, documentation, an FAQ or a methodology post. We found them by web search between 13 and 18 November 2026 and added two widely used SEO suites with AI visibility modules, Ahrefs and Semrush. Eighteen vendors qualified; two were later excluded, leaving 16 (see Limitations).
Reading. For each vendor we read its main methodology page and the pages linked from it that a buyer would reach. Where a vendor publishes a documentation index, we checked it for pages on location, models and changes. Every page and its check date is in the Source Register.
The five questions. Each one is something a buyer needs to reproduce or compare a number:
- 01Channel. Is each engine collected through its API, its consumer interface, or both?
- 02Runs. How many runs per prompt sit behind a reported figure, and how often?
- 03Country and language. Where are answers collected from, and in what language?
- 04Model version. Which model produced the answer?
- 05History. What happens to past data when the method changes?
Classification. Each answer was put in one of three groups.
| Group | Meaning |
|---|---|
| Stated | Specific enough to reproduce, for every engine the vendor tracks |
| Partial | An answer exists but lacks a number or an engine, or another page from the same vendor contradicts it |
| Not stated | No answer on the pages we read |
The Audit at a Glance
Disclosure falls off sharply after the first question. Most vendors say how they collect answers; almost none say which model produced them or what happens to history.
| Question | Stated | Partial | Not stated |
|---|---|---|---|
| 1. Channel | 10 | 5 | 1 |
| 2. Runs | 5 | 9 | 2 |
| 3. Country and language | 4 | 5 | 7 |
| 4. Model version | 1 | 5 | 10 |
| 5. History | 0 | 4 | 11 |
| All five | 20 | 28 | 32 |
The full results, vendor by vendor, alphabetical. Each vendor name links to the page we relied on most; every page read is in the Source Register.
| Vendor | 1. Channel | 2. Runs | 3. Country and language | 4. Model version | 5. History |
|---|---|---|---|---|---|
| Ahrefs Brand Radar | Stated: web interfaces for chatbots; Claude via API | Partial: index monthly, custom prompts up to daily; no run count | Partial: index follows the keyword's country and language; custom prompts not stated | Partial: Claude Sonnet 4.6 named; others "default model" | Not stated |
| Alhena | Not stated | Partial: monthly, weekly or daily by plan; no run count | Not stated | Not stated | Partial: method is versioned; changes that affect comparison are noted |
| Botify | Stated: public websites, logged out, via third-party providers; no API | Stated: one answer per prompt, platform, country and date; daily or weekly | Stated: chosen country; answer language not forced | Not stated | Not stated |
| Demand Genius | Stated: API | Partial: runs repeated and averaged; no count or schedule | Not stated | Not stated | Not stated |
| Evertune | Partial: API for base models, virtual browsers for ChatGPT and Gemini search; other engines not stated | Stated: 100 runs per prompt per model; trackers refresh every 30 days by default | Stated: country and language set per tracker and written into the prompt | Partial: named versions tracked through the API; consumer apps do not expose versions | Not stated |
| Otterly.AI | Stated: public websites; Claude via API | Partial: daily, or weekly on higher plans; no run count | Partial: country per prompt; language not stated | Not stated | Not stated |
| Peec AI | Partial: overview says no APIs; channel list includes a Perplexity API channel | Partial: daily; no run count | Partial: country per prompt; language not stated | Stated: model recorded with each response | Partial: one series is kept across model swaps; method changes not covered |
| Profound | Stated: consumer front end, not model APIs | Stated: once a day per model and region | Partial: region set in configuration and stored with each run; language not stated | Partial: model stored with each run; platform model changes marked on charts (Enterprise); versions not listed | Partial: configuration changes and their effect on past numbers documented in detail; changes to Profound's own method not covered |
| Promptwatch | Stated: live products for search platforms, Google from results pages, provider APIs for API models | Stated: one response per prompt and model per run | Stated: monitor's country and language | Partial: versions named for some API models; live platforms not versioned | Partial: retired model versions keep their collected responses; method changes not covered |
| Scrunch | Partial: browser or API chosen per platform, without saying which | Partial: daily for 14 days, then every 72 hours; no run count | Not stated | Not stated | Not stated |
| Semrush | Partial: "real requests", not LLM APIs; method not named | Partial: tracked prompts daily, brand reports weekly; no run count | Partial: target location; language not stated | Partial: "latest" ChatGPT model in search mode | Not stated |
| Surfer | Stated: the AI apps, not model APIs | Partial: prompts run more than once; no count; daily or weekly by plan | Not stated | Not stated | Not stated |
| Temso | Stated: live consumer interfaces; no API | Partial: rates over repeated runs; no count | Stated: market location, language and session held constant | Not stated | Not stated |
| Topify | Stated: browser for four engines; Claude via API | Stated: once per day per engine | Not stated | Not stated | Not stated |
| Waikay | Partial: API for trends, scraping for Google AI Overviews and Copilot, a mix for the rest | Not stated | Not stated | Not stated | Not stated |
| Wellows | Stated: browser sessions, after moving off APIs | Not stated | Not stated | Not stated | Not stated |
A high count of "Partial" is not a mark against a vendor. Peec AI, Profound and Promptwatch are the only three that address all five questions, yet none of them answers all five in full.
Question 1: API, Interface or Both
Ten of 16 vendors say which channel each engine is collected through, and they do not agree on which channel is right.
| Approach | Vendors | Count |
|---|---|---|
| Interface only | Botify, Profound, Surfer, Temso, Wellows | 5 |
| Mixed by engine | Ahrefs, Evertune, Otterly.AI, Promptwatch, Scrunch, Topify, Waikay | 7 |
| API only | Demand Genius | 1 |
| No API, method not named | Semrush | 1 |
| Pages disagree | Peec AI | 1 |
| Not stated | Alhena | 1 |
Interface-side vendors argue that API answers differ from what a buyer sees, so only the consumer product counts (Botify, Temso, Otterly.AI, Surfer). The case usually made for APIs is that they return repeatable answers that anyone can check. Each channel does something well. They are measuring different things, which is why their numbers should not be compared.
The mixed group mostly collects through the interface and uses an API for engines that are hard to reach otherwise; Claude is the most common API exception (Ahrefs, Otterly.AI, Topify). Three in the group do not say which channel serves some or all of their engines (Scrunch, Waikay, Evertune). Evertune names a channel for four of the eleven model surfaces it lists. Without that, a buyer cannot tell whether a gap between two engines comes from the engines or from the channel.
One detail is worth checking with Peec AI. Its overview says it does not use official APIs, while its list of model channels includes a Perplexity API channel (overview, model channels).
Question 2: How Many Runs Sit Behind a Number
Only one vendor in this set publishes a run count above one, and most publish no count at all.
| What the vendor discloses | Vendors | Count |
|---|---|---|
| A count above one | Evertune | 1 |
| One run per prompt per engine per cycle | Botify, Profound, Promptwatch, Topify | 4 |
| A schedule only (daily, weekly or monthly) | Ahrefs, Alhena, Otterly.AI, Peec AI, Scrunch, Semrush | 6 |
| Runs are repeated, with no number | Demand Genius, Surfer, Temso | 3 |
| Nothing | Waikay, Wellows | 2 |
A single run is a single draw from a changing distribution. That is why the count matters more than the schedule. At one run per cycle, a monthly figure for one prompt on one engine rests on about 30 answers when runs are daily, four or five when weekly, and one when monthly. Cadence also varies inside a single vendor: Ahrefs refreshes its pre-built index monthly while letting custom prompts run up to daily (Ahrefs), and Scrunch runs new prompts daily for 14 days, then every 72 hours (Scrunch).
Three vendors say their scores are rates over repeated runs but give no number (Demand Genius, Surfer, Temso). Waikay's own guide advises five to ten runs per prompt for a stable estimate, but its description of its own method does not say whether it follows that advice (Waikay).
Evertune is the outlier. It samples each prompt 100 times per model, and its trackers refresh every 30 days by default (Evertune). A monthly Evertune figure for one prompt on one engine therefore rests on about 100 answers, more than a daily single-run tool collects in a month.
Question 3: Which Country and Language the Answers Come From
Nine vendors describe some control over country, but only four say how language is handled.
| What the vendor discloses | Vendors | Count |
|---|---|---|
| Country and language both described | Botify, Evertune, Promptwatch, Temso | 4 |
| Country or region set per prompt or project; language not stated | Ahrefs (custom prompts not covered), Otterly.AI, Peec AI, Profound, Semrush | 5 |
| Nothing | Alhena, Demand Genius, Scrunch, Surfer, Topify, Waikay, Wellows | 7 |
The most useful answer came from Botify, and it is a limitation stated openly. Prompts are sent from the campaign's chosen country, but the answer language is not forced, so an engine may reply in the language of the question or of the keyword (Botify). Temso says location, language and session state are held constant for each market (Temso). Ahrefs ties each pre-built prompt to the country and language of the keyword it came from, with English the best covered (Ahrefs).
Evertune also ties each tracker to a country and a language, defaulting to the United States and English, but it applies them by writing them into the prompt text (Evertune, Evertune). That is a different setting from sending the prompt from the chosen country, which is how Botify describes it.
Promptwatch ties each monitor to a country and a language, so a German monitor collects what a user in Germany sees (Promptwatch). Profound sets regions in each customer's configuration and stores the region with every run, and it quotes coverage of 30+ languages and 150+ regions; the pages we read do not say how the answer language is set (Profound).
Country matters even when every market speaks English. Answers differ between markets, sometimes by a wide margin (seoClarity). A UK or Australian brand tracked from an unstated location may be looking at another market's answers.
Question 4: Which Model Version Produced the Answer
Only one vendor documents which model version produced each answer. Five address the question in part and 10 say nothing.
| Vendor | What its documentation says |
|---|---|
| Peec AI | Each response carries the identifier of the model that produced it; a separate channel identifier keeps one series per surface across model swaps |
| Profound | The model is stored with each run, and when a tracked platform changes its model, a dated marker is added to customer charts automatically (Enterprise plan); versions are not listed |
| Promptwatch | Names versions for several API models; live-search platforms are listed by product, not version |
| Evertune | Tracks named versions through its API channels; says consumer apps do not expose the version, and does not say whether the version is stored with each answer |
| Ahrefs | Names Claude Sonnet 4.6 with web search for its API-collected engine; the web-collected chatbots use "the default model" |
| Semrush | Uses the latest ChatGPT model in search mode; no version is named |
For interface collection the gap is partly unavoidable. A logged-out user gets whichever model the platform serves that day, and the vendor has to detect which one it was.
That is why the two strongest answers matter. When a platform swaps its default model, a tracked brand's score can move with no change in the brand or its content. If the vendor records which model answered, as Peec AI does, or marks the date of the swap on the chart, as Profound does, a customer can separate that move from a real one. Without either, the two look the same.
API-side vendors could state a version, because an API call names its model. Evertune says it does so for its API channels; the one API-only vendor in this set does not.
Question 5: What Happens to History When the Method Changes
No vendor explains how past data is treated when its own method changes. Four document related cases.
Profound gives the fullest account, for changes a customer makes. Each answer is stored once and never re-run, while topics, tags and brand lists are applied from the current configuration every time a report loads. Moving a prompt or excluding a brand therefore changes past numbers on every date, and Profound advises exporting any figure that must not change (Profound). Promptwatch says responses from retired model versions stay in a customer's data (Promptwatch). Peec AI keeps one continuous series per surface while the model behind it is replaced (Peec AI). Alhena says its methodology is versioned and that it notes changes affecting comparison with earlier data, without saying whether old figures are restated (Alhena).
None of the four covers the case a buyer can least see: the vendor changing how it collects answers or computes the metric.
The question is not hypothetical, because methods in this market are changing now.
| Change | What the documentation says about earlier data |
|---|---|
| Wellows moved from API collection to browser sessions (Wellows) | Nothing |
| Demand Genius rebuilt its collection pipeline and is rolling it out to customers (Demand Genius) | Nothing |
| Profound's glossary redefined the denominator of its visibility metric between a May 2026 archived copy and a later version, according to the September audit (AuthorityTech) | The page showed only a relative "last updated" label, so a reader could not tell when the definition changed |
A trend line that crosses a method change joins two different measurements. The published API-interface studies found that the two channels share only a minority of named brands, so a channel switch alone can shift a score. A customer cannot see that unless the vendor marks the break.
We also found documentation that disagrees with itself, which is a smaller version of the same problem. Semrush's data page describes its prompt database as updated daily in one section and monthly in its summary (Semrush). Peec AI's overview and its model-channel page disagree on API use, as noted under Question 1. These read as documentation lag, not as intent, and they are the reason "Partial" is the most common result in this audit.
Implications
If you track your own brand
- 01Write down your tool's answers to the five questions. Where the documentation is silent, ask support and keep the reply with your reports.
- 02Work out how many answers sit behind each figure. Prompts, engines, runs and days give the count. Report it next to the percentage.
- 03Set country explicitly for every market you sell in. Do not assume a default, and check which language the answers came back in.
- 04Keep a log of changes. Record the date of every known model update, tool update and prompt edit, and read trend lines against it.
- 05Compare brands inside one tool, never across two. Tools on different channels are measuring different products.
If you are buying a tool or an agency report
- Treat channel as a choice, not a quality mark. Interface collection is closer to what buyers see; API collection is easier to repeat. Decide which you need, then check the vendor matches it for every engine.
- Ask for the run count in writing. A schedule is not a run count.
- Ask how a method change will appear in your account. The best answer is a dated marker on the chart and a statement of whether history was restated.
- Read more than one page, and read each to the end. One contradiction we found sat between two pages of a vendor, another between two sections of a single page.
- Count published limitations in a vendor's favour. The clearest answers in this audit were admissions: answer language is not forced, the platform picks the model, the method will change.
Questions to Ask Before You Buy
These can be sent to a vendor as written. A complete answer to each is short.
| # | Question | A complete answer includes |
|---|---|---|
| 1 | For each engine you track, do you collect through the API, the consumer interface, or a third party? | One channel per engine, and the name of any third-party provider |
| 2 | How many times is each prompt run per engine per cycle, and how many answers does a reported monthly figure rest on? | Two numbers |
| 3 | From which country is each prompt sent, can I set it per prompt, and is the answer language controlled? | The default, the level of control, and what happens when an engine replies in another language |
| 4 | Which model answers on each engine, and do you record it with each response? | A version, or a clear statement that the platform chooses and whether that choice is logged |
| 5 | When you change your method or an engine changes its model, how is that shown in my data? | Whether history is restated, and whether the chart carries a dated marker |
FAQ
How do AI visibility tools collect their data?
In one of two ways, or a mix. Some send prompts to a model's API; others automate the consumer website and record what a user would see. Of the 16 vendors audited, one uses only APIs, five only the interface and seven mix them by engine.
Is API or interface collection more accurate?
They answer different questions. Interface collection records what a buyer sees, including the platform's own search and formatting. API collection is easier to repeat and verify. Published comparisons found the two return largely different brands for the same prompt, so the figures should not be mixed.
How often should an AI visibility tool run each prompt?
There is no agreed standard, and the documentation we read does not set one. Four vendors state one run per engine per cycle. Only Evertune states more, at 100 runs per prompt per model. What a buyer should ask for is the number of answers behind a figure, since that sets how far the figure can be trusted.
Do AI visibility tools track results by country?
Some do. Nine of the 16 vendors describe a country setting or location coverage, and seven say nothing. Language is rarer: four vendors describe how it is handled.
Why did my AI visibility score change when nothing else did?
Possible causes include ordinary run-to-run variation, a change in the model the engine serves, and a change in the tool's own method. Only Profound documents marking platform model changes on customer charts, and no vendor documents how a change to its own method is flagged, so ask your vendor directly.
Does "not stated" mean a tool is unreliable?
No. It means the answer is not on the public pages we read. The tool may handle the issue well and document it privately, or support may answer on request.
Methodology, Limitations and Disclosure
What we did
This is an audit of public documentation. Between 13 and 18 November 2026 we read the methodology, help, FAQ and documentation pages of 18 AI visibility vendors and recorded what each says in answer to five questions. Two were excluded, leaving 16. Each answer was classified as stated, partial or not stated under the rule in the method section above. Where a vendor publishes a documentation index, we searched it for further pages on the five questions. We did not open an account with, run prompts in, or test any tool.
Limitations
- Public pages only. A vendor may answer these questions in a sales call, a contract, an in-app help panel or documentation behind a login. None of that is counted.
- "Not stated" is not "does not do". It records an absence on the pages we read, on the date we read them.
- We read the pages a buyer would reach, not every page. A relevant page elsewhere on a vendor's site could change a classification. Vendors are welcome to point us to one, and we will correct the table.
- Documentation changes. Every result is as of the check date in the Source Register.
- Sixteen vendors is not the market. One public directory, kept by a tracking vendor, lists more than 200 tools (LLMrefs). We included those with a public page on collection, so this set is likely more transparent than the average tool.
- Classification involves judgment. The line between partial and stated is ours. The table gives the wording behind each call so a reader can disagree.
- Documentation is not behaviour. We did not verify that any tool does what its pages say.
- Two vendors were excluded. Rankwize met the selection rule, but its documentation is published only on a preview-style pages.dev address, and its pages contradict each other on how answers are collected: one names the AI providers' own APIs, another a third-party data provider for all five platforms. We could not tell which describes the product, so we left it out rather than classify it.
- A second vendor was excluded. Ataiva's methodology page had been removed (HTTP 404) by 18 November 2026, so readers could no longer check its row against the source.
Disclosure
Freeways is a GEO agency. It sells AI visibility measurement as a service and uses third-party tracking tools in that work, so it has a commercial interest in how these tools are judged. Freeways is not a paying customer of any vendor named here. Vendors are named because the audit rests entirely on documents they published; no figure measured by Freeways is attached to any vendor. Several studies cited for context were published by companies that sell tracking tools.
Source Register
Vendor pages read for the audit. Pages for Evertune and Surfer were checked on 18 November 2026; all others on 13 November 2026.
Twelve of the 27 pages show a date. On the other 15, a reader cannot tell how current the description is.
Studies and directories cited for context, re-checked on 18 November 2026:
- AuthorityTech: AI visibility scores are not comparable, a nine-platform methodology audit
- Brandlight Research Lab: The UI vs API gap
- Surfer: Why you cannot track AI visibility through an API
- Search Engine Journal: AI recommendations change with nearly every query (SparkToro)
- seoClarity: Tracking the volatility of ChatGPT's citations
- LLMrefs: The ultimate list of AI SEO tools
