API-tracked vs real-UI tracking: what your AI visibility tool is actually measuring
Why an engine count on a pricing page hides the real question: was the answer captured from the product people use, or from an API call? A buyer's guide to asking the right questions.
Quick verdict
"We track ten engines" is a sentence that can mean ten different things. Some of those engines may be queried through the same interface a person uses, with live search, sources panels, and product features. Others may be queried through a model API, which returns a model's response without the surrounding product. Both have uses. Only one tells you what a buyer sees in the ChatGPT or Google AI Overviews interface. Promptwatch is explicit about the split, which is a good reason to put it on a shortlist, and a good template for the questions to ask every other vendor.
The two methods
Real-UI monitoring loads the product as a user would, enters the prompt, and captures the answer, including cited sources and any product-specific elements such as shopping carousels or AI Overviews.
API tracking calls a model through its developer interface. It is cheaper, steadier, and easier to scale, and it returns a model's answer without the retrieval and presentation layers a consumer product adds.
Neither is dishonest. The problem arises when a report mixes them without labels.
Where the gap matters
| Situation | Why UI capture matters |
|---|---|
| ChatGPT Search | Citations come from live web retrieval shown in the interface |
| Google AI Overviews and AI Mode | The answer is a product feature, not a bare model reply |
| Shopping and ads | Carousels and sponsored placements exist only in the UI |
| Local answers | Location-aware behavior can depend on the product layer |
| Source analysis | The sources panel is where citation tracking begins |
For a model with no consumer search product, or where you care about how the model itself describes your brand, API sampling is a reasonable proxy.
How Promptwatch splits it
Per the 2026-10-28 reference, Promptwatch monitors these through the real interface: ChatGPT (Search), Google AI Overviews, Google AI Mode, Microsoft Copilot, Gemini, Perplexity, Amazon Alexa, and Meta AI, plus the coding agents Codex and Claude Code.
It tracks these via API alongside them: Claude (Haiku 5.5), DeepSeek, Mistral (Large), Grok 4.5, GPT 5.6 Luna, GPT OSS 120B, Kimi K2.7 Code, GLM 5.2, Inkling, and the OpenCode coding agent.
So Claude, Grok, DeepSeek, and Mistral coverage is API-based. If a vendor tells you otherwise about their own coverage, ask for the method in writing. Every paid Promptwatch plan unlocks all active models, and the free Explore plan is ChatGPT only.
Questions to ask any vendor
- For each engine on your list, is the answer captured from the consumer interface or from an API?
- Does the response include the sources panel, or only a text answer?
- How often does each engine run, and at what volume?
- Can I filter reports by method, so I do not blend them?
- Which engines are add-ons or higher tiers?
Our directory entries show how much this varies. LLM Pulse lists ChatGPT, Perplexity, Google AI Mode, AI Overviews, and Gemini as core engines, with Claude, Copilot, Grok, DeepSeek, Meta AI, and others as paid add-ons. Qwairy's entry notes that Claude, Mistral, and DeepSeek are API samples, not live interfaces. Scrunch AI's Core plan tracks four LLMs and Enterprise adds nine more. See the ranking for the full entries.
When API tracking is the right choice
Do not dismiss it. Use API-tracked engines when:
- You want a stable baseline of how a model describes your brand without search.
- You track a model whose consumer product you do not care about.
- You need volume at low cost for broad coverage.
Use real-UI tracking when you care about citations, ads, shopping, local results, or any element a person would see.
Reading a mixed report
Label every chart by method. A visibility score blended across UI and API engines has no clean interpretation. Promptwatch's heatmap can compare brands against models, which makes it possible to see whether a gap exists on a UI-tracked engine and not on an API one, and that difference often tells you whether the issue is retrieval or knowledge.
| Pattern | Likely meaning |
|---|---|
| Strong on API models, weak in UI search | Models know you, retrieval does not find or cite your pages |
| Weak on API models, strong in UI search | Retrieval works, but you are thin in broad training sources |
| Weak on both | Awareness and content gap |
| Strong on both | Maintain, watch competitors |
Plan notes
Explore is free with 10 prompts and 310 responses, ChatGPT only. Essential at $95 a month includes 50 prompts and 6,000 responses across all active models. Professional at $245 has 150 prompts and 18,000 responses. Business at $579 adds state and city tracking and Local GEO/SEO coordinate targeting, which pins monitors to exact locations for ChatGPT, Google AI Overviews, and Google AI Mode, while other models fall back to country, state, or city.
Who should pick which
| You are... | Recommendation |
|---|---|
| Caring about citations and ads | Real-UI tracking, and ask for the method per engine |
| Wanting a cheap broad baseline | API tracking is fine, labeled as such |
| Comparing vendors on engine count | Compare method per engine, not totals |
| Needing both | Promptwatch, which tracks the UI engines and the API models and labels the split |
| Auditing a competitor's report | Ask which engines were API-sampled |
The decision rule we use: ask for the method first and the count second. A tool that answers plainly has usually thought about it. Promptwatch publishes the split, so it passes this test before the demo even starts.