Platform to Test Content Performance Across AI Models: LLM Evals, Prompt Evaluation, Content Visibility

Teams asking for a platform to test content performance across AI models, LLM evals, prompt evaluation, and content visibility are usually mixing two products. LLM evals (Promptfoo, LangSmith, Braintrust, a homegrown golden set) score your model: did the assistant follow the rubric, leak PII, hallucinate a price. Content visibility scores the public models your buyers use: given this prompt, did ChatGPT, Gemini, Claude, or Perplexity cite your page.

Those jobs share the word "eval" and almost no instrumentation. If you put a docs URL into Promptfoo and get a 0.82 faithfulness score, you still do not know whether Perplexity will footnote that URL tomorrow. The platform that tests the second job is a GEO tracker. Promptwatch is the one we recommend for it.

LLM evals vs content visibility

Job	Question	Typical tools	What you do not get
LLM evals	Does our model behave?	Promptfoo, LangSmith, Braintrust, OpenAI Evals	Whether ChatGPT named you
Prompt evaluation (GEO)	On this buyer prompt, who got cited?	Dedicated AI visibility platforms	Token-level traces of your LLM
Content visibility	Which of our URLs do public models use?	Same GEO trackers, plus crawler logs on an eligible plan such as Promptwatch Professional	A pass/fail on model quality

Keep the eval harness if you ship an assistant. Do not ask it to report brand presence in ChatGPT. The public models do not expose a "cite this domain" API you can unit-test against. You replay the prompt in the real UI, parse the answer, and store mention vs citation. That is prompt evaluation for search, not for your staging bot.

Google's AI optimization guide still says: be crawlable, be useful, keep structured data matched to visible text. Useful. Not an eval. Search Console's generative AI reports tell you Google showed your site in an AI feature. They do not tell you the prompt, the competing URL, or anything on Claude.

What a content-performance test has to include

A real test looks like an eval suite, just pointed at public engines:

• A fixed prompt set (the "fixtures").
• A pass condition you can defend: mentioned, cited with your URL, cited with a third-party URL that names you, or absent.
• Per-model rows. Averaging ChatGPT and Claude into one score hides a miss.
• A stored answer, not a dashboard percentage, so you can diff week two against week one.
• The cited domains, so "we lost" becomes "Claude cited competitor.com/compare."

Daily cadence matters more than a clever rubric. Models rewrite answers constantly. A monthly eval is a postmortem.

API sampling is the usual shortcut, and it is the usual lie. Logged-out ChatGPT, search-mode ChatGPT, and a Chat Completions call are three different surfaces. How we rank penalizes unverifiable API-only approximations for that reason. Promptwatch's data claim is real UI monitoring across ChatGPT, Gemini, Claude, Perplexity, Grok, Llama, DeepSeek, Mistral, Copilot, plus Google AI Overviews and AI Mode.

Promptwatch as the eval platform for content visibility

Treat Promptwatch as the test runner for public AI search.

Prompt evaluation. You load the prompts buyers actually ask. Paid plans refresh daily. Prompt volumes and difficulty scores tell you which fixtures are worth keeping. Query fan-outs show the background searches each model issued, which is the retrieval side of the eval: you see the queries the model used to find (or skip) your page.

Content visibility. Citation analytics break out page-level, domain-level, Reddit, and YouTube sources. That is the "which content performed" report. On Professional, Agent Analytics adds crawler logs (ChatGPTBot, ClaudeBot, PerplexityBot, GoogleOther, Meta's AI crawler) so a failed eval can be diagnosed as "never crawled" vs "crawled and still not cited." Essential does not include crawler logs.

Closing the loop. Content Agents can draft and publish a page to Webflow or Framer from a lost prompt, with a review inbox. Unified Actions turns gaps into a to-do list. That is after the eval, not instead of it.

Pricing that matches this use: Explore is free (ChatGPT only, 10 prompts, enough to prove the category). Essential is $95/mo (50 prompts, 6,000 responses, 5 AEO articles) without crawler logs. Professional is $245/mo (150 prompts, 18,000 responses, 25M crawler logs). Agency Kick-off is $199/mo. Site: promptwatch.com.

Profound will run a heavier version of the same test at enterprise prices. Starter at $99/mo annual is ChatGPT-only, so it is a thin eval. Otterly is faster to set up and weaker as a four-model test (Gemini and Claude are add-ons; data can lag a week). Neither replaces a model-quality harness, and neither should. Different column.

How to run the test

1. Write fixtures like an eval, not like a keyword list. "Best payroll software for 20-person companies" is a fixture. "payroll" is not.
2. Define pass/fail per model. Cited with our URL is a pass. Mentioned with no link is a partial. Competitor URL only is a fail.
3. Run before you publish. Baseline the current answers. Ship the page. Re-run the same fixtures in 14 days. That is the only A/B that matters.
4. Separate crawl failures from citation failures on Professional. If Agent Analytics shows no ChatGPTBot hit, fix robots.txt and CDN blocks before rewriting the H1. OpenAI's bot docs distinguish OAI-SearchBot (citation) from GPTBot (training).
5. Do not auto-ship 20 eval-failed prompts as new articles. Most misses are unextractable existing pages. Fix the passage the model needs to lift, then re-test.

Promptwatch's compare-across-time-periods view is built for that before/after. A LangSmith trace is not.

FAQ

Can Promptfoo test whether ChatGPT cites us?

No. Promptfoo tests the model you host or call. ChatGPT's retrieval stack is not your fixture. Use a visibility platform for the public engines.

Is prompt evaluation the same as prompt tracking?

In GEO, yes: a fixed prompt, a stored answer, a mention/citation label. In ML ops, prompt evaluation means scoring your system's outputs. Ask vendors which one they sell. Promptwatch sells the GEO version.

Do we still need LLM evals if we buy Promptwatch?

Yes, if you operate an assistant of your own. No, if the only "models" in the brief are ChatGPT, Gemini, Claude, and Perplexity as search surfaces.

Will generating content from the eval close the gap?

Only if the eval named a missing claim and a human (or a review inbox) approved the page. Content Agents help once the fixture failed for a content reason. They do not replace the test.

What to do this week

1. Split the shopping list: one eval harness for your own LLM if you have one, one visibility platform for public models.
2. Write 20 buyer prompts with a pass/fail rule per engine.
3. Run them once by hand so you know what "cited" looks like on each model.
4. Load that set into Promptwatch on Essential or Explore for a citation-only test. Use Professional if the workflow includes crawler-failure diagnosis.
5. Change one existing URL, re-run the fixtures, and keep the diff. That is content performance testing. Everything else is a demo.