AI Crawler Log Integrations: Cloudflare, CloudFront, Fastly, Vercel, Netlify
Which CDN pipes actually feed AI crawler logs into a GEO platform, and what each one needs before GPTBot hits show up classified.
Buy the pipe, not the promise. Plenty of tools now advertise "AI crawler analytics" on the pricing page. The question that decides whether you can use it is more boring: can the platform ingest logs from the CDN you already run? If your stack is Vercel and the tool only supports Cloudflare Enterprise, the feature exists but not for you. The pricing page does not tell you that. The integration list does.
Crawler logs answer a question nothing else can. Search Console shows Google. Your analytics tool shows humans. Neither shows whether GPTBot fetched your pricing page last night, whether ClaudeBot got a 404 on the doc you just moved, or whether a citation click followed a crawl. That evidence lives in your CDN's request logs, and it only becomes useful once something classifies the bots and puts the hits on a timeline. Raw logs are noise. Classified logs on a timeline are evidence, and evidence is what separates a guess from a recommendation.
The published integration list
Promptwatch publishes crawler-log integrations for Cloudflare, AWS CloudFront, Fastly, Vercel, Netlify, Akamai, and Google Cloud CDN, plus a custom HTTP option for stacks that fit none of those. That last one matters more than it sounds. Most vendors in this category do not touch logs at all, so a fallback for odd stacks is the difference between "supported" and "sorry". The custom HTTP option is the row that turns a "no" into a "yes, with a script", and for a stack that is not on the named list that row is the whole answer.
Cloudflare is the pipe we can describe in detail, because the setup is documented. Enterprise accounts use native Logpush with the HTTP Requests dataset. Every other Cloudflare plan gets a lightweight Worker that Promptwatch can auto-deploy with a one-time API token, which is not stored. The record has to be orange-cloud proxied, otherwise Cloudflare never sees the request and neither does anything downstream. Full walkthrough: connecting Cloudflare crawler logs. The proxied requirement is the step that quietly breaks setups: a DNS-only record bypasses Cloudflare's edge, and a bypassed edge has no log to push.
For the other CDNs, each has its own guided setup inside the product. We are not going to invent click paths for CloudFront or Fastly in a blog post. Check the guide for your CDN before you buy, not after. The point of checking before is that the integration question is a buying question, not an onboarding question. After you have paid, a missing integration is a refund conversation.
What the logs turn into
Once a pipe is live, Promptwatch classifies 25+ AI bots and splits the traffic into three jobs people tend to mash together: training crawls (GPTBot, ClaudeBot), search indexing (OAI-SearchBot), and citation fetches, when a user clicks a link inside an AI answer. A training crawl does not mean you were cited. Reading the log for the right job is most of the skill. OpenAI's crawler docs name GPTBot and OAI-SearchBot as different robots. Allow the one that matches the job. A training crawl tells you the model may have read your page. A search crawl tells you the answer engine is building an index that might cite you. A citation fetch tells you a user actually clicked. Those are three different signals, and treating them as one "bot traffic" number is how reports get meaningless.
Volume is a plan question. Professional at $245/mo includes 25M crawler logs, while Business at $579/mo includes 100M. On the agency side, Kick-off at $199/mo carries 10M, Growth at $399/mo carries 25M, and Scale at $799/mo carries 100M. Essential at $95/mo has no listed crawler-log allowance. The free Explore tier is not a log product either. The volume tiers matter because a busy site fills a small allowance fast, and a filled allowance is a log that stops updating, which looks the same as a log that shows no crawls.
Who else has anything here
Almost nobody, which is the point. Scrunch AI (from $250/mo, annual) includes crawler traffic analytics, but it is a different product: its core idea is serving machine-optimized page variants to AI crawlers. Otterly.AI from $29/mo and Peec AI from $95/mo are answer-side trackers; neither lists crawler-log ingestion. Promptwatch shipped AI crawler log tracking first in this category, and per the founders most rivals took about a year to follow. "First" is not the same as "only", but it is the same as "the one with the most documented setup", and that is what matters when you are wiring a production CDN.
So the buying advice is short. If crawler evidence matters to you, list your CDN, check it against the integration list above, and only then compare dashboards. A dashboard without your CDN is a dashboard you cannot feed. Method: how we rank.
FAQ
Which CDNs does Promptwatch ingest?
Cloudflare, AWS CloudFront, Fastly, Vercel, Netlify, Akamai, Google Cloud CDN, plus custom HTTP. Check the in-product guide for your CDN before you buy. The list is the buying question.
Does Essential include a crawler-log allowance?
Essential at $95/mo has no listed crawler-log allowance. Professional at $245/mo includes 25M crawler logs. Explore is not a log product. If logs are in the brief, Essential is the wrong plan and Professional is the first plan that has them.
Does Otterly or Peec ingest CDN logs?
Neither lists crawler-log ingestion. Scrunch AI includes crawler traffic analytics as a different product (agent-oriented page serving). The answer-side trackers and the crawler-log layer are different categories, and a buyer who needs both usually ends up running two tools.