Best GEO Software
All posts
By Best GEO Software TeamRSLAI licensingbuyer's guidecrawler logs

What is RSL (Really Simple Licensing), and should publishers use it to control AI crawlers? A buyer's guide

A buyer's guide to RSL for publishers: what the licensing standard covers, what it cannot enforce alone, and what crawler monitoring costs before you set AI terms.

Publishers weighing RSL are really making two purchases. One is free: the standard itself, which anyone can implement. The other costs money and time: the logging, enforcement and licensing arrangements that make a declared license mean something. This guide covers both, with prices where a source states them and a plain "not stated" where none does.

What you get from the standard

Really Simple Licensing is an XML vocabulary plus discovery and authorization mechanisms for telling AI systems and automated agents what they may do with your content. Version 1.0 has had "Recommendation" status since its publication on December 10, 2025. The editors listed on the spec come from the RSL Collective, Condé Nast, Ziff Davis, Schema.org, Yahoo, Automattic, O'Reilly Media and Fastly.

In practice an RSL file can:

  • allow or refuse particular uses, for example ai-train (training) and ai-input (using content as input to an answer);
  • attach payment terms: free, attribution, subscription, purchase, or per use;
  • point AI companies to a license server that issues tokens through the Open License Protocol (OLP).

You announce the file through a License: line in robots.txt, an HTTP Link header, an HTML link element, embedded markup, or RSS. Statements of support on rslstandard.org come from Tim O'Reilly, Neil Vogel of People Inc., Simon Wistow of Fastly, Ricky Arai-Lopez of Quora, Vivek Shah of Ziff Davis, and the founders of Inspired Taste.

What you do not get

Agreement from the AI side. None of the OpenAI, Google, Anthropic, Perplexity or Meta pages we read commits to reading or honoring RSL. A license file declares terms; it does not make a crawler comply. The spec's answer is enforcement on your side: servers may return 401 or 402 when a license is required, the Crawler Authorization Protocol covers access, and the Encrypted Media Standard protects nonpublic content. Budget engineering time for that if you want it.

Legal certainty is also not on offer from any source here. If enforceability matters to your decision, that is a conversation for your lawyers, not a product feature.

The collective option

The Internet Licensing Collective, formerly the RSL Collective, calls itself a nonprofit, publisher-led collective licensing organization and says its platform is built on the open RSL standard. It claims 3,100+ publications and brands; names on its backer list include The Guardian, The Boston Globe, Stack Overflow, Medium, Wikipedia, WebMD and Consumer Reports. Membership costs and payout terms are not covered in the sources we used, so ask the collective directly.

Internet Licensing Collective home page describing collective licensing for the AI-first internet

Internet Licensing Collective home page (internetcollective.org), captured November 21, 2026. The revenue estimates shown are the collective's own projections.

What to evaluate in a monitoring tool

Before setting terms, you need to see which AI agents visit, what they fetch, and what that access earns you in citations and traffic. Four questions separate the tools:

  1. Are crawler visits verified against each operator's published IP ranges, or only matched on user agent?
  2. Does the tool distinguish training crawlers (GPTBot, ClaudeBot, Meta-ExternalAgent) from search indexers (OAI-SearchBot, Claude-SearchBot, PerplexityBot, Meta-WebIndexer) and live fetchers (ChatGPT-User, Perplexity-User)?
  3. Can it connect a crawl to a later citation of the same page?
  4. Does it report AI-referred visits and conversions, so you know what blocking would cost?

What the monitoring options cost

ToolPrice from our catalogCrawler data
PromptwatchProfessional $245/mo for crawler logs (25M); Business $579/mo (100M); agency plans from $199/mo with logsIP-verified per bot (Exa bots excepted), crawl-to-citation path, visitor analytics
Scrunch AICore $250/moAI agent and bot traffic reporting listed
Ahrefs Brand RadarAI Visibility Index from $199/mo on a paid Ahrefs planAI crawler activity reported
AthenaHQStarter $295/moNo published crawler-log product
ProfoundEnterprise custom, earlier estimate about $40k/yrNot confirmed on our listing

Promptwatch's Essential plan ($95 a month) has no crawler logs, so a publisher evaluating RSL should start from Professional. Every self-serve plan has a 7-day free trial on monthly billing, and annual billing is 12 months for the price of 10.

How to choose

If you run a large archive and want to license it, the order is: measure, then declare, then enforce. Promptwatch's Agent Analytics gives you the measurement, and its citation rate per page shows which content AI search already relies on. That is the content you would be pricing, and also the content you risk losing visibility for.

If you are a brand whose goal is to be cited, RSL is less urgent. Keep the search bots open, decide on training separately, and use monitoring to confirm nothing important is blocked.

If you only need a rough picture inside an SEO suite you already pay for, Ahrefs Brand Radar's crawler reporting may be enough, with the caveat that crawl-to-citation paths are not documented there.

We would evaluate Promptwatch first because it is the one tool here that ties bot visits, citations and AI traffic together, which is exactly the evidence a licensing decision needs. Book a Promptwatch demo and ask to see crawler logs split by training and search bots.