Skip to content

Free tools

Check how AI crawlers see your site

Free checks and references for founders: see whether ChatGPT, Claude and Perplexity can read your site, write the robots.txt you want, check that a bot is real, and find which visits came from AI. No account needed.

AI Health Check

See if AI crawlers can reach a page, and what to fix if they can't.

Example result: crawler access good, structured data missing, meta description good.

Run the check

robots.txt Checker

See which AI crawlers your robots.txt allows or blocks.

Example result: GPTBot allowed, ClaudeBot allowed, CCBot blocked.

Check robots.txt

AI Crawler Policy Generator

Build a robots.txt that blocks AI training and keeps AI search and Google.

Example result: AI training blocked, AI search and classic search allowed.

Build your robots.txt

AI Bot Verification Tester

Paste an IP and user agent from your logs and see if the bot is real.

Example result: Googlebot verified, a request claiming GPTBot not verified.

Verify a bot

AI Crawler Directory

Every major AI crawler: who runs it, what it collects, how to block it.

Example result: GPTBot is AI training, OAI-SearchBot is AI search, ChatGPT-User is a user fetch.

Browse crawlers

Source Classifier

Paste referrer URLs and see which visits came from AI, search or social.

Example result: chatgpt.com is AI, google.com is search, reddit.com is social.

Classify referrers

Where to start

  1. Can AI crawlers reach your pages?

    Run the AI Health Check on your homepage. It reads your robots.txt, structured data and page basics in one pass.

    AI Health Check

  2. Which crawlers does your robots.txt allow?

    If the first check flags crawler access, the robots.txt Checker shows the rule for each AI crawler and the exact file we read.

    robots.txt Checker

  3. Want different rules?

    Pick a policy in the AI Crawler Policy Generator and download a robots.txt that keeps Google and AI search, and blocks training.

    AI Crawler Policy Generator

  4. Is AI already sending you people?

    Paste the referrers from your analytics into the Source Classifier to see which visits came from AI answers.

    Source Classifier

Saw a bot in your logs? Look it up in the AI Crawler Directory, or check it's real with the AI Bot Verification Tester.

These checks show what your site asks for. Referscope shows what actually happens: every verified AI crawler visit and every person an AI answer sends you.

Less guessing. Clearer growth.

Connect your site today and see your first verified AI crawler by tomorrow.

Explore ReferscopeBecome a design partner