Skip to content

AI crawler

What is cohere-training-data-crawler?

cohere-training-data-crawler is a token attributed to Cohere that trackers describe as a training data crawler.

AI trainingNot documentedNo way to verify
Operator
Cohere
Purpose
AI training. Collects pages to train AI models. Blocking it does not remove you from AI answers.Cohere does not document it. The purpose comes from third-party trackers.
robots.txt token
cohere-training-data-crawler
Sample user agent
Not documented by the operator. Cohere publishes no crawler page; match on the token only.User agents change with versions. Match on the token, not the full string.
Follows robots.txt
Not documented by the operator. Cohere publishes no crawler documentation.
How to verify
Cohere doesn't publish a way to verify it.
Sources
Cohere publishes no documentation for it.
Last reviewed

Allow or block cohere-training-data-crawler

Block cohere-training-data-crawler
User-agent: cohere-training-data-crawler
Disallow: /

Asks cohere-training-data-crawler not to crawl any page. Well-behaved crawlers stop; it does not stop anyone who ignores robots.txt.

Allow cohere-training-data-crawler
User-agent: cohere-training-data-crawler
Allow: /

Lets cohere-training-data-crawler crawl everything. You only need this if a broader rule would otherwise block it.

Should you block it?

Blocking cohere-training-data-crawler asks Cohere not to use new crawls of your pages for training. It doesn't remove you from AI answers.

Write a full policy with the AI Crawler Policy Generator

cohere-training-data-crawler can't be verified

Cohere doesn't publish IP ranges, DNS names or signatures for cohere-training-data-crawler. The user agent is the only signal, and anyone can copy it, so treat a request that claims to be cohere-training-data-crawler as unproven.

Questions about cohere-training-data-crawler

Is cohere-training-data-crawler safe to allow?

cohere-training-data-crawler reads public pages; it does not log in or submit forms. Whether to allow it is about how your content is used, not security.

How do I know a request is really cohere-training-data-crawler?

You can't, from the request alone. Cohere doesn't publish IP ranges, DNS names or signatures for it, so the user agent is the only signal, and anyone can copy that.

How often does cohere-training-data-crawler visit?

It depends on your site; Cohere doesn't publish a schedule. Referscope shows the real visits.

Related crawlers

All AI crawlers

Free tools for cohere-training-data-crawler

AI Crawler Policy Generator

Build a robots.txt that blocks AI training and keeps AI search and Google.

Build your robots.txt

AI Bot Verification Tester

Paste an IP and user agent from your logs and see if the bot is real.

Verify a bot

robots.txt Checker

See which AI crawlers your robots.txt allows or blocks.

Check robots.txt

Less guessing. Clearer growth.

Connect your site today and see your first verified AI crawler by tomorrow.

Explore ReferscopeBecome a design partner