Skip to content

Search crawler

What is YandexBot?

YandexBot is Yandex's main indexing crawler for Yandex search.

Classic searchPartly followsReverse DNS
Operator
Yandex
Purpose
Classic search. Search engines. Blocking them removes you from search results.
robots.txt token
YandexBot
Sample user agent
Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)User agents change with versions. Match on the token, not the full string.
Follows robots.txt
Partly. Yandex says YandexBot follows the general rules, but some Yandex robots may ignore robots.txt, including rules for User-agent: Yandex.
How to verify
Reverse DNS on yandex.ru, yandex.net or yandex.com, confirmed with a forward lookup.Yandex does not publish an IP list.
Sources
Yandex documentation
Last reviewed

Allow or block YandexBot

Blocking YandexBot removes your pages from Yandex search results, so we don't offer that snippet. If you need to, write the rule by hand.

Allow YandexBot
User-agent: YandexBot
Allow: /

Lets YandexBot crawl everything. You only need this if a broader rule would otherwise block it.

Should you block it?

Keep YandexBot allowed. Blocking it removes your pages from Yandex search results.

Write a full policy with the AI Crawler Policy Generator

How to verify a YandexBot request

  1. Run a reverse DNS lookup on the IP (host 192.0.2.1, with the IP from your log).
  2. Check the hostname ends in yandex.ru or yandex.net or yandex.com.
  3. Run a forward lookup on that hostname and check it returns the same IP. Both steps must pass.

Yandex does not publish an IP list.

Test YandexBot requests

Questions about YandexBot

Is YandexBot safe to allow?

YandexBot reads public pages; it does not log in or submit forms. Whether to allow it depends on whether you want Yandex's product to use your pages.

How do I know a request is really YandexBot?

Check the request against what Yandex publishes: Reverse DNS on yandex.ru, yandex.net or yandex.com, confirmed with a forward lookup. The user agent alone proves nothing.

How often does YandexBot visit?

It depends on your site; Yandex doesn't publish a schedule. Referscope shows the real visits.

Related crawlers

All AI crawlers

Free tools for YandexBot

AI Crawler Policy Generator

Build a robots.txt that blocks AI training and keeps AI search and Google.

Build your robots.txt

AI Bot Verification Tester

Paste an IP and user agent from your logs and see if the bot is real.

Verify a bot

robots.txt Checker

See which AI crawlers your robots.txt allows or blocks.

Check robots.txt

Less guessing. Clearer growth.

Connect your site today and see your first verified AI crawler by tomorrow.

Explore ReferscopeBecome a design partner