Skip to content

Search crawler

What is Baiduspider?

Baiduspider is Baidu's search crawler that adds pages to the Baidu index.

Classic searchPartly followsReverse DNS
Operator
Baidu
Purpose
Classic search. Search engines. Blocking them removes you from search results.
robots.txt token
Baiduspider
Sample user agent
Baidu publishes the token but not a full user agent string.User agents change with versions. Match on the token, not the full string.
Follows robots.txt
Partly. Baidu says Baiduspider obeys robots.txt, but Baiduspider-video does not support it yet and its ads crawlers do not follow it.
How to verify
Reverse DNS on baidu.com or baidu.jp, confirmed with a forward lookup.
Sources
Baidu documentation
Last reviewed

Allow or block Baiduspider

Blocking Baiduspider removes your pages from Baidu search results, so we don't offer that snippet. If you need to, write the rule by hand.

Allow Baiduspider
User-agent: Baiduspider
Allow: /

Lets Baiduspider crawl everything. You only need this if a broader rule would otherwise block it.

Should you block it?

Keep Baiduspider allowed. Blocking it removes your pages from Baidu search results.

Write a full policy with the AI Crawler Policy Generator

How to verify a Baiduspider request

  1. Run a reverse DNS lookup on the IP (host 192.0.2.1, with the IP from your log).
  2. Check the hostname ends in baidu.com or baidu.jp.
  3. Run a forward lookup on that hostname and check it returns the same IP. Both steps must pass.
Test Baiduspider requests

Questions about Baiduspider

Is Baiduspider safe to allow?

Baiduspider reads public pages; it does not log in or submit forms. Whether to allow it depends on whether you want Baidu's product to use your pages.

How do I know a request is really Baiduspider?

Check the request against what Baidu publishes: Reverse DNS on baidu.com or baidu.jp, confirmed with a forward lookup. The user agent alone proves nothing.

How often does Baiduspider visit?

It depends on your site; Baidu doesn't publish a schedule. Referscope shows the real visits.

Related crawlers

All AI crawlers

Free tools for Baiduspider

AI Crawler Policy Generator

Build a robots.txt that blocks AI training and keeps AI search and Google.

Build your robots.txt

AI Bot Verification Tester

Paste an IP and user agent from your logs and see if the bot is real.

Verify a bot

robots.txt Checker

See which AI crawlers your robots.txt allows or blocks.

Check robots.txt

Less guessing. Clearer growth.

Connect your site today and see your first verified AI crawler by tomorrow.

Explore ReferscopeBecome a design partner