Skip to content

Link preview bot

What is Twitterbot?

Twitterbot is the bot X uses to read card markup and build the preview when someone posts your link.

Link previewFollows robots.txtNo way to verify
Operator
X
Purpose
Link preview. Builds the preview card when someone shares your link in a chat or social app.
robots.txt token
Twitterbot
Sample user agent
X says it uses the Twitterbot user agent with a version, such as Twitterbot/1.0, but publishes no full string.User agents change with versions. Match on the token, not the full string.
Follows robots.txt
Yes, according to X. X says that if a page is blocked, no card is shown.
How to verify
X doesn't publish a way to verify it.
Sources
X documentation
Last reviewed

Allow or block Twitterbot

Block Twitterbot
User-agent: Twitterbot
Disallow: /

Asks Twitterbot not to crawl any page. Well-behaved crawlers stop; it does not stop anyone who ignores robots.txt.

Allow Twitterbot
User-agent: Twitterbot
Allow: /

Lets Twitterbot crawl everything. You only need this if a broader rule would otherwise block it.

Should you block it?

Blocking Twitterbot breaks the preview card when someone shares your link on X.

Write a full policy with the AI Crawler Policy Generator

Twitterbot can't be verified

X doesn't publish IP ranges, DNS names or signatures for Twitterbot. The user agent is the only signal, and anyone can copy it, so treat a request that claims to be Twitterbot as unproven.

Questions about Twitterbot

Is Twitterbot safe to allow?

Twitterbot reads public pages; it does not log in or submit forms. Whether to allow it depends on whether you want X's product to use your pages.

How do I know a request is really Twitterbot?

You can't, from the request alone. X doesn't publish IP ranges, DNS names or signatures for it, so the user agent is the only signal, and anyone can copy that.

How often does Twitterbot visit?

It depends on your site; X doesn't publish a schedule. Referscope shows the real visits.

Related crawlers

All AI crawlers

Free tools for Twitterbot

AI Crawler Policy Generator

Build a robots.txt that blocks AI training and keeps AI search and Google.

Build your robots.txt

AI Bot Verification Tester

Paste an IP and user agent from your logs and see if the bot is real.

Verify a bot

robots.txt Checker

See which AI crawlers your robots.txt allows or blocks.

Check robots.txt

Less guessing. Clearer growth.

Connect your site today and see your first verified AI crawler by tomorrow.

Explore ReferscopeBecome a design partner