Skip to content

Search crawler

What is DuckDuckBot?

DuckDuckBot is DuckDuckGo's crawler for DuckDuckGo search results.

Classic searchFollows robots.txtIP list
Operator
DuckDuckGo
Purpose
Classic search. Search engines. Blocking them removes you from search results.
robots.txt token
DuckDuckBot
Sample user agent
DuckDuckBot/1.1; (+http://duckduckgo.com/duckduckbot.html)User agents change with versions. Match on the token, not the full string.
Follows robots.txt
Yes, according to DuckDuckGo.
How to verify
Check the IP against DuckDuckGo's published list.
Sources
DuckDuckGo documentationDuckDuckGo IP list
Last reviewed

Allow or block DuckDuckBot

Blocking DuckDuckBot removes your pages from DuckDuckGo search results, so we don't offer that snippet. If you need to, write the rule by hand.

Allow DuckDuckBot
User-agent: DuckDuckBot
Allow: /

Lets DuckDuckBot crawl everything. You only need this if a broader rule would otherwise block it.

Should you block it?

Keep DuckDuckBot allowed. Blocking it removes your pages from DuckDuckGo search results.

Write a full policy with the AI Crawler Policy Generator

How to verify a DuckDuckBot request

  1. Find the IP address of the request in your server or CDN logs.
  2. Get DuckDuckGo's published list (DuckDuckGo IP list).
  3. Check whether the IP falls inside one of the ranges. If it doesn't, the request is not from DuckDuckBot, whatever its user agent says.
Test DuckDuckBot requests

Questions about DuckDuckBot

Is DuckDuckBot safe to allow?

DuckDuckBot reads public pages; it does not log in or submit forms. Whether to allow it depends on whether you want DuckDuckGo's product to use your pages.

How do I know a request is really DuckDuckBot?

Check the request against what DuckDuckGo publishes: Check the IP against DuckDuckGo's published list. The user agent alone proves nothing.

How often does DuckDuckBot visit?

It depends on your site; DuckDuckGo doesn't publish a schedule. Referscope shows the real visits.

Related crawlers

All AI crawlers

Free tools for DuckDuckBot

AI Crawler Policy Generator

Build a robots.txt that blocks AI training and keeps AI search and Google.

Build your robots.txt

AI Bot Verification Tester

Paste an IP and user agent from your logs and see if the bot is real.

Verify a bot

robots.txt Checker

See which AI crawlers your robots.txt allows or blocks.

Check robots.txt

Less guessing. Clearer growth.

Connect your site today and see your first verified AI crawler by tomorrow.

Explore ReferscopeBecome a design partner