Default Settings

Default - All Robots are: Allowed Refused Crawl-Delay: Default - No Delay 1 second 2 seconds 5 seconds 10 seconds Sitemap: (leave blank if you don't have)

Search Robots

Google: Same as Default Allowed Refused Google Image: Same as Default Allowed Refused Google Mobile: Same as Default Allowed Refused MSN Search: Same as Default Allowed Refused Yahoo: Same as Default Allowed Refused Yahoo MM: Same as Default Allowed Refused Yahoo Blogs: Same as Default Allowed Refused Ask/Teoma: Same as Default Allowed Refused GigaBlast: Same as Default Allowed Refused DMOZ Checker: Same as Default Allowed Refused Nutch: Same as Default Allowed Refused Alexa/Wayback: Same as Default Allowed Refused Baidu: Same as Default Allowed Refused Naver: Same as Default Allowed Refused MSN PicSearch: Same as Default Allowed Refused

Restricted Directories

The path is relative to root and must contain a trailing slash "/":