robots.txt details

saved: size: 483 kb md5 checksum: 001e2604d2ccbd338eeb1df5022d2fca

websites using this robots.txt

content


User-agent: *
Disallow: /
Crawl-delay: 500

User-agent: Googlebot
Disallow:
Disallow: /adm/
Disallow: /administration/
Disallow: /admin/
Disallow: /adminportal/
Disallow: /drupal/
Disallow: /joomla/
Disallow: /cgi-bin/
Disallow: /wp-admin/
Disallow: /wp-content/
Disallow: /trackback/
Disallow: /blackhole/
Disallow: /mint/
Disallow: /feed/
Disallow: /outbound/
Disallow: /js/
Disallow: /css/
Disallow: /files/
Disallow: /cms/
Disallow: /blogs/wp-admin/
Disallow: /blogs/wp-content/