robots.txt details

saved: size: 460 kb md5 checksum: ad75064b142c23087616c91890b3b57d

websites using this robots.txt

content


User-agent: *
Disallow: /admin/
Disallow: /blogs/
Disallow: /comments/flag/
Disallow: /accounts/
Disallow: /obitsform/
Disallow: /mailform/
Disallow: /mailfriend/
Disallow: /random/
Disallow: /right2know/crime/mugs/
Disallow: /events/search/
Disallow: /search/
Disallow: /staff/
Disallow: /videos/sets/
Disallow: /manifest.json/

User-agent: MJ12bot
Disallow: /

User-agent: Yandex
Disallow: /

User-agent: YandexBot
Disallow: /

User-agent: Magnetbot
Allow: /