robots.txt details

3eb0b17ca2da1ae961600b97bcffcd19

saved: | size: 300kb | checksum: 3eb0b17ca2da1ae961600b97bcffcd19

websites using this robots.txt

content


User-agent: *
Crawl-delay: 30
Disallow: /admin*
Disallow: /storage/*

User-agent: MJ12bot
Disallow: /

User-agent: SemrushBot
Disallow: /

User-agent: AhrefsBot
Disallow: /

User-agent: YandexBot
Disallow: /

User-agent: PetalBot
Disallow: /

User-agent: MegaIndex.ru
Disallow: /