robots.txt details

saved: 2019-01-28 08:14:43 size: 71 kb md5 checksum: 930e65bba6c106e6402a04f7f659afed

websites using this robots.txt

detailed.com

content

raw json

User-agent: *
Disallow: /tweets

User-agent: SemrushBot
Crawl-delay: 10