robots.txt details

saved: size: 303 kb md5 checksum: 39289c125febc7f45199958a241fb42c

websites using this robots.txt

content


User-agent: msnbot-media/1.1
Disallow: /

User-agent: msnbot-media/2.0b
Disallow: /

User-agent: scrapy
Disallow: /

User-agent: Diffbot
Disallow: /

User-agent: proximic
Disallow: /*&ampshare=*

User-agent: *
Disallow: /action/
Disallow: /sn/
Disallow: /elastic/
Disallow: /search?
Disallow: /p-api/*