robots.txt details

saved: size: 399 kb md5 checksum: 0583d66ea5dab52664b3e981b0be015b

websites using this robots.txt

content


User-agent: Yandex
Crawl-delay: 2
Disallow: /search/
Clean-param: utm_source&utm_medium&utm_term&utm_campaign /

User-agent: Googlebot
Disallow: /search/
Disallow: /*?*utm_source=
Disallow: /*?*utm_medium=
Disallow: /*?*utm_term=
Disallow: /*?*utm_campaign=

User-agent: Slurp
Crawl-delay: 8
Disallow: /search/
Disallow: /*?*utm_

User-agent: *
Crawl-delay: 10
Disallow: /search/
Disallow: /*?*utm_