robots.txt viewer

rozetka.com.ua @ 2019-01-09 12:42:07


website: http://rozetka.com.ua
robots.txt: rozetka.com.ua/robots.txt

/robots.txt (659B)

User-Agent: *

Disallow: /cgi-bin
Disallow: /search/
Disallow: /*sort=*
Disallow: /*utm_medium=rooms*
Disallow: /*promotions/top_section*
Disallow: /*tab=*
Disallow: /*gclid=*
Disallow: /*yclid=*
Disallow: /*/delivery/
Disallow: /*utm_trafmaguid=*
Disallow: /comparison/
Disallow: /*/comparison/
Disallow: /landing_pages/*
Disallow: /*kits_costs*
Disallow: /landing-pages/*
Disallow: /lazy_load/*
Disallow: /*/recent.rss
Disallow: /ua/
Allow: /recent_recommends/action=getGoodsDetailsJSON/?goods_ids
Disallow: /*?*

User-agent: Yahoo
Disallow: /

User-agent: MJ12bot
Disallow: /

User-agent: Mediapartners-Google
Disallow: /

User-agent: AhrefsBot
Disallow: /