robots.txt details

saved: size: 693 kb md5 checksum: 1c10d02a1f29f31dc80e85eaeddaa247

websites using this robots.txt

content


Sitemap: https://www.taf.hr/sitemap/taf.hr.xml
Sitemap: https://www.taf.hr/pub/sitemap/storelocator_2.xml
Sitemap: https://www.taf.hr/blog/sitemap_index.xml
Sitemap: https://www.taf.hr/corpo/sitemap_index.xml
Sitemap: https://www.taf.hr/zones/sitemap_index.xml
Disallow: /lib/
Disallow: /*.php$
Disallow: /pkginfo/
Disallow: /report/
Disallow: /var/
Disallow: /catalog/
Disallow: /customer/
Disallow: /sendfriend/
Disallow: /review/
Disallow: /*SID=
Disallow: /catalogsearch/result/?q=*
Disallow: /deca/*
Disallow: /ajax/
Disallow: /extendedshelf/
User-agent: bingbot
Crawl-delay: 1
User-agent: msnbot
Crawl-delay: 1
User-agent:  msnbot-media
Crawl-delay: 1
User-agent:  Yandex
Crawl-delay: 1