robots.txt details

106a1780923f1397ea252d91f0b5b6e2

saved: | size: 464kb | checksum: 106a1780923f1397ea252d91f0b5b6e2

websites using this robots.txt

content


User-agent: *
Disallow:
Crawl-delay: 1
Host: https://tools.seo-auditor.com.ru
Sitemap: https://tools.seo-auditor.com.ru/SiteMapIndex.xml
Sitemap: https://tools.seo-auditor.com.ru/SiteMap.xml
Sitemap: https://tools.seo-auditor.com.ru/SiteMapVirusTotal.xml
Sitemap: https://tools.seo-auditor.com.ru/SiteMapCheckIP.xml
Sitemap: https://tools.seo-auditor.com.ru/SiteMapSiteIP.xml
User-agent: Yahoo! Slurp
Crawl-delay: 100
User-agent: Slurp
Crawl-delay: 100