robots.txt details

saved: size: 139 kb md5 checksum: 613c4da73bf925b1af8e029003d0ead9

websites using this robots.txt

content


User-agent: *
Crawl-delay: 1
Allow: /$
Allow: /?hl=
Allow: /h$
Allow: /h?hl=
Allow: /u/0/h
Allow: /u/0/h?hl=
Disallow: /?hl=*&
Disallow: /