robots.txt details

saved: size: 446 kb md5 checksum: 000526d5b5477955d65f1c35bb61a0f5

websites using this robots.txt

content


User-agent: Googlebot-News
Disallow: /angebote/

User-agent: *
Disallow: /zeit/
Disallow: /templates/
Noindex: /templates/
Disallow: /hp_channels/
Disallow: /send/
Disallow: /suche/
Noindex: /suche/
Disallow: */comment-thread?
Noindex: */comment-thread?
Disallow: */liveblog-backend*
Noindex: */liveblog-backend*

User-agent: Baiduspider
Disallow: /

User-agent: GrapeshotCrawler
crawl-delay: 3

Sitemap: https://www.zeit.de/gsitemaps/index.xml