User-agent: Googlebot-News Disallow: /angebote/ User-agent: * Disallow: /zeit/ Disallow: /templates/ Disallow: /hp_channels/ Disallow: /send/ Disallow: /suche/ Disallow: /rezepte/suche/ Disallow: */comment-thread? Disallow: */liveblog-backend* User-agent: Baiduspider Disallow: / User-agent: GrapeshotCrawler crawl-delay: 3 Sitemap: https://www.zeit.de/gsitemaps/index.xml