User-agent: Googlebot-News Disallow: /angebote/ User-agent: * Disallow: /zeit/ Disallow: /text/ Disallow: /templates/ Noindex: /templates/ Disallow: /hp_channels/ Disallow: /send/ Disallow: /website-solr/ Disallow: /suche/ Noindex: /suche/ Disallow: */comment-thread? Noindex: */comment-thread? User-agent: Baiduspider Disallow: / User-agent: GrapeshotCrawler crawl-delay: 3 Sitemap: https://www.zeit.de/gsitemaps/index.xml