User-agent: * Crawl-delay: 1 Disallow: /mailbag/ Disallow: /newsletters/issue/ Disallow: /617/ Disallow: /session/ Disallow: /html/ User-agent: magpie-crawler* Disallow: /