User-agent: * Disallow: /web-briefing Disallow: /web-briefing-three Disallow: /divya-testing Disallow: /one-hundred-edition-nine Disallow: /tour Disallow: /email-sign-ups Disallow: /one-hundred-edition-ten Disallow: /stage Disallow: /templates/two Disallow: /templates/one Disallow: /one-hundred-container-ports-2018/data Disallow: /one-hundred-container-ports-2019/data Disallow: /one-hundred-edition-eleven User-agent: MJ12bot Disallow: / User-agent: * Disallow:/rss/ User-agent: * Disallow:/data-tools/ User-agent: * Crawl-delay: 1 User-agent: Twitterbot Disallow: