User-agent: 80legs Disallow: / User-agent: 008 Disallow: / User-agent: * Crawl-delay: 5 # videos page disallow: /videos/play.php