User-agent: * Disallow: /users/auth/twitter* Disallow: /users/auth/github* Disallow: /report-abuse?url=* Disallow: /connect/@* Disallow: /search?q=* Disallow: /search/?q=* Disallow: /listings*?q=* Disallow: /mod/* Disallow: /mod?* Disallow: /admin/* Sitemap: https://dev-to-uploads.s3.amazonaws.com/sitemaps/sitemap.xml.gz