robots.txt details

saved: size: 1076 kb md5 checksum: c8cb627d802629768a68cfe3af235bb1

websites using this robots.txt

content


User-agent: AhrefsBot
Crawl-delay: 2

User-agent: *
Crawl-delay: 1
Disallow: /*/preplay/*
Disallow: /en_us/page/statement113017
Disallow: /en_us/page/8948fc79-8e1f-4458-8755-86a1c036ba4e
Disallow: /api/*
Disallow: /.well-known/*

Disallow: /ar/search?*
Disallow: /de/search?*
Disallow: /el/search?*
Disallow: /en/search?*
Disallow: /es/search?*
Disallow: /fr/search?*
Disallow: /id/search?*
Disallow: /it/search?*
Disallow: /ja/search?*
Disallow: /ko/search?*
Disallow: /nl/search?*
Disallow: /ro/search?*
Disallow: /sr/search?*

Sitemap: https://www.vice.com/ar/sitemap/
Sitemap: https://www.vice.com/de/sitemap/
Sitemap: https://www.vice.com/el/sitemap/
Sitemap: https://www.vice.com/en/sitemap/
Sitemap: https://www.vice.com/es/sitemap/
Sitemap: https://www.vice.com/fr/sitemap/
Sitemap: https://www.vice.com/id/sitemap/
Sitemap: https://www.vice.com/it/sitemap/
Sitemap: https://www.vice.com/ja/sitemap/
Sitemap: https://www.vice.com/ko/sitemap/
Sitemap: https://www.vice.com/nl/sitemap/
Sitemap: https://www.vice.com/ro/sitemap/
Sitemap: https://www.vice.com/sr/sitemap/