User-agent: gsa-crawler Disallow: User-agent: Googlebot Disallow: User-agent: Twitterbot Disallow: User-agent: Baiduspider Disallow: User-agent: bingbot Crawl-delay: 5 User-agent: UptimeRobot Disallow: User-agent: archive.org_bot Disallow: User-agent: ia_archiver Disallow: User-agent: MJ12Bot Crawl-delay: 10 User-agent: YandexBot Crawl-delay: 10 User-agent: Filpboard Crawl-delay: 10 User-agent: * Disallow: / # Sitemap Full Path Sitemap: https://www.parliament.nz/xmlsitemap