1User-agent: AI2Bot2User-agent: Ai2Bot-Dolma3User-agent: Amazonbot4User-agent: anthropic-ai5User-agent: Applebot6User-agent: Applebot-Extended7User-agent: Brightbot 1.08User-agent: Bytespider9User-agent: CCBot10User-agent: ChatGPT-User11User-agent: Claude-Web12User-agent: ClaudeBot13User-agent: cohere-ai14User-agent: cohere-training-data-crawler15User-agent: Crawlspace16User-agent: Diffbot17User-agent: DuckAssistBot18User-agent: FacebookBot19User-agent: FriendlyCrawler20User-agent: Google-Extended21User-agent: GoogleOther22User-agent: GoogleOther-Image23User-agent: GoogleOther-Video24User-agent: GPTBot25User-agent: iaskspider/2.026User-agent: ICC-Crawler27User-agent: ImagesiftBot28User-agent: img2dataset29User-agent: ISSCyberRiskCrawler30User-agent: Kangaroo Bot31User-agent: Meta-ExternalAgent32User-agent: Meta-ExternalFetcher33User-agent: OAI-SearchBot34User-agent: omgili35User-agent: omgilibot36User-agent: PanguBot37User-agent: PerplexityBot38User-agent: PetalBot39User-agent: Scrapy40User-agent: SemrushBot-OCOB41User-agent: SemrushBot-SWA42User-agent: Sidetrade indexer bot43User-agent: Timpibot44User-agent: VelenPublicWebCrawler45User-agent: Webzio-Extended46User-agent: YouBot4748Disallow: /