Sitemap: https://cybora.tech/sitemap.xml# Global base rulesUser-agent: *Allow: /wp-admin/admin-ajax.phpAllow: /*/*.cssAllow: /*/*.jsDisallow: /wp-admin/Disallow: /wp-includes/Disallow: /readme.htmlDisallow: /license.txtDisallow: /xmlrpc.phpDisallow: /wp-login.phpDisallow: /wp-register.phpDisallow: */disclaimer/*Disallow: *?attachment_id=Disallow: /*?filtering=# Search Engine VisibilityUser-agent: GooglebotAllow: /User-agent: bingbotAllow: /User-agent: SlurpAllow: /User-agent: DuckDuckBotAllow: /User-agent: BaiduspiderAllow: /User-agent: YandexAllow: /# AI Bot RestrictionsUser-agent: GPTBotDisallow: /User-agent: OAI-SearchBotDisallow: /User-agent: Google-ExtendedDisallow: /User-agent: anthropic-aiDisallow: /User-agent: ClaudeBotDisallow: /User-agent: Claude-WebDisallow: /User-agent: FacebookBotDisallow: /User-agent: Meta-ExternalAgentDisallow: /User-agent: Meta-ExternalFetcherDisallow: /User-agent: BytespiderDisallow: /User-agent: Applebot-ExtendedDisallow: /User-agent: Bingbot-AIDisallow: /User-agent: cohere-aiDisallow: /User-agent: DiffbotDisallow: /User-agent: OmgilibotDisallow: /User-agent: img2datasetDisallow: /User-agent: CCBotDisallow: /User-agent: Kangaroo BotDisallow: /User-agent: PanguBotDisallow: /User-agent: Sidetrade indexerDisallow: /User-agent: webz.ioDisallow: /User-agent: ScrapyDisallow: /User-agent: TimpibotDisallow: /User-agent: PerplexityBotDisallow: /User-agent: YouBotDisallow: /User-agent: ChatGPT-UserDisallow: /User-agent: AmazonbotDisallow: /User-agent: AI2BotDisallow: /User-agent: ApplebotDisallow: /User-agent: PhindBotDisallow: /# Bot & Scraper Protection (Basic List)User-agent: python-requestsDisallow: /User-agent: curlDisallow: /User-agent: wgetDisallow: /User-agent: libwww-perlDisallow: /User-agent: Go-http-clientDisallow: /User-agent: Apache-HttpClientDisallow: /User-agent: ScrapyDisallow: /User-agent: JavaDisallow: /# Block Author ArchivesUser-agent: *Disallow: /author/Disallow: /?author=Disallow: *?replytocom=Disallow: *?replytocom# E-commerce Basic CleanupUser-agent: *Disallow: /cart/Disallow: /checkout/Disallow: /my-account/Disallow: /wishlist/Disallow: /compare/Disallow: /account/Disallow: /order-received/Disallow: /order-pay/Disallow: /view-order/# Allow CSS/JSUser-agent: *Allow: /*.css$User-agent: *Allow: /*.js$# Allow ImagesUser-agent: *Allow: /*.jpg$User-agent: *Allow: /*.jpeg$User-agent: *Allow: /*.png$User-agent: *Allow: /*.gif$User-agent: *Allow: /*.webp$User-agent: *Allow: /*.bmp$User-agent: *Allow: /*.ico$User-agent: *Allow: /*.svg$# Block Search & PaginationUser-agent: *Disallow: /?s=Disallow: /*?s=Disallow: /search/Disallow: /page/Disallow: /*?p=Disallow: /*?paged=Disallow: /*?page=# Block common crawl trap parametersUser-agent: *Disallow: /*?p=*Disallow: /*&p=*Disallow: /*?preview=*User-agent: *Allow: /ads.txtUser-agent: *Allow: /app-ads.txt# Custom RulesCrawl-delay: 5# Generated by Better Robots.txt# https://better-robots.com# Crawl governance based on the Interpretive Governance framework# Reference: https://interpretive-governance.org/