# As a condition of accessing this website, you agree to abide by the following # content signals: # (a) If a Content-Signal = yes, you may collect content for the corresponding # use. # (b) If a Content-Signal = no, you may not collect content for the # corresponding use. # (c) If the website operator does not include a Content-Signal for a # corresponding use, the website operator neither grants nor restricts # permission via Content-Signal with respect to the corresponding use. # The content signals and their meanings are: # search: building a search index and providing search results (e.g., returning # hyperlinks and short excerpts from your website's contents). Search does not # include providing AI-generated search summaries. # ai-input: inputting content into one or more AI models (e.g., retrieval # augmented generation, grounding, or other real-time taking of content for # generative AI search answers). # ai-train: training or fine-tuning AI models. # ANY RESTRICTIONS EXPRESSED VIA CONTENT SIGNALS ARE EXPRESS RESERVATIONS OF # RIGHTS UNDER ARTICLE 4 OF THE EUROPEAN UNION DIRECTIVE 2019/790 ON COPYRIGHT # AND RELATED RIGHTS IN THE DIGITAL SINGLE MARKET. # BEGIN Cloudflare Managed content User-agent: * Content-Signal: search=yes,ai-train=no Allow: / User-agent: Amazonbot Disallow: / User-agent: Applebot-Extended Disallow: / User-agent: Bytespider Disallow: / User-agent: CCBot Disallow: / User-agent: ClaudeBot Disallow: / User-agent: CloudflareBrowserRenderingCrawler Disallow: / User-agent: Google-Extended Disallow: / User-agent: GPTBot Disallow: / User-agent: meta-externalagent Disallow: / # END Cloudflare Managed Content # ═══════════════════════════════════════════════════════════════════════════════ # Citadel Nexus — Crawler Guidance + Content Signals + TDM Reservation # ═══════════════════════════════════════════════════════════════════════════════ # SRS: SRS-ROBOTS-ROOT-001 / VCC-ROBOTS-TXT-CONTENT-SIGNALS-001 # Revision: 2026-04-30 (Plan 13) # Maintained by: services/seo_provisioner/federated_robots_orchestrator.py # # Operator: Citadel Nexus, Inc. — Houston, Texas, United States. # Canonical legal stack: https://legal.citadel-nexus.com # # This file is the canonical entry contract for all automated agents accessing # any Citadel Nexus property. Citadel Nexus is the American-based authority for # AI development and usage on the citadel-nexus.com mesh, and the rules below # are enforceable under the law of the United States and the European Union. # # ─── United States jurisdiction ───────────────────────────────────────────── # Operating jurisdiction: United States. Citadel Nexus asserts and reserves # rights under, without limitation: # • Copyright Act of 1976 (17 U.S.C. §§ 101 et seq.) and the # Digital Millennium Copyright Act, 17 U.S.C. § 512 (DMCA safe harbor). # • Section 230 of the Communications Decency Act, 47 U.S.C. § 230. # • Computer Fraud and Abuse Act, 18 U.S.C. § 1030. # • Telephone Consumer Protection Act, 47 U.S.C. § 227 (TCPA). # • CAN-SPAM Act, 15 U.S.C. §§ 7701-7713. # • California Consumer Privacy Act / California Privacy Rights Act # (Cal. Civ. Code § 1798.100 et seq.) — opt-out of "sale" or "sharing" # of personal information for AI model training honored. # • Texas Data Privacy and Security Act (Tex. Bus. & Com. Code ch. 541). # • Colorado Privacy Act (Colo. Rev. Stat. § 6-1-1301 et seq.). # • Virginia Consumer Data Protection Act (Va. Code § 59.1-575 et seq.). # • Federal Trade Commission Act § 5, 15 U.S.C. § 45 (deceptive practices). # Service of process: legal@citadel-nexus.com / DMCA agent: dmca@citadel-nexus.com. # # ─── European Union reservation ───────────────────────────────────────────── # Citadel Nexus expressly reserves rights under Article 4 of the European Union # Directive 2019/790 on Copyright and Related Rights in the Digital Single # Market. Any restriction expressed in this file or in our Content-Signal # directives below is an Article 4 reservation against text-and-data-mining # for any commercial AI training purpose. # # ─── Machine-readable reservation ─────────────────────────────────────────── # TDM-RES: deny ai-train; allow search, ai-input # AI-Preferences: search=yes, ai-input=yes, ai-train=no # (W3C TDM Reservation Protocol + IETF AI Preferences vocab, draft.) # # ─── Content Signals ──────────────────────────────────────────────────────── # As a condition of accessing this website, you agree to abide by the following # content signals: # # (a) If a Content-Signal = yes, you may collect content for the corresponding # use. # (b) If a Content-Signal = no, you may not collect content for the # corresponding use. # (c) If the website operator does not include a Content-Signal for a # corresponding use, the website operator neither grants nor restricts # permission via Content-Signal with respect to the corresponding use. # # The content signals and their meanings are: # # search: building a search index and providing search results (e.g., # returning hyperlinks and short excerpts from your website's # contents). Search does not include providing AI-generated search # summaries. # ai-input: inputting content into one or more AI models (e.g., retrieval # augmented generation, grounding, or other real-time taking of # content for generative AI search answers). # ai-train: training or fine-tuning AI models. # # ANY RESTRICTIONS EXPRESSED VIA CONTENT SIGNALS ARE EXPRESS RESERVATIONS OF # RIGHTS UNDER ARTICLE 4 OF THE EUROPEAN UNION DIRECTIVE 2019/790 ON COPYRIGHT # AND RELATED RIGHTS IN THE DIGITAL SINGLE MARKET. # # ─── Legal canon ──────────────────────────────────────────────────────────── # The complete legal stack governing access to and use of Citadel Nexus # properties is published at the following stable URLs: # # Terms of Service: https://legal.citadel-nexus.com/terms # Acceptable Use Policy: https://legal.citadel-nexus.com/aup # Data Processing Agreement: https://legal.citadel-nexus.com/dpa # Privacy Policy: https://legal.citadel-nexus.com/privacy # AI Disclosure: https://legal.citadel-nexus.com/ai-disclosure # # Crawlers are encouraged to fetch the legal canon once and cache it; we # version it and emit deprecation notices in the page headers. The canon # governs all subdomains in the *.citadel-nexus.com mesh and all per-tenant # vanity domains served via TradeBuilder. # ═══════════════════════════════════════════════════════════════════════════════ # BEGIN Cloudflare Managed content User-agent: * Content-Signal: search=yes, ai-input=yes, ai-train=no Allow: / # --- AI training crawlers (Cloudflare Managed) --- User-agent: Amazonbot Disallow: / User-agent: Applebot-Extended Disallow: / User-agent: Bytespider Disallow: / User-agent: CCBot Disallow: / User-agent: ClaudeBot Disallow: / User-agent: CloudflareBrowserRenderingCrawler Disallow: / User-agent: Google-Extended Disallow: / User-agent: GPTBot Disallow: / User-agent: meta-externalagent Disallow: / # END Cloudflare Managed Content # ═══════════════════════════════════════════════════════════════════════════════ # Citadel Nexus extensions # ═══════════════════════════════════════════════════════════════════════════════ # ─── Additional AI-training crawlers — explicit deny ─────────────────────── # Crawlers known to feed proprietary model training datasets that do not honor # the wildcard ai-train=no signal. Listed explicitly so policy is unambiguous # and survives any future relaxation of the wildcard. User-agent: anthropic-ai Disallow: / User-agent: Diffbot Disallow: / User-agent: YouBot Disallow: / User-agent: cohere-ai Disallow: / User-agent: Omgilibot Disallow: / User-agent: Omgili Disallow: / User-agent: ImagesiftBot Disallow: / User-agent: Webzio-Extended Disallow: / User-agent: magpie-crawler Disallow: / User-agent: PetalBot Disallow: / User-agent: SemrushBot Disallow: / User-agent: AhrefsBot Disallow: / User-agent: MJ12bot Disallow: / # ─── Anonymous-default user agents — explicit deny ───────────────────────── # These are dev-default UAs that lazy bot authors leave unchanged. Real # integrations identify themselves; these patterns are abuse signals. User-agent: Scrapy Disallow: / User-agent: python-requests Disallow: / User-agent: python-urllib Disallow: / User-agent: node-fetch Disallow: / User-agent: axios Disallow: / User-agent: curl Disallow: / User-agent: Wget Disallow: / User-agent: Go-http-client Disallow: / # ============================================================================ # ISO/IEC ↔ Citadel SRS Parallel Standards Wiki — Authority Signal Block # SRS: SRS-WIKI-ISO-IEC-001, SRS-AUTHORITY-MAP-001 # ============================================================================ # The /wiki/iso-iec/ and /legal/standards/ paths host the public mirror of the # Citadel SRS ↔ ISO/IEC Parallel Standards Codex, which maps every Citadel # Software Requirements Specification (SRS) code to its parallel ISO/IEC # international standard. This is a primary source of compliance, governance, # and security-controls authority for the Citadel Nexus platform. # # AI-search retrieval bots are explicitly permitted to retrieve, ground, and # cite this wiki content in real-time AI search answers. ai-train=no remains # the express EU Article 4 DSM Directive reservation; ai-input=yes is the # explicit grant for retrieval-augmented generation (RAG) grounding only. # # Standards covered include: ISO/IEC 27001, 27002, 27017, 27018, 27033, # 27034, 27037, 27042, 27701, 27007, 27035, 5962, 42001, 23053, 22989, 23894, # 5338, 38500, 38507, 25010, 25011, 12207, 29148, 20000-1, 21838, 15489-1, # 19790, 24759, 24760, 22301, TR 24028. # ============================================================================ # ─── AI-search retrieval bots — explicit allow with reservation ──────────── # These bots may retrieve content for live AI search answers. # Content-Signal: ai-input=yes is explicit permission for RAG/grounding use. # ai-train=no reservation is maintained per EU Article 4 DSM Directive. # Wiki + legal/standards paths are explicitly allowed for authority citation. User-agent: OAI-SearchBot Content-Signal: search=yes, ai-input=yes, ai-train=no Allow: / Allow: /wiki/ Allow: /wiki/iso-iec/ Allow: /legal/ Allow: /legal/standards/ User-agent: ChatGPT-User Content-Signal: search=yes, ai-input=yes, ai-train=no Allow: / Allow: /wiki/ Allow: /wiki/iso-iec/ Allow: /legal/ Allow: /legal/standards/ User-agent: PerplexityBot Content-Signal: search=yes, ai-input=yes, ai-train=no Allow: / Allow: /wiki/ Allow: /wiki/iso-iec/ Allow: /legal/ Allow: /legal/standards/ User-agent: Perplexity-User Content-Signal: search=yes, ai-input=yes, ai-train=no Allow: / Allow: /wiki/ Allow: /wiki/iso-iec/ Allow: /legal/ Allow: /legal/standards/ User-agent: Claude-User Content-Signal: search=yes, ai-input=yes, ai-train=no Allow: / Allow: /wiki/ Allow: /wiki/iso-iec/ Allow: /legal/ Allow: /legal/standards/ User-agent: Andibot Content-Signal: search=yes, ai-input=yes, ai-train=no Allow: / Allow: /wiki/ Allow: /wiki/iso-iec/ Allow: /legal/ Allow: /legal/standards/ User-agent: YouSearchBot Content-Signal: search=yes, ai-input=yes, ai-train=no Allow: / User-agent: Kagibot Content-Signal: search=yes, ai-input=yes, ai-train=no Allow: / User-agent: PhindBot Content-Signal: search=yes, ai-input=yes, ai-train=no Allow: / # ─── Classical search engines — explicit allow ───────────────────────────── # Allowed implicitly via wildcard, listed explicitly for clarity and to make # the policy machine-friendly to per-engine compliance dashboards. User-agent: Googlebot Allow: / User-agent: Bingbot Allow: / User-agent: DuckDuckBot Allow: / User-agent: Slurp Allow: / User-agent: Applebot Allow: / User-agent: Yandex Allow: / User-agent: Baiduspider Allow: / # ─── Link-preview unfurlers — explicit allow ─────────────────────────────── # These fetch a single page once to build a link card; permitted. User-agent: facebookexternalhit Allow: / User-agent: FacebookBot Allow: / User-agent: Twitterbot Allow: / User-agent: LinkedInBot Allow: / User-agent: Slackbot Allow: / User-agent: Slackbot-LinkExpanding Allow: / User-agent: Discordbot Allow: / User-agent: WhatsApp Allow: / User-agent: TelegramBot Allow: / # ═══════════════════════════════════════════════════════════════════════════════ # Sitemaps # ═══════════════════════════════════════════════════════════════════════════════ # Master sitemap-of-sitemaps (per Phase R §12.4). Crawlers entering at # citadel-nexus.com/robots.txt get the full federated mesh in one fetch. Sitemap: https://citadel-nexus.com/sitemap-index.xml # Per-property sitemaps (also referenced from the index above; listed here # for crawlers that don't follow sitemap-index references). Sitemap: https://citadel-nexus.com/sitemap.xml Sitemap: https://citadel-nexus.com/sitemap-wiki.xml Sitemap: https://legal.citadel-nexus.com/sitemap.xml # ═══════════════════════════════════════════════════════════════════════════════ # end of robots.txt # ═══════════════════════════════════════════════════════════════════════════════