# Athenic — robots.txt # # Reflex serves this file at /robots.txt by copying everything under # `src/assets/` into the frontend's public dir at build time. Crawlers # look here first to decide which paths to index. # # Policy: # - All public marketing pages (/, /pricing, /customers/*, /docs/*, # /terms-of-service, /privacy-policy, /cookie-policy) are open to # crawlers — that's the entire reason this domain exists. # - Everything authenticated or transactional is disallowed: auth # flows, the in-app surface (/chat, /o/*, /share/*, /super-admin/*), # the email-verification gate, the settings page, and the # server-side API routes Reflex exposes under /_next/* and /_event. # These pages either require login (so a crawler hits a redirect # loop) or expose nothing useful for search snippets. # - Sitemap pointer at the bottom names the absolute URL Google # Search Console expects. Must be kept in sync with ATHENIC_WEBSITE_BASE_URL # in rxconfig.py. User-agent: * Disallow: /auth/ Disallow: /verify-email Disallow: /chat Disallow: /settings Disallow: /o/ Disallow: /share/ Disallow: /super-admin/ Disallow: /_next/ Disallow: /_event/ Allow: / Sitemap: https://athenic.com/sitemap.xml