LLM agents need sites to respect 'Accept: text/plain' (www.skeptrune.com)

Developer skeptrune implemented an agent-friendly web option: when an HTTP Accept header includes text/plain or text/markdown before text/html, the site serves plain Markdown (or text) instead of full HTML. He built this for Astro sites by converting built HTML to Markdown with the @wcj/html-to-markdown-cli, wiring a post-build Bash script and package.json steps, and adding a Cloudflare Worker that inspects Accept headers, prefers /markdown/* files, falls back to /html/*, and serves sitemap.xml at the root for agents. A Caddyfile example is included for simpler reverse-proxy setups. Why it matters: Markdown reduces context tokens dramatically (the Bun team reported ~10x token drop), cutting scraping and LLM inference costs for pay-per-token systems and making content cheaper for agents to ingest — which can increase scraping, inclusion in training data, and visibility to AI assistants (what the author calls GEO). For the AI/ML community this is a pragmatic optimization: sites that respect Accept: text/plain allow agents to consume semantic, low-noise text instead of bulky markup, saving compute and improving downstream parsing. The approach is lightweight, works with static site generators, and provides implementation patterns (CLI conversion, worker logic, Caddy config) for adopters.