Fire Your Fetch Tool — Let your Agent Use a Browser (www.robert-glaser.de)

Agents’ built‑in “fetch” tools are still murky about whether they execute client‑side JavaScript, which matters hugely for SPAs where content is hydrated in the browser. Codex’s docs are ambiguous; Anthropic’s Claude Code clearly separates WebSearch (find sources) from WebFetch (retrieve a page) and explicitly notes its web fetch tool does not execute site JS. That gap means many agent fetches return thin, pre‑JS DOMs and can mislead downstream reasoning or information extraction. A practical fix is to render pages once with a headless browser and hand the model a normalized artifact. Example: run Chrome headless to dump the post‑JS DOM with chrome --headless=new --dump-dom "https://example.com", then convert to model‑friendly text via uvx markitdown or strip HTML tags to save tokens. This delegates networking/JS execution to the browser, keeps context small and stable, and avoids pulling extra MCP tool descriptions into the model prompt. Recommended agent behavior: use the built‑in search tool for discovery, but fetch actual page content with a local headless browser (fallback to curl/wget if needed). Caveats: requires a local browser, attention to privacy/security of fetched pages, and small wrapper scripts to normalize output and reduce token overhead.