Agents Design Is Still Hard (lucumr.pocoo.org)

An experienced practitioner sums up recent hands‑on lessons: building production agents remains messy and requires bespoke engineering. Higher‑level SDKs (they used Vercel AI SDK provider abstractions) can simplify wiring but break down when real tool use and provider‑side tools are involved—messaging formats, cache control and opaque errors push you back to target provider SDKs directly. Caching behavior differs drastically by platform (Anthropic enforces explicit cache points), and explicit cache management, while more work, yields predictable cost and enables strategies like branching conversations and context editing. Reinforcement injected into the loop after tool calls (reminders of objectives, hints for retries, state change notices) does heavier lifting than expected, especially for parallel processing and recovery from failures. Key technical patterns and implications: treat agents as custom loops, not as off‑the‑shelf abstractions; use subagents/subinferences to isolate iterative failures (run risky steps until success and only surface summaries); implement a shared virtual file system so all tools (code execution, inference, image generators) read/write common paths to avoid dead ends. Context editing can remove noisy failure traces but will invalidate caches. Output should be an explicit tool (tracked and enforced), though steering tone via a secondary LLM often hurts quality. Model choice still matters: Haiku/Sonnet are strong tool callers, Gemini 2.5 is useful for document/image subtools, and cheaper token costs don’t necessarily mean cheaper agent loops. Overall: expect significant custom engineering and hard testing/evals.