Show HN: Streaming Structured Output Is Still Broken (schnabl.cx)

A Show HN post argues that while LLMs can stream text and can produce structured output, none of the major APIs (OpenAI, Anthropic, Google Gemini) currently support reliable, incremental streaming of structured data—so developer expectations break in different ways. In a practical agent loop the author describes, streaming just the "action" could halve step latency (8–10s → ~5s), but streaming implementations either drop parsed output and token stats (Gemini), force manual JSON reconstruction (OpenAI’s delta events), or emit partial JSON that isn’t Pydantic-friendly (Anthropic). Switching from non-streaming to streaming often silently breaks the API contract. This is significant because most production pipelines rely on typed schemas (Pydantic) and need predictable, incremental updates for UI, routing, and latency-sensitive agents. Workarounds—ijson/JsonRiver for incremental JSON parsing, pydantic_ai or LangDiff’s per-field event hooks—demonstrate patterns but add substantial boilerplate and still fail to convey field-level deltas, defaults vs. updates, or nested-change routing. The problem is an interface design gap, not a model limitation: the community would benefit from APIs that emit typed, structured delta events, field-level diffs, and final parsing/metrics alongside raw text to make streaming structured output reliable and ergonomic.