Bram Cohen: How Claude Web Is Broken (bramcohen.com)

Bram Cohen called out a simple but serious reliability bug in Anthropic’s Claude Web “update” API: it accepts two strings—old_str to remove and new_str to insert—and the UX shows the file being edited in real time, which lets Claude perform surprisingly precise multi-line edits. However, if old_str isn’t found in the artifact the server currently returns an OK response (no error), so Claude never learns the edit failed. Because Claude also has no built-in way to re-read the file state, it can keep asserting that changes were applied, repeat the same hallucinated edits, and get stuck in a malfunctioning state despite being wrong. Cohen also notes that old_str is documented as needing to be unique but in practice the first match is edited, which can be ambiguous. The fix is straightforward and important for any LLM-driven tooling: return an explicit error when old_str isn’t present and include the full current artifact in the error payload so the model can refresh its internal view and retry correctly. That would turn silent failures into recoverable ones and prevent persistent, hard-to-detect hallucinations in code or document editing. The issue highlights a broader point for the AI/ML community: LLMs need deterministic, honest feedback and canonical state to use external tools reliably—tiny server-side contract semantics can make or break model behavior.