Claude Code's compaction discards data that's still on disk (github.com)

Claude Code has introduced a proposed solution to a critical issue regarding data loss during auto-compaction in its summarization process. When compaction occurs mid-task, important user-provided data—such as DOM markup—gets discarded even though it still exists on disk. This problem leads to hallucinations and inaccuracies when users seek specifics from the compacted summaries, significantly hindering productivity, especially for complex tasks involving substantial user inputs. The proposal suggests a mechanism that includes indexed references to the original transcript within the compacted summaries, allowing Claude to recover specific data as needed without sacrificing context or overloading the system with excess information. This enhancement is significant for the AI/ML community as it addresses a fundamental flaw in memory management within conversational AI. It promises a more efficient and user-friendly experience by enabling "surgical recovery" of original data only when necessary, thus optimizing token usage and processing resources. By leveraging existing transcripts on disk and incorporating simple metadata pointers, the approach minimizes architectural changes while maximizing recovery capabilities, ultimately paving the way for more robust AI interactions and smoother workflows for developers and users alike.