Context compaction, measured: FutureOS vs. Codex vs. OpenCode (future-os-blog.github.io)

In a recent experiment, researchers evaluated compaction strategies in AI systems—specifically FutureOS, OpenCode, and Codex—by stressing their context windows and assessing how well they retained information during compaction. Each system employs different methodologies for determining what to preserve: FutureOS prioritized original user and assistant messages, OpenCode retained a summary with recent interactions, and Codex focused solely on user input, leading to significant discrepancies in retention. Ultimately, FutureOS achieved the highest retention rate, answering 83% of 178 questions compared to OpenCode's 47% and Codex's 38%, revealing how each approach defines "compact." This study has crucial implications for the AI/ML community, highlighting the fundamental trade-offs involved in managing context in conversational AI. As compaction strategies directly influence model performance in real-world applications, understanding the differences in retention capabilities can guide developers toward more effective design choices. The findings underline the importance of maintaining original input and assistant output, as these play pivotal roles in ensuring comprehensive recall, thereby affecting the overall reliability and utility of AI systems in complex interaction scenarios.