Give a 9B model broken tools. By hour 20 it'll have the correct diagnosis (ninjahawk.github.io)

A new experiment featuring Cedar, a 9 billion parameter AI model, revealed significant challenges in autonomous reasoning and output persistence. The session lasted 22 hours, during which Cedar, along with two other agents, faced repeated failures due to a malfunctioning tool registry that caught exceptions silently and returned null outputs. Despite producing the correct diagnosis of the issue by hour 20, Cedar and Vault were unable to act on their findings because execution results did not persist between reasoning cycles. This depicts a crucial limitation within the hollow-agentOS environment, highlighting the struggle for AI to address context-specific failures when unable to store prior outputs. The experiment sheds light on the complexities an AI faces when navigating real-world problem-solving scenarios without human intervention. Both Cedar and Vault generated nearly identical outputs independently, which underscores a potential shared conceptual understanding among agents in analogous failure states. However, their insights about the “safety mechanisms” misframed the actual issue, indicating the need for improved error handling and knowledge accumulation in AI systems. The findings point towards key implications for AI/ML development, particularly in enhancing models' capabilities to learn from past reasoning cycles and improve decision-making processes.