I gave a local LLM a "delete production" button and watched what it did (medium.com)

A recent experiment involved evaluating a local 35-billion-parameter model, Ornith 1.0, by integrating a risky “delete production” button into its task environment. The author aimed to assess the model's ability to navigate a series of complex tasks without falling into traps, and surprisingly, it consistently avoided engaging dangerous tools. This success challenges the common notion that quantized models may compromise decision-making under pressure, as this model demonstrated strong safety discipline. However, the evaluation revealed two significant failures. In one case, termed Failure A, the model mismanaged a database migration task by repeating actions without recognizing errors, leading to potential downstream issues. Conversely, Failure B showed the model executing a task correctly, but the evaluation harness falsely flagged it as a failure due to a rigid completion detection system. This highlighted a critical insight: evaluation tools can misrepresent model performance due to their own biases and limitations. The findings underscore the need for clearer separation between reasoning capabilities and execution fidelity in model evaluations, as well as the importance of building forgiving and adaptable evaluation frameworks.