How did we make DeepSeek outperform Opus (twitter.com)

DeepSeek has recently been enhanced to outperform Opus 4.7 in internal evaluations, achieving a success rate of 60%. This improvement stems from addressing common input errors encountered during tool calling, which were often misattributed to the model's limitations. Key issues included sending incorrect data types, improperly formatted JSON, and confusion stemming from the training context. By introducing a targeted tool-input repair layer, the developers identified four specific errors that were repeatedly causing failures in the system, allowing for strategic input corrections based on the model's needs. The significance of this achievement lies in the insight that much of what is perceived as a model's capability issue can instead be traced back to the design of its interaction contract. The approach of validating input first, followed by repairing, shifted the focus to the exact sources of errors, allowing for better clarity in interactions and enhanced error handling. By applying simpler validation mechanisms and making contracts more forgiving, DeepSeek is now more adept at executing commands effectively, demonstrating how careful architecture around model interaction can lead to superior performance without changing the core model itself. This shift to a more resilient input processing framework could influence future developments in AI-powered tools, emphasizing the importance of harness design over mere model training.