We measured Opus 5.5 for 30 days. Here's what we found (blog.nathanlangley.dev)

LiveNerf, an open-source project aimed at assessing whether AI models like Opus 5.5 degrade in performance after release, has completed a 30-day measurement period without detecting significant accuracy decline. Initiated shortly after the model's launch, LiveNerf examined 34 consecutive days of question-answering accuracy against a baseline, revealing that Opus 5.5 maintained an average accuracy of approximately 58% over two comparison windows—a finding that echoes the broader concerns within the AI community about potential "nerfing" or performance degradation over time. The significance of this study lies in its independent approach to evaluating model performance, countering the anecdotal complaints often tied to new releases. Although daily variations were observed, these did not indicate a sustained change in accuracy. The methodology involved a fixed set of challenging questions, alongside control measures, highlighting that while the overall accuracy remained stable, nuances like reasoning effort—measured through output tokens—could reflect unreported internal adjustments by developers. LiveNerf plans to extend its analysis to capture more granular changes in individual capabilities and foster a more robust framework for ongoing model evaluation in the AI/ML landscape.