Learning from context is harder than we thought (hy.tencent.com)

Recent research reveals that while language models have advanced in tasks like solving complex mathematical problems or passing exams, they struggle significantly with real-time contextual learning—an essential skill for practical applications. Current AI systems primarily rely on static knowledge encoded during training, rather than adapting and learning from new contexts presented during use. This creates a disconnect, as many real-world tasks require dynamic reasoning and context absorption, akin to how humans work. To address this challenge, researchers have introduced CL-bench, a new benchmark designed to measure language models' ability to learn from context in 500 scenarios and evaluate various reasoning tasks. Results indicate that even leading models like GPT-5.1 perform poorly, solving less than 25% of tasks, illustrating a critical gap in their capabilities. This research underscores a paradigm shift needed within the AI/ML community to prioritize context learning and improve model reliability in dynamically evolving settings. The ultimate goal is to enhance AI performance by focusing on memory mechanisms that allow models to retain insights gleaned from contextual interactions, thereby reducing their reliance on fixed pre-trained knowledge.