GPT-6 Astra edited 131,072 weights in Qwen3.5-0.8B, with no backpropagation (twitter.com)

GPT-6 Astra has successfully edited 131,072 weights in the Qwen3.5-0.8B model, achieving a remarkable increase in MATH-500 accuracy from 7.9% to 48.1%, all without utilizing backpropagation. This innovative approach, referred to as Weight-Space Programming, involves analyzing the model's logs to identify and address reasoning loops while maintaining the integrity of productive reasoning. The changes predominantly target the neuron layers responsible for writing outputs, allowing the model to reduce repetitive responses and enhance its performance in mathematical tasks significantly. This advancement is significant for the AI/ML community as it demonstrates a novel method to refine model behavior and improve accuracy with minimal alterations to weights—only 0.016% of the total. By focusing on the model's internal structures and dynamics rather than standard retraining methods, this technique not only boosts mathematical problem-solving capabilities by up to six times but also enhances instruction following by three times. The implications of this research suggest a potential shift in how models can be fine-tuned and optimized, offering a path forward for developing more efficient and effective AI systems.