I tried to break weight programming (flecart.github.io)

A recent exploration into "weight programming" reveals significant insights about advanced AI models like Astra and Opus 5.5. The author successfully replicated results from a prominent MIT study, confirming that these models can manipulate weights directly, akin to compiling simple code. While Astra adeptly solved the Breakout game by optimizing actions based on direct weight manipulation, Opus 5.5 struggled, showcasing a limitation when tasked with complex reasoning. Interestingly, when faced with tasks requiring higher complexity, both models demonstrated a reluctance to attempt solutions, with reasoning length increasing proportional to task difficulty, highlighting intrinsic constraints in these systems. This investigation is vital for the AI/ML community as it sheds light on the boundaries of current transformer models and the potential for future advancements. The findings suggest that rather than relying solely on traditional backpropagation, alternative algorithms may become viable for optimizing weights more efficiently. Furthermore, the ability of models to internally verify their reasoning steps introduces an intriguing aspect of model robustness, suggesting that verification may serve as a defense against potential errors during processing. Overall, these insights into weight programming and complexity limits could refine the understanding of LLM capabilities and inspire innovative approaches to developing more efficient AI systems.