Show HN: ContinualCode – a coding agent that updates its weights from feedback (sdan.github.io)

A new coding agent named ContinualCode has been introduced, allowing users to directly update its weights through iterative feedback. Built upon frameworks like Tinker and SDPO, this model learns from corrections in real-time, making it significantly more interactive and responsive compared to traditional reinforcement learning methods. Users can deny a tool call, provide a correction, and the agent adjusts its behavior accordingly by running a gradient step before retrying the task, enhancing its coding capabilities with each interaction. This development is significant for the AI/ML community as it tackles conventional limitations in training models from human feedback. Traditional methods rely on scalar rewards that yield limited learning signals, while ContinualCode utilizes a self-teaching approach that re-evaluates each generated token based on user feedback, providing a per-token advantage. The model is also designed to prevent catastrophic forgetting by updating only specific low-rank parameters, thus maintaining the integrity of existing knowledge. The potential for faster and more efficient learning without external reward models positions this coding agent as a pioneering tool in AI-driven software development.