Experiences with Local Models for Coding (martinfowler.com)

A recent exploration by Thoughtworks technologists into the use of local AI models for coding has highlighted both the potential and challenges of running smaller models directly on developer machines. The author detailed their hands-on experiences with models like Qwen3.6 and Gemma 4 while performing coding tasks, emphasizing that the choice of task plays a critical role in the effectiveness of these models. Using setups with an M3 Max and M5 Pro machines equipped with 48GB and 64GB of RAM, respectively, the author identified a viability funnel that included factors such as RAM limitations, response speed, task complexity, and code quality. Significantly, the study revealed that while smaller local models can handle simpler tasks reasonably well, they struggle with more complex demands, particularly when extensive context or file changes are involved. The findings from manual and automated evaluations suggested mixed results, with varying performance based on machine specifications and task characteristics. The exploration emphasizes the still-evolving landscape of AI-assisted coding, indicating that while these local models are not yet plug-and-play solutions, they offer valuable learning opportunities and demand more direct engagement from developers, thus shifting the focus back to fundamental coding practices and careful evaluation of AI-generated outputs.