Writing Code vs. Shipping Code: Productivity Effects Across AI Tool Generations (www.nber.org)

A recent study titled "Writing Code vs. Shipping Code" evaluates the productivity effects of various generations of AI coding tools on software development, using data from over 100,000 GitHub developers. The research highlights that while tools like autocomplete, interactive coding agents, and autonomous coding agents boost coding activity significantly—by 40%, 140%, and 180%, respectively—the actual impact on project completion and software releases is markedly lower. The cumulative productivity gain of 180% drops to just 50% for the number of projects and 30% for final software releases, indicating that while AI improves coding efficiency, human bottlenecks in the production process limit the overall output. This study is significant for the AI/ML community as it underscores the complex interplay between AI tools and human effort in software engineering. With an estimated 0.25 elasticity of substitution between AI and human labor, the findings suggest a strong complementarity where human skills remain crucial despite the prowess of AI coding tools. As developers and companies adopt these advanced tools, understanding their limitations in translating task-level gains to fully shipped products is vital for improving workflows and maximizing the return on AI investments in the software development lifecycle.