From Code Foundation Models to Agents and Applications: A Comprehensive Survey (arxiv.org)

A recent publication titled "From Code Foundation Models to Agents and Applications: A Practical Guide to Code Intelligence" explores the transformative impact of large language models (LLMs) in automated software development. The guide highlights how tools such as GitHub Copilot and Claude Code have evolved from traditional rule-based systems to advanced Transformer architectures, achieving remarkable performance improvements on coding benchmarks. It offers a detailed analysis of the entire lifecycle of code LLMs, from data curation to post-training, revealing the interplay between various techniques, including code pre-training and reinforcement learning. This work is significant for the AI/ML community as it addresses the growing need to bridge the gap between theoretical research and real-world applications in software development. By critically examining both general and code-specialized LLMs, the authors identify challenges surrounding code correctness, security, and contextual awareness within large codebases. Furthermore, the experiments conducted in the guide provide insights into scaling laws, hyperparameter sensitivities, and model architectures, pointing to promising research directions that align with practical engineering needs. This contribution is poised to enhance the deployment and efficacy of AI-driven coding solutions, ultimately supporting the evolution of software engineering practices.