Accelerating Large-Scale Test Migration with LLMs (airbnb.tech)

Airbnb has successfully migrated nearly 3,500 React component test files from Enzyme to React Testing Library (RTL) in just six weeks, a task initially estimated to take 1.5 years of manual work. This large-scale automation leveraged large language models (LLMs) to refactor the test files while preserving their original intent and code coverage, showcasing the potential of LLMs for complex code transformations. The migration utilized a structured approach that involved breaking down the process into discrete validation and refactor steps, implementing retry loops with dynamic prompting, and enriching the context fed into the LLMs to facilitate better understanding and outcomes. The significance of this achievement for the AI/ML community lies in its demonstration of how LLMs can effectively handle intricate coding challenges that require a deep understanding of existing systems. By fine-tuning their prompts and incorporating extensive contextual data, Airbnb reached a remarkable 97% success rate in automating the migration process, with the remaining files easily addressed through manual corrections. This case highlights the promising future of LLM-driven development tools, encouraging broader application of AI technologies in software engineering to enhance productivity and streamline complex code migrations.