Translating Pandas to Polars using LLMs (pola.rs)

Recent advancements in Large Language Models (LLMs) have streamlined the migration process for developers transitioning from the pandas library to Polars, a data manipulation library. By inputting pandas scripts into models like Claude Opus 4.8, developers can receive Polars code that not only runs efficiently but also exhibits idiomatic use of the Polars API. Significantly, this process can lead to substantial improvements in runtime performance, often achieving faster execution on less expensive hardware. The success of LLMs in producing high-quality translations depends on their ability to translate intent rather than merely replicating the structure of the original code. Researchers analyzed different bodies of pandas code and identified persistent challenges in translation quality, notably when LLMs retained pandas-like patterns instead of leveraging Polars' more efficient constructs. For instance, instead of using the `over` method to apply window functions directly, models occasionally generated code mimicking pandas' structure, resulting in unnecessary complexities. To tackle these issues, a "Polars skill" has been developed and deployed to nudge LLMs toward more idiomatic translations. Evaluations showed that while improvements were noted, the skill is not a complete solution, but rather a step forward in enhancing the translation capabilities of LLMs in the AI/ML community.