I ported one program to 10 languages to see how an LLM thinks (www.alexv.lv)

A recent experiment involved porting a 500-line Go bank statement analyzer into ten different programming languages using the LLM Claude, with a focus on understanding the model's internal processing. The study revealed interesting insights into how LLMs approach coding tasks, highlighting factors such as deliberation, confidence in API usage, and the implications of language design constraints. While the LLM successfully predicted its confidence and anticipated fixes with high accuracy, it stumbled with line count predictions, suggesting it better understands conceptual confidence than practical coding effort. The findings underscore the significance of training data density and API uncertainty on an LLM's performance. Languages like Java 8, characterized by limited design choices, resulted in minimal deliberation and effortless execution, while languages with more flexibility, like Odin, faced challenges due to compounded small uncertainties among many API calls. This experiment sheds light on how LLMs can effectively perform coding tasks while revealing the hidden complexities that influence their outputs, ultimately providing valuable insights for developers and researchers in the AI/ML community.