From Hand-Crafted to LLM-Based Variation Operators in Metaheuristics (camilochs.github.io)

A recent tutorial explores the innovative use of large language models (LLMs) as variation operators in metaheuristic algorithms, moving away from traditional hand-crafted approaches. This shift allows the generation and modification of candidate solutions through dynamic model calls, tailored by specific types of conditioning information. The tutorial introduces a novel framework that categorizes variation operations based on two descriptors: the type of prompt conditioning (Numeric, Symbolic, Linguistic) and artifact persistence (Transient, Amortized, Transfer), offering a comprehensive lens for practitioners to classify and implement LLM-assisted techniques effectively. This development is significant for the AI/ML community as it opens new avenues for enhancing heuristic search methods by integrating LLMs, supporting more sophisticated and adaptable solution generation. The tutorial provides practical resources, including a build template, method surveys, and a cost-aware decision guide, facilitating the application of LLMs in addressing complex optimization problems like the Travelling Salesman Problem. By addressing the critical prompt-conditioning choices and their implications on computational cost and efficacy, this work sets a structured foundation for future research and implementation in algorithmic design and optimization.