Append filler tokens, answer harder questions (twitter.com)

Recent findings suggest that appending filler tokens, such as the repetition of dots, can significantly enhance the performance of large language models (LLMs) when tackling complex multi-hop reasoning questions. For instance, a question like "Who won the Nobel Prize for Chemistry in (1900 + Mozart's age when he died)?" often stumps LLMs due to their inability to execute immediate logical reasoning. However, introducing a series of filler tokens before the question can trigger the model's underlying reasoning mechanisms, allowing it to arrive at the correct answer. This discovery is pivotal for the AI/ML community as it highlights a novel strategy to improve the interpretability and performance of LLMs in handling complicated queries. By manipulating input structures, researchers can leverage existing models more effectively, potentially reducing the need for extensive architectural changes or retraining. This approach may open avenues for further exploration into enhancing LLM capabilities, suggesting that even minor adjustments in input can lead to significant improvements in reasoning performance.