Do well-written, clear instructions beat few-shotting for tiny-LLMs? (softwaredoug.com)

A recent exploration into prompt strategies for tiny language models (LLMs), such as GPT-4.1-nano, reveals that well-crafted rule-based instructions outperform few-shot prompting when handling complex tasks like spelling correction in product search queries. By evaluating zero-shot, few-shot, rule-based, and hybrid prompting techniques on a labeled furniture search dataset, the study found that pure rules-driven prompts achieved the highest accuracy (~96%), surpassing those augmented with examples or few-shot demonstrations. This counters common expectations in the AI/ML community that more examples invariably enhance model performance. Technically, the research highlights that small LLMs with limited context capacity may forget or misapply rules when overloaded with examples, leading to more spelling and brand name correction errors. The experiment underscores the importance of clear, concise instructions—particularly for resource-constrained models—while cautioning against overcomplicating prompts with excessive demonstrations. These findings have practical implications for deploying tiny LLMs in large-scale, cost-sensitive applications like e-commerce search, where maintaining accuracy without prompt bloat is crucial. Overall, the work urges practitioners to rigorously evaluate prompting strategies rather than relying on assumptions that few-shot always beats well-structured rule prompting for lightweight models.