Degradation of Multi-Task Prompting Across Six NLP Tasks and LLM Families (www.mdpi.com)

Researchers presented a comprehensive evaluation of six prominent large language models (LLMs) through an incremental multitask prompting framework, revealing critical insights into performance dynamics as task complexity escalates. The study highlights significant variations in model behavior, especially the DeepSeek R1 7B, whose JSON formatting accuracy increased with task complexity despite a notable drop when integrating additional tasks like Topic Extraction. Conversely, tasks requiring nuanced semantic understanding, such as machine translation and emotion analysis, exhibited consistent degradation, underscoring challenges in resource allocation among competing cognitive demands. The findings are crucial for the AI/ML community as they shed light on how task interference affects performance across different architectural designs. The Llama 3.1 8B model showed improved translation quality when combined with sentiment analysis, illustrating potential synergies between correlated tasks. However, the overall results indicate that LLMs may struggle with fine-grained semantic tasks under multitasking conditions, exposing limitations in current models and suggesting that further architectural innovations or training methodologies could be necessary to enhance performance in complex, multi-faceted environments.