Lotus root is not Fried tofu: giving an on-device food model better eyes (littletheta.com)

In a recent development for the Macro Trainer app, the team encountered significant issues with its food recognition model, which mistakenly identified images of sliced lotus root and other vegetables as fried tofu. This misclassification highlighted limitations in the app’s photo food logging system, originally powered by a general language model that lacked specialized knowledge of diverse food items. The app's developers realized that while the software could provide decent results for common foods, it often struggled with less conventional items, demonstrating a tendency to confidently make incorrect assertions. To address these shortcomings, the team integrated Qwen3-VL, a vision-language model specifically designed for food recognition. This new model significantly outperformed the previous one, correctly identifying 12 out of 14 food items compared to only 4 by Gemma. This upgrade not only enhanced the accuracy of food identification, it also streamlined processing times, reducing them from 14-20 seconds to just 2-6 seconds per plate. Additionally, the Metro Trainer app now processes images and textual advice through two separate models, improving overall performance and user experience while managing memory usage more efficiently. This shift marks a crucial step in refining AI systems to better handle specific tasks in real-world settings, emphasizing the need for specialized models in applications that require detailed visual analysis.