Exploiting food embeddings for ingredient substitution (github.com)

A new study presented at the International Conference on Health Informatics 2021 introduces innovative models for ingredient substitution in cooking, harnessing food embeddings through Food2Vec and FoodBERT. These models aim to facilitate healthier cooking by suggesting alternatives that optimize nutrients, avoid allergens, or cater to personal tastes. Notably, the multimodal versions of these models combine text and image data, enhancing their capability to recommend suitable substitutes based on dietary needs. This research is pivotal for the AI/ML community as it showcases the integration of natural language processing and image analysis in the culinary domain, providing a framework for leveraging embeddings in real-world applications. FoodBERT, particularly in its multimodal iteration, demonstrated superior performance in identifying ingredient substitutes according to human evaluations and ground truth comparisons. This opens avenues for future applications in personalized nutrition and health, allowing users to navigate ingredient choices intelligently and inclusively. The findings and associated resources are available through a dedicated GitHub repository, encouraging further exploration and potential collaboration in the field.