Can Claude teach me to make coffee? (www.lesswrong.com)

A recent Reddit experiment explored whether AI language models, specifically Claude, could effectively guide a user through making coffee using images of their environment. The participant shared their living space pictures while seeking step-by-step instructions to locate the kitchen and coffee-making tools. Although the AI made a series of misidentifications and procedural mistakes, it successfully led the user through multiple rooms, showcasing its conversational aptitude and potential as a real-time navigation assistant. This exploration highlights the significant capabilities and limitations of large language models (LLMs) in practical applications. While Claude demonstrated an understanding of dialogue and provided contextual guidance, it struggled with spatial reasoning and did not maintain a coherent world model throughout the process. The experiment underscores the necessity for improving AI models to better interpret visual information and integrate it with task instructions, suggesting promising avenues for enhancing AI’s applicability in hands-on, real-world tasks.