TypeSafe's Jev Can't See. I Made It Guess What I Drew Anyway (mikulskibartosz.name)

In an innovative experiment, a user tasked TypeSafe's text-only model, Jev, with interpreting drawings by converting them into text-based representations. Despite Jev's limitations in processing visual data, the user tested its capabilities using the Quick, Draw! dataset, resulting in a comparison of Jev's performance against that of Claude Sonnet 5, a model proficient in image recognition. The findings revealed Jev's accuracy at 35% when reading vector data despite being significantly outperformed by Sonnet, which scored around 91% with actual images. This experiment highlights the potential and limitations of text-based AI models in scenarios traditionally dominated by vision models. While Jev's ability to achieve more than random guessing (10%) demonstrates some efficacy in shape recognition through SVG data, its struggles with pixel-based formats like base64 underscore the importance of spatial information in image interpretation. The results suggest that while Jev can engage in simple tasks, it lacks the sophistication needed for practical sketch recognition, emphasizing that text models are not yet a viable substitute for visual processing in more complex recognition tasks.