Show HN: Jev-phone – driving iOS and Android apps with jev (github.com)

A new project called Jev-phone has emerged, which allows users to control both iOS and Android apps without writing any text. Utilizing TypeSafe's System One model named Jev, this innovative tool interprets the screen's elements to execute commands such as searching for coffee shops in Maps, saving articles on Wikipedia, and creating contacts, all through simple function calls. Notably, Jev handles every interaction via the accessibility tree, responding to real-time changes and ensuring accuracy by validating actions before finalizing them. This development is significant for the AI/ML community as it showcases a novel application of machine learning models in real-world interactions with mobile devices, emphasizing the potential of AI to streamline user experiences. With impressive performance metrics—including tasks completed in as little as 16.5 seconds—Jev demonstrates an efficient, accessible architecture that could lead to broader use cases for automation in mobile environments. The project is currently in early access, with tools readily available for developers to experiment with, opening the doors for further advancements in hands-free operation of mobile applications.