I built the Playwright for desktop apps. 80% token savings (github.com)

An innovative new CLI tool called agent-desktop has been launched for native desktop automation, specifically designed for AI agents using Rust. This tool allows structured interaction with any application through operating system accessibility trees—eliminating the need for screenshots or pixel matching. Its key features include 53 commands for a wide range of functionalities (observation, interaction, window management) and a progressive skeleton traversal method that achieves 78–96% token savings when querying dense applications like Slack or VS Code. The significance of agent-desktop lies in its ability to simplify automation tasks by providing an efficient and versatile interface for AI applications. By operating via a native CLI without runtime dependencies and facilitating direct access to accessibility APIs, it greatly enhances performance and reduces overhead for developers. The tool can execute tasks across various operating systems, including macOS and Linux, and is accessible in a straightforward manner through package managers or direct cloning from GitHub. With structured JSON outputs and extensive error-handling capabilities, agent-desktop represents a substantial advancement in desktop automation for AI/ML developers.