Show HN: GUI-tool – zero-dependency Rust CLI for desktop GUI automation (github.com)

A new cross-platform Rust CLI tool called "gui-tool" has been introduced for desktop GUI automation, enabling AI agents to interact with applications by naming grid cells instead of relying on pixel coordinates. This innovative approach overcomes a common limitation among automation tools that require understanding pixel positions from screenshots, providing a labeled grid overlay on the desktop. Users can zoom into specific grid cells for precise targeting, making it an effective solution for automating apps without a CLI or where accessibility data is incomplete or inaccurate, such as games and custom-drawn UIs. The significance of gui-tool lies in its zero-dependency setup and native OS API use, which ensures security and reliability. It simplifies the automation workflow for AI agents like Claude Code and Codex, eliminating the need for complex accessibility trees. With its capability to perform pixel-precise actions and return structured JSON outputs, gui-tool empowers developers and researchers to debug applications and automate workflows that were previously challenging due to missing or faulty accessibility features.