I made a zero cost browser-use tool – let AI click and type on webpages for you (github.com)

A new browser-use tool, browser-use-wasm, has been developed to automate interactions with web pages, allowing AI to click and type on webpages without requiring a backend. This innovative system operates entirely client-side in the browser using WebAssembly (WASM) and WebGPU, utilizing a model called ShowUI-2B to interpret user goals and execute actions directly on the live Document Object Model (DOM). Users can engage with web pages through a simple interface that captures screenshots and infers actions based on visual inputs, enhancing automation ease and accessibility. The significance of this tool lies in its zero-cost operation and enhanced performance, as it eliminates the need for remote servers and backend processing. By leveraging WASM, browser-use-wasm can effectively manage browser automation with minimal latency and high reliability. Key technical features include the use of SnapDOM for precision screenshot capturing, dynamic model switching, and a robust API for developers to integrate automation functionalities into their applications. This development not only democratizes access to AI-driven web interactions but also sets a new benchmark for browser-based automation tools in the AI/ML community.