Open source tool to collect data for computer use agent models (github.com)

A new open-source, privacy-first app (GitHub: bobcoi03/computeruse-data-collection) makes it easy to record standardized computer-use data—keyboard and mouse events, optional screen MP4 video, and audio—to train and evaluate AI agents that operate on desktops. It ships as a simple GUI or CLI (python3 -m computeruse_datacollection or computeruse-collect), stores everything locally by default under ~/computer_use_data/, and exports selected sessions as zip files for sharing. Recordings follow a documented JSONL schema (events.jsonl with timestamped keyboard/mouse events, plus metadata.json and screen_recording.mp4), support selective recording, and offer config-driven privacy/performance options (anonymize_text, blur_sensitive_areas, screen_fps, max_storage_gb). Requirements: macOS/Linux (Windows soon), Python 3.8+, ~100 MB app size, minimal RAM, and ffmpeg for MP4. This tool matters because it tackles three major bottlenecks for agent research: fragmented formats, privacy-invasive collection, and closed-source pipelines. By standardizing schemas and keeping data local (no automatic uploads), it enables reproducible dataset creation, cross-project interoperability, and session replay for debugging or supervised training. Built for auditability (MIT license) and community contribution, it lowers barriers to collecting high-quality human–computer interaction traces while foregrounding consent and ethical use—users must grant OS permissions and review/export data themselves—making it practical for researchers, companies, and volunteers to build safer, more general computer-use agents.