Record and Replay, teach AI agents desktop workflows by showing them once (github.com)

A new tool called the MCP server has been launched, allowing users to record desktop workflows once and transform them into reusable AI agent skills through a streamlined process. The workflow is recorded through native accessibility events and screen captures, then compiled into structured JSON and human-readable markdown files (SKILL.json and SKILL.md), making it easy for AI agents to execute complex tasks in the future by simply referring to these compiled skills. The technology supports cross-platform functionality across macOS, Windows, and Linux, ensuring accessibility and usability in various environments. This development is significant for the AI/ML community as it enhances the efficiency of training AI agents through a human-in-the-loop approach. By demonstrating a task once, users can create a self-contained artifact that captures not only the steps of a workflow but also contextual video when available. This capability simplifies the deployment of AI agents in real-world applications, streamlining workflows and reducing the need for extensive repetitive training. The integration of LLMs for compilation allows for automated skill versioning and improved adaptability, marking a step forward in AI's ability to learn and mimic human behavior effectively.