Building a (useful) voice assistant I can call from my watch (blog.jamesolds.me)

A developer built a realtime voice AI assistant you can call from an Apple Watch in under a day that talks to GPT-5 Realtime and integrates with Todoist, Gmail, and GitHub — and open-sourced the code at github.com/oldsj/assistant. The UX is simple: add a “🤖 Assistant” contact as a watch complication, tap or tell Siri “call Assistant,” and you get an interactive, hands-free conversation that can create and edit tasks, read email, and query repos. The project demonstrates how current tooling makes practical, wrist-first AI assistants feasible for personal productivity. Technically, the stack uses Twilio to handle inbound calls over WebSockets, a Fly.io-hosted bot that proxies to GPT-5 Realtime for voice, and Zapier’s MCP server to attach Todoist/Gmail/GitHub without building separate connectors. Authentication is enforced by Twilio number checks and signature validation; semantic_vad turn detection solved noisy-VAD issues; latency is noticeable when calling MCP tools but remains conversational. It’s cheap to run on free/low-cost tiers, and rapid prototyping was accelerated with Claude Code. Next steps include hooking a knowledge base (Obsidian), proactive outbound calls, and smarter briefing logic — showing that lightweight, secure, extensible watch-first AI assistants are now practical for developers and power users.