Show HN: ContextD – OCRs your screen activity, use it with LLMs via local API

ContextD, a newly launched macOS application, revolutionizes how users interact with their screen activity by continuously capturing screenshots, extracting text using optical character recognition (OCR), and summarizing that information with a local language model (LLM). This innovative tool captures the screen every two seconds, only processing the changes to minimize data usage, and stores the extracted text in a local SQLite database for easy access. ContextD runs a personal API that allows users to interact with their activity summaries, supporting full-text search and enriched prompts, while ensuring all data remains on the user's machine.

This app is significant for the AI and machine learning community as it demonstrates an efficient application of LLMs in practical scenarios, such as summarizing user interactions and providing contextual information for productivity. By leveraging tools like Claude Haiku, ContextD operates affordably at approximately $2 per day and emphasizes user control by requiring minimal external interactions purely for summarization. The app is developed in Swift and designed for macOS 14 and above, highlighting its accessibility to macOS users seeking enhancement in their daily computing activities.