Millfolio – my take on local/hybrid AI (millfolio.app)

Millfolio is a new local AI tool designed for personal data analysis without compromising privacy. Unlike traditional methods that send data to AI models, Millfolio inverts this process by sending a program to the data, which runs entirely on a Mac. It employs a local model that indexes user files and generates a de-identified schema, allowing a frontier model (Claude) to create a query program that accesses this schema without ever seeing the actual data values. This ensures that sensitive information remains on-device, significantly enhancing privacy. This innovation is significant for the AI/ML community as it proposes a novel architecture for local, private AI usage, challenging conventional cloud-based models. Built using Mojo, a new compiled language, Millfolio includes a custom inference engine optimized for GPU performance, allowing it to run efficiently on standard hardware like a Mac mini. While Millfolio is still a personal project and needs scrutiny regarding its privacy claims, particularly about the de-identification boundary, it demonstrates a unique approach to harnessing AI on personal data and minimizes reliance on external providers.