Local AI needs to be the norm (unix.foo)

A growing trend among developers is the reliance on cloud-hosted AI models, such as those provided by OpenAI or Anthropic, which leads to software that is fragile and privacy-invasive. The argument is made that local computing capabilities should be utilized instead, leveraging the powerful chips in devices that can handle tasks without needing constant server interaction. The author emphasizes that routing data to third-party AI providers introduces complexities like data retention issues and dependency on external services, undermining software stability and user trust. A practical example is the development of The Brutalist Report's iOS client, which utilizes Apple’s local model APIs to generate article summaries on-device, eliminating the need for server communication. This approach not only enhances privacy and responsiveness but also simplifies the application’s architecture. Apple has made significant strides in improving local AI tools, allowing developers to request typed output without relying on unstructured data formats. The conclusion stresses that while local models may not rival cloud models in intelligence, they can efficiently perform specific tasks like summarization and classification directly on the user's device, fostering a trustworthy user experience without compromising data privacy.