Run local LLMs on multiple Android, iOS phones (github.com)

A new cross-platform AI chat client, PrivateLM, has been developed using Flutter, allowing users to run local large language models (LLMs) on their Android and iOS devices. This app integrates local inference capabilities with cloud API access, enabling users to execute GGUF models offline on Android through GPU-accelerated inference with Vulkan and switch to cloud services like OpenAI and Anthropic when additional computational power is needed. Notably, all chat data and settings are stored locally, enhancing user privacy and device control. The significance of this development lies in its potential to democratize access to powerful AI capabilities on mobile devices, reducing reliance on constant internet connectivity. The app supports multimodal chat with text and images and includes smart auto-configuration to optimize performance based on device specifications. Though the iOS version is sideloadable and aimed at devices with sufficient RAM, the focus on local processing could inspire further advancements in mobile AI applications, positioning them as viable alternatives to traditional cloud-based services.