NobodyWho is an inference engine that lets you run LLMs locally and efficiently (github.com)

NobodyWho has unveiled a powerful inference engine that enables users to run Large Language Models (LLMs) locally and efficiently, eliminating the need for API keys and recurring fees. This innovation allows developers to integrate various chat LLMs like Gemma, Qwen, and Mistral directly into their applications. Key features include multimodal input support—allowing for image and audio data integration—text-to-speech synthesis, and speech-to-text capabilities using Whisper. This versatility is powered by GPU-accelerated inference via Vulkan or Metal, ensuring fast performance across multiple operating systems. The significance of NobodyWho lies in its approach to on-device AI, which promotes privacy and efficiency by processing data locally. The engine supports thousands of pre-trained LLMs in the GGUF format and allows for seamless model downloading from platforms like Hugging Face. Its type-safe tool calling also automates the generation of structured grammars, making integration straightforward across popular programming environments such as Kotlin, Swift, Python, and more. As AI applications increasingly require robust and privacy-conscious solutions, NobodyWho represents a substantial leap forward for developers looking to leverage LLMs without relying on external services.