ModelMesh – Native macOS app to shard models across Macs via Llama-RPC (modelmesh.app)

ModelMesh has introduced a native macOS application designed to enable users to shard AI models across multiple Macs, leveraging unified memory pooling to run models that exceed a single machine's RAM capabilities. This innovative platform utilizes Thunderbolt 5 connections for optimal performance, with a fallback to TCP over Ethernet, ensuring seamless operation. Notably, the application features direct Objective-C++ bindings to llama.cpp and a native AppKit UI, which eliminates the overhead associated with Electron or Python environments. This development is significant for the AI/ML community as it facilitates more efficient model deployment and execution, enabling developers to harness collective computational power without compromising data privacy—model weights, prompts, and inference traffic remain confined to the local network. With tools for managing multi-node RPC clusters, GGUF model management, and real-time memory telemetry, ModelMesh empowers developers to utilize high-performance computing setups while integrating effortlessly with existing development tools and scripts. This marks a step forward in optimizing distributed AI workflows, particularly for macOS users.