Show HN: Sanctuary – Offline LLM Running on Apple Silicon Metal (iOS) (testflight.apple.com)

A new project, Sanctuary, has emerged that enables offline large language model (LLM) capabilities on Apple Silicon devices utilizing Metal. This groundbreaking approach allows users to run LLMs without needing continuous internet access, tailoring efficiency for mobile developers and users during the testing phase of applications. Utilizing Apple's powerful architecture maximizes performance while minimizing latency, making it a notable advancement for the AI and machine learning community. The significance of Sanctuary lies in its potential to bridge the gap between advanced AI applications and user accessibility, especially for iOS and macOS platforms. With an offline capability, developers can test LLM functionalities securely and more efficiently, enhancing the in-app experience without relying on cloud services. This initiative not only promotes more robust application testing within Apple's ecosystem but also aligns with growing concerns over data privacy and connectivity challenges, heralding a more localized approach to deploying AI technologies on consumer devices.