Exploring Apple's On-Device Model: Twenty Billion Parameters in Flash (www.airealist.ai)

Apple has unveiled its latest on-device language model, Core Advanced, capable of 20 billion parameters, of which 1 to 4 billion can be active at once. This model integrates directly with macOS 27, allowing it to enhance Siri's dictation capabilities and various apps through swift calls to the Apple Intelligence framework. Developed in collaboration with Google, Core Advanced signifies a notable advancement in bringing sophisticated AI functionalities to consumer hardware while maintaining low energy consumption and a small memory footprint. This model operates from flash storage and utilizes a mixture-of-experts approach, significantly reducing memory usage compared to traditional models. The significance of Core Advanced lies in its ability to perform a range of tasks such as summarization, entity extraction, and creative writing, all while operating efficiently on standard consumer Macs. By employing innovative techniques like keeping model weights in flash storage and selectively activating only a subset of parameters for each request, Apple achieves performance that scales without the extreme resource demands typically associated with traditional models. The system is designed to run efficiently on Apple’s Neural Engine, consuming approximately one-third of the energy of comparable open models while delivering competitive performance on designated tasks, though it has limitations in areas such as mathematical reasoning and coding. This development showcases Apple's strategic focus on integrating advanced AI capabilities directly into consumer devices, effectively democratizing access to powerful machine learning tools.