Show HN: Running MiniCPM-v4.7 35B-A3B on a 2020 Android phone with 8 GB RAM (github.com)

A developer has successfully run the MiniCPM-V 4.7 35B-A3B model on a 2020 Android phone with 8 GB of RAM, marking a significant milestone for on-device AI/ML applications. This achievement stems from an experimental fork of the MiniCPM repository that features optimizations for mobile platforms, including patched CPU loading and cleanup processes. The capability to run such a large multimodal model locally, with features such as streamed experts from SSD, represents a breakthrough in making advanced machine learning models more accessible on everyday devices. This development is significant as it not only demonstrates the potential of advanced AI models to operate on smartphones but also highlights the feasibility of multimodal capabilities—combining text and vision processing—on limited hardware. The MiniCPM-V 4.7 model is designed to intelligently load components as needed, reducing the memory footprint and allowing for smoother operation. Key technical details include the use of a 35 billion parameter model, which typically requires high-end processing power, now streamlined for mobile usage, setting a precedent for future innovations in AI for portable devices.