No Cloud, No Waiting: Tool-Calling Agents on Consumer Hardware with LFM2-24B-A2B (www.liquid.ai)

LFM2-24B-A2B has been announced as a groundbreaking tool-dispatching AI model that operates entirely on consumer hardware, enabling users to execute tasks without relying on cloud services. This is particularly significant for industries like finance and healthcare, where data privacy is paramount, and latency can impact user experience. The model allows for the rapid selection and execution of various tools, achieving an average response time of around 385 milliseconds for tool selections, while maintaining user data securely on-device. A desktop agent, called LocalCowork, has been developed to demonstrate LFM2-24B-A2B in action. This agent can perform a range of functions, such as security scanning, document processing, and system information retrieval, with a focus on seamless user interaction. For example, it can quickly scan folders for sensitive information, log every tool call, and generate compliance reports without any internet dependency. Although the model shows promise with an 80% accuracy rate in single-step tool selection, there are challenges in multi-step operations, which can be addressed through further training. Overall, this development offers a strong foundation for local AI agents, enhancing both performance and privacy in AI applications.