Chatbot Running on an ESP32-S3 (github.com)

A groundbreaking development in artificial intelligence has been achieved with the introduction of Cardputer AI, a fully functional chatbot that operates solely on the ESP32-S3 microcontroller. This tiny chip, which boasts only 512 KB of RAM, enables the chatbot to engage in meaningful, coherent conversations without necessitating an internet connection or external cloud services. Unlike traditional chatbots that rely on extensive data centers, Cardputer AI integrates an 8-million-parameter language model that allows it to maintain engaging dialogue and even respond with simple stories, all while fitting into a credit card-sized device. The significance of Cardputer AI lies in its demonstration of how advanced conversational abilities can be achieved within the constraints of low-cost hardware, posing exciting implications for the AI/ML community. It challenges prevailing assumptions about the size and resource requirements of conversational agents by effectively balancing complexity and efficiency. With the entire model encapsulated in just 5.7 MB and executing approximately five tokens per second, this development opens the door for innovative applications of AI in personal devices, education, and interactive entertainment, all while being entirely offline and accessible to anyone with the necessary hardware.