A wall that listens: the local-LLM pipeline behind an AI party in Vilnius (vania-novikau.me)

A unique AI-driven installation called the Dynamic AI Party is being tested at the ZOLAK Micro Art Residence in Vilnius, where a wall integrating live input from attendees creates an interactive experience. Utilizing a projection system that reacts to both movement and speech, the wall displays highlighted words from the conversation, instantly reflecting the atmosphere of the room. This initial test showcases the integration of a local language model (LLM) that processes real-time audio and video input, producing outputs that enhance the interactive nature of the gathering. Significantly, this project represents a shift from traditional AI art, which often relies on pre-rendered content or prompts from a single operator. By harnessing local computing power, including components like the Coral Edge TPU for motion detection and an RTX 3090 for processing LLM decisions, the system facilitates a seamless, closed-loop interaction with minimal latency. This innovative setup not only showcases the potential of localized AI applications in dynamic environments but also emphasizes the capabilities of hybrid hardware solutions, making live audience interaction a central feature of the experience.