Stt+LLM+TTS voice chat server on RTX 3050 Ti 4GB: 11.9s voice-to-voice (lilting.ch)

A new voice chat server setup has been successfully demonstrated on an RTX 3050 Ti laptop with only 4GB VRAM, achieving a voice-to-voice response time of 11.9 seconds. This innovative system integrates speech-to-text (STT) using Qwen/Qwen3-ASR-0.6B, a large language model (LLM) via an API from the Qwen Ambassador program, and text-to-speech (TTS) using Irodori-TTS-500M-v3. By consolidating both STT and TTS on the same device and utilizing asynchronous job handling, the server can efficiently manage audio processing and response generation, significantly reducing latency. This advancement is significant for the AI/ML community as it showcases the potential of combining powerful AI models on modest hardware to create responsive voice interfaces. Through technical optimizations, such as the reduction of unnecessary processing and the synchronization of STT and TTS tasks, the system minimizes response time without compromising on output quality. This setup not only highlights the capabilities of current AI technologies but also opens pathways for developing more efficient and accessible voice communication tools on budget-friendly hardware.