Nvidia PersonaPlex 7B on Apple Silicon: Full-Duplex Speech-to-Speech in Swift (blog.ivan.digital)

Nvidia has announced the release of PersonaPlex 7B on Apple Silicon, enabling full-duplex speech-to-speech interactions using native Swift. This advancement leverages a single model architecture to facilitate real-time conversation capabilities by processing audio input and output simultaneously, eliminating the traditional multi-step pipeline of transcription, processing, and synthesis. The PersonaPlex 7B model operates at approximately 68ms per step, allowing for faster-than-real-time performance, and employs an innovative approach using 4-bit quantization, resulting in a lightweight model size of about 5.3 GB. This development is significant for the AI/ML community as it redefines voice assistant interactions by streamlining the conversation workflow into a unified system. In doing so, it not only reduces latency associated with voice processing but also preserves emotional and prosodic qualities often lost in conventional methods. The implementation of key features such as multilingual synthesis and enhanced performance optimizations further exemplifies the potential of PersonaPlex to revolutionize user experiences in AI conversations, making seamless, natural interactions a reality across various applications.