Streaming Speech Synthesis Without the Trade-Offs: Meet StreamFlow (arxiv.org)

Researchers have introduced StreamFlow, a groundbreaking neural architecture designed to enhance streaming speech synthesis by addressing the limitations of existing token-to-waveform generation methods. Traditional approaches using semantic tokens and flow matching often suffer from audio quality degradation in real-time applications due to their dependence on a global receptive field. StreamFlow overcomes these challenges by implementing a local block-wise receptive field strategy utilizing diffusion transformers (DiT). By segmenting the sequence into blocks and employing block-wise attention masks, StreamFlow efficiently handles historical dependencies while maintaining high audio quality. This innovation is significant for the AI/ML community as it enables streaming speech synthesis with improved performance that rivals non-streaming methods. The architecture achieves a remarkable first-packet latency of just 180 ms, making it highly suitable for interactive applications where real-time feedback is crucial. Experimental results show that StreamFlow not only enhances speech quality compared to existing streaming techniques but also maintains competitive inference times during long-sequence generation. Overall, this development represents a notable advancement in generating high-fidelity speech in real-time, paving the way for more sophisticated audio applications.