Dots.tts: 2B-parameter continuous, end-to-end autoregressive TTS system (rednote-hilab.github.io)

Dots.tts has been introduced as a groundbreaking text-to-speech (TTS) system, boasting a colossal 2 billion parameters. This fully continuous, end-to-end autoregressive system employs a unique architecture that combines a semantic encoder, a large language model (LLM), and an autoregressive flow-matching acoustic head, all operating without discrete tokens, thereby enhancing fluidity and naturalness in speech synthesis. Its performance is remarkable, achieving the best average results across multiple benchmarks, including an impressive word error rate (WER) of 0.94% for Mandarin and 1.30% for English. The significance of dots.tts for the AI and machine learning community lies in its status as an open-source system that sets new standards for TTS technology. With the highest average speaker similarity score of 83.9 on the MiniMax multilingual benchmark, it showcases capabilities in generation stability, voice cloning, and emotional expressiveness, making it a versatile tool for applications in multilingual environments. This advancement not only promises improved accessibility in TTS applications but also paves the way for more nuanced human-computer interactions, highlighting the ongoing evolution of AI in natural language understanding and synthesis.