Unofficial fork of Microsoft's VibeVoice after repo withdrawal (github.com)

Following Microsoft’s removal of the official VibeVoice repository, the AI community has launched an unofficial, community-maintained fork to preserve and advance this cutting-edge expressive speech synthesis framework. VibeVoice stands out for its ability to generate long-form, multi-speaker conversational audio—such as podcasts—directly from text, addressing key limitations in traditional Text-to-Speech (TTS) systems around scalability, speaker consistency, and natural dialogue flow. The model leverages novel continuous speech tokenizers operating at an ultra-low frame rate (7.5 Hz) that balance high audio fidelity with computational efficiency, enabling synthesis of up to 90 minutes of speech with up to four distinct voices. Technically, VibeVoice employs a next-token diffusion architecture powered by a Large Language Model (LLM) to understand context and dialogue, combined with a diffusion head to produce high-quality acoustic details. Open-sourced model weights are available for its 1.5B and larger variants, with unofficial training and fine-tuning code forthcoming. The fork integrates with Hugging Face Transformers and provides tools like VibePod for end-to-end podcast generation from simple text prompts. Users should note some current limitations—such as occasional instability with Chinese speech, emergent capabilities like singing, and the absence of overlapping speech modeling—as well as ethical considerations given the high fidelity of synthetic voices and their potential misuse. This community-driven effort ensures VibeVoice remains accessible for AI/ML researchers focused on advancing expressive, multi-speaker TTS technologies.