CorentinJ: Real-Time Voice Cloning (2021) (github.com)

CorentinJ’s Real-Time Voice Cloning is an open-source implementation of SV2TTS (Transfer Learning from Speaker Verification to Multispeaker TTS) packaged with a real-time WaveRNN vocoder — originally developed as the author’s master’s thesis. The system uses a three-stage pipeline: a GE2E encoder that extracts a fixed speaker embedding from a few seconds of audio, a Tacotron-based synthesizer that conditions on that embedding to produce mel-spectrograms from arbitrary text, and a WaveRNN vocoder that generates waveform audio in real time. The repo includes pretrained models (auto-downloadable), demos (demo_cli.py and demo_toolbox.py), and instructions for running on Windows/Linux with PyTorch, ffmpeg, and Python ~3.7; LibriSpeech/train-clean-100 is recommended for quick experiments. This project is significant because it demonstrated practical, few‑seconds voice cloning using transfer learning and provided a reproducible baseline that accelerated research and experimentation in multispeaker TTS. It’s a useful toolkit for prototyping and education, but the author notes it’s aged compared with commercial SaaS and newer research — for state-of-the-art open-source alternatives see PapersWithCode or projects like Chatterbox (2025 SOTA). Practical implications include democratized voice cloning for accessibility and content creation, faster iteration for model developers, and the associated ethical/privacy considerations as synthesis quality improves.