Show HN: Autofit2 – End-to-end pipeline for multilingual text classification (github.com)

A new end-to-end pipeline called Autofit2 has been introduced for multilingual text classification, leveraging few-shot learning techniques to achieve impressive precision rates of 95–99% with minimal labeled data. This pipeline supports over 50 languages and can work with pretrained models for 20 languages, showcasing its scalability through integration with Common Crawl for additional language support. The system is built using SetFit and SBERT embeddings, providing a fully automated process that handles everything from data preprocessing and model fine-tuning to evaluation and deployment—all orchestrated through a single JSON configuration file. Autofit2's significance lies in its potential to democratize access to advanced multilingual classification capabilities, allowing researchers and developers to train models efficiently without requiring extensive programming knowledge. The features promoting reproducibility—including model card generation detailing training specifics, intended use, and performance metrics—further enhance its value. Additionally, the automatic tracking of CO₂ emissions during model training underscores a commitment to sustainability in AI development. The structured configuration allows for flexible adjustments to be made easily, making Autofit2 a promising tool for accelerating multilingual NLP projects in a variety of applications.