Fine-Tune Black Box Embedding Models (arxiv.org)

Mafin (Model Augmented Fine-tuning) is a new method for improving retrieval embeddings when the base encoder is a black-box (e.g., hosted or closed-source) that you cannot fine-tune. The authors target Retrieval-Augmented Generation pipelines, where off-the-shelf embeddings often underperform on domain-specific semantics. Rather than modifying the inaccessible model, Mafin attaches a small, trainable embedding module that augments or adapts the black-box outputs so they better reflect task-specific similarity. Experiments show this lightweight augmentation substantially boosts retrieval and downstream RAG performance while only training a modest additional component. Technically, Mafin is agnostic to label availability: it can be optimized with supervised signals where annotations exist and with unsupervised objectives otherwise, making it broadly applicable. Because only a small model is trained, the approach is compute- and data-efficient and preserves the integrity and privacy of the original embedding provider. For practitioners reliant on API-hosted embeddings, Mafin offers a practical path to domain adaptation, improved semantic retrieval, and reduced hallucination in LLMs without needing access to or retraining of the underlying embedding model.