Learning to Replicate Expert Judgment in Financial Tasks (thinkingmachines.ai)

A recent study from Bridgewater AIA Labs has successfully trained custom models to replicate expert judgment in financial tasks, addressing the challenge of filtering and processing vast amounts of investment-related information. Their proprietary model outperforms several leading large language models (LLMs) like Gemini, Claude, and GPT, achieving average accuracy of 84.7% compared to less than 80% for these frontier models. This significant advancement stems from high-quality human annotations and a unique training process that allows the model to develop an understanding of investment context, crucial for making nuanced decisions that traditional LLMs struggle with. The study highlights a critical shift towards custom models tailored for specific organizational needs, suggesting a future where differentiated intelligence can lead to enhanced productivity in financial decision-making. By utilizing innovative training techniques such as interleaved batching and on-policy distillation, the researchers not only achieved increased accuracy but also reduced inference costs significantly—by 13.8 times per task. This work showcases the practical implications of fine-tuning LLMs with expert insights, indicating that well-constructed, context-aware models can offer substantial advantages in performance and operational efficiency over generic LLMs in the financial sector.