Condorcet's Theorem and an LLM Jury: Diminishing returns as group sizes increase (arxiv.org)

Recent research highlights significant advances in mitigating biases inherent in large language models (LLMs) through innovative crowd-based strategies. While LLMs are powerful tools for generating text, they often reflect the biases present in their training data. The study revealed that simply averaging responses from multiple LLMs can lead to worse bias outcomes due to a lack of diversity within these artificial crowds. Instead, the researchers propose that locally weighted aggregation methods can enhance both accuracy and bias mitigation by leveraging the strengths of varied input. The study underscores the importance of combining LLMs with human input to harness the "wisdom of diversity." By integrating diverse perspectives from human contributors in conjunction with LLM outputs, hybrid crowds demonstrate significantly improved performance and a reduction in biases, particularly related to gender and ethnicity. These findings suggest an important direction for future AI development, emphasizing the necessity of incorporating diverse human insights to counteract the biases propagating through machine learning models. This research not only challenges existing methods of bias mitigation in AI but also opens avenues for more inclusive AI systems in practice.