We tested eight new decision models. None got more right than the one we use (plainlamp.com)

In a recent evaluation of decision models for document classification, the testing revealed that none outperformed the existing model, Decider V1.2, used by Plainlamp. Despite introducing eight new models, including Google's Gemini Decision 1, the results showed that Decider V1.2 achieved 336 correct classifications out of 339 descriptions, maintaining its lead due to its robust performance. Gemini Decision 1 closely followed with 332 correct classifications, notably taking no wrong classifications but experiencing a higher number of stops due to a lower confidence threshold. This evaluation is particularly significant as it underscores the challenge of improving decision model accuracy, even with advanced iterations, while highlighting the potential of Gemini's distinctive confidence calculation method. The experiment evaluated the models based on specific thresholds for classification confidence, revealing critical insights into model performance across various document types, including papers, policies, and standards. The results are vital for the AI/ML community, as they reflect ongoing efforts to enhance model reliability and identify effective prediction thresholds, which in turn can inform future algorithmic developments. The introduction of competitive models like WhatsApp's Decider Flash V1.3 and Parallel's Decisions Fast, despite promising features such as free usage options, did not yield superior results and were deemed less reliable than the current standard. This reinforces the importance of rigorous testing and assessment in the race to improve AI-driven decision-making frameworks.