Show HN: LOAB – benchmarking AI process fidelity in lending (github.com)

The newly launched LOAB (Lending Operations Agent Benchmark) assesses AI agents' ability to conduct a mortgage process end-to-end, focusing not only on achieving accurate decisions but also on adhering to necessary processes and compliance regulations. Traditional AI benchmarks usually verify if the model reached the correct conclusion, but LOAB emphasizes the importance of process fidelity in lending, where any oversight, such as skipping identity verification, can lead to compliance failures. The current proof-of-concept evaluates three origination tasks and explores how well AI models, including GPT-5.2 and Claude Opus 4.6, meet a strict rubric that measures outcome accuracy, tool usage, agent handoffs, forbidden actions, and the evidence provided. The significance of LOAB lies in its potential to improve compliance and reliability in AI-driven lending workflows. By evaluating AI across multiple components and not just final outcomes, it exposes where models may falter, thereby enabling developers to refine their systems for real-world application. Initial results reveal that both models struggle with process fidelity, indicating that correct conclusions can still stem from flawed processes—a critical insight for ensuring AI systems meet regulatory standards in financial services. As LOAB expands to cover additional lending tasks, it aims to provide a comprehensive framework for assessing AI performance in compliance-heavy environments.