TaxCalcBench: An open source eval for testing if AI can file taxes (github.com)

The launch of TaxCalcBench v2, an open-source evaluation framework for assessing AI's ability to file taxes, marks a significant step in the AI/ML community. Released in June 2026 for the tax year 2025, this update introduces realistic PDF inputs and covers complex state and federal tax scenarios, expanding its applicability. The framework documents the performance of various AI models—including GPT-5.5, Claude Fable 5, and Gemini 3.1 Pro—against 50 test cases with varying reasoning budgets, providing detailed metrics on correctness. The results showcase the varying capabilities of these models, with GPT-5.5 showing the highest accuracy: 54% for strict correct returns and 66% leniently. This benchmarking not only highlights the current state of AI in handling intricate tax scenarios but also draws attention to the challenges still faced. Such evaluations can guide further developments in AI models, targeting improved accuracy and reliability in real-world applications, particularly amid evolving tax regulations.