Parse: LLM Driven Schema Optimization for Reliable Entity Extraction (arxiv.org)

PARSE (Parameter Automated Refinement and Schema Extraction) is a new system that treats JSON schemas not as fixed contracts but as artifacts LLMs can interpret and improve to make structured extraction from text more reliable. It combines ARCHITECT, which autonomously refines and optimizes JSON schemas for LLM consumption while preserving backward compatibility via RELAY (an integrated code-generation layer), with SCOPE, a reflection-based extraction engine that fuses static validation and LLM-based guardrails. The approach addresses shortcomings of prior methods (constraint decoding or RL) that relied on human-designed schemas and often produced hallucinations or inconsistent outputs. On three benchmarks—Schema-Guided Dialogue (SGD), Structured Web Data Extraction (SWDE), and an internal retail dialogue dataset—PARSE yields substantial gains: up to 64.7% improvement on SWDE, around 10% combined improvements across models, and a 92% reduction in extraction errors after the first retry, all while keeping practical latency. For the AI/ML community, PARSE reframes schema design as an optimization problem that LLMs can solve, improving reliability of agent-to-API interactions, reducing hallucinations, and enabling more robust, automatable pipelines for real-world Software 3.0 applications.