Achieving 20%+ improvement in structured extraction using DSPy and GEPA (kmad.ai)

Recent experiments using the DSPy framework combined with the GEPA optimizer have achieved over 20% improvement in structured data extraction tasks compared to traditional large language model (LLM) outputs. Specifically, the study employed a unique setup to extract financial entities from news articles, demonstrating a clear increase in exact match accuracy—from 32.07% for baseline API calls to 54.43% using DSPy with GEPA and BAML adapters combined. This advancement highlights the potential for optimizing LLMs with minimal engineering effort, making sophisticated extraction capabilities more accessible and cost-effective, even on less powerful models. The significance of this development lies in GEPA's ability to provide highly targeted feedback that helps refine extraction instructions, effectively uncovering latent requirements during optimization. The techniques used showcased the iterative improvement process in AI engineering, emphasizing that small adjustments in prompt design can yield substantial gains in functionality. The success in extracting multiple entity types, such as Companies and Products, indicates the importance of precise wording and schema adaptability in structured tasks, and suggests further exploration into optimizing even cheaper models could soon follow.