Context Engineering: Improving AI Coding Agents Using DSPy GEPA (medium.com)

Firebird Technologies walked through applying GEPA — a DSPy-integrated evolutionary prompt optimizer — to improve the coding agents inside their Auto‑Analyst system. They prepared a stratified dataset of real code-execution runs (focusing on four high-frequency agent signatures: preprocessing_agent, data_viz_agent, statistical_analytics_agent, sk_learn_agent) with constraints to avoid overfitting: ≤20% default-system data, ≥10% representation per model provider, and stratification across model_provider, is_successful, and is_default_dataset. Because user datasets aren’t stored, they synthesize realistic pandas DataFrames via an LLM-based "create_synthetic_context" signature so agent code can be executed and evaluated. GEPA evolves prompt modules using LLM reflection and Pareto-style multi-objective selection: initialize a candidate pool, run mini‑batch feedback loops (reflection_minibatch_size=3) where a reflection LM (they used gpt-4o) analyzes execution traces and proposes prompt mutations, then test new candidates on a Pareto set and grow a tree of non-dominated variants. Their metric_with_feedback executes synthetic data + agent code, scores executability and relevance, and collects textual feedback for refinement. Running GEPA (auto="light", num_threads=32) produced improved agent prompts (example: a richer, structured data_viz_agent prompt). The work demonstrates a practical, automated pathway for robust prompt engineering and evaluation in multi-agent ML systems — improving execution reliability across datasets and provider models and offering a reproducible approach for optimizing agent behavior and guardrails.