From hand-tuned Go to self-optimizing code: Building BitsEvolve (www.datadoghq.com)

Datadog described how painstaking, hand-tuned Go optimizations—like removing redundant runtime bounds checks in NormalizeTag (a 25% speedup yielding a 0.75% CPU reduction and tens of thousands in annual savings) and adding an ASCII fast-path to NormalizeTagArbTagValue (over 90% faster and hundreds of thousands saved)—inspired an automated system to scale that expertise. The core lesson: micro-optimizations only pay when they hit hot, autoscaled code paths, and observability-driven evidence about real inputs is essential to choose where to spend effort. To scale those wins, Datadog built BitsEvolve, an agentic evolutionary optimizer (inspired by AlphaEvolve) that mutates code, benchmarks variants with a fitness function, and evolves solutions in an island-model population to balance exploration and exploitation. Orchestrated with Temporal and driven by Datadog’s observability and benchmarking stack, BitsEvolve rediscovered hand-tuned fixes (down to identical code), learned from low-level hints (e.g., panicBounds patterns, -gcflags), and produced concrete wins—Murmur3 ~20% faster, CRC32 improvements after ~50 iterations, and surprising algorithmic finds like fast-doubling for Fibonacci. Crucially, BitsEvolve relies on realistic, production-derived benchmarks (Live Debugger + Cursor) and human-guided constraints to ensure verifiability. The result is a repeatable, auditable pipeline that can democratize high-impact, cost-aware performance tuning across an organization.