Lessons from Building an OTel Normalizer for GenAI (www.groundcover.com)

Groundcover has announced its development of an OTel normalizer aimed at streamlining AI observability for Generative AI (GenAI) applications. The company discovered significant inconsistencies in how different SDKs emit OpenTelemetry attributes, undermining the assumption that OTel provides a standardized approach to capturing GenAI telemetry. By building a normalizer that consolidates data from various instrumentation SDKs and LLM providers, Groundcover seeks to deliver a unified view of GenAI metrics, including models, tokens, and costs, despite the complexities inherent in naming conventions and data structures across the ecosystem. This development is pivotal for the AI/ML community as it highlights the challenges of achieving true observability in an increasingly fragmented environment. Groundcover's findings underscore that while the narrative of standardization exists, the practical realization is fraught with discrepancies stemming from SDK conventions, orchestration frameworks, and provider-specific details. The normalizer aims to alleviate the burden on SRE teams by simplifying data processing and enabling focused root cause analysis, ultimately enhancing the efficiency of AI observability efforts across platforms.