AI should earn its keep: Introducing the AI Productivity Guarantee (cognition.ai)

Cognition has introduced the AI Productivity Guarantee, a groundbreaking initiative aimed at addressing a critical gap in the AI/ML industry: the measurement of tangible value versus mere usage metrics. While companies are heavily investing in AI, they often struggle to quantify the actual benefits gained. To tackle this, Cognition has developed an AI estimator that assesses the productive output of its AI agent, Devin, in engineering tasks. By comparing the output of Devin to human engineering estimates, Cognition can offer a financial guarantee of up to $10 million to customers if they find the engineering value delivered falls short of their investment. This move signifies a shift towards accountability and transparency in the AI market, challenging AI vendors to focus on real outcomes rather than superficial activity metrics. The estimator works by evaluating each completed session to determine whether it resulted in useful output, and then estimating how much time a human engineer would have taken to achieve the same result. By using a standardized approach to measure productive hours rather than lines of code, Cognition aims to provide a clearer picture of ROI in AI-powered engineering. This initiative not only enhances trust in AI solutions but also encourages a broader industry trend towards establishing robust productivity metrics.