The new math of scaling AI across the enterprise (www.businessinsider.com)

Organizations are now grappling with how to scale AI efficiently for large workforces, having previously confirmed its effectiveness in enhancing productivity. The traditional budgeting methods have become inadequate in an AI-driven economy, as many companies are blindsided by costs associated with usage models, leading to higher-than-expected expenses. Jason Banta from Qualcomm highlights the complexity of agentic AI workflows, which can require numerous interactions with models, significantly increasing token consumption and, thus, costs. The growing consumption rate, despite decreasing token prices, challenges organizations to rethink their AI budgeting strategies with an emphasis on "tokenomics". Shifting towards on-device AI is becoming a pivotal solution for reducing overall costs and energy consumption, particularly for high-frequency tasks like transcription or summarization. Gartner advises using smaller, domain-specific models rather than relying purely on cloud solutions to leverage better performance at a lower expense. This approach not only cuts down on inference costs but also enhances data privacy and security by processing information directly on devices. As Banta points out, creating an AI architecture optimized for cost and performance requires companies to evaluate and strategically deploy their AI resources, ensuring a sustainable, scalable solution that meets the unique needs of their workforce.