Inference Is the Last LLM Moat (www.vincentschmalbach.com)

OpenAI and Anthropic have established a significant competitive advantage in the AI/ML landscape through their subsidized inference models, offering developers reliable and cost-effective access to advanced large language models (LLMs). This economic moat hinges on the affordability and speed of their services, which help retain individual developers and small businesses. However, experts warn that the moment these subsidies falter, a significant exodus could occur, with over 50% of users potentially shifting to emerging platforms like DeepSeek, which, despite being less reliable or trustworthy, could provide a cost-effective alternative. The implications of this situation are profound for the AI/ML community. Running LLM inference at scale poses substantial challenges that require expertise and resources, an arena where OpenAI and Anthropic currently excel. However, the landscape is evolving; as more providers enter the market, the barriers to accessing high-quality inference could diminish, eventually normalizing it like web hosting. This shift could allow for greater competition and innovation within the AI space, but it also raises questions about data privacy and reliability, as users might have to navigate a more complex ecosystem of choices in their pursuit of effective AI tools.