Price per 1M tokens is meaningless (janilowski.pl)

A recent analysis has revealed that comparing AI models based solely on the cost per million tokens is misleading. Many companies are currently facing surprising API bills as they discover that the tokenization process varies significantly among different models and even within the same frontier lab. For instance, adjustments in tokenizers can lead to a model processing the same text as a higher number of tokens, inadvertently inflating costs. This inconsistency makes direct price comparisons unreliable, as different tokenizers will yield disparate token counts from identical inputs, affecting overall expenditure. Moreover, the analysis highlights the importance of token efficiency—not just the price per token, but the real output achieved per token consumed. While models like GLM-5.2 are cheaper per token, their lower efficiency can result in higher costs per task completed. For instance, despite being more expensive, GPT-5.5 proved cost-effective regarding its task performance, costing less per completed benchmark despite a nominally higher price. This challenges the notion that cheaper models are always more economical. The findings indicate that deeper analysis of cost-effectiveness, rather than surface-level pricing, should drive model selection in the AI/ML community to avoid poor decisions that could lead to higher operational costs.