Show HN: Scalattice Modelmeter. OSS price and latency bench for open models (scalattice.com)

Scalattice has announced the release of Modelmeter, an open-source web app designed to benchmark its inference API against competitors offering open models. By enabling users to compare pricing and latency across various providers, Modelmeter aims to support the development of more efficient inference networks and ultimately foster a landscape of "inference abundance." The tool allows users to filter live catalog prices, estimate costs for different token mixes, and concurrently run prompts on multiple OpenAI-compatible endpoints, providing insights into latency, tokens per second, and costs—all on a single interface. This tool is particularly significant for the AI/ML community as it addresses the challenges of price competitiveness and latency measurement, crucial aspects of optimizing model deployment. Scalattice emphasizes that Modelmeter is tailored for engineers needing a streamlined way to track dynamic pricing and performance metrics without the hassle of managing individual spreadsheets or one-off notebooks. Users can easily integrate their own OpenAI-compatible hosts, making Modelmeter a versatile tool for evaluating rates and efficiencies, although it is designed as a supplementary resource rather than a replacement for Scalattice’s production services.