I made a tool to filter LLM API providers by speed, quant, context and more (modelgrep.com)

A new tool has been developed to filter language model (LLM) API providers based on key performance metrics such as speed, cost, and context length. The tool presents a comprehensive comparison of various providers, displaying throughput rates, latency times, and pricing for both input and output. For instance, Morph and Perplexity are highlighted for their contrasting performance, with Morph achieving a remarkable throughput of 2957.6 tokens per second at 400 ms latency and $0.90 per million tokens, compared to Perplexity's 2522.3 tokens per second at a significantly higher latency of 25.9 seconds. This tool is significant for the AI/ML community as it empowers developers and researchers to make informed decisions when selecting LLM APIs for their projects, ultimately enhancing the efficiency and effectiveness of AI applications. By providing quantitative metrics across 22 different API providers, it aids users in balancing the trade-offs between speed and cost, a crucial consideration for scaling AI models in real-world applications. The extensive data enables a nuanced understanding of how different providers perform under various conditions, fostering healthier competition and innovation in the rapidly evolving landscape of AI services.