Show HN: What LLM fits my GPU? – VRAM measured on real hardware for 117 models (quantized.uk)

A new tool has been launched that allows users to determine which language models (LLMs) are compatible with their GPU, based on real hardware measurements for 117 models. The platform features a VRAM calculator and guides based on specific GPU configurations, giving users critical insights into the memory requirements of various models, such as the Mistral Small 24B Instruct, which can run on a 16GB card with quantized settings. Users can also see the expected performance in terms of throughput and quality loss, making it easier to balance power consumption and model efficiency. This initiative is significant for the AI/ML community as it democratizes access to high-performance models by clearly outlining what fits on different hardware configurations. By providing meticulously collected data, including memory requirements and performance benchmarks, it addresses a common pain point for developers and researchers who need to optimize their setups for specific applications. The tool empowers users to make informed decisions about model selection, essentially enabling broader experimentation and development especially among those with limited resources or less powerful hardware.