Show HN: Headroom – measure your GPU's true bandwidth ceiling for local AI (ar5en1c.github.io)

A new tool named "Headroom" has been introduced, allowing users to measure their GPU's memory bandwidth ceiling for local AI applications in just 30 seconds. This online test does not require model downloads, using only ~0.4 MB of public tensor metadata to evaluate a device's tokens-per-second (tokens/s) capacity. Notably, it also checks for a GPU driver bug that can cause in-browser LLMs to produce incorrect outputs. By leveraging WebGPU technology in supported browsers, Headroom delivers measurements directly in the browser, keeping user data private. This development is significant for the AI and machine learning community as it highlights the potential performance limitations of current in-browser models and provides a means to identify hardware bottlenecks that could be improved. Headroom measures bandwidth using three independent tests and provides projections for real-world performance based on validated engines, revealing that many in-browser engines currently operate with substantial overhead. For example, an RTX 5070 was shown to achieve around 577 GB/s, only 30% of its ceiling is utilized for actual token generation, underscoring untapped potential in existing hardware. The tool not only aids performance optimization but also contributes to understanding driver and GPU issues, making it valuable for both developers and researchers in the AI/ML space.