Show HN: AI/ML benchmark for local LLM inference and XGBoost training on GPU/CPU (github.com)

A new benchmarking tool for local AI/ML inference and XGBoost training has been introduced, allowing users to evaluate the performance of their consumer GPUs and CPUs with just one command. This tool provides benchmarks for Ollama LLMs (with models ranging from 3B to 14B parameters) and XGBoost on the HIGGS dataset, covering a dataset size from 100k to over 10 million rows. The entire process is streamlined through a YAML configuration file and a Python script, producing an interactive HTML report at the conclusion of testing. This innovation is significant for the AI/ML community as it enhances accessibility to performance metrics, enabling researchers and developers to gain insights into their hardware's capabilities efficiently. The results are automatically recorded and visualized in a Jupyter notebook, while also being showcased on a continuously updated Streamlit dashboard. By sharing encrypted benchmark results, users contribute to a growing database that can help establish industry standards and improve comparative performance evaluations, fostering collaborative advancements in AI and machine learning.