Reusing obsolete Windows 10 PCs for on-premises large language model inference (www.frontiersin.org)

A recent study has explored the potential of repurposing obsolete Windows 10 PCs as on-premises servers for large language model (LLM) inference, spurred by Microsoft’s impending end-of-support for Windows 10. By utilizing an enterprise workstation equipped with a consumer-grade NVIDIA GPU and leveraging tools like Proxmox virtualization, Ollama, and Open WebUI, researchers demonstrated that these repurposed systems could sustain stable inference throughput of 29–65 tokens per second for coding and document-assistant tasks. This method not only preserves data sovereignty and avoids recurring subscription fees associated with cloud services but also presents a more cost-effective solution, especially for organizations facing stringent data-handling regulations. The study highlights the significance of transitioning legacy computing hardware into dedicated AI infrastructure, which can address environmental concerns by mitigating e-waste and reducing carbon footprints compared to newer hardware and cloud alternatives. By analyzing various open-source LLMs, researchers found that while throughput and correctness do not always correlate, certain models performed well under constrained conditions. Their findings suggest that deploying AI locally not only leverages existing resources but also enables organizations to maintain greater control over sensitive data and comply with regulatory requirements, marking an important strategic shift in AI deployment amid the growing scrutiny of cloud services.