Someone on Reddit is running Qwen 27B at ~20 tok/s on a jailbroken PS5's GPU (www.reddit.com)

A Reddit user has successfully run the Qwen 27B AI model on a jailbroken PlayStation 5, achieving an impressive processing speed of approximately 20 tokens per second using the console's GPU. This achievement marks a significant milestone in the accessibility and portability of advanced AI models, demonstrating that high-performance computing resources can be utilized beyond traditional server environments and specialized hardware. The implications of this development are profound for the AI and machine learning communities. It highlights the potential for widespread, cost-effective access to powerful AI capabilities, making it feasible for more individuals and small teams to experiment with and deploy state-of-the-art models. Additionally, it opens up discussions around optimizations and adaptations needed for AI models to run on consumer-grade hardware, pushing the boundaries of what's possible in resource-constrained environments. As the hobbyist community continues to explore such possibilities, it could spark further innovations in AI deployment strategies, with significant implications for democratizing access to powerful machine learning tools.