I checked whether my laptop is wasting RAM (ansnadeem.com)

In a recent discussion on Stripe’s Cheeky Pint podcast, OpenAI CEO Sam Altman expressed concerns about individuals running AI models locally on personal computers, suggesting it leads to inefficient use of limited DRAM resources. Altman highlighted that while data center memory is utilized continuously, personal computers often sit idle for extended periods, underlining the issue of resource wastage in decentralized AI model operation. A user running models on a MacBook Pro with 64 GB of memory documented their experience, revealing they used their setup to process approximately 10.6 million tokens over three months, with a mere 15 hours of GPU activity. The cost-effectiveness of running models locally was notable, averaging only 12 cents in electricity compared to around $5 with cloud providers. Despite the financial advantages, the user cited the importance of data privacy and debugging, finding smaller models valuable for identifying weaknesses in their code. While local inference offers benefits, the infrequent model usage raises significant questions about the practical implications of resource allocation in the AI/ML community.