Auto-grading decade-old Hacker News discussions with hindsight (karpathy.bearblog.dev)

A recent project explores the potential of large language models (LLMs) to auto-grade decade-old Hacker News discussions by analyzing predictions made in December 2015 with the benefit of hindsight. A developer utilized ChatGPT 5.1 Thinking and the Opus 4.5 model to sift through 930 articles and comments, generating a detailed retrospective analysis that includes summaries, predictions, and grades for commenters based on their foresight. This approach not only showcases the power of LLMs in evaluating past discussions but also serves as a unique historical record of technological conversations from ten years ago. The significance of this endeavor lies in its implications for training future LLMs and understanding human forecasting capabilities. The project underscores the potential for predictive analysis to become a structured academic discipline, as suggested by superforecasters, and raises thought-provoking questions about how today’s discussions may be scrutinized in the future. The results, hosted on karpathy.ai/hncapsule, highlight notable events like the launch of OpenAI and the open-sourcing of Swift, while providing insights into which commenters were most and least accurate. This novel use of LLMs enhances our understanding of past technological trends and serves as a reminder of the ongoing impact of today’s conversations.