An Open Letter to Steven Pinker on AI (www.astralcodexten.com)

A recent exchange between Scott Alexander and renowned psychologist Steven Pinker sparked significant discussion in the AI community regarding the implications of superintelligent AI. Alexander challenged Pinker to a debate about the potential dangers of advanced AI after expressing frustration with Pinker’s social media posts, which he perceived as downplaying risks associated with AI development. In a response reflective of his thoughtful style, Pinker acknowledged the discourse but emphasized the importance of reasoned argument over confrontational debates, asserting that his contributions focus on rational skepticism of doom scenarios surrounding AI. This exchange highlights an ongoing debate within the AI/ML community about the nature of intelligence and the trajectory of AI systems. Alexander argues that as AI models evolve, their capabilities will undoubtedly surpass human intelligence, contrary to Pinker’s claims of a philosophical limitation on extrapolating intelligence across generations. He underscores that with each iteration, like from GPT-3 to GPT-6, AI's intelligence has demonstrably improved in significant ways, suggesting a trend that will likely continue. Alexander’s emphasis on empirical evidence and scaling laws for predicting AI advancements challenges the philosophical skepticism surrounding AI’s potential, reaffirming the urgency for critical discussions in the face of rapid technological change.