The Education of a Doomer (borretti.me)

In a recent reflective blog post, an AI enthusiast shares a transformative journey from optimism to skepticism regarding AI's impact on society. Initially excited about the potential of AI to create jobs and enhance living standards, they now express deep concerns about the implications of Artificial General Intelligence (AGI). With the possibility of AI outpacing human capabilities and rendering most occupations obsolete, the author worries about the emergence of a permanent underclass and decreasing political power, as the economic utility of humans declines. The idea that individuals might willingly surrender control to superintelligent AIs further intensifies these anxieties. The technical discourse around AI has shifted dramatically, with advancements in reinforcement learning leading to powerful yet increasingly opaque models. The author notes that as AI systems become more capable, they simultaneously become less understandable, heightening the risk of misalignment. Furthermore, the rise of automated writing and software engineering is seen as diminishing the essential cognitive processes that underpin human creativity and intelligence. This has led to a disempowerment of human capital and critical thinking, raising alarming questions about societal values and the future of meaningful human contribution in an AI-driven world.