Is 'human' a risky AGI target (www.nullsy.com)

The piece critiques a common pattern in LLM discourse: treating "human-like" behavior as the baseline test for progress and then highlighting every way models fall short. It argues this is a risky target because today's neural language models "think" in ways fundamentally different from humans. Examples include asking models to perform tasks that require embodied, real-world grounding (e.g., which side of a door an object is on) versus tasks they already excel at (rapidly synthesizing text, producing photorealistic or stylistically uncanny images). Those mismatched expectations lead to trivializing advances or overlooking capabilities that are alien but powerful. For the AI/ML community this matters because focusing purely on human parity can blind us to emergent, superhuman abilities that won’t resemble human cognition and could outpace our intuitive understanding or control. The author notes that many model achievements could, in principle, be replicated by massive human labor today, but frontier models may soon perform them instantaneously—shifting goalposts and rendering tests like the Turing test obsolete. The takeaway is a call for better instrumentation: early detection, new evaluation frameworks, and methods to extrapolate the benefits and risks of non-human, superhuman capabilities as they appear, rather than insisting every model be judged only by human-like benchmarks.