The Future of Attention (micro.mu)

What happened: The piece traces the arc from the Transformer paper "Attention Is All You Need" to today’s multimodal generative models that don’t just predict text tokens (the mechanism behind GPT) but can now synthesize images, audio and short-form video. New services like Sora 2 are surfacing as attention-first social platforms built around AI‑generated clips—short videos created from prompts today (hours-long or continuous streams remain limited mainly by compute), with recommendation systems primed to keep users swiping. Why it matters and technical implications: This shift turns high-capacity sequence models into content factories that can be looped by human and algorithmic creators into a self-reinforcing feed. Key technical facts: Transformers predict next tokens across modalities, multimodal models ingest and generate text/audio/video, and current limits on duration are chiefly computational. The consequence is an exponentially larger supply of ultra-personalized, possibly deceptive or harmful media—deepfakes, extremist or sexual content, and hyper-addictive micro-entertainment. That raises urgent questions for the AI/ML community about provenance, watermarking, moderation at scale, recommender-system incentives, compute governance and policy. The takeaway: generative video is a powerful technical advance, but without technical safeguards, content policies and platform accountability it risks amplifying harm through attention-economy dynamics.