Karpathy on DeepSeek-OCR paper: Are pixels better inputs to LLMs than text? (twitter.com)

I couldn’t access the original X/Twitter post or article content you referenced (the page returned a browser/JS error), so I don’t have the specific quotes or claims from Andrej Karpathy to summarize. If you can paste the text or provide a working link, I’ll create a precise 2–3 paragraph summary of what he said and its implications. In general, the idea of “image-only input to LLMs” is significant because it reframes multimodal AI: instead of treating images as auxiliary to text, a model would accept purely visual token streams and produce language or actions. Technically this requires a robust visual encoder that converts pixels into tokens compatible with a language transformer (options include ViT-style patch embeddings, learned discrete visual tokenizers, or Perceiver-like interfaces), plus alignment and training objectives that teach the language model to reason over visual tokens (contrastive pretraining, cross-modal generative loss, or end-to-end fine-tuning). Implications include richer visual reasoning and grounding, new dataset and compute demands, different latency/efficiency tradeoffs, and evaluation challenges for coherence and factuality when models must infer context purely from images.