It's Your Job to Understand (jrhawley.ca)

At an Anthropic presentation, the author saw demos of research tools pitched to “augment” and “accelerate” scholarly work: LLM-driven aggregation of experiment results, web scrapers to find papers, and automation for generating figures. The implicit sell is that these systems can lift time‑consuming tasks (literature review, synthesis, visualization) out of researchers’ hands so they can work faster. The presenter framed LLMs as a shortcut to interpretation and context, promising to streamline the messy parts of research. The author pushes back: for scientists and scholars, interpreting data and producing arguments is not a detachable workflow step but the core of the job. Writing, drawing figures, and iterating ideas are methods of thinking that reveal assumptions, refine questions, and change conclusions — processes current LLM pipelines can shortcut but not replace. Technical implications for the AI/ML community include the need to design tools that preserve human-in-the-loop epistemic responsibility, improve transparency (traceable sources, uncertainty quantification), and avoid outsourcing judgment to models that can hallucinate or miss nuance. The piece is both a caution about incentive structures that favor automation over understanding and a call for tools that ask users what they need rather than substituting for their expertise.