Useful Outsourcing is Hard (2024) (gwern.net)

A recent discussion highlights the challenges of effectively "outsourcing" tasks to artificial intelligence, particularly large language models (LLMs). The article posits that just because an LLM is highly capable doesn't mean it automatically provides value for specific tasks or workflows. Many users struggle to leverage these technologies efficiently due to context, overhead, and the unique nature of individual workflows. The author encourages readers to explore whether they could outsource tasks to humans—specifically, to ask, “Could I outsource this to a person working for free?”—to better understand the obstacles they face in delegating to AI. This insight is significant for the AI/ML community as it underscores the gap between capabilities and practical utility, suggesting that the lack of effective AI integration often stems from organizational limitations rather than AI's inherent intelligence. The discussion emphasizes the importance of reconfiguring personal and organizational workflows to harness the full potential of AI. The article concludes that even the most advanced LLMs may struggle to deliver meaningful assistance without contextual understanding and user preparation, thus encouraging a reevaluation of how we think about automation in both human and AI contexts.