Of Hammers and Nails: What AI Can and Cannot Do for a Data Analyst (adamwritesaboutdata.substack.com)

AI and large language models (LLMs) have made significant inroads into the data analysis realm, particularly in speeding up code writing and improving the development of data assets. This enhancement is crucial since the quality of underlying data models is a major differentiator between high-performing data teams and those that merely incur costs. However, while AI can hasten coding and serve as a useful drafting assistant, its ability to accurately answer ad hoc data queries remains inconsistent. Reports show that even with well-structured data, the accuracy rates of AI responses hover around 86%, meaning a notable number of answers can be incorrect—an unacceptable risk for business decision-making. The current limitations of AI in analytics highlight the importance of human attributes such as judgment, context, and institutional knowledge, which machines can't replicate. While AI tools can provide tangible productivity gains, they have yet to automate core analytical skills effectively. Organizations are advised to adopt a balanced approach: leveraging AI where it excels while recognizing its shortcomings. Successful teams will be those that navigate this middle ground wisely, integrating AI tools without over-relying on them for critical analysis tasks.