"what if you don't have the dataset?" (chris-parmer.com)

In a recent exploration, a developer working on Plotly Studio, an LLM-powered analytics and visualization app, was inspired by a conversation during a road trip. The intriguing question posed was, "What if you don't have a dataset?" This led to an unexpected revelation: the app's LLM could autonomously discover primary data sources on the web without user-provided datasets. By querying open-ended questions, the LLM surfaced rich, often obscure datasets such as water temperature data from NOAA buoys, 311 civic complaint data, and extensive macroeconomic statistics from the Federal Reserve. This development is significant for the AI/ML community as it demonstrates the potential of LLMs to enhance data accessibility and promote independent data exploration. Instead of relying on traditional structured datasets, users can tap into a wealth of real-time data sources that would typically remain hidden, thereby enabling more dynamic and informed decision-making. This method not only empowers users to validate narratives against empirical data but also raises questions about the LLM's ability to navigate the sea of unstructured information while maintaining data integrity and quality. The implications for data-driven analysis could reshape how analysts and researchers approach data discovery and usage in their projects.