Open source toolkit to analyze your ChatGPT/Claude usage from exports (github.com)

A new open-source toolkit has been launched to help users analyze their personal usage of language models like ChatGPT and Claude using data exported from these platforms. The toolkit, found at GitHub under `llm-export-analytics`, enables users to run all analyses locally, ensuring that no personal data leaves their device. Users can generate insights such as model adoption timelines, prompt engineering effectiveness, and cost efficiency metrics, which include quality-adjusted Productive Output per Dollar (POE). The toolkit also provides a full pipeline for deep analysis, covering various metrics and methodologies for quantitative assessment of LLM use, while emphasizing privacy through local data processing. This development is significant for the AI/ML community as it empowers individuals to gain deeper insights into their interactions with LLMs without compromising privacy. With features like topic classification, session dynamics, and benchmarking against industry standards, the toolkit presents an opportunity for researchers and practitioners to refine their prompting techniques and demonstrate the cost-effectiveness of AI applications in their workflows. Moreover, the inclusion of methodology papers enhances its credibility, offering structured approaches for measuring LLM performance and encouraging more data-driven decision-making among users.