Classifying AI Website Fetches on Behalf of Users (canonry.ai)

A new development in AI traffic management has emerged with the introduction of Canonry, a tool designed to classify server logs to differentiate between user fetches and AI crawler requests. Traditionally, analytics only capture browser activity post-page load, but in the case of AI fetches, user-agent identity and server-side requests complicate this picture. Canonry addresses this by precisely categorizing events logged by different user actions, AI traffic, and crawler behavior, ensuring that user fetches are tracked separately and accurately. This innovation is significant for the AI/ML community as it provides deeper insights into how AI tools like ChatGPT and Claude interact with web content, while also highlighting potential security concerns—such as the identifying patterns of unauthorized requests masquerading as user fetches. By offering transparent analytics that can help developers understand the nature of their traffic, Canonry enhances data accuracy and security for web providers. With 947 user-fetch requests logged within a week on canonry.ai, the tool's ability to separate traffic sources sheds light on the unique dynamics of AI interactions with web content, presenting both opportunities and challenges for developers and website owners.