We Turned Our Wireshark Wizard into a Markdown File (www.checklyhq.com)

Checkly has officially launched Rocky AI, an AI agent designed to enhance root cause analysis for failed Playwright checks by utilizing large language models (LLMs). Developed over the past six to eight months, Rocky AI focuses on parsing extensive Playwright trace files, often exceeding 100MB, by filtering irrelevant data and converting it into a manageable format for the LLM. The team realized the importance of not overwhelming the model with excessive information, and instead aimed to codify critical triaging processes that experienced engineers typically carry in their heads into a structured markdown framework. This initiative is significant for the AI/ML community as it exemplifies the practical application of AI in optimizing existing SaaS products beyond simple chat interfaces. By automating the analysis of failures, Rocky AI not only improves efficiency but also preserves domain expertise within a semi-structured format. Furthermore, updates like the transition from OpenAI's GPT-4.1 to GPT-5.1 have demonstrated substantial improvements in the quality and reliability of AI outputs, marking a leap forward in the capabilities of AI-driven tools in troubleshooting complex issues. Checkly plans to continue expanding Rocky AI’s skill set and tools, hinting at a broader evolution of AI utility in the analysis and monitoring space.