Why we stopped using an automated SRE agent (blog.neatcontext.com)

After a year of experimentation, a platform engineering team abandoned their fully automated, AI-driven Site Reliability Engineering (SRE) agent due to its failure in effectively managing infrastructure incidents. Initially envisioned to reduce on-call fatigue and improve mean time to resolution (MTTR), the agent struggled with information overload, making incorrect connections between unrelated alerts. Additionally, reliance on outdated documentation led to erroneous conclusions, and the risk of executing incorrect actions in production environments proved too high. Instead of a fully autonomous approach, the team shifted to a lightweight, desktop-first model using tools like NeatContext. This new paradigm emphasizes explicit context delivery, allowing engineers to bundle and filter critical information before it's processed by Large Language Models (LLMs). By creating airtight boundaries around relevant data, the engineers maintained control and ensured accuracy in diagnostics while leveraging LLMs' capabilities. This transition has resulted in a significant reduction in MTTR and a more reliable partnership between humans and AI, highlighting the importance of tailored context over automation in complex environments.