My "blocked-by-default" approach to working with coding agents (oscarswanros.com)

In a recent talk, engineering manager and cave diving enthusiast highlighted the risks associated with working with AI coding agents, drawing parallels between diving safety and software operations. Following an incident where an AI agent accidentally wiped a production database in just nine seconds, he emphasized that the risk culture in technical diving is more mature than that in software development. In diving, practitioners study past fatalities to internalize risk management, whereas software engineers often react to mistakes only after they occur. The discussion introduced a “blocked-by-default” approach, advocating for systems that prevent errors before they can happen, likening it to a safety-focused dive plan that only permits explicit actions. The proposed framework includes five layers of protection that block erroneous operations by default, allowing only explicitly authorized actions, thereby reducing the risk of catastrophic mistakes in software deployments. Each layer is designed to catch potential failures, ensuring actions undergo scrutiny before reaching critical systems. This model serves as a structural safeguard, arguing that while AI agents lack human intuition and emotional weight, purposeful design can mitigate risks effectively. The takeaway for the AI/ML community is to build robust infrastructures that prioritize safe and reliable operations, transforming the right actions into the only options available.