The Control Plane Was the Point: Revisiting Autofz in the LLM Era (yfu.tw)

The recent reflection on autofz, a meta-fuzzer developed during a PhD and accepted at USENIX Security 2023, reveals its lasting significance as the AI/ML community grapples with coordinating multiple algorithmic agents. Initially designed to optimize resource allocation among fuzzers, autofz addresses a timeless challenge: determining how to efficiently distribute a fixed computing budget across various "workers"—now including not just fuzzers, but also static analyzers, code agents, and model variants. With security capabilities becoming more accessible and generating plausible bug candidates easier, the real challenge remains in making sense of the noisy output from these agents and transforming them into reliable, actionable insights. Key to autofz's contribution is its novel control-plane framing which allows the system to dynamically adjust resource allocation based on ongoing performance observed in a preparatory phase. Rather than relying on static decisions at the outset, autofz continuously evaluates which fuzzer is performing best and reallocates resources accordingly, ensuring a more effective search strategy. This dynamic orchestration enables better coverage and bug discovery, outperforming both individual fuzzers and static collaborative approaches in benchmarks. As such, autofz exemplifies how thoughtful orchestration of diverse AI tools can yield superior results, asserting the importance of adaptive strategies in the evolving landscape of security and AI.