The tests are the code now (softwaredoug.com)

Recent developments in AI coding suggest a paradigm shift where tests serve as the primary source code, enabling AI agents to "compile" executable programs from these tests. This innovative approach is significant for the AI/ML community as it emphasizes the role of feedback mechanisms—such as tests and evaluations—in enhancing the quality of code generated by AI systems. By using richer feedback harnesses, developers can mold plausible AI-generated code into accurate, correct outputs. This method shifts the focus from the specifics of coding to ensuring the output meets predetermined correctness criteria defined by tests. The implications of this approach are profound. As AI applications generate solutions that are often 90% correct, the traditional need for explicit instructions diminishes; rather, the emphasis is on capturing the nuances of correctness through test criteria. This concept aligns with the notion of legacy code, where AI-generated outputs are treated as black boxes that developers must manage. The "strangler pattern" can be applied to incrementally improve the code by layering in tests and functionalities, reflecting a top-down engineering approach to address potential issues while maintaining the integrity of the system. This evolution signifies a new era in AI development, where testing and evaluation become integral to the coding process itself.