Experiments in building bespoke tools with AI (knlb.dev)

A recent exploration into using AI for bespoke software development revealed significant challenges and insights regarding the trustworthiness of agent-generated code. The author shared their experiences with AI models like Claude and Codex, expressing frustration over the reliability of the code produced by these agents and the inherent risks associated with deploying it in production environments. This highlights a broader issue within the AI/ML community: while generative models can produce code rapidly, there remains a lack of accountability and the need for developers to maintain oversight and validation of the output. The author introduces the concept of a "vibe limit," articulating their personal threshold for risk in coding, which varies with the complexity of changes. The author's work led to the development of a software platform named "djinn," designed to streamline context sharing and managing AI inference for coding tasks. However, initial attempts to build this platform faced setbacks due to complexity and validation issues. Ultimately, they pivoted towards smaller projects, utilizing AI to manage configuration files effectively. This approach emphasizes the importance of incremental updates to mitigate risk. By fostering personal customization in tool-building, the author underscores a vision for harnessing AI not as a decision-maker, but as an amplifier of individual developer creativity, thus promoting a tailored software development experience.