Startup's new mechanistic interpretability tool lets you debug LLMs (www.technologyreview.com)

San Francisco-based startup Goodfire has launched Silico, a groundbreaking mechanistic interpretability tool that empowers researchers and engineers to scrutinize and adjust AI model parameters during training. This innovative off-the-shelf tool allows for fine-grained control of models, addressing the challenge of understanding and debugging AI behavior, which has often been likened to alchemy rather than a systematic science. By leveraging mechanistic interpretability—an emerging field focusing on the inner workings of AI models—Goodfire aims to transform model development into a more precise engineering process, making model building more accessible to smaller firms and research teams. Silico enables users to examine individual neurons and their interactions within a model, offering insights into how different inputs influence outputs. This capability can help mitigate common issues, such as reducing AI hallucinations and enhancing ethical decision-making. For instance, the tool allows developers to adjust neuron parameters to optimize responses, thereby improving transparency in AI behavior. Goodfire envisions that Silico will democratize access to interpretability techniques, crucial for developing trustworthy models in critical sectors like healthcare and finance, without the need for specialized interpretability teams.