Agentic ML Experimentation at Snap (eng.snap.com)

Snapchat has introduced Alchemist, a human-in-the-loop platform designed to streamline machine learning (ML) experimentation. By enabling ML engineers to set objectives and refine hypotheses, Alchemist reportedly cuts feature selection cycle time in half—from four weeks to just two—while concurrently reducing feature costs by 30%. This innovation is particularly impactful for the AI/ML community, as it enhances the efficiency of the experimentation process, allowing teams to focus more on innovation and less on time-consuming administrative tasks. The Alchemist platform utilizes a deterministic experimental loop where agents assist in proposing and refining hypotheses but require human approval before executing tasks. This ensures high-quality outcomes while minimizing errors associated with automated decision-making. The initial case study showcased the feature selection recipe, where an engineer successfully trimmed model features without sacrificing performance, all while encapsulating the knowledge gained into a durable process for future use. This modular approach promises quicker feedback and knowledge sharing across teams, setting the stage for further advancements in Snapchat's ML initiatives and beyond.