Zepto Scales Customer Support on Databricks and MLflow (www.databricks.com)

Zepto, a rapidly expanding quick-commerce platform in India, has revolutionized its customer support system by implementing a multi-agent AI architecture powered by Databricks and MLflow. Faced with the challenge of processing over 100,000 support tickets daily, Zepto shifted its strategy from merely deploying more agents to developing a robust evaluation framework that enhances the reliability of its AI agents. This framework enables continuous monitoring and improvement while addressing the complexities of fluctuating ticket volumes and diverse customer needs. The significance of this approach lies in its dual-loop model, which connects a development loop with a production loop through a quality gate, ensuring that failures are captured and rectified. By leveraging detailed execution traces, stratified sampling, and a golden dataset for agent evaluation, Zepto has enhanced its agents' accuracy and performance while significantly increasing operational efficiency. This proactive evaluation method not only mitigates risks but also turns insights from production issues into new features, ultimately leading to improved customer satisfaction and reduced operational costs. Thus, Zepto's innovations contribute valuable lessons to the AI/ML community about effective agent management in high-demand environments.