Coinbase Built, Evaluated, and Post-Trained a Fraud Agent (www.coinbase.com)

Coinbase has developed a cutting-edge fraud detection agent, integrating a large language model (LLM) into its Onramp system, which allows users to purchase cryptocurrency without a Coinbase account. This new agent achieved a remarkable 9.6% increase in F1 score performance and reduced end-to-end latency by 55% compared to previous models. In online experiments, the LLM risk agent demonstrated a 30% reduction in fraudulent transactions and a 22% decrease in fraud value by enhancing the review of recent transaction patterns against a backdrop of existing machine learning models and rules. This innovation is significant for the AI/ML community as it showcases the potential of LLMs in real-time fraud detection, particularly in scenarios with limited account histories. By leveraging contextual reasoning rather than traditional feature engineering, Coinbase's approach enables a more nuanced fraud assessment. The LLM evaluates the transaction in the context of prior activities, identifying patterns that straightforward models might miss. This layered, selective utilization of machine learning not only enhances fraud detection capabilities but also illustrates a cost-effective method to improve overall transaction security while maintaining high processing speeds. As Coinbase prepares to examine different LLM versions, this initiative sets an important precedent for continuously optimizing AI-driven fraud prevention strategies.