X algorithm has been open sourced (github.com)

The recommendation algorithm powering the "For You" feed on X has been open-sourced, marking a significant step for the AI/ML community in understanding advanced content ranking systems. This algorithm integrates posts from followed accounts (in-network) with globally discovered content (out-of-network) using a Grok-based transformer model originally adapted from xAI's Grok-1. It uniquely eliminates hand-engineered features, relying solely on the transformer model to analyze user engagement history and predict relevance, optimizing recommendations through a sophisticated scoring mechanism based on predicted engagement probabilities for multiple actions. This release not only democratizes access to cutting-edge recommendation technology but also highlights an evolving trend in AI towards reducing reliance on manual tuning in favor of data-driven approaches. With its architecture incorporating retrieval from both in-network and out-of-network sources and innovative filtering and scoring methods, the system serves as a robust template for future AI-driven recommendation projects. The open-source nature and detailed documentation encourage further exploration and adaptation of these techniques, promising advancements in personalized content delivery across platforms.