Trees to Flows and Back: Unifying Decision Trees and Diffusion Models (arxiv.org)

A recent study has successfully unified two seemingly disparate machine learning model classes: decision trees and diffusion models. This innovative approach reveals that these models, one discrete and hierarchical and the other continuous and dynamic, share a fundamental mathematical relationship. Central to this unification is a new optimization principle called Global Trajectory Score Matching (GTSM), which highlights that gradient boosting can achieve asymptotic optimality under specific conditions. The implications for the AI/ML community are significant. This research introduces two practical applications: \treeflow, a model that leverages this unification to generate high-quality tabular data with double the speed compared to existing methods, and \dsmtree, a distillation technique that efficiently transfers the hierarchical decision logic of trees into neural networks, achieving nearly identical performance to the original models. These advancements not only enhance computational efficiency but also demonstrate the potential for deeper synergies between traditionally distinct modeling approaches, paving the way for more robust AI systems.