What Changed? Pin-pointing behavior shift (world.hey.com)

Turn “what changed?” into a simple supervised task: label every impression before the suspected change as control (0) and every impression after as treatment (1), then train a classifier to predict 0 vs 1 using only impression-level features (geo, publisher, app, creative, campaign, etc.). Because you must not include time as a feature (that would leak the answer), the classifier is forced to find the actual covariates that separate the “old world” from the “new world.” Inspect the model’s top feature importances (or per-feature scores) to pinpoint which dimensions moved the most and likely caused the metric shift. This is powerful and practical: tree-based models (LightGBM/XGBoost/CatBoost or RandomForest) handle mixed types and missing values and provide reliable importances; a typical pipeline uses one-hot encoding or categorical handling, scaling for numerics, and standard classification metrics (classification report, ROC AUC) to ensure the split is predictable. Caveats: importance indicates association not causation, class imbalance or sampling artifacts can mislead, and interactions may hide drivers—use SHAP or per-segment analyses for deeper insight. In short, classification-as-change-detection gives a fast, interpretable root-cause signal for metric regressions when you have impression-level data.