Scaling and Normalizing Arrays – A Practical Guide for Data Preprocessing (ferdo.us)

This post is a practical primer on scaling and normalizing arrays for ML and scientific computing, explaining why mismatched feature ranges harm distance-based and gradient-driven models (KNN, K-Means, SVM, neural nets) and showing how to fix it with common transforms implemented in NumPy and scikit-learn. It clarifies the difference between “scaling” (broader set of rescaling methods) and “normalization” (often meaning min–max or unit-norm), and emphasizes that correct preprocessing can improve convergence, stability, interpretability, and PCA results. It walks through four main techniques with when to use each: Min–Max scaling X' = (X − Xmin)/(Xmax − Xmin) for bounded inputs (sensitive to outliers); Standardization X' = (X − μ)/σ as the default for many models; Robust scaling using median/IQR for outlier-heavy data; and MaxAbs scaling (divide by max absolute value) for sparse inputs like TF–IDF. Key best practices: fit scalers only on training data and then transform test/validation to avoid leakage, apply the same transform across splits, handle scikit-learn’s 2D input expectations, and inverse_transform predictions when you need original units. Rule of thumb: use StandardScaler by default, MinMax for bounded activations, RobustScaler for outliers—small preprocessing steps that often yield large performance gains.