A short note on e-means and jevons paradox (kovashikawa.com)

A recent exploration into e-means has unveiled a groundbreaking approach to clustering that significantly accelerates the k-means algorithm. This new variant efficiently processes a massive dataset of 10 billion DINO embeddings into approximately 16.7 million clusters in just 2.5 epochs, compared to the 25 epochs required by traditional full-batch k-means. The e-means method operates by updating centroids throughout each training pass using mini-batches and exponential moving averages, allowing for faster convergence without sacrificing accuracy. Key innovations include a decaying update schedule, better initialization, and a strategy to reinvigorate underutilized centroids. The implications of e-means for the AI/ML community are substantial, particularly as it lowers the computational burden while maintaining the same clustering objective of minimizing distance to the nearest centroid. This advancement means practitioners can explore multiple cluster configurations more freely, potentially leading to more optimal outcomes without extensive hyperparameter tuning. Although it maintains conventional geometric limitations, such as not being density-based, e-means promises to expand the toolkit available for large-scale data analysis, enabling researchers to experiment more vigorously in their searches for meaningful categories.