Nvidia Kumo Tabular: Open Foundation Model for Tabular Prediction (huggingface.co)

NVIDIA has announced Kumo Tabular, an open foundation model for tabular data now available on Hugging Face. This innovative model performs predictions for both classification and regression tasks without requiring training, tuning, or feature engineering. Kumo Tabular is designed to predict labels from a table of labeled rows in a single forward pass, making it significantly easier for practitioners to work with tabular data, which is prevalent in various enterprise applications. With three model sizes (28M to 215M parameters) and pre-trained exclusively on artificial data, Kumo Tabular ranks first across multiple benchmarks, outperforming traditional methods such as gradient-boosted trees. Technically, Kumo Tabular employs Transformer architecture that utilizes column, row, and in-context attention to understand the structure of the data. Through innovative embedding techniques and temperature scaling for attention, the model can handle large tables efficiently while addressing missing values and other data imperfections common in real-world datasets. This model's ability to leverage background context and perform predictions without updating weights—akin to in-context learning with large language models—marks a significant advancement in tabular data processing, potentially reshaping how machine learning is deployed in the industry. This significant leap opens new avenues for practitioners by streamlining the model lifecycle and enhancing predictive capabilities.