Pulpie: Pareto-Optimal Models for Cleaning the Web (usefeyn.com)

Researchers have unveiled Pulpie, a series of Pareto-optimal models designed to enhance the extraction of main content from HTML pages. Pulpie achieves near-state-of-the-art extraction quality at a fraction of the cost, with its smallest model, Pulpie Orange Small (210M parameters), achieving a ROUGE-5 F1 score of 0.862, closely rivaling the leading extractor Dripper (600M parameters) at 0.864. The architectural innovation lies in Pulpie's encoder, which efficiently labels every HTML block in a single forward pass, leading to remarkable speed—processing 13.7 pages per second on an NVIDIA L4 GPU, compared to Dripper's 0.68. The significance of Pulpie for the AI/ML community is rooted in its potential to dramatically improve the quality of training data and inference context, addressing long-standing issues with noisy web data extraction. Cleaner data is essential for effective model training; studies have shown that improved extraction methods can lead to increases in model accuracy by over a percentage point. Pulpie's advancements not only lower operational costs—cleaning 1 billion pages costs approximately $7,900 versus Dripper's $159,000—but also propose a scalable solution to the extraction bottleneck that exists in language model training and usage, ultimately fostering the development of more powerful AI applications.