Email triage with an embedding-based classifier (adamwiggins.com)

A new research initiative is exploring an embedding-based classifier's ability to personalize email triage by distinguishing important messages from less relevant ones. This approach moves beyond traditional spam-filtering methods by leveraging user-specific preferences to categorize emails into labels such as "priority," "for your information," and "ignore." The process involves hand-labeling a dataset of emails and training a logistic regression model to classify these based on computed embeddings, achieving an accuracy rate of 82% with a high-confidence prediction rate of 93%. Significantly, this project addresses the complexity of individual preferences in email management, potentially enhancing productivity by automating email sorting. The developer also compared this classifier to a fine-tuned LLM approach, finding that while the LLM performed adequately, it required more effort in prompt tuning and lacked the efficiency of the embedding classifier. Looking forward, the research aims to develop methods for effectively collecting user feedback and adapting the classifier to evolving email triage needs, positioning this as a practical tool for enhancing email management in a personal and customizable way.