My trick for getting consistent classification from LLMs (verdik.substack.com)

A developer announced a practical technique and open-source Golang package ("consistent-classifier") for turning noisy, stochastic LLM outputs into deterministic class labels. The core insight: LLMs are lexicographically inconsistent but semantically consistent—so instead of forcing a rigid label set, embed both texts and the generated labels into a vector space (voyage-3.5-lite, 1024-dim), use a vector index (Pinecone) to find nearest neighbors by cosine similarity (default threshold 0.80), and consolidate semantically similar labels with a Disjoint Set Union (DSU) / path-compression strategy to pick a stable root label. Workflow: embed content → vector search → cache hit returns root label; on miss call the LLM (gpt-4.1-mini), store embeddings, cluster labels, and update the index. Benchmarks on 10K tweets show strong benefits: label cardinality fell from ~6,520 (LLM-only) to ~1,381 (vectorized), cache hit rate rose to ~94% by tweet #10,000, and marginal cost per tweet transitions from ~15% higher at start to roughly 10x cheaper at scale as LLM calls are avoided. Latency is initially ~130% worse due to embedding + search overhead, but it improves as cache hits grow. The repo includes adapters, DSU utilities and benchmark scripts — useful for teams needing scalable, consistent labeling when label space is unconstrained.