Show HN: Semlib – Semantic Data Processing (github.com)

Semlib is a new Python library that turns standard functional primitives (map, reduce, sort, filter) into LLM-driven, natural-language data-processing building blocks. Instead of writing code to specify operations, you give each primitive an English description (e.g., "sort by right-leaning" or "How old was {} when he took office?") and Semlib handles the prompting, parsing, concurrency, caching and cost tracking needed to run the subtasks. It exposes typed return values, supports async execution, and shows example workflows for tasks like sorting presidents, extracting attributes, or analyzing large corpora. For the AI/ML community Semlib is significant because it operationalizes a robust pattern for scaling LLM-based pipelines: decompose complex jobs into many small semantic subtasks that can be run in parallel, cached, and assigned to different models. That addresses practical limits of long-context models (feasibility and quality), cuts latency via concurrent and tree-reduce patterns (O(log n) depth), lowers cost by using smaller models per subtask, and improves security by enabling self-hosted/open-model steps. It also supports hybrid pipelines where pure Python handles precise computations while LLMs handle linguistic tasks, making it useful for production data-processing, retrieval, triage, and labeling workflows.