Turn fenic data sandboxes into versioned Hugging Face datasets (huggingface.co)

fenic, a PySpark‑inspired DataFrame framework for production AI and agentic apps, now lets you mount Hugging Face Hub datasets directly into pipelines using an hf:// protocol built into its DataFrameReader. That means teams can read CSV and Parquet files (with glob patterns), reference specific commits, branches or branch aliases (using @revision notation), and mix Hub files with local data in a single read. The integration preserves reproducibility and versioning for dataset-driven ML workflows and makes it trivial to pull public or private Hub assets (private reads require HF_TOKEN) into a single, optimized logical plan. Beyond simple I/O, fenic couples Hub access with its full DataFrame API, automatic query optimization, and built‑in semantic operations: you can add embeddings, run LLM-based analyses, and map semantic functions inline once you configure providers (OpenAI, Anthropic, Cohere, Google). It supports schema merging across files (filling missing columns with nulls and widening types where possible—see docs for limits), and allows specifying language and embedding models per SessionConfig. For practitioners, this unifies dataset versioning, scalable batch processing, and model‑centric enrichment in one framework, reducing ETL friction and improving reproducibility for model training and evaluation.