Show HN: Director-AI – token-level NLI+RAG (github.com)

Director-AI has introduced a groundbreaking real-time guardrail for large language models (LLMs), focusing on mitigating hallucinations during generation. This innovative solution employs a combination of Natural Language Inference (NLI) for contradiction detection and Retrieval-Augmented Generation (RAG) for fact-checking, enabling token-level monitoring of output coherence. If the coherence score drops below a predetermined threshold while generating responses, Director-AI can halt the process mid-stream, preventing unreliable information from reaching users. This development marks a significant advancement in the AI/ML community, addressing the persistent issue of LLM hallucinations. By integrating a dual-entropy scoring mechanism and allowing users to customize the ground truth knowledge base, developers can enhance LLM outputs' reliability. Director-AI uniquely supports real-time streaming halts, a feature that competitors lack, and is adaptable to various backend systems, making it a versatile addition to any AI toolkit. The project is open-source, encouraging further exploration and innovation in safe AI interactions.