Berry: An MCP server that verifies LLM claims against your evidence (strawberry.hassana.io)

Berry has been introduced as a verification-only, evidence-required server designed to improve confidence in the claims made by large language models (LLMs). Unlike traditional AI coding assistants that often generate misleading or incorrect information with high confidence, Berry mandates that any claim must be backed by evidence provided by the user. It achieves this through two core tools: detect_hallucination, which checks if each claim is supported by the given evidence while providing confidence scores, and audit_trace_budget, which assesses the adequacy of that evidence based on its specificity. This innovation is significant for the AI/ML community as it shifts the paradigm from relying on prompting and fine-tuning to enforce reliability in LLM outputs toward a more robust evidence-based verification process. Berry is designed to run locally within a project’s repository, allowing users full control over the types of evidence they wish to validate claims against. By enforcing the standard that unsupported claims must be flagged, Berry aims to reduce the frequency of LLM hallucinations and enhance the accountability of AI-generated outputs, making a meaningful improvement in the reliability of AI systems.