Show HN: Forge – Guardrails take an 8B model from 53% to 99% on agentic tasks (github.com)

Forge, a new reliability layer for self-hosted language models (LLMs), has been introduced to significantly enhance their performance in multi-step agentic tasks, elevating an 8B model's effectiveness from 53% to an impressive 99%. This framework incorporates guardrails like rescue parsing, retry nudges, and enforced step execution, alongside sophisticated context management techniques that optimize VRAM usage. The standout configuration of this system, Minstral-3 8B Instruct Q8 running on llama-server, achieved a top score of 86.5% across a rigorous 26-scenario evaluation suite. The release of Forge is significant for the AI/ML community as it allows developers to improve the reliability of self-hosted models, which traditionally struggle with maintaining task-oriented workflows. Users can seamlessly implement Forge in three ways: as a WorkflowRunner for structured agent loops, as guardrails middleware for their orchestration, or as a drop-in proxy server compatible with OpenAI clients. This flexibility not only supports various backend configurations— including Ollama and Anthropic— but also promotes responsible usage of AI by ensuring that models call tools correctly and manage context efficiently. With robust evaluation metrics and comprehensive documentation, Forge promises to be a transformative tool for developers seeking to leverage LLMs in complex applications.