Run local LLMs with Ruby llama.cpp bindings (www.docuseal.com)

Rllama is a new Ruby gem that wraps the high‑performance llama.cpp shared library, letting Ruby developers load GGUF-format LLMs locally and generate text without calling an external API. Built at DocuSeal to power local semantic search over docs, Rllama provides a simple programmatic API (load_model, generate, close) and an interactive CLI (rllama) that can list, download, or chat with models from local paths or direct Hugging Face URLs. Install with gem install rllama and point it at a .gguf model (example uses a quantized gemma-3-1B Q4_0 file). Technically, Rllama leverages llama.cpp’s efficient CPU inference so you can run quantized GGUF models on laptops, servers, and CI machines for low‑latency, private inference. The gem exposes generation stats (tokens generated, tps, duration) and supports instruction‑tuned and quantized models, making it practical to start with smaller models for local development and scale to larger checkpoints when needed. For the AI/ML community this lowers the barrier for integrating local LLMs into Ruby apps and workflows—reducing API costs, improving privacy, and enabling reproducible, offline experimentation with models hosted on Hugging Face.