Ellora: Enhancing LLMs with LoRA Standardized Recipes for Capability Enhancement (huggingface.co)

Ellora is a new, open collection of standardized, production-ready recipes that use Low-Rank Adaptation (LoRA) to improve LLMs efficiently—without full-finetuning. It codifies lessons from years of work (LoRA’s low-rank adapters, QLoRA quantized finetuning, and 2025’s “LoRA Without Regret” showing LoRA can match full-finetuning while using ~67% of the compute) into six infrastructure-agnostic notebooks. Ellora emphasizes self-supervised data generation (Magpie), rigorous evaluation, and compatibility with PEFT, LoRAX, vLLM, Unsloth, and Hugging Face tools, making parameter-efficient pipelines reproducible for production teams. The six recipes span practical and frontier problems with concrete, benchmarked wins: Recipe 1 recovers quantization loss (INT4+LoRA student perplexity 2.09 vs FP16 1.97; 5.7% gap, 75% memory reduction, 2–3x inference speed); Recipe 2 uses GRPO self-RL to instill chain-of-thought (quality score 3.2→5.6, 75% improvement); Recipe 3 combines synthetic scenarios with real tool execution for robust tool-calling; Recipe 4 scales context from 32K→2M tokens via progressive curriculum (61× increase, vLLM/Unsloth optimizations); Recipe 5 reduces vulnerabilities (vuln score −97%, secure pattern usage +1,420%); Recipe 6 explores execution-awareness (early mean state accuracy 33.3%). Coupled with PE-RLHF results (substantially faster training and lower memory with comparable performance), Ellora offers a practical, reproducible path to deploy LoRA-driven enhancements today—repo: github.com/codelion/ellora.