LLM, give me a JSON. Make no mistakes (nobodywho.ooo)

A new approach has emerged for generating structured JSON outputs from large language models (LLMs), addressing the common issue of inconsistencies and errors in the JSON format. Traditional methods often involve retrying the output until a valid JSON is produced, which can waste computational resources and tokens. This new method refines output generation through constrained sampling, where LLMs create output token by token while immediately rejecting any non-compliant tokens, thus avoiding the inefficiencies of discarding entire responses. Significantly for the AI/ML community, this technique includes the use of JSON schemas and context-free grammars (CFGs) to define expected output formats more precisely. By employing these advanced methods, developers can create grammars that specify acceptable string formats, which helps the LLM avoid mistakes during generation, enhancing both reliability and speed. This transformative approach not only streamlines structured data creation but also opens pathways for more complex applications requiring structured JSON outputs, making it easier for LLMs to accurately serve diverse data needs in various applications.