Json2toon_rs – Fast JSON to Toon Converter in Rust (github.com)

🤖 AI Summary
json2toon_rs is a new Rust crate that provides a fast, spec-compliant converter between JSON and TOON v2.0 (Token-Oriented Object Notation). It offers bidirectional encoding/decoding with deterministic, human-readable output optimized for arrays of uniform objects (tabular data), nested configs, and LLM prompt/structured-data interchange. The implementation emphasizes production safety and performance: minimal allocations, zero unsafe code, comprehensive tests, detailed structured errors, and example-driven documentation. Add it via Cargo and use encode/decode to get perfect round-trips for typical JSON payloads. For AI/ML workflows, the format’s line-oriented, indentation-based syntax and automatic tabular detection make it useful for converting datasets and prompts into compact, deterministic text useful for fine-tuning, evaluation, and prompt engineering where consistent tokenization matters. Key technical features include configurable delimiters (comma/tab/pipe), strict mode (enforces indentation multiples, declared array/row counts, delimiter consistency and valid escapes), canonical number formatting (no exponents/trailing zeros), deterministic quoting and escape handling, UTF-8 support, and preservation of object key order. The decoder implements full TOON v2.0 line-parsing, tabular mapping, expanded lists for mixed arrays, type inference, and robust validation. This makes json2toon_rs a practical, safe building block for ML data pipelines that need compact, readable, and deterministic serialization.
Loading comments...
loading comments...