Tract: Self-contained, TensorFlow and ONNX inference (github.com)

Sonos has unveiled "tract," a powerful neural network inference toolkit previously known as tfdeploy. This toolkit can optimize and execute models from popular frameworks such as TensorFlow, ONNX, and NNEF (Neural Network Exchange Format). Tract successfully passes approximately 85% of the ONNX backend tests, demonstrating compatibility with many real-world models like ResNet and Inception. However, it currently lacks support for certain operators, such as those involving Tensor sequences, and instead focuses on a streamlined approach that prioritizes core Tensor operations, acknowledging that many advanced features may not be necessary in production environments. The significance of tract lies in its ability to bridge the gap between diverse training frameworks and efficient production environments. By supporting NNEF and offering a translation command line for converting TensorFlow or ONNX networks to NNEF/OPL formats, tract enhances the deployment of machine learning models, particularly on resource-constrained devices like Raspberry Pi. Performance benchmarks indicate that tract can significantly outperform traditional TensorFlow Lite setups, confirming its potential as a vital tool in the AI/ML community for optimizing inference on edge devices.