GStreamer 1.28 brings AI inference to your media pipeline (www.collabora.com)

Collabora has announced the release of GStreamer 1.28, integrating advanced AI inference capabilities into its renowned media pipeline framework. This update introduces several key enhancements, including improved support for inference engines like ONNX Runtime and LiteRT, as well as the addition of backend support for VeriSilicon's NPU, which is used in various System-on-Chip (SoC) platforms. These advancements allow developers to process and analyze media through machine learning models more efficiently, streamlining the deployment of AI functionalities in media applications. Significantly, GStreamer 1.28 simplifies the use of machine learning within media pipelines by introducing a new "tensordecodebin" that automates tensor decoding based on model metadata. Enhanced features, such as a tensor ID registry and a new ModelInfo file format, improve usability by minimizing the need for manual file modifications and promoting collaboration within the AI community. With additional model decoders and tools for visualizing results, GStreamer 1.28 stands as a comprehensive solution for developers looking to integrate AI-driven analytics into their media applications, expanding the potential use cases in areas like image classification and object detection.