Show HN: InstinctFlash – Run 5B world-action models in real time on Jetson Thor (github.com)

A significant advancement in AI model deployment has emerged with the announcement of InstinctFlash, a new framework enabling real-time execution of 5 billion parameter world-action models on NVIDIA's Jetson Thor and RTX GPUs. This includes the recently added support for the RTX 5090 and 4090, allowing for efficient desktop inference and WebSocket serving. Users can expect up to a remarkable 33.78× speedup in model performance, particularly highlighted by the LingBot-VA model, which maintains task efficacy even at reduced sampling steps and lower voltage configurations. This development is particularly crucial for the AI/ML community as it streamlines the deployment and optimization of complex robotics models across diverse hardware environments. InstinctFlash integrates a comprehensive Runtime API, allowing for easy setup and reproducibility. Key features include support for FP8 precision, multiple model families, and Python integration for both training and inference. The emphasis on optimizing execution across several layers—ranging from model compression to hardware-specific optimizations—makes this approach highly adaptable, potentially transforming how researchers and developers implement and validate AI models in real-world applications.