RAI: A vendor-agnostic agentic framework for robotics by utilizing ROS 2 tools (github.com)

RAI is a newly released, vendor‑agnostic agentic framework that leverages ROS 2 tooling to develop and deploy Embodied AI on real robots and simulators. It bundles multi‑agent orchestration, human‑robot interaction, and native multi‑modal processing so developers can add GenAI-driven behaviors (language, vision, speech) to existing robotic stacks. The project emphasizes out‑of‑the‑box AI features—natural language manipulation, mission reasoning, navigation—and was showcased at ROSCon 2024 with demos including orchard mission planning, natural‑language control of a Franka Panda using Grounded SAM 2, and autonomous navigation on a Husarion ROSbot XL. Technically, RAI is modular: rai_core (multi‑agent and multi‑modal runtime), rai_whoami (extracts robot embodiment from docs/URDFs/images), rai_asr/rai_tts (speech pipelines), rai_sim (sim integration), rai_bench (benchmarking agents/models/simulators), rai_openset (open‑set detection), rai_nomad (NoMaD navigation integration), and rai_finetune (finetune LLMs on embodied datasets). This stack enables reproducible evaluation, sim‑to‑real workflows, and fine‑tuning of language models on robot data—lowering integration friction for researchers and product teams. Docs, setup guides, demos and a contribution guide are at robotecai.github.io; the project paper is on arXiv (2505.07532).