Physical_Atari: Platform for evaluating RL algorithms on a physical Atari (github.com)

Physical Atari is a new open hardware/software platform for evaluating RL agents on a real Atari 2600+ console, announced to bridge the “reality gap” between the popular ALE simulator and physical systems. It exposes real-world concerns absent from emulators—real-time operation (no turn-taking), camera and actuator latency, sensor/actuation noise, lighting and reflection variability, and ROM/hardware differences—and includes a working implementation of a game‑independent RL agent that reliably surpasses standard benchmark performance within five hours (≈1M frames) on several games. The project’s three aims are to provide a physical testbed, demonstrate learnability under real-time constraints, and surface discrepancies that should inform simulator design and evaluation metrics. Technically, the system couples an Atari 2600+ (HDMI output) to a 60 FPS USB camera (Razer Kiyo Pro) and a control interface that either actuates a physical CX40+ joystick via a servo-based RoboTroller or sends DB9 controller-port signals via a digital I/O module. Observations are rectified and streamed at 60Hz; rewards, lives, and end-of-episode signals are extracted from screen pixels using per-game CRNNs and rule-based validators trained on ALE-rendered frames. The platform includes Docker scripts, performance and profiling guides (NVIDIA Nsight), and careful notes on ROM revisions, NTSC/PAL, bankswitching and robustness trade-offs—making it a practical tool for assessing how simulator-derived advances translate to true hardware.