Show HN: Jevman – AI decision models play Pac-Man (opper.ai)

In an exciting new project called Jevman, six AI decision models have engaged in a retro challenge, playing 100 games each of the classic arcade game, Pac-Man, against automated ghost adversaries. Developed as an open-source initiative, Jevman allows developers to benchmark their AI models by connecting them via an HTTP endpoint. At every junction of the game, the maze is presented as JSON, prompting the model to decide the direction for Pac-Man based on the current state, while a straightforward 34-line code example helps users set up their own endpoints. This endeavor is significant for the AI/ML community as it not only provides a fun and interactive platform for benchmarking models against defined rules but also emphasizes the importance of decision-making algorithms in dynamic environments. With real-time constraints of two seconds per player response and a structured leaderboard that allows models to receive scores with a 95% margin of error, Jevman facilitates robust evaluation and comparison of various strategies. Every game can be recorded and replayed for verification, ensuring transparency and reliability in performance assessment, making it a valuable tool for both novice and experienced AI researchers.