Show HN: Only 1 LLM can fly a drone (github.com)

A recent experiment has shown that only one out of seven large language models (LLMs) could effectively pilot a drone in a 3D voxel world to locate and identify virtual creatures. Inspired by Pokémon Snap, the simulation progresses as a visual-language model (VLM) receives prompts about its environment and issues movement commands based on its observations. The standout performer was Gemini Flash, which successfully adjusted its altitude to identify creatures, while other models such as Claude Opus struggled due to their inability to descend and adjust approach angles. This finding holds significant implications for the AI/ML community, highlighting that the effectiveness of an LLM in practical applications like navigation may not correlate with its size or cost. The experiment suggests that spatial reasoning capabilities might not scale linearly with model complexity, as Gemini Flash—an affordable model—outperformed pricier alternatives. The results raise questions about model training and the potential advantage of smaller models in specific tasks, pointing to the nuance needed in developing LLM-powered agents for real-world navigation challenges. This research invites further exploration into fine-tuning LLMs for specific tasks, as well as potential real-world applications of these models in piloting drones.