Decompiling the Synergy: Human–LLM Teaming in Reverse Engineering [pdf] (www.zionbasque.com)

A recent empirical study titled "Decompiling the Synergy: Human–LLM Teaming in Software Reverse Engineering" investigates the collaboration between human analysts and Large Language Models (LLMs) in the field of software reverse engineering (SRE). Conducted by researchers from Arizona State University and other institutions, the study includes an online survey of 153 SRE practitioners, followed by a detailed analysis involving 48 participants (novices and experts) working on Capture-The-Flag-style challenges. The findings reveal that LLMs significantly enhance the capabilities of novice practitioners, nearly doubling their comprehension rates to match expert levels, while offering mixed results for seasoned analysts, who occasionally fell prey to harmful hallucinations and unhelpful suggestions from the LLMs. This research is significant for the AI/ML community as it highlights both the potential and the limitations of integrating LLMs into complex human-driven processes like SRE. It identifies key areas where LLMs can improve efficiency—reducing algorithm analysis time by 238% and artifact recovery by 66%—while also underscoring the risks of relying too heavily on LLM outputs. By capturing the dynamics of SRE workflows and providing insights into effective LLM usage, the study serves as a foundational step toward optimizing human-LLM collaboration in reverse engineering, offering critical implications for future tool development and educational approaches in the field.