Did it use REA? Checking viral AI reverse-engineering demos (github.com)

A recent evaluation of various AI reverse-engineering demos has revealed significant insights into the usage of the tool REA (reverse-engineer-anything). Conducted by reviewing README files, author posts, and official showcases, the analysis confirmed that 26 cases explicitly documented the use of REA, while three used it partially. However, four cases lacked verification, and six were identified as not using REA at all, but rather other tools like Claude Code or manual reverse engineering methods. This thorough review underscores the importance of accurate documentation and transparency in the AI/ML community, as many claims about tool utilization were found to be unverified or misrepresented. The implications of this assessment are notable, particularly regarding the effectiveness of REA. Two controlled comparisons conducted on a subset of projects demonstrated no significant advantages in problem-solving rates when REA was employed, suggesting that its integration might not provide the anticipated efficiency gains, despite an increased investment of time and resources. This highlights a critical juncture for the AI reverse-engineering community to reflect on methodology effectiveness, optimization strategies, and the need for rigorous validation of claims in a rapidly evolving field.