44% on ARC-AGI-1 in 67 cents (github.com)

A recent update has shown a significant improvement in the performance of the ARC-AGI-1 model, achieving 44% accuracy on the ARC-1 public evaluation. This marks a notable jump from the previous 27.5% performance level, while also reducing the total compute cost to approximately $0.67 using a rented 5090 GPU for 2 hours. The model employs a standard transformer architecture with 75 million parameters, emphasizing its efficiency and scalability at a lower cost compared to previous implementations that required more substantial compute resources like the A100 GPU. This advancement is crucial for the AI/ML community, as it demonstrates the potential of smaller, more accessible models to achieve competitive performance on challenging tasks. The detailed instructions for setting up the model environment, including software dependencies and data handling, highlight an approach that encourages reproducibility and transparency in research. Furthermore, by achieving high performance at a fraction of the cost, this development could democratize access to powerful AI tools, fostering innovation and experimentation within the field.