Turns out, Astra cannot build manufacturing CAD yet (interpretai.tech)

Recent assessments revealed that Astra, an advanced AI model, struggles to produce production-grade Computer-Aided Design (CAD) assemblies, as evaluated against multiple criteria such as geometry, editability, and manufacturability. The testing included various task families and graded tasks, with scores showcasing significant performance discrepancies among competing models. For instance, while GPT-6's AstraCodex scored 40% in one task family, leading peers like Grok and Claude in some areas, none exceeded the critical 60% pass mark, underscoring their limitations in manufacturing applications. This outcome is significant for the AI/ML community as it highlights the current challenges AI models face in specialized fields like CAD design, which requires not only high accuracy but also practicality in real-world applications. The inability of these models to consistently deliver on crucial manufacturing criteria indicates that while AI technology has advanced, it still needs further refinement for industry-specific tasks. Addressing these gaps could lead to better integration of AI systems into manufacturing, potentially revolutionizing how designs are produced and optimized in various sectors.