The New American AI Model Designed to Be Customized (blog.bytebytego.com)

Thinking Machines has introduced a novel AI model named Inkling, which features a unique architecture that allows for real-time collaboration between coding agents and continuous integration checks. By operating within the agent’s coding loop, Inkling provides architectural context prior to code generation and verifies output as it's produced, effectively reducing token consumption by 36% and defects by an impressive 92%. This model comprises 66 layers, each with 256 experts—only six of which activate for any given input—representing a significant shift towards efficient processing in large-scale AI applications. The significance of Inkling lies in its design choices, such as the use of a Mixture of Experts system that decouples model storage from operational costs. This allows Inkling to utilize 975 billion parameters without overwhelming computational requirements, as only about 41 billion are engaged for each token processed. Additionally, Inkling maintains a vast context window of one million tokens while employing novel routing and attention mechanisms that optimize expert selection and balance usage, mitigating issues like routing collapse that can hinder performance. These innovations position Inkling as a versatile tool for developers, facilitating model customization and further advancing the capabilities of AI/ML technologies.