How AST-grep Rewrote Tree-sitter in Rust and Made It 30% Faster (astgrep.com)

In a remarkable advancement for the AI/ML community, ast-grep has successfully rewritten the C core of Tree-sitter in Rust, achieving up to 30% faster parsing speeds. This significant upgrade leverages AI assistance, utilizing ChatGPT to guide the code transformation. The new Rust implementation not only enhances raw parsing and tree traversal speeds—resulting in a 22% increase in overall performance for ast-grep—but also preserves compatibility with existing language grammars and structure. The rewrite eliminates unnecessary features for ast-grep's specific use cases, such as incremental parsing, optimizing the runtime for complete file analysis rather than real-time editor interactions. The implications of this project are substantial, as it demonstrates how AI can streamline complex software rewriting processes, making previously daunting optimization tasks feasible for individual developers. The Rust version, while marginally heavier in memory usage, significantly reduces peak resident memory in most scenarios compared to its C predecessor. This ambitious reengineering effort positions ast-grep to enhance its capabilities in code analysis and development environments, ultimately paving the way for faster, more efficient AI-driven coding tools.