A high-performance document search engine built in Rust with WebAssembly support (github.com)

A new high-performance document search engine has been unveiled, developed in Rust with WebAssembly (WASM) support. This engine integrates full-text search capabilities using Finite State Transducers (FST) alongside FSST compression, allowing for efficient storage and rapid fuzzy matching. An interactive demo showcases its ability to search through 50,000 articles from the AG News dataset directly in browsers, highlighting its design for performance with a compressed index size of just 5.20 MB, search speeds of 1-3 milliseconds per query, and an indexing time of approximately 1.1 seconds. The significance of this tool for the AI/ML community lies in its combination of advanced search algorithms and efficient data handling, enabling real-time search functionality without requiring server resources. The use of FST for keyword matching with Levenshtein distance enhances the system's fuzzy matching abilities, while FSST compression reduces storage demands by 60-80%. The ability to compile into a standalone WASM module means developers can easily integrate this powerful search functionality into web applications, making it a noteworthy contribution to the ecosystem of serverless search technologies.