HNSW vector search beyond available RAM for ESP21P4 (github.com)

The recent announcement of nn20db introduces a groundbreaking SDK for conducting million-scale vector searches on compact devices, specifically targeting the ESP32-P4 hardware and other Linux-based systems. This innovative solution allows users to perform vector searches directly from low-cost persistent storage, circumventing the need for large RAM capacities or cloud infrastructure. The approach is engineered for applications in edge AI search, embedded retrieval systems, and scenarios where local processing power is limited, effectively broadening the accessibility of advanced AI technologies in constrained environments. Significantly, nn20db leverages the Hierarchical Navigable Small World (HNSW) algorithm for approximate nearest-neighbor searches, capable of operating efficiently beyond available RAM. It supports various distance metrics such as Euclidean, cosine, and Jaccard, making it versatile for different use cases. The SDK comprises Python bindings for Linux experiments and multiple demo applications to facilitate user onboarding. As a beta release, it invites feedback from the AI/ML community, particularly those involved in embedded AI and offline retrieval. This development enhances local-first search capabilities, potentially transforming how AI applications are deployed on resource-restricted devices.