Redis's Homegrown JSON Parser (github.com)

Redis has announced a significant upgrade to its vector sets feature with a new homegrown JSON parser designed to improve performance and reduce code complexity. This new parser replaces the existing cJSON library, which required parsing entire JSON objects and converting them into structures that were incompatible with Redis' virtual machine. The new implementation allows for zero-allocation parsing, focusing specifically on the required fields, which dramatically streamlines the process and eliminates the need for external dependencies. The technical implications are substantial: the new specialized parser not only sheds approximately 3,000 lines of code but also delivers a remarkable 3.5x speedup in executing queries on large JSON datasets. For example, when querying over a dataset of one million JSON objects with multiple fields, the efficiency gained is evident. This upgrade positions Redis as a highly performant choice for handling vector storage, especially for applications needing quick, user-facing improvements. Additionally, the new parser is resilient to imperfect JSON structures, ensuring broader compatibility while maintaining high performance.