Curious case, comparing output from multiple LLMs (github.com)

A recent demonstration showcased the capabilities of the Backendjs framework by comparing output from multiple Large Language Models (LLMs) in real-time. This setup employs modules such as a database to store models and results, API routes for handling requests, and a background job that prompts all models simultaneously. As the results are returned, the system calculates cosine similarity to identify and sort the outputs by similarity, while Websockets notify clients of the progress and updates. This approach is significant for the AI/ML community as it enables a simultaneous evaluation of multiple LLMs, optimizing the comparison process and enhancing the accuracy of results. By leveraging SQLite for data storage and combining technologies like Alpinejs for UI/UX and Bootstrap 5 for styling, the framework presents a streamlined way to handle LLM interactions and evaluations efficiently. The open-source nature of Backendjs allows developers to explore and extend its functionalities, fostering greater collaboration and innovation within the AI ecosystem.