Show HN: Experimentplatform, A/B testing images with LLMs (github.com)

A new React-based platform called Experimentplatform has been launched, designed specifically for A/B testing of images utilizing large language models (LLMs) for evaluation. Users can upload two images, pose questions about them, and receive ratings on a scale from 1 to 10. The platform supports both simulated evaluations and real LLM interactions via providers like Ollama, allowing for more robust results. Additionally, it features real-time tracking of experiments, enhancing user engagement and allowing for immediate insights. This platform is significant for the AI/ML community as it combines image processing with advanced statistical analysis techniques, specifically Welch's t-test, to assess the significance of image differences based on user feedback. The integration of LLMs for subjective evaluations introduces a novel approach to understanding visual preferences, making it a powerful tool for researchers and marketers. The technical components include real-time feedback mechanisms and customizable sample sizes, with the added capability to use vision models for more nuanced evaluations, pushing the boundaries of traditional image A/B testing.