Ask Me Twice: A longitudinal archive of AI chatbot responses (wayback-api.archive.org)

A new project titled "Ask Me Twice" has launched, creating a longitudinal archive that systematically records responses from various AI chatbots to a curated set of questions. This evolving dataset currently includes 137 questions posed to 44 different models, resulting in nearly 469,000 responses. Users can query this extensive database over HTTP or through bulk exports, facilitated by a machine-readable OpenAPI schema. The significance of this initiative lies in its ability to provide valuable insights into how AI models evolve over time in their responses to the same questions, encouraging transparency and accountability in AI development. By employing a Model Context Protocol server, AI agents can directly query the archive, facilitating more in-depth analysis and comparison of responses. This resource serves as a critical tool for researchers and developers in the AI/ML community, enabling them to track changes, identify trends, and better understand the nuances of AI behavior in response to user inquiries.