Awesome Vintage LLMs (github.com)

A curated compilation of vintage large language models (LLMs) has been released, showcasing models that are trained on text from specific historical periods, thus preserving the vocabulary, worldview, and assumptions unique to those eras. Coined by Owain Evans, the term "vintage LLM" refers to models whose training data has a defined knowledge cutoff, enabling researchers to explore concepts such as counterfactual scenarios and the historical evolution of language. Notable examples include MonadGPT, a fine-tuned 7B-parameter chatbot trained on early-modern texts, and the larger 13B-parameter Talkie model, which draws from pre-1931 texts to create a conversational AI reflecting a historical linguistic style. This emerging field holds significant promise for the AI/ML community, particularly in digital humanities, social sciences, and behavioral research. By providing a means to conduct experiments that adhere to historical contexts, vintage LLMs allow for contamination-free benchmarks and genuine insights into past ideologies. The development of these models not only emphasizes the interplay between technology and history but also raises questions about the biases present in our understanding of the past, as they preserve and highlight the distinct cultural narratives of their respective time periods.