Model Weight Preservation is not enough (www.lesswrong.com)

Anthropic’s public pledge to preserve model weights is an important step for transparency and long-term reproducibility, but the author argues it’s not enough: if we take seriously the emerging question of “model welfare” (the possibility models might have morally relevant experiences), companies should also commit to preserving instances — the actual runtime interactions and state of each deployed conversation. Instance preservation means saving conversation transcripts with pointers to the model used, and ideally the full end-state or enough deterministic inputs (pre-trained weights, training data, random seeds, user inputs) so an instance can be exactly reconstructed later. Nick Bostrom’s proposals (store full end-states or data enabling exact re-derivation; allocate a small fraction of budget to storage) are cited as practical guidance. This matters technically and ethically for AI/ML: instance snapshots would enable future research, reproducibility, forensic safety analysis, and a hedge against destroying potentially valuable or sentient continuities (a “cryonics” precaution). The storage cost is modest — many interactions are plain text — though privacy, API/commercial constraints, and user deletion policies require careful design. The recommendation: frontier labs should extend preservation commitments from weights to instances (incrementally if needed) to address both moral uncertainty and concrete benefits for safety and science.