The Personalization in ChatGPT Is Creepy (jamesoclaire.com)

A user reports growing unease with ChatGPT’s personalization after the model began surfacing small, previously mentioned details — like working on a parents’ off-grid solar home — in otherwise unrelated answers about Tailscale or homelab routers. The system also blends IP-based location signals into product advice in a way that feels intrusive, and the writer worries that pasted code and conversational fragments act as persistent fingerprints linking their identity to past chats. The core complaint: the model “remembers” and reuses personal nuggets across queries in ways that feel incorrect, emotionally jarring, or privacy-invasive. For the AI/ML community this underscores tensions between helpful personalization and user trust. Technically, it points to usage of persistent chat histories, session/context windows, and metadata (like IP/location) to tailor outputs — which can produce unwanted memory, inaccurate personalization, or implicit product placement. Remedies include stronger memory controls, clearer opt-outs, ephemeral contexts, and privacy techniques (local inference, on-device models, differential privacy, encrypted logs). The author hopes efficient local models will enable private AI chats, but notes that practical, budget-friendly on-device solutions remain limited today. The episode is a timely reminder that personalization features need transparent controls and privacy-preserving architectures to maintain user confidence.