Show HN: Who's in the weights? – which people 13 language models know (whos-in-the-weights.vercel.app)

A recent exploration titled "Who's in the weights?" evaluates how well 13 different language models recognize various individuals based on their confidence scores, which range from 0 to 100. The study assessed 291 people, showcasing that recognition is loosely correlated with how often these individuals are looked up on Wikipedia. Interestingly, while models generally align on ranking well-known individuals, they exhibit significant variability in confidence: some models are certain about almost everyone, while others rarely recognize anyone, emphasizing the subjective nature of 'knowledge' in AI systems. This analysis is significant for the AI and machine learning community as it highlights how different models can interpret and evaluate the same data differently, which is crucial for applications such as personalized AI assistants and social media analytics. Furthermore, the effects of name overlap and occupational categories on recognition were examined, revealing that athletes tend to lag behind others in recognition, while shared names modestly impact visibility. Overall, this work underscores the complexities of training and implementing language models for human recognition, suggesting broader implications for bias and reliability in AI-driven insights.