Visualizing LLM embeddings on a sphere (github.com)

A new project has emerged that showcases an interactive 3D visualization of OpenAI's text embeddings, featuring 1002 words across 15 different domains. Using the text-embedding-3-small model, the embeddings are first reduced to 3D dimensions through PCA and UMAP, then projected onto a sphere utilizing UMAP's spherical output metric. This approach allows for an engaging exploration of text embeddings in a novel format, which is accessible live at nofone.io/experiment/3dembed. This visualization is significant for the AI/ML community as it addresses common issues with traditional embedding methods, particularly clustering artifacts that arise in flat 3D representations. By employing a haversine distance metric, the project effectively maps embeddings directly onto the surface of a sphere, thus enhancing data interpretation and clarity. Users can customize the word list and UMAP parameters using a straightforward Python pipeline, making it easy to regenerate embeddings without prior API integration. This tool not only facilitates immediate insight into textual similarities but also serves as a foundation for further advancements in embedding visualization techniques.