ML-pokedex – experiments trying to train some models directly on RDF (alganet.github.io)

A recent exploration dubbed "ML-pokedex" has introduced experiments using language models trained directly on RDF (Resource Description Framework) data, specifically focusing on a Pokémon knowledge graph. These experiments, which run entirely in the browser, enable the models to recall and visualize Pokémon attributes—such as types and abilities—by memorizing the entirety of the graph. Notably, while the neural network underperforms compared to traditional data compression techniques like gzip, the insights drawn from the model's learning process present intriguing possibilities for engaging with the Semantic Web. This initiative highlights the potential for language models to engage with structured data in novel ways, emphasizing their capability to generate coherent information autonomously. For instance, the models can invent new Pokémon by generating a plausible set of attributes, revealing their natural language processing strengths. Furthermore, the training process illustrates a layered understanding, as the model progressively learns syntax, vocabulary, and reasoning structures from the RDF data. These findings indicate a significant step towards enhancing AI's ability to manipulate structured knowledge, which could have broad implications for future AI applications in various domains.