English Lexicon Time Machine (github.com)

🤖 AI Summary
English Lexicon Time Machine is an open-source pipeline that visualizes the historical growth of English as a living “prefix galaxy” by combining Wiktionary lemmas with Google Books 1‑gram shards. A zero‑config bash setup (./setup.sh) creates a Python virtualenv, streams 26 1‑gram shards and a 1.4GB Wiktionary dump, extracts lemmas, infers first‑use years, aggregates prefix counts into a trie, and renders 220 radial frames at 1080p which are encoded into an MP4 and GIF. Everything checkpoints to disk (lemmas.tsv, first_years.tsv, prefix_counts.jsonl, prefix_positions.json) and is tarball‑ready, so heavy work is cached and reruns focus only on rendering. The codebase exposes tunable flags (--min-radius, --max-radius, --base-edge-alpha, --start-progress) and includes commands for frame rendering and Neo4j import for further graph analysis. For the AI/ML community this is both a compelling data‑viz demo and a reproducible research asset: the pipeline yields time‑stamped lexical data useful for diachronic NLP, lexicon modeling, morphological/prefix frequency analysis, and graph/GNN experiments. Its streaming, shard‑aware ingestion and artifact checkpointing are practical patterns for scaling corpus workflows; the Neo4j ingest enables knowledge‑graph exploration of first‑use signals. Beyond aesthetics, the project provides a repeatable, remixable dataset and visualization that researchers can repurpose for temporal lexical studies, educational demos, or as a substrate for downstream ML experiments.
Loading comments...
loading comments...