Lecture Summarization by Extracting Content from Whiteboards (2018) (github.com)

Researchers released code for a 2018 system that automatically extracts handwritten notes, math and sketches from lecture whiteboards to produce keyframe summaries and make lecture videos searchable. The repo adapts a deep-learning scene-text detector (AccessMath-TextBoxes) to detect handwritten content, pairs it with an SSD-based person detector to identify and remove speaker occlusions, and provides end-to-end scripts to export frames, run text detection, perform temporal reconstruction and refinement, resolve conflicts, and generate final summary keyframes and evaluation outputs. The release includes an annotation tool for ground-truth labeling and is distributed under the GNU Public License. This matters for AI/ML and educational tech because it automates indexing and retrieval of lecture content that’s otherwise buried in video, supporting accessibility, study, and large-scale analysis of classroom material. Technically, the pipeline uses Caffe and PyTorch (training available), plus OpenCV/ffmpeg, NumPy, SciPy and PyGame; it requires the AccessMath dataset and pretrained models. The code reproduces the paper’s evaluation flow (including the Table 2 reconstruction vs. raw pipelines) and provides concrete scripts for every stage—making it straightforward for researchers to reproduce, adapt, or extend whiteboard text detection, occlusion recovery, and video summarization methods.