EyesOff: I Built a Screen Contact Detection Model (ym2132.github.io)

EyesOff is a lightweight screen-contact detector built to stop shoulder-surfing by flagging when someone in your webcam feed is looking at your screen. Because no public dataset existed for this specific binary task (looking vs. not-looking), the author hand-labelled over 20k face images gathered from FFHQ, a Kaggle selfie set and a Video Conferencing Dataset (VCD, 160 people) and developed a fast annotation tool. They use YuNet for face detection to crop faces (enabling per-face inference in multi-person scenes) and followed the Eye-Contact-CNN paper’s labeling boundaries (direct or slightly off-camera counts as looking) while enforcing person-exclusive train/test splits to avoid leakage. Technically the model is an EfficientNet-B0 (≈5.3M parameters) trained in two stages: phase 1 pre-trains on gaze regression using synthetic pitch/yaw labels generated by OpenVINO’s gaze_estimation_adas_0002 fed with eye crops and MediaPipe head-pose/landmarks; phase 2 fine-tunes for the binary screen-contact classification. Many layers are unfrozen except early blocks (first four frozen) to aid generalization. The resulting model runs on CPU, achieves ~71% accuracy across close and mid-range distances, and avoids per-user calibration—trading some accuracy for a smooth, always-on UX. Limitations include an assumed top-of-screen webcam position, some face-detection quirks at full resolution, and scope for smaller architectures (e.g., a YuNet-derived classifier) and broader datasets in future work.