Hands-Free JavaScript: Write Code by Whistling (velato.net)

Researchers demoed a hands-free programming prototype that translates whistled melodies into JavaScript code: users whistle a first “root” note to set the reference pitch, then whistle interval-based commands (2nd, 3rd, 4th, etc.) that map to programming primitives. The interface (called Velato in the demo) provides a lexicon and playback examples so users can hear how each token should sound; relative intervals encode variables, prefix expressions, digits (as single notes avoiding the 5th), conditionals (mapped to the 3rd), math operations (mapped to the 5th), and I/O (6th). The system converts the stream of pitched notes into a block-like representation and a JavaScript program in real time, effectively creating a musical domain-specific language for code entry. This approach is significant for accessibility and HCI: it offers an alternative input modality for people with motor impairments, hands-busy scenarios, or creative coding contexts, and demonstrates how symbolic programming can be built on top of perceptual features (relative pitch) rather than phonemes. Key technical implications include robust pitch-tracking, transposition-invariance via a root note, prefix expression parsing to disambiguate streams, and a compact lexicon that uses interval roles rather than absolute notes. Challenges remain around noise tolerance, learning curve, and expressive coverage compared to full-text coding, but the demo highlights a novel, low-bandwidth interaction channel for programming and multimodal code generation.