Virtual Rock Star

Computer technology can now translate your air guitar moves into music, with no strings attached.

Photo courtesy of Teemu Mäki-Patola Become an instant rock star: A web cam recognizes the orange gloves and software tracks your hands as you rock out.

Forget learning to play the guitar: now you can become a rock star by donning a pair of orange gloves and playing the air guitar. If you haven't been taken seriously in the past as you blast your favorite rock song and impersonate the band's lead guitarist, you may well be now when your antics actually generate music. Finnish researchers are now making it possible for aspiring performers to play an invisible guitar and have their intentions translated into music, thanks to technology involving a webcam and several pieces of software.

The virtual air guitar originated from an EU-funded research project called Algorithms for the Modeling of Acoustic Interactions (ALMA) started in 2002. The project focused on the future of musical instruments and several universities were developing physical sound models: mathematical models of real-world objects that produce sound. The virtual air guitar was developed from the model of a single vibrating guitar string by researchers Aki Kanerva, Juha Laitinen and Teemu Maki-Patola.

"We developed several prototypes of user interfaces for these sound models, the first of which was a very rough and simple air guitar. My main responsibility was the musical intelligence: the module that interprets playing gestures in a musical context, to ensure that the sound you hear is cool guitar music," says Kanerva.

The virtual air guitar is made up of several pieces of software but the first step is an input system to read the movements of the air guitarist. The key components are the orange gardening gloves worn by the performer and a webcam connected to gesture recognition software. The researchers chose the colour orange since it's a colour not usually found in people's clothes and would make the person's hands a unique colour that the software could track as they play the air guitar.

Photo courtesy of Teemu Mäki-Patola This screenshot taken from the webcam software shows the guides and the guitar centreline that it constructs.

For the performer to play a song that has no wrong notes, the gestures of the unskilled guitarist have to be intelligently mapped to guitar chords and playing techniques. The guitarist can press a pedal to choose between chord and solo mode: chord mode will map his movements to one of four chords and solo mode gives him access to a pentatonic minor scale, the most common rock and blues solo scale.