College of Engineering Unit:
Music is universally appreciated as an art form; it is considered a communication vehicle that unites people and is an essential cultural element. Yet, currently, there exists a deficit in social and musical events that provide accessibility for the Deaf & Hard-of-Hearing (DHH). Often these events rely on the audience’s ability to hear the music or message. Those with hearing impairments do not get to enjoy it or can only get a small glimpse of it. One reason may be that the event organizers overlook accessibility or lack the resources to provide equal access to DHH attendees.
Nonetheless, approximately 360 million people worldwide have partial or complete hearing loss, so there is a pressing need for alternatives. For decades, different approaches have been developed to help bridge the gap between hearing and non-hearing, usually utilizing a metaphor or alternate representation of the musical content. The two most common means are haptic and visual because they leverage two very rich sensors in the human body, visual and tactile. However, the current solutions can be expensive, bulky, and awkward.
The AudioLux project addresses this issue by allowing DHH patrons to visualize music. It is the result of an effort by a deaf musician to make the visualization of music accessible and affordable. This software uses LED strips to display various patterns of light and colors that represent changes in a song’s volume, pitch, tempo, and formant (vocal) qualities.
What sets the AudioLux device apart from existing light shows is that it processes audio in real-time and distills parameters from it that reveal the music’s character. Besides pitch and volume, the AudioLux can segment the incoming audio into specific frequency bands to sense percussion and several vocal patterns.
This project was a collaboration between two teams. An on-campus team worked on the audio processing and light-pattern algorithms and development, while this team (Online Campus) focused on providing an embedded web server and application to configure the device.
Project Website(s):
Industry Sponsor(s):