Imagine firefighters trying to navigate through an unfamiliar, burning building full of suffocating smoke and deafening noise. Firefighting is exceedingly dangerous, and the ability for first responders to maintain communications in hostile environments can literally mean life or death.
Developing technologies to improve the safety and efficiency of firefighters is challenging because smoke and noise affect human senses, and they impair computer vision algorithms that provide navigational guidance.
Ruling out audio and visual approaches, an interdisciplinary research team from Carnegie Mellon University is exploring haptic interfaces as alternative communication channels for firefighters. A haptic interface is a system that enables people to interact with a computer through their body movements.
Yang Cai, a senior systems scientist in Carnegie Mellon CyLab and the director of the Visual Intelligence Studio, heads the team that modified a firefighter’s helmet by adding haptic actuators, a control box with a radio module, and other devices to provide real-time information to firefighters. They named their prototype the “Haptic Helmet.”
-
Florian Alber, Yang Cai (principle investigator), and Sean Hackett (left to right) show variations of the helmet, which were used throughout the multiphase completion. The red helmet on the right was used in the live-testing scenario in the firefighting facility.
The helmet communicates directions to a firefighter by sending a buzz to front, back, or sides of the head, indicating to move forward, stop, or turn left or right. The haptic signals are sent to the helmet from a RF remote controller that’s in the vicinity or from a virtual reality simulator via a cable.
The impetus for this project was the National Institute of Standards and Technology (NIST) Public Safety Communication Research (PSCR) Division’s Haptic Interfaces for Public Safety Challenge. The contest assessed the use of virtual reality environments as a development tool for creating safety technologies. The challenge also investigated the relevancy of haptic interfaces for assisting first responders. In November 2019 the CMU team won the challenge and an additional award for their prototype being the “most commercially promising.”
“We wanted to keep the design very simple and have very rapid prototyping. Our solution to the problem was the simplest, and it can be low cost,” said Cai.
During the multi-phase contest, the helmet was tested in virtual reality scenarios, and in a final challenge, the helmet was live-tested in a firefighter training facility in Colorado. In a series of trials, different firefighters would put on the helmet and make their way through a pitch-black, smoke-filled building. A CMU researcher would follow the firefighter through the dark with a thermal imaging camera so he could send directions to the helmet. This was done because the contest was constructed to keep the testing focused on the haptic system and not the contestants’ wireless radio systems. During the trials, the helmet performed efficiently, and the firefighters quickly made it through the course.
“At the competition, half of the judges were real firefighters, and overall, they liked our approach because it was the most intuitive. If you got a buzz on the left side of your head, you look towards the left. Being intuitive is important because firefighting is really stressful,” said Florian Alber, a CMU research engineer.
Other winning traits were that the haptics were embedded into an existing piece of gear so it took no extra time to put on the helmet. It was easy for first responders to learn how to use the system, too.
“It only took us a minute to explain to the firefighters what each of the commands mean,” said Sean Hackett, a CMU research engineer.
The helmet was on display at the NIST’s PSCR booth at the 2020 Consumer Electronics Show in Las Vegas. In the future, the team will work with Pittsburgh firefighters, gathering feedback on the helmet to improve its functionality in broader firefighting activities.
The CMU research team includes Cai; Hackett; Alber; Roberta Klatzky, professor of psychology and human-computer interaction; Mel Siegel, professor emeritus in the Robotics Institute; Ben Graham, undergraduate student in electrical and computer engineering; and Weizhe Guo, intern in the Visual Intelligence Studio. Lenny Weiss, MD, from the University of Pittsburgh School of Medicine acted as a medical adviser during the project.