How Stephen Hawking reclaimed his voice—and helped others do the same

One of Hawking’s greatest legacies is the work he did on technologies to assist people with disabilities.

Future Tense is a partnership of Slate, New America, and Arizona State University that examines emerging technologies, public policy, and society.

By Sethuraman Panchanathan, Slate March 22, 2018

Stephen Hawking, renowned physicist and best-selling author, died on March 14 at the age of 76. Hawking is well-known as one of the greatest scientists of our time, whose discoveries transformed our knowledge of black holes and whose popular science texts inspired millions. He is also well known for having amyotrophic lateral sclerosis, a disease that left him almost completely paralyzed, and for losing his ability to speak after a bout of pneumonia that required a tracheotomy.

Fortunately for us all, Hawking refused to let his disabilities prevent him from sharing his brilliant ideas and insights with all of us. His beliefs about communication were beautifully summarized in, of all places, a 1993 commercial:

Speech has allowed the communication of ideas, enabling human beings to work together to build the impossible. Mankind’s greatest achievements have come about by talking, and its greatest failures by not talking. It doesn’t have to be like this. Our greatest hopes could become reality in the future. With the technology at our disposal, the possibilities are unbounded. All we need to do is make sure we keep talking.

He lived that philosophy. He worked tirelessly with engineers, software developers, interaction designers, and many others to co-create technologies that would enable him to give inspiring lectures, transact emails, explore the internet, and write insightful books.

His success serves as a powerful example of how people and machines can work symbiotically to unleash human potential. We can empower people across the entire range of abilities to express their creativity and engage in intellectual pursuits. While humans have always used tools and technologies to enhance their abilities, new developments in areas like artificial intelligence, machine learning, and human-machine symbiosis can advance this goal far more effectively, more efficiently, and faster. This increases accessibility to people across the ability spectrum and geographical boundaries.

Hawking was able to communicate with the world through a computer tablet mounted on his wheelchair. A program called ACAT (Assistive Context-Aware Toolkit), developed by Intel, allowed him to select characters from an on-screen keyboard by twitching his cheek muscle. An infrared sensor in his glasses detected this movement. A predictive text algorithm, customized to Hawking’s writings and lectures, sped up the painstaking process. These words could then be saved to text or vocalized through a separate voice synthesizer. ACAT also allowed Hawking to operate his entire computer, surfing the internet, connecting with friends through Skype and more. Intel released ACAT as open-source software in 2015 so that anyone could access and adapt the program for their own needs.

The development of ACAT was a highly iterative process that took several years, a team of experts, and close collaboration with Hawking. For example, Hawking had trouble adapting to the initial predictive text application. After many attempts to improve it, the team ended up incorporating a third-party algorithm that worked better.

The key to developing assistive technologies is building them with, not just for, the people who are intended to use them. Better still is when these technologies can be designed and built in partnership and by people who will also use them. For example, at Arizona State University, where I work, a legally blind student named David Hayden needed help capturing class lecture notes from the blackboard. Off-the-shelf technologies did not completely meet his needs, so he worked with a team at ASU’s Center for Cognitive Ubiquitous Computing to create his own device called Note-Taker. (Disclosure: ASU is a partner with Slate and New America in Future Tense.)

This kind of inclusive multidisciplinary collaboration produces comprehensive, meaningful, innovative, and person-centered solutions with broad impact. For example, after Hayden developed his Note-Taker, sighted students were keen to use it, as he had arguably the best course notes from the various lectures.

All technologies at their core can be thought of as assistive technology, since we use them to enhance our abilities. For instance, we are developing devices in CUbiC that provide haptic (tactile) feedback for people who are blind. This was based on feedback we received from our collaborators who are visually impaired. They emphasized the importance of not using the audio/speech modality as it typically overloads their auditory perception. We therefore chose to work with the sense of touch instead to communicate augmentative cues.

There is significant interest in using this technology to enhance situational awareness for sighted individuals. We are increasingly saturated with visual and auditory inputs. For example, soldiers in the battlefield or air-traffic controllers need to be alerted of an impending danger or disaster in addition to what they are able to glean from the audio-visual information. We need a way to convey important additional information that doesn’t compete with the auditory and visual perception.

This crossover also works in reverse. Predictive text was developed to make it easier to send messages on early cellphone keypads. But it has also proven invaluable to Hawking and others with similar disabilities. In both cases, the technology serves to enhance the ability of users to convey messages faster.

By collaborating with machines, Stephen Hawking reclaimed his voice—one that is as recognizable to all of us as those of other celebrities. He served as a role model not only for aspiring scientists, but also for people who do not want their accomplishments bound by the limits of their bodies.

However, not everyone with a disability has a team of Intel scientists or a university laboratory available to them. This is where some of the latest advances can have tremendous impact.

Machine learning is a process by which computers are not only programmed to do a task, but to learn from and adapt to individual users. It is, in a sense, a team of programmers that could be available to everyone—if we choose to take advantage of the latest developments in ways that serve humankind. In his speech at Websummit 2017, Hawking cautioned that new artificial intelligence technologies could be either the best or the worst thing to happen to humanity. We could create solutions to disease and poverty. Or we could create new ways to destroy and oppress.

We are all responsible to ensure that we identify the opportunities and challenges of new technologies and focus our development efforts in a way that provides opportunities for everyone. Technology typically moves in the direction that people steer it. So let us follow Hawking’s approach: Keep engaging. Keep speaking.

Sethuraman Panchanathan is senior vice president for knowledge enterprise development at Arizona State University.

Source Slate

Also see
The quest to save Stephen Hawking’s voice in The San Francisco Chronicle
Stephen Hawking’s Intel speech system is now open source in The Next Web
Assistive Context-Aware Toolkit: ACAT in Cyberpunk