Forget the cell phone screen, a forearm is all you’ll need to read

Researchers report haptic interface for mobile technology.

Kelsey Henry, Purdue University July 2, 2018

Communication could step beyond reading a cellular phone screen with a new technique by Purdue College of Engineering researchers to learn and read messages through a person’s sense of touch.

Hong Tan, professor of electrical and computer engineering, was the lead professor on the haptic research that developed a method to receive messages by learning to interpret signals such as a buzzing sensation and others through the skin on the forearm.

The research results were presented Friday June 15 at the Proceedings of EuroHaptics 2018 conference in Pisa, Italy.

The yearlong project was done with researchers from the Massachusetts Institute of Technology and Facebook Inc.

Tan, founder and director of Purdue’s Haptic Interface Research Laboratory, said while the research lends itself to being used by hearing-impaired and visually impaired users, the study is also being looked at on general terms for any number of possible uses.

“We are collaborating with Facebook through the company’s Sponsored Academic Research Agreement. Facebook is interested in developing new platforms for communication and the haptic research we are doing has been promising,” she said.

“I’m excited about this… imagine a future where you’re able to wear a sleeve that discreetly sends messages to you – through your skin – in times when it may be inconvenient to look at a text message,” Tan said. “I’m really hoping this takes off as a general idea for a new way to communicate.

Speech Through Skin: Haptic Communications Research. Purdue engineers lead by Hong Tan, professor in the School of Electrical and Computer Engineering, developed a method to receive messages by learning to interpret signals such as a buzzing sensation and others through the skin on the forearm. The yearlong project was done with researchers from the Massachusetts Institute of Technology and Facebook Inc. and the results were presented at the Proceedings of EuroHaptics 2018 conference in Pisa, Italy. Purdue Engineering. Youtube Jun 20, 2018

“When that happens, the hearing-impaired, the visually-impaired, everyone can benefit.”

In the study, subjects used a material cuff encircling the forearm from the wrist to below the elbow. The instrument, wrapped around the test subject’s non-dominant arm, featured 24 tactors that, when stimulated, emitted a vibration against the skin, changing quality and position in the process.

Tan said the 39 phonemes (units of sound in a language that distinguish one word from another) in the English language were mapped using signals from specific tactors. The sounds of consonants such as K, P and T were stationary sensations on different areas of the arm while vowels were indicated by stimulations that moved up, down or around the forearm.

We used anything that can help you establish the mapping and recognize and memorize it,” Tan said. “This is based on a better understanding of how to transmit information through the sense of touch in a more intuitive and effective manner.”

Twelve subjects learned haptic symbols through the phoneme-based method at a schedule of 100 words in 100 minutes for the research, while 12 others learned using a word-based system with the haptic signals on their arm.

Animation of haptic symbols for 15 English vowel phonemes. To be used as a reference for the IEEE Transactions on Haptics paper titled A Phonemic-Based Haptic Display for Speech Communication, by Charlotte M. Reed, Hong Z. Tan, Zachary D. Perez, E. Courtenay Wilson, Frederico M. Severgnini, Jaehong Jung, Juan S. Martinez, Yang Jiao, Ali Israr, Frances Lau, Keith Klumb, Robert Turcott, and Freddy Abnousi. Youtube Apr 19, 2018

Research results showed the phonemes worked better than a word-based approach by providing a more consistent path for user learning in a shorter period of time. Performance levels varied greatly among the test subjects for each method, but with the phoneme-based approach at least half could perform at 80 percent accuracy while two subjects reached 90 percent accuracy.

Tan said using phonemes was more efficient compared to letters, noting there are less phonemes in a word compared with the number of letters.

She said the reasoning behind the project is the thought that there are many ways to encode speech into feelings. The project goal is to demonstrate one way that actually works.

“For this research, the learning progress is one of the key things,” Tan said. “With this, not only do we have a system that works, but we’re able to train people within hours rather than months or even years.”

For the study, a concentrated, intense training regimen was developed where the test subjects worked for about 10 minutes a day. They were then tested on their progress.

“It is more efficient than if they sit here for three hours to study,” Tan said. “You can’t keep good concentration for that long.”

“People who are hearing-impaired may be motivated to spend additional time to training themselves, but the general public probably doesn’t have the patience,” she added.

The research was funded by Facebook Inc.

Source Purdue University

References

A Comparative Study of Phoneme- and Word-Based Learning of English Words Presented to the Skin, A Comparative Study of Phoneme- and Word-Based Learning of English Words Presented to the Skin, Prattichizzo D, Shinoda H, Tan H, Ruffaldi E, Frisoli A. (eds) Haptics: Science, Technology, and Applications. EuroHaptics 2018. Lecture Notes in Computer Science, vol 10894. Springer, Cham. doi: 10.1007/978-3-319-93399-3_53

Human Factors For The Design Of Force-Reflecting Haptic Interfaces, PDF, Hong Z. Tan, Mandayam A. Srinivasan, Brian Eberman, Belinda Cheng. Reprinted from DSC-Vol.55-1, Dynamic Systems and Control, Editor: Clark J. Radcliffe, Book No. G0909A-1994. The American Society of Mechanical Engineers

Also see
Researchers develop technology that uses pulses to send messages through the skin Purdue University
“D” for danger! Speech sounds convey emotions Medical Xpress