Brain-computer interface advance allows fast, accurate typing by people with paralysis

Clinical research led by Stanford University investigators has demonstrated that a brain-to-computer hookup can enable people with paralysis to type via direct brain control at the highest speeds and accuracy levels reported to date.

By Bruce Goldman, Stanford University February 21, 2017

The report ^[1] involved three study participants with severe limb weakness — two from amyotrophic lateral sclerosis, also called Lou Gehrig’s disease, and one from a spinal cord injury. They each had one or two baby-aspirin-sized electrode arrays placed in their brains to record signals from the motor cortex, a region controlling muscle movement. These signals were transmitted to a computer via a cable and translated by algorithms into point-and-click commands guiding a cursor to characters on an onscreen keyboard.

Each participant, after minimal training, mastered the technique sufficiently to outperform the results of any previous test of brain-computer interfaces, or BCIs, for enhancing communication by people with similarly impaired movement. Notably, the study participants achieved these typing rates without the use of automatic word-completion assistance common in electronic keyboarding applications nowadays, which likely would have boosted their performance.

One participant, Dennis Degray of Menlo Park, California, was able to type 39 correct characters per minute, equivalent to about eight words per minute.

A clinical research publication led by Stanford University investigators has demonstrated that a brain-to-computer hookup can enable people with paralysis to type via direct brain control at the highest speeds and accuracy levels reported to date. YouTube Feb 21, 2017

‘A major milestone’

This point-and-click approach could be applied to a variety of computing devices, including smartphones and tablets, without substantial modifications, the Stanford researchers said.

“Our study’s success marks a major milestone on the road to improving quality of life for people with paralysis,” said Jaimie Henderson MD, professor of neurosurgery, who performed two of the three device-implantation procedures at Stanford Hospital. The third took place at Massachusetts General Hospital.

Henderson and Krishna Shenoy PhD, professor of electrical engineering, are co-senior authors of the study, which was published online Feb. 21 in eLife. The lead authors are former postdoctoral scholar Chethan Pandarinath, PhD, and postdoctoral scholar Paul Nuyujukian MD PhD, both of whom spent well over two years working full time on the project at Stanford.

“This study reports the highest speed and accuracy, by a factor of three, over what’s been shown before,” said Shenoy, a Howard Hughes Medical Institute investigator who’s been pursuing BCI development for 15 years and working with Henderson since 2009. “We’re approaching the speed at which you can type text on your cellphone.”

“The performance is really exciting,” said Pandarinath, who now has a joint appointment at Emory University and the Georgia Institute of Technology as an assistant professor of biomedical engineering. “We’re achieving communication rates that many people with arm and hand paralysis would find useful. That’s a critical step for making devices that could be suitable for real-world use.”

Shenoy’s lab pioneered the algorithms used to decode the complex volleys of electrical signals fired by nerve cells in the motor cortex, the brain’s command center for movement, and convert them in real time into actions ordinarily executed by spinal cord and muscles.

“These high-performing BCI algorithms’ use in human clinical trials demonstrates the potential for this class of technology to restore communication to people with paralysis,” said Nuyujukian.

Life-changing accident

Millions of people with paralysis reside in the United States. Sometimes their paralysis comes gradually, as occurs in ALS. Sometimes it arrives suddenly, as in Degray’s case.

Now 64, Degray became quadriplegic on Oct. 10, 2007, when he fell and sustained a life-changing spinal-cord injury. “I was taking out the trash in the rain,” he said. Holding the garbage in one hand and the recycling in the other, he slipped on the grass and landed on his chin. The impact spared his brain but severely injured his spine, cutting off all communication between his brain and musculature from the head down.

“I’ve got nothing going on below the collarbones,” he said.

Degray received two device implants at Henderson’s hands in August 2016. In several ensuing research sessions, he and the other two study participants, who underwent similar surgeries, were encouraged to attempt or visualize patterns of desired arm, hand and finger movements. Resulting neural signals from the motor cortex were electronically extracted by the embedded recording devices, transmitted to a computer and translated by Shenoy’s algorithms into commands directing a cursor on an onscreen keyboard to participant-specified characters.

The researchers gauged the speeds at which the patients were able to correctly copy phrases and sentences — for example, “The quick brown fox jumped over the lazy dog.” Average rates were 7.8 words per minute for Degray and 6.3 and 2.7 words per minute, respectively, for the other two participants.

A tiny silicon chip

The investigational system used in the study, an intracortical brain-computer interface called the BrainGate Neural Interface System, [CAUTION: Investigational Device. Limited by Federal Law to Investigational Use]. represents the newest generation of BCIs. Previous generations picked up signals first via electrical leads placed on the scalp, then by being surgically positioned at the brain’s surface beneath the skull.

An intra-cortical BCI uses a tiny silicon chip, just over one-sixth of an inch square, from which protrude 100 electrodes that penetrate the brain to about the thickness of a quarter and tap into the electrical activity of individual nerve cells in the motor cortex.

This is like one of the coolest video games I’ve ever gotten to play with.

Henderson likened the resulting improved resolution of neural sensing, compared with that of older-generation BCIs, to that of handing out applause meters to individual members of a studio audience rather than just stationing them on the ceiling, “so you can tell just how hard and how fast each person in the audience is clapping.”

Shenoy said the day will come — closer to five than 10 years from now, he predicted —when a self-calibrating, fully implanted wireless system can be used without caregiver assistance, has no cosmetic impact and can be used around the clock.

“I don’t see any insurmountable challenges.” he said. “We know the steps we have to take to get there.”

Degray, who continues to participate actively in the research, knew how to type before his accident but was no expert at it. He described his newly revealed prowess in the language of a video game aficionado.

“This is like one of the coolest video games I’ve ever gotten to play with,” he said. “And I don’t even have to put a quarter in it.”

The study’s results are the culmination of a long-running collaboration between Henderson and Shenoy and a multi-institutional consortium called BrainGate. Leigh Hochberg MD PhD, a neurologist and neuroscientist at Massachusetts General Hospital, Brown University and the VA Rehabilitation Research and Development Center for Neurorestoration and Neurotechnology in Providence, Rhode Island, directs the pilot clinical trial of the BrainGate system and is a study co-author.

“This incredible collaboration continues to break new ground in developing powerful, intuitive, flexible neural interfaces that we all hope will one day restore communication, mobility and independence for people with neurologic disease or injury,” said Hochberg.

Stanford research assistant Christine Blabe was also a study co-author, as were BrainGate researchers from Massachusetts General Hospital and Case Western University.

Bruce Goldman is a science writer for the medical school’s Office of Communication & Public Affairs.

Source Stanford University

 

References

1. High performance communication by people with paralysis using an intracortical brain-computer interface, Chethan Pandarinath, Paul Nuyujukian, Christine H Blabe, Brittany L Sorice, Jad Saab, Francis R Willett, Leigh R Hochberg, Krishna V Shenoy, Jaimie M Henderson. eLife 2017;6:e18554. doi: 10.7554/eLife.18554
2. Cortical control of a tablet computer by people with paralysis, Paul Nuyujukian, Jose Albites Sanabria, Jad Saab, Chethan Pandarinath, Beata Jarosiewicz, Christine H. Blabe, Brian Franco, Stephen T. Mernoff, Emad N. Eskandar, John D. Simeral, Leigh R. Hochberg , Krishna V. Shenoy, Jaimie M. Henderson. PLOS One. November 21, 2018 https://doi.org/10.1371/journal.pone.0204566
3. Restoration of reaching and grasping movements through brain-controlled muscle stimulation in a person with tetraplegia: a proof-of-concept demonstration, Ajiboye AB, Willett FR, Young DR, Memberg WD, Murphy BA, Miller JP, Walter BL, Sweet JA, Hoyen HA, Keith MW, Peckham PH, Simeral JD, Donoghue JP, Hochberg LR, Kirsch RF. Lancet. 2017 May 6;389(10081):1821-1830. doi: 10.1016/S0140-6736(17)30601-3. Epub 2017 Mar 28. Full text
4. A high performing brain-machine interface driven by low-frequency local field potentials alone and together with spikes, Stavisky SD, Kao JC, Nuyujukian P, Ryu SI, Shenoy KV. J Neural Eng. 2015 Jun;12(3):036009. doi: 10.1088/1741-2560/12/3/036009. Epub 2015 May 6. Full text
5. A Nonhuman Primate Brain–Computer Typing Interface, Paul Nuyujukian, Jonathan C. Kao, Stephen I. Ryu, Krishna V. Shenoy. Proceedings of the IEEE Volume: PP, Issue: 99. Print ISSN: 0018-9219 Online ISSN: 1558-2256DOI: 10.1109/JPROC.2016.2586967
6. Neural control of cursor trajectory and click by a human with tetraplegia 1000 days after implant of an intracortical microelectrode array, Simeral JD, Kim SP, Black MJ, Donoghue JP, Hochberg LR. J Neural Eng. 2011 Apr;8(2):025027. doi: 10.1088/1741-2560/8/2/025027. Epub 2011 Mar 24. Full text

What it’s like to type with my thoughts

By Jessica Hamzelou, New Scientist 21 February 2017

“I have no function below my collar bones,” says John Gardener*, who damaged his spinal cord in a fall 10 years ago. “I rely on devices,” he says.

Gardener, who lives in California, is unable to move any of his limbs. Although he is able to communicate by speaking, he is not able to use voice commands to do everything he wants to do on a computer, such as move a cursor or respond to emails. Instead, he uses an eye-tracking device to do these things. Gardener was keen to volunteer for a trial of a device that would make these tasks easier for him and others in similar situations. The device, developed by a collaboration called “Braingate” together with researchers at Stanford University, uses an implant to record brain activity. This then feeds a signal, through a wire, to a computer (see main story).

Gardener had the implant surgically placed onto his brain under anaesthetic, and felt no pain afterwards. The most testing part of the procedure was having the mount for the external components of the device screwed into his skull. “It was the sound, and the feeling of vibration, like someone was using a power drill,” he says.

Once Gardener had recovered from surgery, the team was able to train the device to recognise patterns of brain activity that appeared when Gardener thought about moving his limbs. This activity could then be used to direct a cursor on a screen.

Gardener’s first task was to move the cursor to a specific target. “When I first tried, the cursor wiggled and fell to the bottom of the screen,” he says.

Gardener learned to control the cursor by imagining his hand on top of a ball, rolling it one way or another. He quickly got the hang of it. When faced with a keyboard he was able to move the cursor across the screen to the desired letter and select it by thinking about clicking his fingers.

At the end of the first training session, Gardener was able to type on average 8.5 words per minute. “It’s at least five times as fast as the system I’ve been using,” he says.

His existing system is called a “SmartNav” – essentially a pair of glasses that monitors his head movements to allow him to control the movement of a cursor on a screen. But sending emails isn’t easy, says Gardener. “A short 20-word email takes 15 minutes,” he says.

At the moment, his brain implant is only being used for research. Before such devices hit the general market, they will have to be cordless, and be less labour intensive to set up.

The first of such devices will probably help people with paralysis to communicate, but others might help them control robotic limbs, or aspects of their home environment.

*Name has been changed

Source New Scientist

The study was funded by the National Institutes of Health (grants R01DC014034, R011NS066311, R01DC009899, N01HD53404 and N01HD10018), the Stanford Office of Postdoctoral Affairs, the Craig H. Neilsen Foundation, the Stanford Medical Scientist Training Program, Stanford BioX-NeuroVentures, the Stanford Institute for Neuro-Innovation and Translational Neuroscience, the Stanford Neuroscience Institute, Larry and Pamela Garlick, Samuel and Betsy Reeves, the Howard Hughes Medical Institute, the U.S. Department of Veterans Affairs, the MGH-Dean Institute for Integrated Research on Atrial Fibrillation and Stroke and Massachusetts General Hospital. Stanford’s Office of Technology Licensing holds intellectual property on the intercortical BCI-related engineering advances made in Shenoy’s lab. Stanford’s departments of Neurosurgery and of Electrical Engineering also supported the work. Shenoy and Henderson are members of Bio-X and the Stanford Neuroscience Institute.

Also see
Paralyzed Individuals Operate Tablet with Brain Implant IEEE Spectrum
Mind-reading typing tool for paralysed people is fastest yet New Scientist
A symphony of the mind – Special Projects Braingate
Brain Implant Allows Paralyzed Woman to Spell Out Messages With Her Mind Gizmodo
New Record: Paralyzed Man Uses Brain Implant to Type Eight Words Per Minute IEEE Spectrum
The brain-computer duel: Do we have free will? Science Daily
Listening in on the brain: A 15-year odyssey Stanford Medicine
BrainGate neural interface reaches 1,000-day milestone Brown University