The Dawn of Brain-Machine Interfaces
Brain implants are giving hope to the disabled and revolutionizing neuroscience
Matthew Nagle can move a cursor on a computer screen with only the power of his thoughts. It’s a remarkable feat for anyone, but especially momentous for Nagle, who is paralyzed from the neck down.
Millions of brain neurons fire when the 25-year-old wills his hand to move, but until last year, those messages were stranded shortly after leaving his brain. Now, a pill-sized sensor inserted into the motor area of Nagle’s brain extracts a tiny fraction of these signals and sends them to a computer. The computer translates the brain’s message, “move arm left” or “move arm right,” into cursor movement. Soon, Nagle doesn’t think about moving his arm, but moves the cursor directly.
Nagle’s is the first brain-machine interface (BMI) to be placed inside a human brain. The earliest BMIs were developed in animals, and systems that read brainwaves from the scalp rather than from surgical implants have been tested in humans for several decades.
While many hailed the human trial as a milestone, the real wow-factor is in the potential of this nascent technology. With further refinement, a BMI could give Nagle more than limited cursor movement. By feeding brain signals to muscle electrodes, prosthetic limbs, or an exoskeleton suit, it could someday restore motor functions—giving him the ability to pick up an object or even walk.
The BMI field is growing rapidly. While Nagle’s BMI takes signals from his brain, other implanted BMIs pass signals into the brain to restore lost senses such as hearing, touch, or vision. These developments are drawing from and inspiring advances in neuroscience, computer science, and engineering. BMIs are also giving scientists a new way to study the inner workings of the brain as we move, feel, remember, and think. The results are already changing our understanding of how the brain works.
THE LANGUAGE OF THE BRAIN
Reach out and grab an object near you. To carry out this seemingly simple task, your brain has to deal with rapidly changing information on arm position, arm speed, and hand grasping and lifting forces—all coordinated with changing feedback from your hand and eyes.
Scientists’ limited understanding of how this unfolds in the brain goes something like this: When you move your hand, there is a flurry of activity in the arm area of your motor cortex, located near the top of your head. Millions of neurons “talk” by firing action potentials— spikes of electrical current that sound like a buzzing or crackling when amplified and played through a speaker. The message is contained in the firing rate of individual neurons—100 spikes per second means something different than 5 spikes per second. And scientists have discovered that they can “hear” a single neuron talk by placing a tiny electrode right next to it in the brain tissue.
In the 1980s, work by Apostolos Georgopoulos, MD, PhD, now at the University of Minnesota, opened the door for motor BMIs. He implanted a microelectrode array in a monkey’s brain and recorded single-neuron firing rates as the monkey reached in different directions. Using a simple mapping, he correlated the arm’s changing three-dimensional position (x,y,z) to the shifting pattern of neural spiking.
“Georgopoulos showed fairly convincingly that broad populations of cells in the motor cortex participate in varying degrees,” says Bill Heetderks, MD, PhD, associate director of Extramural Science Programs at the National Institute of Biomedical Imaging and Bioengineering (NIBIB). That’s good news for building a motor BMI, Heetderks explains. To generate precise control signals you have to record the firings of hundreds of neurons in the brain. But you don’t have to be too strict about where you stick the electrodes. You’re likely to be listening to relevant neural chatter as long as you’re somewhere in the arm area.
EARLY ANIMAL WORK: THE CHANGING BRAIN
In 1999, a group of researchers including Miguel Nicolelis of Duke University, demonstrated a simple BMI system that allowed rats to control a lever with their minds. In 2000, Nicolelis extended this work by having monkeys remotely control a prosthetic arm located in a distant city.
Since then, several groups have trained monkeys to move cursors or prosthetics in two and three dimensions using BMI systems. In the lab run by Andrew Schwartz, PhD, professor of neurobiology and bioengineering at the University of Pittsburgh, monkeys can feed themselves slices of orange and zucchini with a robotic arm wired to their brains.
The monkeys assimilate the cursors or arms with ease, moving them as fluidly as we would a paintbrush or a tennis racket. “It surprises me how quickly cells adapt to thinking ‘my purpose in life is to move this dot on the screen’ as opposed to ‘my purpose in life is to move this limb,’” Heetderks says.
Amazingly, with feedback, the brain learns to adjust the firing pattern of neurons to improve BMI handling. This finding firmly quashes the old theory that brain patterns are fixed in childhood. Brain plasticity will smooth the progress of BMI research, Heetderks says.
Some researchers even speculate that they could scramble the code that transforms neural spiking to cursor position and, with enough training, the brain could relearn how to fire its neurons to control the cursor. So far, such tests in monkeys have flopped as monkeys quickly get frustrated and give up.
The plasticity of the brain has surprised scientists so much that it is no longer ludicrous to wonder if neurons from a nonmotor area of the brain, such as the visual cortex, could drive a prosthetic arm.
FROM MONKEYS TO HUMANS: THE LATE STONE AGE
In 2001, four researchers from Brown University formed a company with the mission of bringing brain-interface technology to humans. The company, Cyberkinetics, Inc., of Foxborough, Massachusetts, launched its first human trial in 2004 with Matthew Nagle.
“We thought that there were certain limitations with animals that we couldn’t overcome until we got to humans,” says Cyberkinetics cofounder Nicholas Hatsopoulos, PhD, now assistant professor of organismal biology and anatomy at the University of Chicago. You cannot ask monkeys to do complex cognitive tasks or tell you what they are feeling, he explains.
Surgeons implanted a tiny array of 100 electrodes, each narrower than a human hair, into Nagle’s motor cortex. These electrodes act like 100 tiny microphones suspended in the brain tissue, each able to hear up to a few neurons. Surgery took less than three hours, and there were no complications.
To calibrate the system, Nagle is told to think about moving his hand left or right, as a computer associates the brain pattern with direction. Once calibrated, the computer reads signals out of Nagle’s brain and moves the cursor accordingly. Soon, Nagle moves the cursor without thinking about his arm. He can check a mock email program, play a simple video game, even draw a crude circle. And he can whistle or talk—tasks that fire other neurons in his brain—without losing control of the cursor.
“The patient was thrilled initially,” Hatsopoulos says. “Here’s a case where a patient who hasn’t moved anything in several years can actually move something intentionally.” But Nagle soon wanted more, Hatsopoulos admits. “I can’t say it’s perfect.”
Nagle can only use the system when technicians are available. The system has to be calibrated for each use, due to implant shifting; also, Nagle’s head has to be plugged into the computer via a bundle of wires. Cyberkinetics is working on automating the system and making it wireless. Besides Nagle, a second patient has been implanted with the system, but no data are available about him yet. Heetderks calls the trial a “real milestone.” It shows for the first time that the system works in somebody who is paralyzed, not just in intact monkeys, he says.
But not everyone agrees that the Cyberkinetics trial was a milestone. “We are appalled by what’s going on,” says Miguel Nicolelis, MD, PhD, of Duke University, who did much of the pioneering work on BMIs in animals. Nicolelis is professor of neurobiology, biomedical engineering, and psychological and brain sciences at Duke and co-director of Dukes Center for Neuroengineering. He calls the trial a marketing stunt. Because Nagle could work a computer without the BMI (mechanisms exist for moving a cursor using one's voice or eyes), the benefits of the implant do not outweigh the risks of neurosurgery, he says. Opening the skull and inserting electrodes into the brain is the only way to record neural spiking, but it risks infection or bleeding.
Nicolelis contends that the trial was also scientifically unnecessary, since imaging studies already showed that quadriplegic patients have a working motor cortex. “We didn’t need to put electrodes in their heads to answer that.”
Nicolelis has his sights set on human trials in the next one to three years, but he is waiting for a few key pieces: he wants a completely wireless system that can drive a prosthetic arm with several degrees of freedom. He says no system out there is ready for prime time.
“Implantation methods are probably in the late stone age,” agrees Jonathan Wolpaw, MD, a research physician at the Wadsworth Center at the New York State Department of Health, in Albany. “Basically what we do now, is we put nails in the brain. They’re very fine, tiny, well-machined nails, but they are in fact nails.”
Nagle is not the first human to control a cursor with his thoughts, but previous systems had much lower resolution. In 1998, a stroke victim named Johnny Ray, who could communicate only by blinking, received a two-electrode implant that allowed him to select letters and icons with his brain. The device read from only a few neurons, unlike the array in Nagle’s brain.
Additionally, since 1991, many people have driven cursors with their brainwaves, which are recorded noninvasively. The noninvasive approach is more commonly called a brain-computer interface (BCI).
The firing of millions of neurons generates an electrical noise that can be recorded by electrodes placed on the scalp. These collective signals do not give precise information about three-dimensional arm trajectories, but can be used to control a computer cursor. The impetus of this technology is helping people who are “locked-in,” explains Wolpaw, who heads the BCI program at the Wadsworth Center. Locked-in patients have normal brain function but no muscle control—giving them no means of communicating with the outside world. The condition can develop as a result of a brainstem stroke, a high spinal cord injury, or amyotrophic lateral sclerosis (ALS or Lou Gehrig’s disease).
BCI systems unlock these patients,
allowing them to move a cursor, surf the internet, and select letters at a slow rate—up to 1-2 words per minute. “If they can do even a couple of letters a minute, that’s a big step because they’ve had some control returned to them,” Wolpaw says.
The Wadsworth BCI is an electrode cap that fits over the skull and is wired to a computer. Patients begin training by imagining movement—running, playing baseball, sticking out their tongue—to depress idling rhythms in the brain (known as mu and beta rhythms). A computer moves a cursor up or down according to the strength of these rhythms. After hundreds of trials, most people can voluntarily move the cursor up and down or right and left.
Recently, however, Wolpaw’s group has achieved two-dimensional cursor movement. Using independent control signals from mu and beta rhythms, patients can steer a cursor to one of eight target points on a screen, with up to 90 percent accuracy.
The brain is now achieving its intent through nonmuscular output, Wolpaw says. But control requires concentration, making it hard to multitask.
It’s a problem that Heetderks says is a major drawback of noninvasive systems. “If there is going to be a breakthrough in that area, it’s going to be the ‘walking and chewing gum at the same time’ issue,” he says.
Despite this limitation, Wolpaw’s recent successes have surprised invasive BMI researchers. “I would claim that they’ve been able to do better with their noninvasive electrodes than the vast majority of the invasive electrode work,” says Krishna Shenoy, PhD, who works on invasive BMIs as an assistant professor of electrical engineering at Stanford University. “It’s harder to motivate monkeys,” he adds.
Invasive technology gives quicker learning and more resolution than noninvasive systems, but these advantages have not been reflected in significantly better performance, Shenoy says. He calls it a “kick in the butt” for the invasive community to optimize their systems.
Electrodes have a limited shelf life in the brain; they get gummed up with immunological factors and stop picking up signals after anywhere from a month to a few years. A big challenge for invasive BMIs is to make implants reliably last a decade, the benchmark for other implantables like cardiac stents. “It’s a bit risky for the whole field to be betting on it, but there are good people working on that,” Shenoy says.
Researchers will also need to fit all the electronics for sensing, amplifying, and decoding signals on one tiny implantable chip that can wirelessly transmit to a prosthetic arm or other device. If you have all this stuff in the brain, you’d want a pretty dexterous prosthetic arm, too, Shenoy says, but it does not exist yet.
Though BMI research has focused on arm movement, the Holy Grail of motor BMIs would be locomotion, Nicolelis says. Eventually he hopes BMIs will drive an exoskeleton suit to restore walking in someone who is paralyzed. For now, he is working on taking brain signals from rats on a treadmill to drive a robot with arms and legs. To do this successfully, Nicolelis must pull many different control signals from multiple arrays placed in different regions of a rat’s brain. Because he’s reading from multiple areas, he’s also been able to show something surprising about what happens in the brain when it commands an arm to move.
Current textbooks present a hierarchical picture of arm reaching; in short, the posterior parietal cortex (deep in the brain) launches the intention to move; then the pre-motor cortex draws up an exact plan for moving; then the motor cortex moves the muscles. However, reading from these three brain regions simultaneously, Nicolelis has found a different picture: the spiking sometimes appears directly in the motor cortex or simultaneously in all three areas. Touch follows a similarly surprising path, he’s found.
“My students were dying to see a beautiful linear sequential activation pattern—they wanted a Science paper. I said: ‘Sorry kids, we don’t see it.’ And we got a Science paper anyway.”
CONVERSATIONS WITH THE BRAIN
The implant in Nagle’s brain was originally developed at the University of Utah for a different purpose: to send signals into the brain for artificial vision. Talking to the brain is not much different from listening to it—you just have to know the language, the pattern of electrical spiking that causes the brain to hear and see.
More than 50,000 people have already had a cochlear implant placed in their ears. This device transforms sound waves into electric pulses that the brain can understand. Current implants have 22 electrodes that directly stimulate the neurons of the auditory nerve in the inner ear. That’s enough for some people who were once profoundly deaf to talk on the telephone.
Vision researchers hope to someday match this success, but the eye is more complex than the ear. To see the letter “E”, light reflected off the letter is focused onto 100 million photoreceptor cells (cones and rods) at the back of your eye, which convert light into a pattern of electric pulses. Adjacent cells in your eye then compress the image 100-fold, process it and route it down the optic nerve to your brain. When the message reaches the visual cortex, you see “E”.
Vision BMI systems bypass breaks in this pathway by feeding digitized pictures from an eyeglass-mounted video camera to an implant. Implants in the retina bypass degenerated photoreceptors; implants in the visual cortex bypass damage to the optic nerve. These systems will probably not work in those blind since birth, since their brains never learned how to see.
Typical implants transmit a pattern of electric pulses that match the pixels of light and dark from the incoming image. More electrodes give more resolution. The brain recognizes this pattern because photoreceptors in the retina and neurons in the visual cortex are organized to correspond to the lay of the land. For example, the top right of your visual field is “seen” by a cluster of neurons in a particular location in your visual cortex.
Since as early as 1978, scientists at the Dobelle Institute of Long Island, New York, have placed electrodes in the visual cortex of a handful of blind patients. Patients see phosphenes— bright spots of light that scientists deem the “starry night effect”—that confer enough information to avoid bumping into objects.
Second Sight, Inc. of Sylmar, California, has recently implanted a 16-electrode array on the surface of the retina (epi-retinal) that directly stimulates the optic nerve. Patients can recognize patterns, horizontal and vertical lines, and direction of motion, says Daniel Palanker, PhD, who works on artificial vision as an assistant professor of ophthalmology at Stanford University.
“It’s a big step forward,” he says. Palanker has plans for a sub-retinal implant, to directly replace damaged photoreceptors; this design enters the vision pathway at its earliest point, without missing the eye’s natural image processing. Such an implant could confer 20/80 visual acuity, he estimates, but it is still early in development.
To truly recreate vision, the brain needs to receive multiple images per second. Similarly, to control a prosthetic limb, the brain has to send out multiple control signals per second. In both cases, the challenge is to interpret huge amounts of streaming data that are noisy, high-dimensional and changing over time, says Michael Black, PhD, professor of computer science at Brown University; he teaches a computationally oriented class on BMIs at Brown.
“You have to sit down and try to figure out: how does the activity of these hundreds of cells relate to these tens or hundreds of parameters of behavior or motion. That requires machine learning and computer science techniques: visualization, modeling, a lot of probability and statistics.”
The complexity of the human brain presents a great opportunity for computer scientists, he says. “Everyone’s running to get involved in computational molecular biology at the moment, but the brain is really a growth area for computer science,” he says.
BRAVE NEW WORLD
Unlocking the brain—the very seat of human consciousness—is both fascinating and frightening. Our imaginations quickly slip the bounds of science and drift into science fiction, conjuring up brain chips that boost memory and intelligence, drive cars and planes, and leave us susceptible to mind-control.
The reality is that brain interfaces are very much in their technological infancy, and the focus is medical. Scientists are working on implantable chips to enhance memory, but the technology—which is still being tested in cells—is aimed at those with Alzheimer’s disease or head injury. The Defense Advanced Research Projects Agency (DARPA) heavily funds BMI research with a view toward military uses, but the agency isn’t chasing cyborg soldiers so much as prosthetic limbs for amputees coming back from war.
“It’s important to keep that perspective in mind in terms of what we’re trying to do here,” Heetderks says. “It’s to improve quality of life, not to make some kind of a Super Human.”