Brain-computer interface (BCI) research is producing new augmentative communication and control technology for people with severe neuromuscular disorders, such as amyotrophic lateral sclerosis (ALS), brainstem stroke, cerebral palsy, and high-level spinal cord injury (Wolpaw, 2009). The primary goal is to give these extremely disabled users, who may be unable to breathe or move their eyes, nonmuscular communication and control capabilities so that they can express their wishes to caregivers, use word-processing programs or other software, or even control neuroprostheses. Present-day BCIs determine the intent of the user by analyzing electrical signals recorded from the scalp (electroencephalography [EEG]), or from electrodes surgically implanted on the cortical surface (ECoG) or within the brain (neuronal action potentials [spikes] or local field potentials [LFPs]). Alternatively, BCIs may determine the intent of the user by analyzing signals that reflect brain metabolic activity and are recorded by functional magnetic resonance imaging (fMRI), functional near-infrared spectroscopy (fNIRS), or positron emission tomography (PET). Simple EEG-based BCI systems are just beginning to come into everyday use by people with ALS.

Most BCI research has been based on four assumptions that originate from the common misconception that BCIs read minds. The first assumption is that an intended action is fully represented in the cerebral cortex, that the cortex contains information that completely defines all important aspects of the action. This assumption implies that a properly designed analysis of cortical electrical activity can describe the intended action accurately and in detail. The second assumption is that high-resolution cortical signals (i.e., neuronal action potentials, or spikes) are the best measures of cortical activity and can provide the most accurate and detailed description of an intended action. The third assumption, which derives from the first two, is that the best BCI is one that records spikes and decodes them to determine the person's intent. Finally, the fourth (and often unacknowledged) assumption is that effective BCI performance does not require continued learning or adaptation by either the user's brain or the BCI system.

In reality, none of these assumptions is consistent with existing and emerging knowledge. In reality, the activity that underlies an intended action is distributed from the cortex to the spinal cord. Many different cortical areas participate, as do subcortical, midbrain, brainstem structures, and spinal cord circuits. The final output that emerges from spinal motoneurons to activate the muscles that produce the action reflects the contributions of numerous regions. The primary motor cortex, which is the focus of most BCI development efforts, is only one of the players in this production (albeit a major one). In addition, the contributions of each individual area change continually as the CNS adapts to achieve and maintain optimal performance. As a result, the signals recorded from a single area provide an incomplete and inconstant picture of the intended action. As a result, methods for tracking both spontaneous and adaptive changes in the signals being measured by the BCI, and for encouraging and guiding adaptation so as to improve the correlation between the signals and the BCI user's intention, are essential components of effective BCI development.

Furthermore, it is not clear which brain signals can best reflect an action. Neuronal action potentials are high-resolution signals that have certainly proved very useful in basic-science studies of CNS function. However, the question as to which of the available signals, from EEG to spikes, are best for BCI uses is an empirical one, and can be resolved only by investigation. Indeed, at present, and contrary to common expectations, the level of movement control achieved in human subjects by an EEG-based BCI is comparable to that achieved by a cortical neuron-based BCI. Figure 1 illustrates this surprising finding. It shows the distributions of target-acquisition times for two studies of center-out two-dimensional cursor control in humans, one using an EEG-based BCI (Wolpaw & McFarland, 2004) and the other using a cortical neuron-based BCI (Hochberg et al., 2006). Also shown is the distribution of times for normal muscle-based joystick control of the cursor. The two BCI studies had comparable protocols, and they produced almost identical distributions of target-acquisition times. Compare also the videos at http://www.bciresearch.org/html/2d_control_8tn.html and http://www.nature.com/nature/journal/v442/n7099/suppinfo/nature04970.html Both BCIs are slower and much less consistent than the joystick, despite the substantial BCI training given in each study. Such inconsistency is typical of BCI studies (e.g., compare supplementary Videos 1 and 8 of Hochberg et al.). The most remarkable feature of Figure 1 is the very close similarity of the distributions for the two types of BCI, one being a BCI that used single-neuron activity recorded within the cortex and the other being a BCI that used EEG recorded from the scalp. This similarity suggests that their inconsistency was not due signal resolution (which was high for the neuronal BCI and low for the EEG BCI), but rather to one or more other factors that limit both BCI methods.

Brain–Computer Interface Research Comes of Age: Traditional Assumptions Meet Emerging Realities

All authors

Jonathan R. Wolpaw

https://doi.org/10.1080/00222895.2010.526471

Published online:

20 November 2010

FIGURE 1 Distributions of target-acquisition times (time from target appearance to target hit) on a two-dimensional center-out cursor-control task for a conventional joystick (solid), an EEG-based brain–computer interface (BCI; dashed), and a cortical neuron-based BCI (dotted). The EEG- and neuron-based BCIs have similar distributions, and both are slower and far less consistent than the joystick. For both BCIs in some trials, the target is not hit even in the 7 s allowed. Such inconsistency is typical of movement control by present-day BCIs, regardless of which brain signals they use. Joystick data and neuron-based BCI data from Hochberg et al. (2006). EEG-based BCI data from Wolpaw and McFarland (2004).

Display full size

Although motor control has traditionally been thought to be highly localized in cortex (Woolsey, 1958), recent work has shown that movements are controlled by distributed cortical networks that include many areas (Aflalo & Graziano, 2006; Dum, 2005; Dum & Strick, 2002; Ledberg, Bressler, Ding, Coppola, & Nakamura, 2006; Meier, Afalo, Kastner, & Graziano, 2008), which interact through synchronous oscillations (Bullmore & Sporns, 2009; Salinas & Sejnowski, 2001; Sejnowski & Paulsen, 2006; Zhang, Wang, Bressler, Chen, & Ding, 2008). This new insight suggests that present BCI movement control may be inconsistent because it uses signals from only one cortical area. EEG-based control has focused on signals over sensorimotor cortex (e.g., Wolpaw & McFarland, 2004), and neuron-based control has usually focused on neurons from a few cubic mm of motor cortex (e.g., Hochberg et al., 2006). The inconsistent BCI performances seen in Figure 1 may indicate the limits of the adaptation possible for a single area, whether the activity in that area is detected as EEG or as neuronal action potentials. The consistency of the joystick data in Figure 1 may result from the complementary contributions of many brain areas. If so, BCI consistency may be improved by recording signals from multiple cortical areas and using appropriate adaptive algorithms to combine them to control movement. By removing the limitation imposed on BCIs that use a single area, this strategy could permit the control capacities of the various signal types to be more fully realized and might achieve more consistent performance.

In sum, BCIs do not read minds but, rather, they allow their users to develop new skills. Unlike normal motor skills, these new BCI skills are executed by brain signals rather than muscles. Nevertheless, similar to normal motor skills, their acquisition and maintenance depend on the continual interactions of the CNS with the outcomes produced by its signals, and they reflect the activity-dependent adaptive plasticity that these interactions induce in the CNS. Thus, the proper objective in BCI development is to find signals the user can control, maximize that control, and translate it into action accurately and reliably.

This reality implies that BCI research should explore the full range of available brain signals to find those signals that people can best control, and should focus on developing signal-analysis methods and user training protocols that facilitate and increase that control. At the present time, it is clear that the full range of electrical signals, from EEG, through ECoG, to local field potentials and neuronal action potentials, warrants careful evaluation for BCI use.

The unreliable performance typical of present BCI methods, regardless of whether they use high- or low-resolution recording methods, is emerging as perhaps the single most difficult problem. This problem raises fundamental questions about the determinants of consistent brain function and about the demands on that function posed by BCIs. Its solution is particularly important for the realization and dissemination of BCI systems of significant practical value for people with severe motor disabilities.

BCI research and development is coming of age. The assumptions that dominated its infancy came from other fields and served other purposes, and they are now dropping away as the field confronts and engages its own key issues. The results of this critical transition should greatly affect the ultimate scientific significance and practical success of this exciting new field.