Single Trial Predictors for Gating Motor-Imagery Brain-Computer Interfaces Based on Sensorimotor Rhythm and Visual Evoked Potentials

For brain-computer interfaces (BCIs) that utilize visual cues to direct the user, the neural signals extracted by the computer are representative of ongoing processes, visual evoked responses, and voluntary modulation. We proposed to use three brain signatures for predicting success on a single trial of a BCI task. The first two features, the amplitude and phase of the pre-trial mu amplitude, were chosen as a correlate for cortical excitability. The remaining feature, related to the visually evoked response to the cue, served as a possible measure of fixation and attention to the task. Of these three features, mu rhythm amplitude over the central electrodes at the time of cue presentation and to a lesser extent the single trial visual evoked response were correlated with the success on the subsequent imagery task. Despite the potential for gating trials using these features, an offline gating simulation was limited in its ability to produce an increase in device throughput. This discrepancy highlights a distinction between the identification of predictive features, and the use of this knowledge in an online BCI. Using such a system, we cannot assume that the user will respond similarly when faced with a scenario where feedback is altered by trials that are gated on a regular basis. The results of this study suggest the possibility of using individualized, pre-task neural signatures for personalized, and asynchronous (self-paced) BCI applications, although these effects need to be quantified in a real-time adaptive scenario in a future study.

1. Introduction

The goal of a brain-computer interface (BCI) is to interpret the intention of a subject directly from neuronally generated signals, in order to develop a direct communication pathway between the brain and the external environment. Successful BCI implementation has enormous potential for improving the quality of life of certain disabled groups, including those with spinal cord injury and locked-in syndrome induced by stroke or late-stage amyotrophic lateral sclerosis (Birbaumer, 2006).

Motor-imagery is a specific type of mental rehearsal applied for BCI use, in which the intention of the subject is determined by their imagination of movement of a specific part of the body e.g., feet, arms, or tongue. Imagination of movement in these limbs results in changes in the sensorimotor rhythms (SMR), which are recorded using electroencephalography (EEG) from motor and premotor cortical areas. SMRs are commonly defined in two frequency bands, mu and beta. The actual frequencies are subject specific, but typically occur around 8–12 and 18–25 Hz, respectively (Pfurtscheller and da Silva, 1999; McFarland et al., 2000). The modulation of these rhythms in response to overt as well as imagined movement has been well documented (McFarland et al., 2000; Miller et al., 2010), and their utility in controlling BCIs of substantial complexity has been demonstrated (McFarland et al., 2010).

Success in controlling an SMR-based BCI system depends on the user's ability modulate these motor rhythms from a baseline state. The reasons for why some individuals are able to do this well and why others are not is an area of active research, with differences in imagery ability, baseline oscillations, brain structure, and motivation being implicated as possible factors (Halder et al., 2013). One such predictor of BCI performance is the level of baseline SMR amplitude, which has been shown to be positively correlated with the success of the individual in completing the task (Blankertz et al., 2010). Within individuals, high SMR amplitude has further been shown to be correlated with more successful single trial performance (Maeder et al., 2012).

EEG oscillations such as the SMR have been given roles as modulatory gating mechanisms for information transfer between the cortex and subcortical structures such as the thalamus (Niedermeyer and da Silva, 1999), and may explain the dependence of imagery performance on these rhythms. Using transcranial magnetic stimulation (TMS), low amplitude oscillations were shown to be representative of an excitable state of the motor cortex (Sauseng et al., 2009). These findings indicate that the regulation of SMRs, specifically the mu rhythm, is likely to condition the cortex for forthcoming perception or action (Thut and Miniussi, 2009). A BCI that relies on changes in mu rhythm may be affected by pre-trial mu amplitude simply because high mu before the trial offers opportunity for larger decreases from baseline level. However, the ability to form motor imagery may also be modulated by oscillations representing motor network excitability.

Early work by Bishop demonstrated that the phase of ongoing cortical oscillations partially predicted the seemingly random variations in evoked potentials due to visual stimulation (Bishop, 1932). This demonstration led to the hypothesis that these oscillations represented cyclical cortical excitability. Later studies showed how these periods of excitability could be demonstrated by consistent changes in perceptual or motor thresholds. Phase and amplitude of the occipital alpha at the time of visual stimulus presentation has been shown to be predictive of stimulus detection (Niedermeyer and da Silva, 1999; Thut and Miniussi, 2009). Somatosensory cortical evoked responses to painful stimuli are also facilitated by the amplitude of 8–13 Hz mu motor rhythms (Ploner et al., 2006). Further evidence comes from the work of Kruglikov and Schiff (2003), who showed that auditory evoked potential morphology is a function of broadband EEG phase.

Visual evoked potentials (VEP)s are present in the scalp EEG as a result of the visual cue presented to the user in a synchronous BCI task. In motor-imagery tasks, this evoked response may be considered a source of artifact and removed or ignored in further processing. However, these evoked responses have been shown to be modulated by region of fixation (Hillyard and Anllo-Vento, 1998; Treder and Blankertz, 2010), and serve as a marker of visual attention. Visual attention is controlled by a distributed network of cortical and subcortical areas which act to provide “bias signals” that enhance or suppress the responses to visual stimuli (Hillyard and Anllo-Vento, 1998). The distinction between fixation and attention in the context of a BCI is an important one. In a cued BCI paradigm, attention is required for the subject to understand the cue and react to it, but fixation on that cue is not absolutely necessary (Treder and Blankertz, 2010). Of the components of the VEP, the N1, N2c, and P3b peaks have been shown to be influenced by visual attention (Niedermeyer and da Silva, 1999; Treder and Blankertz, 2010; Railo et al., 2011). Increases in amplitude of the first two components and a decrease in the latency of the P3b have been shown to be correlated with visual attention. Error-related potentials are also present in the EEG during BCI tasks. These evoked components of the EEG are generated after viewing feedback incongruent with the target response in both cursor movement and P300 tasks (Spüler et al., 2012; Iturrate et al., 2013). Although post-trial retraction based on error detection has been shown to improve BCI performance (Spüler et al., 2012), in this study, evoked potentials prior to the primary task were the focus.

The aim of this study was to identify intra-individual predictors of single trial BCI success. By finding brain signatures which correlate with improved performance, we can attempt to increase the accuracy and bit rate achieved with the interface. One possible way to do this is by gating trials with low predicted performance. By gating, we prevent classification from taking place if the signature of brain activity predicts poor future performance on the task. We evaluated the utility of three of these brain signatures, which we term “gating variables”: the amplitude and phase of ongoing motor rhythms, and the visually evoked potential produced in response to the cuing mechanism. We show that mu amplitude at the beginning of the trial, which is generally discarded during motor-imagery feature selection, is correlated with performance within a subset of subjects. Aside from one subject, the visual evoked potential did not associate with subsequent imagery performance, demonstrating that fixation to the cue does not modulate level of success on the task. Despite the potential for gating trials using mu amplitude, the utility of these features in an offline gating simulation was limited in its ability to produce an increase in device throughput.

2. Materials and Methods

2.1. Experimental Setup

A commercial EEG recording system (Guger technologies, www.gtec.at) was used to acquire data from subjects. Data were sampled at 256 Hz and band pass filtered at 0.1–30 Hz. This bandpass range was chosen to preserve VEP components and oscillations in the mu band. Data were recorded within the Simulink environment in MATLAB (version 2009b) and stored on a notebook computer (Dell Latitude E6400) running Windows XP. Subjects were seated in a chair facing an LCD monitor which displayed cuing and feedback information. The experimental protocol was approved by the Institutional Review Board of Penn State University.

2.2. BCI Paradigm

Nine right-handed volunteers, all male with ages ranging from 18 to 37, participated in a cue-paced, one-dimensional center-out motor imagery task. Participants were recruited from the local area by means of a flier with no restrictions on gender. Channels FC3, FC4, C5, C3, C1, C2, C4, C6, CP3, CP4, P5, P3, P1, P2, P4, P6, PO3, and PO4 were recorded, in addition to three electrooculogram (EOG) electrodes placed on the lateral canthi as well as just above the nasion. All channels were referenced to linked earlobes, and ground was placed at Fpz. Each subject performed four sessions over a 2 week period. Each session lasted ~1.5 h.

During each session, the subject performed five runs of 60 trials each, divided equally between left, right, and no-target cues that were presented in a randomized sequence. Data from the first run of each session, the training run, was used to construct a classifier so the remaining four testing runs could be used to give feedback to the subject as they performed the task. After each feedback run, the classifier was updated with data corresponding the twenty additional trials of the left and right classes.

In each trial, the subject was cued by an arrow pointing in the left or right direction. Arrows were displayed symmetrically in the visual field to minimize asymmetry in the evoked potential due to cue type. This asymmetry, which can last up to 500 ms and is observable in C3 and C4 electrode locations over the motor cortex (Dias et al., 2010), can falsely contribute to BCI classification accuracy independent of user-driven modulation (Dias et al., 2009). Subjects were instructed to perform imagery of an object-oriented grasping action for the hand corresponding to the direction of the arrow being displayed. If no arrow appeared, the subjects were informed to relax and were given no visual feedback.

In each trial of the training run, a fixation cross would first appear at which time the subject would relax. If this were followed by an arrow cue 1 s later, the subjects were instructed to perform imagery while the arrow remained on screen for 2 s. Each trial was followed by a random inter-trial period of 1–2 s. During the testing runs, a target appeared on the far side of the screen in the direction of the arrow, and during the 2 s in which they performed imagery, a cursor moved on the screen to provide the subject with feedback.

The output of the classifier served as feedback which informed the subject how closely their EEG signals of intent matched templates for left and right cues developed in the training run. This was done by summing the squared distance D of the band power in trial n from the m ∈ (left, right) template band power over all time points t and features f (Equation 1). Each template μ consisted of mu and beta band powers from Laplacian-transformed channels C3, C4, P3, and P4 during the period of motor imagery, making up f = 8 total band power features. In the calculation of distance,

$\begin{array}{lll}\hfill D_{n,m}(t)={\displaystyle \sum _{f=1}^8}{\left(x_{n,f}(t)-{\mu }_{m,f}(t)\right)}^2,& & \left(1\right)\end{array}$

x_{n, f} represents a band power feature from a single trial. The feedback at each time point was calculated as the log ratio of the left distance over the right distance (Equation 2), as

$\begin{array}{lll}\hfill Feedbac{k}_n(t)=Feedbac{k}_n(t-1)+\text{log}\left(\frac{D_{n,left}(t)}{D_{n,right}(t)}\right).& & \left(2\right)\end{array}$

This control algorithm drove the cursor to the left when this ratio was less than one, and to the right when the ratio was greater than one. Feedback was accumulated over the length of the trial and then reset to the center of the screen at the beginning of the next trial. Z_n, the success of the subject at performing imagery during trial n, was defined according to Eqution (3),

$\begin{array}{cc}\begin{array}{l}{Z}_n=\{\begin{array}{ll}Feedbac{k}_n(T),\hfill & \text{if Cue}=right\hfill \\ -Feedbac{k}_n(T),\hfill & \text{if Cue}=left,\hfill \end{array}\hfill \end{array}& \left(3\right)\end{array}$

where the time index T marks the end of the trial. This Euclidean distance-based classifier, unconventional in the LDA- and SVM- dominated field of BCI, was used due to a previous study which compared the performance with conventional classifiers (Geronimo et al., 2011). Over the four sessions, each subject completed 16 test runs consisting of 960 trials, with 320 trials each for left, right, and no-target cues. The data belonging to the roughly 320 trials of left and right cues were analyzed offline using the methods described below.

2.3. Artifact Correction

The first step in offline data preprocessing was removal of eye-related artifacts that resulted from blinking or eye movement. This was a two-step process including artifact reduction and trial rejection. Artifact reduction was accomplished by linear regression (Schlögl et al., 2007). This least-squares method assumes the linear superposition of neural and artifact sources to produce the measured signal. Assuming the independence of the artifact sources and the neural sources, data can be used to find a weight matrix, which represents the projection of noise sources onto neural sources. We used the 18 EEG channels as our recorded signal Y, and the three EOG channels as the noise sources U, to find the weight matrix W, and solve for decontaminated neural sources S (Equation 4).

$\begin{array}{ll}{\text{Y}}_{18\times T}={\text{S}}_{18\times T}+{\text{W}}_{18\times 3}·{\text{U}}_{3\times T}.& \left(4\right)\end{array}$

Here, T is the length of the data segment from which the weight matrix is calculated. In practice, this method may be suboptimal if there is significant leakage of the task-relevant EEG data into the designated noise channels, rendering the assumption of independence void. To minimize this effect, we solved for W using data which was sampled from when the EOG channel exceeded 75 μV, as in the case during a blinking event.

Following artifact reduction, trials were rejected if channel FC3 displayed absolute amplitude of greater than 50 μV. To control for artifact in unintentional movements, bipolar electrodes were placed on the forearms of a subset of the subjects to record electromyographic (EMG) muscle activity during the recording session. EOG and EMG were both analyzed offline to rule out possible contamination from overt eye and arm movements.

2.4. Extraction of Gating Variables

2.4.1. Amplitude and Phase of Peri-Stimulus mu

Data in channels C3 and C4 were filtered with a Laplacian spatial filter to localize the mu rhythms specific to the motor cortex (McFarland et al., 1997). Peak mu frequency in these channels was found for each subject using the multitaper spectral analysis method (Mitra and Bokil, 2008). The EEG data in these channels were filtered using a zero-delay filter in a 2 Hz range around the peak mu frequency. A Hilbert transform was applied to this band-limited signal to generate the analytic signal S_a(t), comprising a real part S(t) made up of the original data, and an imaginary part H(t), its Hilbert transform (Equation 5),

$\begin{array}{ll}{S}_a(t)=S(t)+iH(t).& \left(5\right)\end{array}$

The amplitude of the signal A(t) was calculated as the Euclidean norm of the real and imaginary parts of the analytic signal (Equation 6).

$\begin{array}{ll}A(t)=\sqrt{S^2(t)+H^2(t)}.& \left(6\right)\end{array}$

The instantaneous phase ϕ(t) was calculated as the four quadrant inverse tangent of the ratio of the imaginary part of the analytic signal to the real part (Equation 7). Phase ranged from −π to π.

$\begin{array}{ll}\varphi (t)=arctan\frac{H(t)}{S(t)}.& \left(7\right)\end{array}$

For each trial n, the amplitude and phase of mu in C3 and C4 at the time of cue presentation were extracted (Figure 1). Mu amplitude was the first gating variable. Linear regression between trial success and amplitude was used to determine association. We assumed that the success at the end of the trial was a linear function of the natural-log-transformed peri-stimulus mu amplitude (Equation 8). Log transformation was performed on the spectral features to enforce the normality of the data as well as reduce outlier effects (Gasser et al., 1982).

$\begin{array}{ll}{Z}_n=\alpha \times \text{ln}(A({\tau }_n))+\beta .& \left(8\right)\end{array}$

FIGURE 1

Figure 1. Phase and amplitude extraction of a single EEG trial. (A) Trial n of EEG data. (B) Signal was filtered in the subject-specific mu band. (C) The phase of the band-limited signal at τ_n, the time of cue presentation, is marked by the vertical dashed line. (D) Amplitude of the band-limited signal.

In this equation, α and β are the parameters of the regression, and A(τ_n) is the magnitude of the mu rhythm at the time of the cue of trial n. We used the slope of the regression α, as the measure of correlation between mu amplitude and trial success.

Mu phase was the second gating variable. We assumed trial success was a cosine function of the phase of the mu rhythm.

$\begin{array}{ll}{Z}_n=\gamma \times cos(\varphi ({\tau }_n)-\delta )+ϵ.& \left(9\right)\end{array}$

Here, γ, δ, and ϵ are regression parameters specifying the amplitude, phase, and offset of the fitted cosine, and ϕ(τ_n) represents the phase of the mu rhythm at the time of the cue of trial n. We specified the frequency of the cosine to be one cycle per 2π of phase. To find the parameters of this cosine model, we performed a non-linear regression using the MATLAB function nlinfit.m, with initial parameter estimates of γ, δ, and ϵ as 1, π, and 0. The correlation measure between mu phase and trial success was the amplitude of this fitted curve, γ.

2.4.2. Match to VEP Template

The final gating variable was related to the quality of the VEP. In order to define whether the subject produced an evoked response during a single trial that represented good fixation, we needed to define a template that represented a typical VEP waveform for that subject.

Following 0.1–30 Hz bandpass filtering, we first performed phase-matched control trial correction, following the methodology defined by Kruglikov and Schiff (2003). This was done because the phase of EEG at the time of the cue presentation introduces an averaging bias. Individual trials were corrected using phase-matched control trials, or trials in which a similar alpha phase was evident but in which there was no stimulus presented. By subtracting out the average of control trials of the same phase group, defined in 45°phase bins, we remove the predominant alpha rhythm and are left with a trace that better reflects the underlying neural correlate of the VEP (Figure 2). Such a procedure removes peri-stimulus bias by phase of the evoked potential in all subjects (Figure 3).

FIGURE 2

Figure 2. Construction of the VEP template from phase-corrected left cue trials, channel PO4-F4, subject 8. (A) A single trial of VEP data. (B) Phase-matched correction by subtraction of a non-cued trial in the same phase group (falling within a same 45°window, black line) results in the phase-corrected trial (red line). (C) Uncorrected trials of the same phase group and their average. (D) Corrected trials of the same phase group and their average. (E) All eight phase groups and the average resulting VEP (black trace). Regions with thick horizontal lines indicate times during which the mean VEPs between the eight phase groups were significantly different as determined by ANOVA (p < 0.05, Bonferroni corrected). (F) The final template for this subject is the average of the phase corrected trial groups (black line).

FIGURE 3

Figure 3. Uncorrected (left) and phase match corrected (right) VEPs for all subjects. This represents for all subjects what is Figures 2E,F for subject 8. Colored lines are the eight phase groups, and the black trace is the resulting average VEP. Regions with thick horizontal lines indicate times during which mean VEPs between the eight phase groups were significantly different as determined by ANOVA (p < 0.05, Bonferroni corrected).

Once corrected for phase, templates for left and right cues were defined by averaging EEG data over trials corresponding to each cue type for bipolar channels PO3-FC3 and PO4-FC4. This referencing scheme was the closest approximation to the standard Oz-Fz (Odom et al., 2010) we could accomplish with our electrode montage. The timing of the template ranged from 100 to 400 ms after cue presentation.

A metric describing the match of individual trials to the template was created using a matched filter approach. This technique comes from radar communication as a method for maximizing the probability of detecting a target waveform in the presence of Gaussian noise (McClellan and Purdy, 1978). Although a derivation can be found elsewhere (Levanon and Mozeson, 2004), the optimal filter for maximizing the signal to noise ratio of a signal generated by a linear, time invariant system with added Gaussian noise is the time-reversed version of the transmitted signal. This time-reversed template is called the matched filter. We applied the template as the transmitted signal s(t), and the phase-corrected single trial data as r_n(t). The output of the matched filter y_n(t) was the convolution of the trial data with the time-reversed template (Equation 10). The output of the matched filter that we used was the central value of this convolution, normalized to the magnitude of the template (Equation 11). This value, which we called the matched filter value (MFV), was related to the signal to noise ratio of the VEP, and served as the third gating variable. Trials with large MFV were interpreted as being trials which the subject produced a robust VEP.

$\begin{array}{rc}{y}_n(t)=r_n(t)*s(-t).& \left(10\right)\end{array}$

$\begin{array}{rc}MF{V}_n=\frac{y_n(0)}{\left|\right|s\left|\right|}.& \left(11\right)\end{array}$

Here, * represents the convolution function, and y_n(0) is the central value of the output of the convolution. The magnitude of the template s, in this case the Euclidean norm, is denoted by vertical brackets.

Similarly to the previous gating variables, we determined the relationship between the MFV and success on the BCI task. Again, we assumed that the trial success was a linear function of the MFV (Equation 12).

$\begin{array}{ll}{Z}_n=\zeta (MF{V}_n)+\eta .& \left(12\right)\end{array}$

The slope of this regression, ζ, reflects the correlation between of the outcome of the trial and the match to a VEP template of good fixation.

2.5. Permutation Testing for Significance

The three gating variables were tested for significant correlation with trial success in each subject. Because the dependent variable, the trial success, was not normally distributed, we chose to perform a non-parametric permutation test to determine the statistical significance of the regression parameters. We first computed the test statistic Q_obs, the slope of the linear regression (α, ζ) or the amplitude of the fitted cosine (γ). Then we shuffled the Z success outcomes between the trials and used the same fitting procedures to calculate the permuted statistics, Q(k) for each permutation k = 1…K, where K = 1000.

This permutation was performed with each gating variable from EEG features in two channels. Correction for multiple comparisons was done for each hypothesis test in the following manner, as described in more detail in Lage-Castellanos et al. (2010). For each permutation k, both the maximum Q_max(k) and minimum Q_min(k) values were chosen from the two Q(k) statistics evaluated for both channels. This resulted in K-values making up each of the maximum and minimum empirical distributions of the randomized data set. The calculation of p-statistics, the probability of Q_obs belonging to the null distribution of the permuted set, is given in Equation (13).

$\begin{array}{lll}\hfill p_{high}=\frac{{\displaystyle {\sum }_{k=1}^{1000}}(H(Q_{max}(k)-Q_{obs}))+1}{1000+1}.& & \left(13\right)\end{array}$

Here, H is the Heaviside function. Ones are added to the numerator and denominator to include the Q_obs in our null distribution. p_low was also calculated similarly using Q_min to test for significance from both tails of the null distribution. Findings of either p_high or p_low < 0.025 are significant deviations from chance. The generic process of trial ranking and permutation testing for significance is shown Figure 4.

FIGURE 4

Figure 4. This flowchart shows a generic example of the permutation test for determining significant relationships between gating variables and trial success. (1) Test data from left and right trials are sorted using the rankings determined by each gating method. Shown here, the trials are ranked according to mu amplitude at the time of the cue, A(τ_n). Trials with high A(τ_n) are shown on the bottom of the block. The feedback at the end of these trials is also shown, with left trials in blue and right trials in red. Feedback was subsequently simplified to trial success, Z. (2) The observed test statistic, Q_obs is the slope of the regression that relates mu amplitude to trial success. The permuted test statistic Q(k) was computed K = 1000 times after permuting the success scores of the trials to Z_{k, n}. (3) The k^th value for the maximum and minimum null distributions Q_max and Q_min were found from Q(k) across both channels tested. Finally the p-value of the test statistic distribution is computed as the percentage of values in this null distribution exceeding that of the observed test statistic.

2.6. Trial Gating Simulation

To simulate the effect of utilizing the potential gating variables online, we computed how the accuracy and bit rates would change for each subject if a portion of the trials having the least predictive value for task completion were removed. This predictive value was determined from the output of a linear discriminant analysis (LDA) classifier, which had been trained with the gating variables derived from channels in the right and left hemispheres as the input and trial success as the outcome variable. This simulation was run using individual gating variables determined to have significant predictive value by the permutation test, as well as with a combination of these gating variables. Trials in the test set that were classified as having the lowest predictive value were gated, meaning they were stopped before the imagery period began. The simulation was repeated at different thresholds, from gating no trials up to gating 70% of the total trials. Gated trials were skipped 500 ms after stimulus presentation and were interpreted as no-decision while taking 1.5 s (1 s before cue and 0.5 s after) to complete. Allowed trials resulted in a decision while taking 4 s to complete. Accuracy was defined as the number of allowed trials performed correctly over the total number of allowed trials.

Four-fold cross validation was used to find an average of the accuracy of the gating procedure at each threshold. In each fold of the classification scheme, a single session was used as the test set, while trials from the remaining sessions were used to train the classifier. The accuracy was calculated as the number of trials completed correctly out of the total number of trials attempted after gating. The bits per trial, B, was calculated for an N = 2 choice task having accuracy P following the form of Equation (14) from Wolpaw et al. (2002).

$\begin{array}{rc}B={log}_{2}N+P{log}_{2}P+(1-P){log}_2\frac{(1-P)}{(N-1)}.& \left(14\right)\end{array}$

The bit rate BR in bits/min was calculated using Equation (15).

$\begin{array}{rc}BR=B∕T_{\mu },& \left(15\right)\end{array}$

where the average trial time in minutes, T_μ, was increased to represent a penalty for disposing trials. T_μ was calculated from the number of allowed trials, N_a, and the gated trials, N_g, using Equation (16).

$\begin{array}{rc}{T}_{\mu }=(4N_a+1.5N_g)∕60N_a.& \left(16\right)\end{array}$

Significant differences in the baseline vs. gated accuracies and bit rates were calculated by a non-parametric Wilcoxon rank sum comparison of means over the four cross-validation folds.

3. Results

3.1. BCI Performance

The operation of the BCI was successful for eight of the nine subjects, whose online accuracies for each session are given in Table 1. Subject 2, with an average accuracy of 50.4%, indicative of random BCI control using this method, was not considered successful at operating the BCI system. Because we cannot be sure there are any signals relevant to motor imagery produced by this individual, they were omitted from further analysis. Subject 3 showed a significant learning effect, with others displaying a smaller or non-existent effect. LDA with 10-fold cross validation was performed for features derived from EOG and EMG channels. Mean classification accuracy for all eight subjects using EOG features was 51.7 ± 2.9% and mean classification accuracy for the four out of eight subjects with recorded EMG features was 49.2 ± 0.8%. Bounds are standard deviations.

TABLE 1

Table 1. Online performance.

3.2. Mu Amplitude and VEP Correlation Associate with Trial Success

Trials were ranked according to the three gating variables of mu amplitude, mu phase, and MFV. Subject-specific mu frequencies for Laplacian-derived channels C3 and C4 are given in Table 2. Figure 5 shows the grand average results for the study in right hemisphere channels, in which trials are subdivided into eight groups based on ranking by each gating variable. Average EEG traces for each group are shown in subplots (a), (d), and (g), while mu suppression from baseline is given in the second row in subplots (b), (e), and (h). For groups ranked by baseline mu amplitude, there is a significant difference in the level of suppression during imagery (Figure 5B, black bar indicates ANOVA with p < 0.05, Bonferroni corrected for 768 time points). MFV shows a positive association between gating variable group and trial success (Figure 5I, r² = 0.25, not significant), mu phase shows no association, and the positive correlation between trial success and mu amplitude is significant (Figure 5C, r² = 0.86, p = 0.001). Positive correlations between trial success and both mu amplitude and MFV are also evident in the left hemisphere channels (Figure 6), of which the correlation between MFV and trial success is significant (Figure 6I, r² = 0.56, p = 0.032).

TABLE 2

Table 2. Subject specific mu frequencies.

FIGURE 5

Figure 5. Three EEG features were considered as potential gating variables (shown here are the right hemisphere features): mu amplitude in C4 at stimulus presentation (left column, A–C), mu phase in C4 at stimulus presentation (middle column, D–F), and match to PO4-FC4 VEP template (right column, G–I). For each feature, all recorded trials were divided into eight groups, from low feature value to high, represented by the eight colored lines/points. The first row is the average of the EEG in these groups. The second row is the average mu suppression from baseline in C4 in these groups. The last row is the mean and standard error of the success across all trials by group.

FIGURE 6

Figure 6. Three EEG features were considered as potential gating variables (shown here are the left hemisphere features): mu amplitude in C3 at stimulus presentation (left column, A–C), mu phase in C3 at stimulus presentation (middle column, D–F), and match to PO3-FC3 VEP template (right column, G–I). Refer to Figure 5 for further description.

On an individual basis, the permutation test procedure determined that subjects 4, 6, 8, and 9 produced in at least one channel a slope of regression, α, which was significantly greater than that produced by the random null distribution, indicating a significant positive correlation between mu amplitude and motor imagery task success (Table 3). Non-linear regression produced no significant cosine fits between mu phase and trial performance. Finally, only subject 9 displayed a significant correlation between the MFV gating variable, with high values of the gating variable in both channels PO3-FC3 and PO4-FC4 corresponding to greater trial success.

TABLE 3

Table 3. Results of the permutation test with the three gating variables.

3.3. Trial Gating Simulation

Table 4 gives the average of the maximum accuracies and bit rates achieved by each subject in the four-fold simulation procedure of gating with the potential gating features. In most instances where the permutation test indicated a subject would benefit from gating trials of low value, the simulation of offline gating yielded an increased accuracy. Bolded values show where the gated accuracy or bit rate of the four-folds of the simulation procedure were significantly larger than the non-gated values (Wilcoxon rank sum test, p < 0.05). The only significant increase in accuracy was seen for subject 9, where gating ~60% of trials based on the match to VEP resulted in improvement to 100% accuracy. Figures 7C,D show how the accuracy and bit rate change for each subject as a function of the number of trials that are gated based on a combination mu amplitude and MFV. In comparison, Figures 7A,B show how the accuracy and bit rate are affected by gating of random trials.

TABLE 4

Table 4. Simulated gating results.

FIGURE 7

Figure 7. Average accuracy and bit rates from the four-folds of the gating simulation. (A) Average accuracy achieved for each subject after gating up to 70% of trials randomly. (B) Resulting bit rate. (C) Average accuracy achieved for each subject after gating of trials up to 70% based on a linear combination of mu rhythm amplitude and MFV features. (D) Resulting bit rate.

4. Discussion

4.1. Prospects for Gating Features

In BCI, single trial identification of user intent is a difficult task; nevertheless, we have shown that ongoing oscillatory activity of the motor cortex contains additional information that co-varies with motor-imagery task performance. Of the three features explored, the technique of selecting for trials with high mu amplitude was the most consistently correlated with trials in which users are most likely to achieve higher performance.

The topic of oscillatory electrical brain phenomena influencing motor excitability is controversial (Mäki and Ilmoniemi, 2010), but the evidence is stronger for cortical excitability changes to be correlated with SMR amplitude than with phase. The lack of correlation between the phase of the mu rhythm and subsequent suppression efficacy is in agreement with other studies which found that phase had little effect on mu suppression (Mäki and Ilmoniemi, 2010). Whereas, the phase at the time of cue presentation did have an effect on early evoked potential morphology (Figure 2E), it is not surprising that the effects generally do not propagate 1–3 s after the cue, during which mu suppression is evident. On the other hand, all subjects displaying significant correlation between mu amplitude and BCI task success performed better on individual trials when the amplitude of mu was high over channels C3 and C4 at the onset. These results build upon the those of Blankertz et al. (2010), who demonstrated that baseline SMR amplitude distinguished good and poor performers, and Maeder et al. (2012), who showed the same effect of SMR amplitude on BCI performance within subjects.

The relationship between pre-stimulus SMR amplitude and performance could be due to two reasons. Our classifier was based on reduction of mu from a baseline level to an attenuated level, and as a result, trials with larger baseline mu amplitude have a greater potential for mu power reduction. In addition, the amplitude of the mu rhythm reflects the state of excitability of its generating network (Mäki and Ilmoniemi, 2010). While our finding is in disagreement with studies that point to a desynchronized cortex as one primed for motor output (Sauseng et al., 2009; Mäki and Ilmoniemi, 2010), Blankertz et al. (2010) showed that a successful predictor of imagery performance across subjects was an increased resting state mu rhythm. In a follow up study, they showed that this performance increase was associated with larger recruitment of motor and premotor regions. They concluded that recruiting more synchronized neurons for motor imagery led to higher resting mu amplitude as well as higher performance (Halder et al., 2011). In our study we show that not only is this relationship between resting mu amplitude and imagery performance valid between subjects, but also on a single-trial basis within subjects.

The third tested gating variable did not produce consistent evidence to warrant widespread use as a metric for predicting motor-imagery success, although the potential for this was evident in one user. Risner et al. showed that, following phase-matched correction of VEP data using unstimulated controls, average VEPs in different phase groups display the same VEP morphology (Risner et al., 2009). We also found this to be the case for early evoked potentials, although after phase-matched control trial correction, there was some phase dependence in later evoked potentials in certain individuals (Figure 3, subjects 4 and 8). Because the MFV was calculated from phase-corrected VEP data, the value of this metric can be interpreted to be free of oscillatory alpha bias and instead can be associated with attention to the visual task. However, using a subject-specific VEP template to define the MFV, we were unable to find a consistent correlation between this marker of attention and the subsequent modulation of motor rhythms. Although subject 9 did display a relationship between high MFV and good performance, overall there was no group-wide trend. This finding has important implications for BCI research. It demonstrates that fixation to the visual cue is not critical for task success in most users. For these users, control over the BCI communication device should not be limited by their ability to directly fixate with cues on the screen, a critical allowance in the case of severe oculomotor impairment that could occur in stroke or late stage neurodegenerative disease.

4.2. Trial Gating Simulation

The goal of this study was to identify certain brain signals present during a motor imagery BCI experiment that could be used as a supplementary control mechanism to improve accuracy, speed, and even allow for asynchronous timing.

To show this might be possible, we attempted to use these signals to predict trials with low probability of successful completion, and gate them so that they aborted before finishing. In the majority of cases where a correlation between the gating variables and trial success was present, offline simulation of the gating procedure failed to produce increases in accuracy or bit rate for the motor imagery BCI system. The notable exception was seen in subject 9, where significant increases in accuracy were accomplished by discarding trials with low MFV. Because this user was already quite high performing, throwing away 60% of the trials to increase accuracy from 97 to 100 percent was detrimental overall to the information transfer rate of the BCI (Figures 7C,D).

In all other cases, the gating procedure did yield improvements to system accuracy, but these increases were not significantly above the baseline scenarios in the cross-validation procedure. Furthermore, the gains in accuracy were not great enough to overcome the extra time it takes to gate a trial, resulting in slower system speed. Utilizing a combination of gating features as part of the gating system also resulted in improved performance for subject 9, although the level of improvement was less than that resulting from gating by MFV alone. In fact, for most subjects, gating with a combination of multiple features was less effective than use of individual variables that correlated with BCI performance. This demonstrates that for a gating system such as this to be successful, the mechanism of gating must be personalized to the user based on brain signatures that correlate with performance.

The discrepancy in results between the permutation procedure and the gating simulation highlights the distinction between what we are looking for in our study, specifically, a correlation between pre-task EEG features and task performance, and the use of this knowledge in an online BCI. We show that this type of gating is successful for a number of subjects to improve accuracy on the motor imagery task. However, the increase in accuracy needs to be substantial for this method to be useful.

4.3. Study Limitations

Although the oscillation of cortical rhythms is associated with the depolarization and hyperpolarization of large groups of neurons, at the level of EEG the linkage with motor cortical excitability may be tenuous; at this level of measurement from scalp, both excitatory and inhibitory neurons contribute to oscillatory behavior (MacKay, 1997). Consequently, broadly recorded signatures of rhythmic activity may be unable to describe excitability in a local group of motor neurons responsible for hand imagery. This is especially true when we attempt to use the excitability of the motor region as a predictor of SMR modulation that occurs on the order of 500–1000 ms later.

The lack of findings for the MFV gating variable could be due to poor characterization of individuals' VEPs. The recording parameters included a 0.1–30 Hz bandpass filter, which is narrower than the clinical recommendation of 1–100 Hz for identifying individual VEP peaks (Odom et al., 2010). As a result, these peaks may have undergone some attenuation and blurring, and may not have been the ideal template for assessing attention with small variations in peak amplitude.

A substantial limitation of our study is the offline analysis of BCI data. Feedback is an integral part of the BCI system, and we cannot assume that the user will respond similarly when faced with a scenario where trials are gated on a regular basis. Just as poor feedback on a trial may have led to subsequent good performance on the following trial, we might expect a gated trial to produce increased vigilance or promote focus for the upcoming trial. This is a limitation that cannot be reconciled by offline analysis; these effects need to be quantified in a real-time adaptive scenario in a future study. What remains to be seen is how the feedback of the gating procedure affects the stationarity of these signals, and whether trial gating is an effective means for boosting information transfer during these tasks.

Author Contributions

AG, MK, and SS made substantial contributions to the conception and design of the study. AG facilitated the recruitment of participants, data collection, and analysis. AG drafted the article. MK and SS revised it critically for important intellectual content. All authors provided their final approval for this version to be published.

Funding

NIH grant: 1 K25 NS061001 01A2.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

References

Birbaumer, N. (2006). Breaking the silence: brain-computer interfaces (BCI) for communication and motor control. Psychophysiology 43, 517–532. doi: 10.1111/j.1469-8986.2006.00456.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Bishop, G. H. (1932). Cyclic changes in excitability of the optic pathway of the rabbit. Am. J. Physiol. Legacy Content 103, 213–224.

Google Scholar

Blankertz, B., Sannelli, C., Halder, S., Hammer, E. M., Kübler, A., Müller, K.-R., et al. (2010). Neurophysiological predictor of SMR-based BCI performance. Neuroimage 51, 1303–1309. doi: 10.1016/j.neuroimage.2010.03.022

PubMed Abstract | CrossRef Full Text | Google Scholar

Dias, N., Jacinto, L. R., Mendes, P., and Correia, J. (2009). “Visual gate for brain-computer interfaces,” in Annual International Conference of the IEEE Engineering in Medicine and Biology Society, 2009 (Minneapolis, MN), 532–535.

PubMed Abstract

Dias, N., Kamrunnahar, M., Mendes, P., Schiff, S., and Correia, J. (2010). Feature selection on movement imagery discrimination and attention detection. Med. Biol. Eng. Comput. 48, 331–341. doi: 10.1007/s11517-010-0578-1

PubMed Abstract | CrossRef Full Text | Google Scholar

Gasser, T., Bächer, P., and Möcks, J. (1982). Transformations towards the normal distribution of broad band spectral parameters of the EEG. Electroencephalogr. Clin. Neurophysiol. 53, 119–124.

PubMed Abstract | Google Scholar

Geronimo, A., Schiff, S., and Kamrunnahar, M. (2011). “A simple generative model applied to motor-imagery brain-computer interfacing,” in Neural Engineering Conference (Cancun: IEEE EMBS).

Halder, S., Agorastos, D., Veit, R., Hammer, E., Lee, S., Varkuti, B., et al. (2011). Neural mechanisms of brain-computer interface control. Neuroimage 55, 1779–1790. doi: 10.1016/j.neuroimage.2011.01.021

PubMed Abstract | CrossRef Full Text | Google Scholar

Halder, S., Ruf, C. A., Furdea, A., Pasqualotto, E., De Massari, D., van der Heiden, L., et al. (2013). Prediction of P300 BCI aptitude in severe motor impairment. PLoS ONE 8:e76148. doi: 10.1371/journal.pone.0076148

PubMed Abstract | CrossRef Full Text | Google Scholar

Hillyard, S. A., and Anllo-Vento, L. (1998). Event-related brain potentials in the study of visual selective attention. Proc. Natl. Acad. Sci. U.S.A. 95, 781–787.

PubMed Abstract | Google Scholar

Iturrate, I., Montesano, L., and Minguez, J. (2013). Task-dependent signal variations in EEG error-related potentials for brain-computer interfaces. J. Neural Eng. 10, 026024. doi: 10.1088/1741-2560/10/2/026024

PubMed Abstract | CrossRef Full Text | Google Scholar

Kruglikov, S. Y., and Schiff, S. J. (2003). Interplay of electroencephalogram phase and auditory-evoked neural activity. J. Neurosci. 23, 10122–10127.

PubMed Abstract | Google Scholar

Lage-Castellanos, A., Martínez-Montes, E., Hernández-Cabrera, J. A., and Galán, L. (2010). False discovery rate and permutation test: an evaluation in ERP data analysis. Stat. Med. 29, 63–74. doi: 10.1002/sim.3784

PubMed Abstract | CrossRef Full Text | Google Scholar

Levanon, N., and Mozeson, E. (2004). “Matched filter,” in Radar Signals (Hoboken, NJ: John Wiley and Sons), 20–33.

Google Scholar

MacKay, W. A. (1997). Synchronized neuronal oscillations and their role in motor processes. Trends Cogn. Sci. 1, 176–183.

PubMed Abstract | Google Scholar

Maeder, C., Sannelli, C., Haufe, S., and Blankertz, B. (2012). Pre-stimulus sensorimotor rhythms influence brain - computer interface classification performance. IEEE Trans. Neural Syst. Rehabil. Eng. 20, 653–662. doi: 10.1109/TNSRE.2012.2205707

PubMed Abstract | CrossRef Full Text | Google Scholar

Mäki, H., and Ilmoniemi, R. J. (2010). EEG oscillations and magnetically evoked motor potentials reflect motor system excitability in overlapping neuronal populations. Clin. Neurophysiol. 121, 492–501. doi: 10.1016/j.clinph.2009.11.078

PubMed Abstract | CrossRef Full Text | Google Scholar

McClellan, J. H., and Purdy, R. J. (1978). “Applications of digital signal processing to radar,” in Applications of Digital Signal Processing, ed A. V. Oppenheim (Upper Saddle River, NJ: Prentice–Hall), 250.

McFarland, D., Miner, L., Vaughan, T., and Wolpaw, J. (2000). Mu and beta rhythm topographies during motor imagery and actual movements. Brain Topogr. 12, 177–186. doi: 10.1023/A:1023437823106

PubMed Abstract | CrossRef Full Text | Google Scholar

McFarland, D. J., McCane, L. M., David, S. V., and Wolpaw, J. R. (1997). Spatial filter selection for EEG-based communication. Electroencephalogr. Clin. Neurophysiol. 103, 386–394.

PubMed Abstract | Google Scholar

McFarland, D. J., Sarnacki, W. A., and Wolpaw, J. R. (2010). Electroencephalographic (EEG) control of three-dimensional movement. J. Neural Eng. 7, 036007. doi: 10.1088/1741-2560/7/3/036007

PubMed Abstract | CrossRef Full Text | Google Scholar

Miller, K. J., Schalk, G., Fetz, E. E., den Nijs, M., Ojemann, J. G., and Rao, R. P. N. (2010). Cortical activity during motor execution, motor imagery, and imagery-based online feedback. Proc. Natl. Acad. Sci. U.S.A. 107, 4430–4435. doi: 10.1073/pnas.0913697107

PubMed Abstract | CrossRef Full Text

Mitra, P., and Bokil, H. (2008). Observed Brain Dynamics. Oxford University Press. Available online at: http://chronux.org.

Niedermeyer, E., and da Silva, F. L. (1999). Niedermeyer's Electroencephalography: Basic Principles, Clinical Applications, and Related Fields. Philadelphia, PA: Lippincott Williams & Wilkins.

Google Scholar

Odom, J., Bach, M., Brigell, M., Holder, G., McCulloch, D., Tormene, A., et al. (2010). ISCEV standard for clinical visual evoked potentials (2009 update). Doc. Ophthal. 120, 111–119. doi: 10.1007/s10633-009-9195-4

PubMed Abstract | CrossRef Full Text | Google Scholar

Pfurtscheller, G., and da Silva, F. H. L. (1999). Event-related EEG/MEG synchronization and desynchronization: basic principles. Clin. Neurophysiol. 110, 1842–1857.

PubMed Abstract | Google Scholar

Ploner, M., Gross, J., Timmermann, L., Pollok, B., and Schnitzler, A. (2006). Oscillatory activity reflects the excitability of the human somatosensory system. Neuroimage 32, 1231–1236. doi: 10.1016/j.neuroimage.2006.06.004

PubMed Abstract | CrossRef Full Text | Google Scholar

Railo, H., Koivisto, M., and Revonsuo, A. (2011). Tracking the processes behind conscious perception: a review of event-related potential correlates of visual consciousness. Conscious. Cogn. 20, 972–983. doi: 10.1016/j.concog.2011.03.019

PubMed Abstract | CrossRef Full Text | Google Scholar

Risner, M. L., Aura, C. J., Black, J. E., and Gawne, T. J. (2009). The visual evoked potential is independent of surface alpha rhythm phase. Neuroimage 45, 463–469. doi: 10.1016/j.neuroimage.2008.12.031

PubMed Abstract | CrossRef Full Text | Google Scholar

Sauseng, P., Klimesch, W., Gerloff, C., and Hummel, F. (2009). Spontaneous locally restricted EEG alpha activity determines cortical excitability in the motor cortex. Neuropsychologia 47, 284–288. doi: 10.1016/j.neuropsychologia.2008.07.021

PubMed Abstract | CrossRef Full Text | Google Scholar

Schlögl, A., Keinrath, C., Zimmermann, D., Scherer, R., Leeb, R., and Pfurtscheller, G. (2007). A fully automated correction method of EOG artifacts in EEG recordings. Clin. Neurophysiol. 118, 98–104. doi: 10.1016/j.clinph.2006.09.003

PubMed Abstract | CrossRef Full Text | Google Scholar

Spüler, M., Bensch, M., Kleih, S., Rosenstiel, W., Bogdan, M., and Kübler, A. (2012). Online use of error-related potentials in healthy users and people with severe motor impairment increases performance of a P300-BCI. Clin. Neurophysiol. 123, 1328–1337. doi: 10.1016/j.clinph.2011.11.082

PubMed Abstract | CrossRef Full Text | Google Scholar

Thut, G., and Miniussi, C. (2009). New insights into rhythmic brain activity from TMS EEG studies. Trends Cogn. Sci. 13, 182–189. doi: 10.1016/j.tics.2009.01.004

PubMed Abstract | CrossRef Full Text | Google Scholar

Treder, M., and Blankertz, B. (2010). (C)overt attention and visual speller design in an ERP-based brain-computer interface. Behav. Brain Funct. 6, 28. doi: 10.1186/1744-9081-6-28

PubMed Abstract | CrossRef Full Text | Google Scholar

Wolpaw, J. R., Birbaumer, N., McFarland, D. J., Pfurtscheller, G., and Vaughan, T. M. (2002). Brain-computer interfaces for communication and control. Clin. Neurophysiol. 113, 767–791. doi: 10.1016/S1388-2457(02)00057-3

PubMed Abstract | CrossRef Full Text | Google Scholar