Classifying multiple types of hand motions using electrocorticography during intraoperative awake craniotomy and seizure monitoring processes—case studies

In this work, some case studies were conducted to classify several kinds of hand motions from electrocorticography (ECoG) signals during intraoperative awake craniotomy & extraoperative seizure monitoring processes. Four subjects (P1, P2 with intractable epilepsy during seizure monitoring and P3, P4 with brain tumor during awake craniotomy) participated in the experiments. Subjects performed three types of hand motions (Grasp, Thumb-finger motion and Index-finger motion) contralateral to the motor cortex covered with ECoG electrodes. Two methods were used for signal processing. Method I: autoregressive (AR) model with burg method was applied to extract features, and additional waveform length (WL) feature has been considered, finally the linear discriminative analysis (LDA) was used as the classifier. Method II: stationary subspace analysis (SSA) was applied for data preprocessing, and the common spatial pattern (CSP) was used for feature extraction before LDA decoding process. Applying method I, the three-class accuracy of P1~P4 were 90.17, 96.00, 91.77, and 92.95% respectively. For method II, the three-class accuracy of P1~P4 were 72.00, 93.17, 95.22, and 90.36% respectively. This study verified the possibility of decoding multiple hand motion types during an awake craniotomy, which is the first step toward dexterous neuroprosthetic control during surgical implantation, in order to verify the optimal placement of electrodes. The accuracy during awake craniotomy was comparable to results during seizure monitoring. This study also indicated that ECoG was a promising approach for precise identification of eloquent cortex during awake craniotomy, and might form a promising BCI system that could benefit both patients and neurosurgeons.

1. Introduction

Electrical recordings from surface of brain cortex [i.e., electrocorticography (ECoG)], have been recognized as a promising signal resource not only for clinical application (e.g., eloquent cortex identification during awake craniotomy, Roland et al., 2010, or epileptic seizure localization) but also for brain-computer interface (BCI) research (Schalk and Leuthardt, 2011). Clinical and BCI research may benefit from each other. Clinical research gave the access to the valuable ECoG signal, and BCI research could enhance our understanding of the ECoG characteristic and expand its application.

For a pure BCI-purpose implantation, it would be highly necessary to demonstrate reliable neuroprosthetic control during surgical implantation (i.e., during awake craniotomy), in order to verify the optimal placement of electrodes (Fifer et al., 2014). Non-invasive methods of functional mapping (e.g., fMRI) could only be used for gross surgical planning, but intraoperative verification of control with ECoG would be extremely useful to refine the final implantation site. This would help to avoid the need for re-implantation because the patient was unable to control the neuroprosthetic.

Precise identification of eloquent cortex was a clinical necessity prior to surgical resections adjacent to motor cortex. ECoG signals could serve as a useful adjunct to cortical stimulation mapping during the intraoperative setting (Roland et al., 2010; Kamada et al., 2014). It also possessed the potential to increase the identification efficiency and resolution of the motor cortex (Leuthardt et al., 2007), and reduce the amount of necessary electrocortical stimulation during the awake craniotomy process. However, motion tasks within these studies were simply opening and closing the contralateral hand, and dexterous hand motion types may have the potential for more precise localization of eloquent cortex.

There was a general agreement that low-frequency (< 25 Hz) components of the electroencephalography (EEG) decrease in amplitude after onset of movement over the contralateral sensorimotor cortex, and this finding was later found in subdural ECoG (Miller et al., 2007). Compared with EEG, additional increases of ECoG in the amplitude of higher frequency components (< 35 Hz) were described in relation to movement (Crone et al., 1998; Miller et al., 2007), and those changes had a more localized spatial distribution on the surface of brain cortex than low-frequency changes. Signals at higher frequencies were demonstrated to carry more detailed information about hand movements (Schalk and Leuthardt, 2011), thereby providing more critical information which was hardly accessible with EEG-based BCI.

On the early stage, ECoG-based BCI focused on classifying simply movements of different body parts, such as hand, foot and tongue movements based on neural oscillatory dynamics (Graimann et al., 2003). Subsequently, predicting the kinematic and kinetic parameters of human hand and finger motions aroused much interest. Spatially distinct brain regions were specific to individual fingers (Miller et al., 2009), and the flexion time courses were highly specific to fingers (Kubánek et al., 2009). Individual finger flexion and movements were decoded with particularly high correlation coefficient (0.46 in Liang and Bougrain, 2012, 0.42 in Flamary and Rakotomamonjy, 2012), and multiple types of hand posture and grasps have been classified with “macro” (Pistohl et al., 2012; Chestek et al., 2013) and “micro” (Bleichner et al., 2014) electrode grids. Five isometric hand postures could be classified correctly with an average of 77.6% (20% chance) with three subjects (Chestek et al., 2013), and for four complex hand gestures, two participants achieved 97 and 74% (25% chance) classification accuracy respectively (Bleichner et al., 2014). However, all these studies were carried out after the implantation.

We here test the hypothesis that reliable identification of multiple types of hand motions can be realized on subjects under awake craniotomy, which is comparable to the decoding results of subjects during epileptic seizure monitoring. This is the first step toward the dexterous neuroprosthetic control during surgical implantation, in order to verify the optimal placement of electrodes that are implanted for pure BCI-purpose, and aimed for more precise localization of eloquent cortex during intraoperative surgical process.

Considering the clinical condition and current studies for multiple hand motion types decoding (Liang and Bougrain, 2012; Bleichner et al., 2014), three types of simple hand motions (i.e., grasp, thumb-finger motion and index-finger motion) were performed by four subjects (two brain tumor subjects undergoing an awake craniotomy, two epilepsy subjects during seizure monitoring, for one epilepsy subject in good condition, two more hand motion types, i.e., hand unfold and wrist motion, were performed). To further validate the decoding results, two methods were applied in this study. Method I: autoregressive (AR) model with burg method was applied to extract features, and additional waveform length (WL) feature has been considered, finally the linear discriminative analysis (LDA) was used as the classifier. Method II: stationary subspace analysis (SSA) was applied for data preprocessing, and the common spatial pattern (CSP) was used for feature extraction before LDA decoding process.

2. Methods

2.1. Subjects

The experiment was conducted on five right-handed human subjects. Subject P0 (Figure 1A) suffered from intractable epilepsy, and underwent temporary placement of subdural electrode arrays, however, he has already underwent resection of brain tumor before been diagnosed with epilepsy (shown within the black circle of Figure 1A), Hand motion information could not be decoding from ECoG signal of P0, thus Subject P0 was excluded and four subjects were considered in the study.

FIGURE 1

Figure 1. Photo of ECoG grids placed on the brain cortex surface. (A) Grid photo of Subject P0, the black circle indicates the resection cortex of brain tumor; (B) Grid photo of Subject P1.

Subject P1 (male, 14 years old), P2 (male, 30) suffered from intractable epilepsy, and they underwent temporary placement of a subdural electrode array in order to localize the epileptic seizure focus and map brain function prior to surgical resection (Figure 1B). The epileptic focus was found to be left posterior frontal for P1 and right for P2. Platinum grid electrodes (4 mm electrode diameter and 1 cm inter-electrode distance, 87 contacts for P1, and 95 contacts for P2) were subdurally implanted above the region of corresponding hemisphere. The sites of electrode implantation were exclusively based on the requirements of the clinical evaluation.

Subject P3 (male, 60), P4 (female, 61) suffered with brain tumor, and they were undergoing an awake craniotomy for the treatment of tumors adjacent to motor and speech cortex. Platinum grid electrodes (4 mm electrode diameter and 1 cm inter-electroded,16 contacts for P3 and 52 contacts for P4) were subdurally implanted above the region of the left hemisphere for both of the subjects.

All the subjects were recruited from Huashan Hospital and were informed about the whole experiment process. This study was approved by the Ethics Committee of Huashan Hospital. All subjects signed the informed consent forms by themselves or legal guardians before participating in the experiments.

2.2. ECoG Data Recording

ECoG data of subjects P1, P3, P4 were recorded from the electrode grid connected to the Stellate eAMP 64 (Canada) system, and CEEGRAPH (128+ channal XL 2) was used on subject P2. The signals were amplified, bandpass filtered between 0.15~100 Hz, digitized at 250, 256, 200, and 2000 Hz for P1~P4, respectively.

2.3. Experimental Design

2.3.1. Setup for Subjects P1, P2

Subject P1 was lying in a semi-recumbent position in a hospital bed. He was instructed to perform the given tasks with the hand contralateral to the implant in response to a sound clip. A total of 100 trials were performed by subject P1 in 20 runs. Each run contained 5 trials which indicated different hand motion types (Figure 2), i.e., hand unfold (T1), hand grasp (T2), thumb-finger motion (T3), wrist motion (T4), and index-finger motion (T5). For each trial, the subject repeated to do a given motion type for 5 s, e.g., repeating to perform hand grasp with normal speed, and then following a relaxation period with 5 s. Data were collected for a total period of about 20 min, which yielded an average of 20 trials for each hand motion type. During the data collecting process, any other movements including facial or head movements were avoided.

FIGURE 2

Figure 2. The full run experiment of subject P1. T1, hand unfold; T2, hand grasp; T3, thumb-finger motion; T4, wrist motion; T5, index-finger motion. Each trial lasts 5 s, thereby 50 s were needed for the full run.

Subject P2 did the similar tasks as P1, and three hand motion types were executed : hand grasp, thumb-finger motion, index-finger motion. A total of 60 trials were performed in 20 runs, and thus yielded an average of 20 trials for each types.

2.3.2. Setup for Subjects P3, P4

Subject was in the operating room, and was instructed to perform the given tasks with the hand contralateral to the implant in response to the voice commands. Three motion types were performed in separate run (Figure 3), i.e., run I for hand grasp, run II for thumb-finger motion and run III for index-finger motion. Twenty trials were performed for each run, and 60 trials during the whole experiment. For each trial, subject performed the given task just once instead of repetitively performing the task (as P1, P2), the execution time last for 2~3 s and then following a relaxation period with 2~3 s. Data were collected for a total period of about 5 min, which yielded an average of 20 trials for each motion type.

FIGURE 3

Figure 3. Full run experiment of subject P3, P4. RUN I, hand grasp; RUN II, thumb-finger motion; III, index-finger motion. Each run contained 20 trials, each trial lasts 4~6 s.

2.4. Offline Analysis

Before decoding of hand motion types, the data during the awake craniotomy and seizure monitoring process was analyzed offline to investigate which electrodes and frequency ranges carry more task-relevant information. The raw signal was re-referenced to the common average by subtracting the mean value over all channels at each time point. The autoregressive model of order 60 with Burg method (Harris, 1967) was applied to estimate the power spectrum for each data segment. Logarithmic power of 1 Hz frequency bins from 1~100 Hz were used to calculated the R²-values (Sheikh et al., 2003) between all relevant combinations of two hand motions for each frequency bin at each electrode. Channels with high R²-values indicated more task-relevant information.

2.5. Algorithms for Decoding Hand Motion Types

To further validate the decoding results, two methods (Method I and Method II) were applied in this study.

2.5.1. Method I: AR-LDA with WL Feature

Before decoding of hand motions, the frequency ranges and electrodes that carry more task-relevant information need to be selected. The raw signal was re-referenced to the common average by subtracting the mean value over all channels at each time point. The autoregressive model of order 60 with Burg method (Harris, 1967) was applied to estimate the logarithmic power spectrum for each data segment. We calculated the R²-values (Sheikh et al., 2003) between all relevant combinations of two hand motions for each 10 Hz frequency bin at each electrode. To reduce dimensionality of the input space, 5 electrodes with 5 frequency ranges within each electrode were selected for classification according to the R²-values. Thus, 25 features (AR feature) were selected for a single trial.

In this study, we also considered another informative feature: Waveform Length feature. WL is a measure of signal complexity (Tkach et al., 2010), and has been proved to be a robust and efficient feature for electromyography (EMG) (Tkach et al., 2010) and EEG (Lotte, 2012) signals. The WL feature could be extracted from an ECoG signal as follows (Tkach et al., 2010):

$\begin{array}{cc}wl=log({\displaystyle {\sum }_{i=1}^{N-1}\left|x_{i+1}-x_i\right|})=log({\displaystyle {\sum }_{i=1}^{N-1}\left|\Delta x_i\right|})& \left(1\right)\end{array}$

where wl is the WL feature, |x| is the absolute value of x, N is the number of sample points for a data segment. We extracted wl feature from the 5 selected electrodes (the same electrodes as the previous step), and combined the wl feature to the AR feature. Thus, 30 features were determined for a single trial.

And finally, LDA was used as the classifier.

2.5.2. Method II: dSSA-mulCSP-LDA

Non-stationarity brain sources cause differences between the distributions of electrophysiological signals over time and in particular between the calibration and the application phase (von Bunau et al., 2010), Stationary Subspace Analysis (SSA) (von Bünau et al., 2009) can be used to restrict the decoding to the stationary brain sources. SSA is limited when applying it to multi-class data, and dSSA (Samek et al., 2012; Liu et al., 2015) that trades-off stationarity and discriminativity was used in this study. Common spatial pattern (CSP) (Fukunaga, 1990; Ramoser et al., 2000) has been widely used in BCI literature, mathematically it is realized by simultaneous diagonalization of the covariance matrices for the two classes. For multi-class decoding, CSP with one-to-one strategy has been proved (Liu et al., 2009). In a pre-processing step we apply dSSA to the calibration data and then mulCSP+LDA on the estimated s-sources. The details are shown as follows.

SSA (von Bünau et al., 2009) is a novel method to factorize a high-dimensional time-series signal x(t) into its stationary S^s(t) and non-stationary Sⁿ(t) components.

$\begin{array}{ll}x(t)=AS(t)=\left[A^s{A}^n\right]{\left[S^s(t)S^n(t)\right]}^{-1}& (2)\end{array}$

where A is an invertible matrix. The goal of dSSA is to minimize the distance measured as Kullback-Leibler Divergence D_KL, between the distribution of the estimated s-sources in each epoch (described by first two moments) and the standard normal distribution. The objective function of dSSA measures the divergence can be written as

$\begin{array}{cc}L(R)={\displaystyle {\sum }_{i=1}^{n_g}{D}_{KL}}[N({\overline{u}}_i,{\overline{\sum }}_i)\Vert N(\overline{u},\overline{\sum })& \left(3\right)\end{array}$

where n_g is the number of groups, $N({\overline{u}}_i,{\overline{\sum }}_i)$ is the average distribution in group i, $N(ū,\overline{\sum })$ is the average distribution of all groups and R is a rotation matrix.

For CSP method, the stationary ECoG signal is represented as S_k with dimensions ch×len, where ch is the number of recording electrodes (i.e., the channel numbers of the signal), and len is the number of sample points in one set. The normalized spatial covariance matrix of the ECoG can be obtained from

$\begin{array}{ll}{C}_k=\frac{S_k{S}_k^T}{trace(S_k{S}_k^T)}& (4)\end{array}$

where k is the trial number, $S_k^T$ denotes the transpose of the matrix S_k, and $trace(S_k{S}_k^T)$ is the sum of the diagonal elements of the matrix $S_k{S}_k^T$. Let

$\begin{array}{cc}{C}_l={\displaystyle {\sum }_{k\in I_l}{C}_k}{C}_r={\displaystyle {\sum }_{k\in I_r}{C}_k}& \left(5\right)\end{array}$

where C_l, C_r is the average covariance matrix, I_l and I_r are the two index sets of the separate classes (e.g., hand grasp and hand unfold). Set C = C_l + C_r, the eigenvalue decomposition of C

$\begin{array}{ll}C=U_c{\displaystyle \sum _{}}{U}_c^T& (6)\end{array}$

where $\sum _{}$ is the diagonal matrix, U_c is the eigenvector matrix. The average covariance matrix C_l, C_r can be transformed as

$\begin{array}{cc}{S}_l=P{C}_l{P}^T=U{\displaystyle {\sum }_{l\in }{U}^T};S_r=P{C}_r{P}^T=U{\displaystyle {\sum }_{r\in }{U}^T}& \left(7\right)\end{array}$

where $P=\sqrt{\sum _{}^{-1}}{U}_c^T$ is the whitening matrix, $\sum _l+\sum _r=I$. The projection matrix W could be gained from W = U^TP, the rows of W are called spatial filters, and the columns of W⁻¹ are called spatial patterns. To the k-th trial, the filtered signal Z_k = WS_k is uncorrelated. In this work, the log variance of the first n rows and last n rows (n = 3 or 4) of Z_k corresponding to largest n eigenvalues and smallest n eigenvalues are chosen as feature vectors. One-to-one strategy were employed for three classes decoding in this study, 3 spatial filters were achieved, and finally, obtained 6n dimension (i.e., 3*2n) feature vector for each trial.

Finally, LDA was used as the classifier.

For both method I and method II, a 10 × 10 fold cross validation was adopted to evaluate the classification accuracy among different tasks, which was described as follows: trials were randomly permutated firstly, then equally divided into ten partitions, each partition was used as an unknown test set which was classified by the classifier trained with the remaining nine partitions, and a classification accuracy for each partition was achieved. This process was repeated ten times, and 100 classification accuracy indexes were generated. The average value of these 100 indexes was the final classification accuracy.

3. Results

3.1. Spatial/Spectral Selection and Analysis

To determine the most discriminative features (electrodes and frequency ranges), R²-values of the training data set were calculated for all relevant class combinations and for each electrode and each frequency bin. Exemplary R²-values for two-class combinations with 1 Hz frequency range bins are shown in Figures 4, 5. Subjects P1~P4 were implanted with 87, 95, 16, 52 electrodes, respectively.

FIGURE 4

Figure 4. R²-values for Subject P1 and P2 with two different combinations of classes (Grasp vs. Thumb-finger, Grasp vs. Index-finger, Thumb-finger vs. Index-finger) during seizure monitoring. The power spectrum shown here was calculated with a frequency bin width of 1 Hz for each electrode. The selected frequency bins of P2 with combination of thumb-finger and index-finger, for example, were 64, 65, 97, 98, 99 within electrodes 3, 4, 9, 10, 16. (A) R²-values of Subject P1; (B) R²-values of Subject P2.

FIGURE 5

Figure 5. R²-values for Subject P3 and P4 with two different combinations of classes (Grasp vs. Thumb-finger, Grasp vs. Index-finger, Thumb-finger vs. Index-finger) during awake craniotomy. The power spectrum shown here was calculated with a frequency bin width of 1 Hz for each electrode. The selected frequency bins of P3 with combination of Grasp and index-finger, for example, were 64, 65, 97, 98, 99 within electrodes 3, 4, 9, 10, 16. And similarly, the selected frequency bins of P3 with combination of thumb-finger and index-finger were 2, 14, 18, 35, 65 within electrodes 4, 11, 12, 13, 14. (A) R²-values of Subject P3; (B) R²-values of Subject P4.

As shown in Figure 5A, for example, the selected frequency bins and electrodes of P3 for “Grasp vs. index-finger” and “thumb-finger vs. index-finger” are different, indicating that different types of hand motion is induced by different cortical areas. Figures 4, 5A show the R²-values of Subject P1, P2, and P3 with two different combinations of classes, the movement-related information is highly localized in a several electrodes, which is in line with previous ECoG studies (Leuthardt et al., 2004). For “Grasp vs. Thumb-finger” and “Thumb-finger vs. Index-finger” of Subject P4 (Figure 5B), the activity in the low frequency band is widespread, and most electrodes over the grid contain movement-related information. For the higher frequency ranges (>80 Hz), the activity is localized, and only a few electrodes contain movement-related information.

Figure 6 shows the R²-values for two-class combinations (motion state and idle state) with 1 Hz frequency range bins. For both seizure monitor (P2) and awake craniotomy (P4) signal, the activity in the alpha (8~12 Hz) and beta (13~28 Hz) frequency bands was widespread, and lots of electrodes contained movement-related information. Within the high frequency ranges (>50 Hz), the activity was localized to a few electrodes, and this result was in accordance with the existing literature studies (Spüler et al., 2014).

FIGURE 6

Figure 6. R²-values for Subject P2, P4 with combinations of motion and idle state. Integrating all types of hand motions duration as motion state, and all the relax interval as idle state. (A) R²-values of Subject P2; (B) R²-values of Subject P4.

3.2. Decoding of Hand Motion Types

Different class combinations with 10 Hz frequency range bins were tested by cross-validation to estimate how well different hand motions can be decoded. Results from the cross-validation with combination of three and two classes of subjects P1~P4 are presented in Tables 1, 2 with method I and method II, respectively. “Two-class mean” in Tables 1, 2 is the average of all two-class combinations, i.e., average of Thu-Ind, Thu-Gra and Ind-Gra (Thumb, Index, and Grasp are abbreviated with Thu, Ind, and Gra respectively).

TABLE 1

Table 1. Three and two classes classification results from the offline analysis applying method I.

TABLE 2

Table 2. Three and two classes classification results from the offline analysis applying method II.

Applying method I, the classification accuracy of three classes for P1~P4 were 90.17, 96.00, 91.77, and 92.95% respectively. And the “two-class mean” of P1~P4 were 96.92, 97.08, 94.22, and 93.76% respectively. For method II, the frequency band of the stationary signal was 50~80 Hz for P1, P3, P4, and 8~25 Hz for P2. The classification accuracy of three classes for P1~P4 were 72.00, 93.17, 95.22, and 90.36% respectively. And the “two-class mean” of P1~P4 were 77.19, 95.35, 97.06, and 94.13% respectively.

Additional analysis was performed to classify more complex hand motion types. Results of P1 with combinations of 2~5 classes are illustrated in Figure 7, and only class combinations with classification accuracy higher than 70% are presented. The best two-class accuracy was achieved for the combination of grasp and thumb-finger motion with accuracy of 98.75%, the best three-class was combination of grasp, wrist motion and index-finger motion with accuracy of 94.17%, while the best four-class was combination of grasp, thumb-finger motion, wrist motion and index-finger motion with accuracy of 81.25%. The five-class accuracy was acceptable with accuracy of 75.40%.

FIGURE 7

Figure 7. Classification accuracies of Subject P1 with combinations of 2~5 classes of hand motions. The power spectrum was calculated with a frequency bin width of 10 Hz for each electrode. Only class combinations with accuracy higher than 70% were presented. T1~5 means hand unfold, hand grasp, thumb-finger motion, wrist motion and index-finger motion respectively. T1T3 is the combination of T1 and T3, and so on for the rest combination. “ALL” is the combination of all the five motion types.

3.3. Comparison of Method I and II in Awake Craniotomy Condition

For awake craniotomy signal (P3, P4), method II achieved a better decoding performance than method I, as shown in Figure 8. Particularly, three-class accuracy of P3 was 91.77% with method I and 95.22% with method II, i.e., 3.75% improvement. For Ind-Gra (P3) and Thu-Ind (P4), it had 12.01, 6.01% improvement, respectively. Results indicated that method II was a good algorithm for awake craniotomy signal decoding. Because of the two different clinical conditions (under anesthetic and not), ECoG signals during awake craniotomy and seizure monitoring may have some difference. Subjects was under anesthetic state during awake craniotomy, and both volatile and intravenous anesthetics had deep influence on the central nervous system (Engelhard and Werner, 2006). With increase of end-expiratory concentrations, the spectral edge frequency decreased, the total power and relative power in the delta and theta band increased and the power in the beta band decreased (Schwender et al., 1998; Gugino et al., 2001). These two different clinical conditions may gave a reason to the better decoding performance of method II. And also, the decoding results indicated that method I and II were robust enough for both clinical conditions.

FIGURE 8

Figure 8. Classification accuracies of Subject P3, P4 (awake craniotomy) applying method I (AR-LDA with WL feature) and method II (dSSA-mulCSP-LDA). “P3-mean” is the average of all two class combination of Subject P3, i.e., average of thu-Ind, Thu-Gra and Ind-Gra. “P4-mean” is similar to “P3-mean.” The power spectrum used in method I was calculated with a frequency bin width of 10 Hz for each electrode. The frequency band of the stationary signal used in method II was 50~80 Hz for P3, P4.

AR-LDA and mulCSP-LDA methods have been widely used in EEG and MEG signal decoding, this study gave an improvement based on the two methods for ECoG signal processing. WL feature was considered in the AR-LDA method, and SSA was applied in the mulCSP-LDA method. Table 3 showed the decoding results of awake craniotomy signal (A1: AR-LDA with WL, A2: traditional AR-LDA; B1: SSA-mulCSP-LDA, B2: traditional mulCSP-LDA). With WL feature, the three-class accuracy of P3 was 91.77%, while 78.67% without WL feature. For Thu-Ind, 5.03 and 3.10% improvements were obtained for P3 and P4. Applying SSA process, the three-class accuracy of P4 was 90.36%, while 86.05% with traditional method. And 3.05% improvement was obtained for P4 with Thu-Ind.

TABLE 3

Table 3. Decoding results of awake craniotomy signal applying traditional AR-LDA or CSP-LDA methods, and results using improved methods.

4. Discussion

Results (Tables 1, 2) obtained by two different methods verified the possibility of decoding multiple hand motion types with high accuracy. Two kinds of subjects were involved in this study, i.e., during awake craniotomy and seizure monitoring processes. Although the clinical condition (Engelhard and Werner, 2006) and signal characteristics (Schwender et al., 1998; Gugino et al., 2001) were different, both methods were robust enough to achieve a high decoding accuracy. Especially, subjects undergoing awake craniotomy achieved a comparable results to the subjects during seizure monitoring. For awake craniotomy subjects P3, P4, the accuracy of three classes were 91.77, 92.95% using method I, and 95.22, 90.36% using method II. It is also important to note that, similar hand motions are considered, like index-finger motion and thumb-finger motion. The accuracies of P1~P4 (index-finger vs. thumb-finger) were 98.75, 99.00 97.50, and 81.27% respectively (Table 1). For P1, the five-class accuracy was 75.40%, which once again showed the possibility of decoding complex hand motions with ECoG signal. These results of the seizure monitor signal were consistence with previous studies (Pistohl et al., 2012; Chestek et al., 2013; Bleichner et al., 2014).

So far, additional important questions need to be answered to accelerate research of ECoG-based BCI, and the ECoG feature selection was one of those (Schalk and Leuthardt, 2011). As AR feature and CSP feature were commonly used for neural electrophysiological signals analysis, two methods, i.e., AR-LDA and mulCSP-LDA, were applied in this studies to further validate the decoding results. Preliminary results indicated that the later one was a good algorithm for awake craniotomy signal decoding. However, this case study mainly focused on the feasibility of decoding awake craniotomy ECoG signal, the best feature and optimal decoding algorithm for intraoperative ECoG signal need to be intensively studied in further researches.

With substantial theoretical and empirical evidence, ECoG-based implant could enhance the functional capability of disabled patients by enabling their ability to modulate their environment, communicate, or control a prosthesis. Looking into the future, as the risk profile is low enough, it may also become reasonable to contemplate implants that augment capabilities in normal functioning adults (Schalk and Leuthardt, 2011). For a pure BCI-purpose implantation, it would be highly necessary to demonstrate reliable neuroprosthetic control during surgical implantation (i.e., during awake craniotomy), in order to verify the placement of electrodes and troubleshoot any technical difficulties during the intraoperation (Fifer et al., 2014). Classifying multiple hand motion types during an awake craniotomy was the crucial first step toward the dexterous neuroprosthetic control during the time of surgical implantation.

For clinical condition, the human ECoG signals were mainly measured on patients with intractable epilepsy and brain tumor. ECoG has been verified as an effective approach for precise identification of eloquent cortex prior to surgical resections adjacent to motor cortex (Roland et al., 2010; Schalk and Leuthardt, 2011; Kamada et al., 2014). Speech and motor cortex could be localized broadly and grossly in real time during an awake craniotomy (Roland et al., 2010). By taking multiple hand motion types into consideration, it has the potential for more precise identification of eloquent cortex, and thus reduces the risk of motor related paralysis after surgical resection.

As the motor imagery is associated with actual movement in ECoG signals (Miller et al., 2010) and it is very hard (and sometimes unable) for patients to execute actual moment during awake craniotomy, ECoG signal may be a good approach for eloquent cortex identification based on motor imagery or intended movement. Using motor imagery ECoG signal during awake craniotomy to localize the eloquent cortex in real time can be a useful BCI system, which may help both patients and neurosurgeons.

ECoG decoding in this case study was completed off-line, so real-time decoding should be performed during an awake craniotomy for optimal placement of electrodes and on-line identification of precise eloquent cortex. In addition, for more precise cortex localization, micro-electrode grids should be taken into consideration, so as to obtain ECoG signal with higher spatial resolution.

5. Conclusion

This study verified the possibility of decoding multiple hand motion types during an awake craniotomy, and its accuracy was comparable with subjects during seizure monitoring. Even similar hand motions, like index-finger motion and thumb-finger motion, could be decoded with accuracy of 98.75, 99.00, 97.50, and 81.27% for P1~P4, respectively, which once again showed the possibility of decoding complex hand motions with ECoG signal. Two methods were applied to further validate the decoding results, both of the methods could achieve high decoding accuracy, and method II showed good performance for awake craniotomy signal decoding. We finally suggest that, ECoG signal during awake craniotomy could be used for intraoperative BCI study and precise localization of eloquent cortex, and intensive real-time study should be performed during awake craniotomy.

Author Contributions

DZ and LC conceived the study, DZ and LC designed the experiment, TX, ZW, and LC conducted the experiments. TX, ZW, and DZ analyzed the data. TX, DZ, and XZ co-drafted the paper, XZ coordinated the work.

Funding

This work is supported by the National Basic Research Program (973 Program) of China (Grant No.2011CB013305, No. 2015CB755500) and the National Natural Science Foundation of China (Grant No.51475292, No.51375296).

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgment

We thank Lin Yao, Jianwei Liu, Zhifeng Shi, and Guangye Li for the help in this work. In particular, we thank the patients who volunteered to be subjects in our experiments.

References

Bleichner, M. G., Freudenburg, Z. V., Jansma, J. M., Aarnoutse, E. J., Vansteensel, M. J., and Ramsey, N. F. (2014). Give me a sign: decoding four complex hand gestures based on high-density ecog. Brain Struct. Funct. doi: 10.1007/s00429-014-0902-x. [Epub ahead of print].

PubMed Abstract | CrossRef Full Text | Google Scholar

Chestek, C. A., Gilja, V., Blabe, C. H., Foster, B. L., Shenoy, K. V., Parvizi, J., et al. (2013). Hand posture classification using electrocorticography signals in the gamma band over human sensorimotor brain areas. J. Neural Eng. 10:026002. doi: 10.1088/1741-2560/10/2/026002

PubMed Abstract | CrossRef Full Text

Crone, N. E., Miglioretti, D. L., Gordon, B., and Lesser, R. P. (1998). Functional mapping of human sensorimotor cortex with electrocorticographic spectral analysis. ii. event-related synchronization in the gamma band. Brain 121, 2301–2315. doi: 10.1093/brain/121.12.2301

PubMed Abstract | CrossRef Full Text | Google Scholar

Engelhard, K. Werner, C. (2006). Inhalational or intravenous anesthetics for craniotomies? pro inhalational. Curr. Opin. Anesthesiol. 19, 504–508. doi: 10.1097/01.aco.0000245275.76916.87

PubMed Abstract | CrossRef Full Text | Google Scholar

Fifer, M. S., Hotson, G., Wester, B. A., McMullen, D. P., Wang, Y., Johannes, M. S., et al. (2014). Simultaneous neural control of simple reaching and grasping with the modular prosthetic limb using intracranial eeg. Neural Syst. Rehabil. Eng. IEEE Trans. 22, 695–705. doi: 10.1109/TNSRE.2013.2286955

PubMed Abstract | CrossRef Full Text | Google Scholar

Flamary, R., and Rakotomamonjy, A. (2012). Decoding finger movements from ecog signals using switching linear models. Front. Neurosci. 6:29. doi: 10.3389/fnins.2012.00029

PubMed Abstract | CrossRef Full Text | Google Scholar

Fukunaga, K. (1990). Introduction to Statistical Pattern Recognition. San Diego, CA: Academic Press.

Graimann, B., Huggins, J. E., Schlögl, A., Levine, S. P., and Pfurtscheller, G. (2003). Detection of movement-related patterns in ongoing single-channel electrocorticogram. Neural Syst. Rehabil. Eng. IEEE Trans. 11, 276–281. doi: 10.1109/TNSRE.2003.816863

PubMed Abstract | CrossRef Full Text | Google Scholar

Gugino, L., Chabot, R., Prichep, L., John, E., Formanek, V., and Aglio, L. (2001). Quantitative eeg changes associated with loss and return of consciousness in healthy adult volunteers anaesthetized with propofol or sevoflurane. Br. J. Anaesth. 87, 421–428. doi: 10.1093/bja/87.3.421

PubMed Abstract | CrossRef Full Text | Google Scholar

Harris, B. (1967). Spectral Analysis of Time Series. New York, NY: Wiley.

Kamada, K., Ogawa, H., Kapeller, C., Prueckl, R., and Guger, C. (2014). “Rapid and low-invasive functional brain mapping by realtime visualization of high gamma activity for awake craniotomy,” in Engineering in Medicine and Biology Society (EMBC), 2014 36th Annual International Conference of the IEEE (Chicago, IL: IEEE), 6802–6805. doi: 10.1109/EMBC.2014.6945190

PubMed Abstract | CrossRef Full Text

Kubánek, J., Miller, K., Ojemann, J. G., and Wolpaw, J. R, Schalk, G. (2009). Decoding flexion of individual fingers using electrocorticographic signals in humans. J. Neural Eng. 6:066001. doi: 10.1088/1741-2560/6/6/066001

PubMed Abstract | CrossRef Full Text

Leuthardt, E., Schalk, G., Wolpaw, J., Ojemann, J., and Moran, D. (2004). A brain computer interface using electrocorticographic signals in humans. J. Neural Eng. 1, 63–71. doi: 10.1088/1741-2560/1/2/001

PubMed Abstract | CrossRef Full Text | Google Scholar

Leuthardt, E. C., Miller, K., Anderson, N. R., Schalk, G., Dowling, J., Miller, J., et al. (2007). Electrocorticographic frequency alteration mapping: a clinical technique for mapping the motor cortex. Neurosurgery 60, 260–271. doi: 10.1227/01.NEU.0000255413.70807.6E

PubMed Abstract | CrossRef Full Text | Google Scholar

Liang, N., and Bougrain, L. (2012). Decoding finger flexion from band-specific ecog signals in humans. Front. Neurosci. 6:91. doi: 10.3389/fnins.2012.00091

PubMed Abstract | CrossRef Full Text | Google Scholar

Liu, G. Q., Huang, G., and Zhu, X. Y. (2009). Application of csp method in multi-class classification. Chin. J. Biomed. Eng. 28, 935–940. doi: 10.3969/j.issn.0258-8021.2009.06.024

CrossRef Full Text

Liu, J., Sheng, X., Zhang, D., Jiang, N., and Zhu, X. (2015). Towards zero re-training for myoelectric control based on common model component analysis. IEEE Trans. Neural. Syst. Rehabil. Eng. doi: 10.1109/tnsre.2015.2420654. [Epub ahead of print].

PubMed Abstract | CrossRef Full Text | Google Scholar

Lotte, F. (2012). “A new feature and associated optimal spatial filter for eeg signal classification: waveform length,” in Pattern Recognition (ICPR), 2012 21st International Conference on (Tsukuba: IEEE), 1302–1305.

Miller, K., Zanos, S., Fetz, E., Den Nijs, M., and Ojemann, J. (2009). Decoupling the cortical power spectrum reveals real-time representation of individual finger movements in humans. J. Neurosci. 29, 3132–3137. doi: 10.1523/JNEUROSCI.5506-08.2009

PubMed Abstract | CrossRef Full Text | Google Scholar

Miller, K. J., Leuthardt, E. C., Schalk, G., Rao, R. P., Anderson, N. R., Moran, D. W., et al. (2007). Spectral changes in cortical surface potentials during motor movement. J. Neurosci. 27, 2424–2432. doi: 10.1523/JNEUROSCI.3886-06.2007

PubMed Abstract | CrossRef Full Text | Google Scholar

Miller, K. J., Schalk, G., Fetz, E. E., den Nijs, M., Ojemann, J. G., and Rao, R. P. (2010). Cortical activity during motor execution, motor imagery, and imagery-based online feedback. Proc. Natl. Acad. Sci. U.S.A. 107, 4430–4435. doi: 10.1073/pnas.0913697107

PubMed Abstract | CrossRef Full Text | Google Scholar

Pistohl, T., Schulze-Bonhage, A., Aertsen, A., Mehring, C., and Ball, T. (2012). Decoding natural grasp types from human ecog. Neuroimage 59, 248–260. doi: 10.1016/j.neuroimage.2011.06.084

PubMed Abstract | CrossRef Full Text | Google Scholar

Ramoser, H., Muller-Gerking, J., and Pfurtscheller, G. (2000). Optimal spatial filtering of single trial eeg during imagined hand movement. Rehabil. Eng. IEEE Trans. 8, 441–446. doi: 10.1109/86.895946

PubMed Abstract | CrossRef Full Text | Google Scholar

Roland, J., Brunner, P., Johnston, J., Schalk, G., and Leuthardt, E. C. (2010). Passive real-time identification of speech and motor cortex during an awake craniotomy. Epilepsy Behav. 18, 123–128. doi: 10.1016/j.yebeh.2010.02.017

PubMed Abstract | CrossRef Full Text | Google Scholar

Samek, W., Muller, K., Kawanabe, M., and Vidaurre, C. (2012). “Brain computer interfacing in discriminative and stationary subspaces,” in Engineering in Medicine and Biology Society (EMBC), 2012 Annual International Conference of the IEEE (San Diego, CA: IEEE), 2873–2876. doi: 10.1109/EMBC.2012.6346563

CrossRef Full Text

Schalk, G., and Leuthardt, E. C. (2011). Brain-computer interfaces using electrocorticographic signals. Biomed. Eng. IEEE Rev. 4, 140–154. doi: 10.1109/RBME.2011.2172408

PubMed Abstract | CrossRef Full Text | Google Scholar

Schwender, D., Daunderer, M., Klasing, S., Finsterer, U., and Peter, K. (1998). Power spectral analysis of the electroencephalogram during increasing end-expiratory concentrations of isoflurane, desflurane and sevoflurane. Anaesthesia 53, 335–342. doi: 10.1046/j.1365-2044.1998.00332.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Sheikh, H., McFarland, D. J., Sarnacki, W. A., and Wolpaw, J. R. (2003). Electroencephalographic (eeg)-based communication: eeg control versus system performance in humans. Neurosci. Lett. 345, 89–92. doi: 10.1016/S0304-3940(03)00470-1

PubMed Abstract | CrossRef Full Text | Google Scholar

Spüler, M., Walter, A., Ramos-Murguialday, A., Naros, G., Birbaumer, N., Gharabaghi, A., et al. (2014). Decoding of motor intentions from epidural ecog recordings in severely paralyzed chronic stroke patients. J. Neural Eng. 11:066008. doi: 10.1088/1741-2560/11/6/066008

PubMed Abstract | CrossRef Full Text

Tkach, D., Huang, H., and Kuiken, T. A. (2010). Research study of stability of time-domain features for electromyographic pattern recognition. J. Neuroeng. Rehabil. 7:21. doi: 10.1186/1743-0003-7-21

CrossRef Full Text

von Bünau, P., Meinecke, F. C., Király, F. C., and Müller, K.-R. (2009). Finding stationary subspaces in multivariate time series. Phys. Rev. Lett. 103:214101. doi: 10.1103/PhysRevLett.103.214101

PubMed Abstract | CrossRef Full Text

von Bunau, P., Meinecke, F. C., Scholler, S., and Muller, K. (2010). “Finding stationary brain sources in eeg data,” in Engineering in Medicine and Biology Society (EMBC), 2010 Annual International Conference of the IEEE (IEEE), 2810–2813.

Google Scholar