# LEARNED BRAIN SELF-REGULATION FOR EMOTIONAL PROCESSING AND ATTENTIONAL MODULATION: FROM THEORY TO CLINICAL APPLICATIONS

EDITED BY: Sergio Ruiz, Ranganatha Sitaram, Niels Birbaumer and Francisco Javier Zamorano PUBLISHED IN: Frontiers in Behavioral Neuroscience

#### *Frontiers Copyright Statement*

*© Copyright 2007-2016 Frontiers Media SA. All rights reserved. All content included on this site, such as text, graphics, logos, button icons, images, video/audio clips, downloads, data compilations and software, is the property of or is licensed to Frontiers Media SA ("Frontiers") or its licensees and/or subcontractors. The copyright in the text of individual articles is the property of their respective authors, subject to a license granted to Frontiers.*

*The compilation of articles constituting this e-book, wherever published, as well as the compilation of all other content on this site, is the exclusive property of Frontiers. For the conditions for downloading and copying of e-books from Frontiers' website, please see the Terms for Website Use. If purchasing Frontiers e-books from other websites or sources, the conditions of the website concerned apply.*

*Images and graphics not forming part of user-contributed materials may not be downloaded or copied without permission.*

*Individual articles may be downloaded and reproduced in accordance with the principles of the CC-BY licence subject to any copyright or other notices. They may not be re-sold as an e-book.*

*As author or other contributor you grant a CC-BY licence to others to reproduce your articles, including any graphics and third-party materials supplied by you, in accordance with the Conditions for Website Use and subject to any copyright notices which you include in connection with your articles and materials.*

> *All copyright, and all rights therein, are protected by national and international copyright laws.*

*The above represents a summary only. For the full conditions see the Conditions for Authors and the Conditions for Website Use.*

ISSN 1664-8714 ISBN 978-2-88919-980-8 DOI 10.3389/978-2-88919-980-8

# About Frontiers

Frontiers is more than just an open-access publisher of scholarly articles: it is a pioneering approach to the world of academia, radically improving the way scholarly research is managed. The grand vision of Frontiers is a world where all people have an equal opportunity to seek, share and generate knowledge. Frontiers provides immediate and permanent online open access to all its publications, but this alone is not enough to realize our grand goals.

# Frontiers Journal Series

The Frontiers Journal Series is a multi-tier and interdisciplinary set of open-access, online journals, promising a paradigm shift from the current review, selection and dissemination processes in academic publishing. All Frontiers journals are driven by researchers for researchers; therefore, they constitute a service to the scholarly community. At the same time, the Frontiers Journal Series operates on a revolutionary invention, the tiered publishing system, initially addressing specific communities of scholars, and gradually climbing up to broader public understanding, thus serving the interests of the lay society, too.

# Dedication to Quality

Each Frontiers article is a landmark of the highest quality, thanks to genuinely collaborative interactions between authors and review editors, who include some of the world's best academicians. Research must be certified by peers before entering a stream of knowledge that may eventually reach the public - and shape society; therefore, Frontiers only applies the most rigorous and unbiased reviews.

Frontiers revolutionizes research publishing by freely delivering the most outstanding research, evaluated with no bias from both the academic and social point of view. By applying the most advanced information technologies, Frontiers is catapulting scholarly publishing into a new generation.

# What are Frontiers Research Topics?

Frontiers Research Topics are very popular trademarks of the Frontiers Journals Series: they are collections of at least ten articles, all centered on a particular subject. With their unique mix of varied contributions from Original Research to Review Articles, Frontiers Research Topics unify the most influential researchers, the latest key findings and historical advances in a hot research area! Find out more on how to host your own Frontiers Research Topic or contribute to one as an author by contacting the Frontiers Editorial Office: researchtopics@frontiersin.org

# **LEARNED BRAIN SELF-REGULATION FOR EMOTIONAL PROCESSING AND ATTENTIONAL MODULATION: FROM THEORY TO CLINICAL APPLICATIONS**

Topic Editors:

**Sergio Ruiz,** Pontificia Universidad Católica de Chile, Chile **Ranganatha Sitaram,** Pontificia Universidad Católica de Chile, Chile **Niels Birbaumer,** University of Tübingen, Germany **Francisco Javier Zamorano,** Universidad del Desarrollo, Chile

The cover image is an artist's depiction of self-regulation of the bilateral anterior insulae in a psychopath during real-time fMRI neurofeedback training and the ensuing effective connectivity changes. The central part of the image shows the regions of interest (ROIs) in the brain (two yellow squares are placed on the left and right anterior insulae); the thermometer indicates the changes in the hemodynamic signals in the two ROIs; and the two schematics of the brain on either side of the central image show the brain's connections during weak and strong periods of performance of brain selfregulation, respectively, in the course of neurofeedback training.

Figure created by Pradyumna Sepúlveda, Ranganatha Sitaram and Sergio Ruiz

Mounting evidence in the last years has demonstrated that self-regulation of brain activity can successfully be achieved by neurofeedback (NF). These methodologies have constituted themselves as new tools for cognitive neuroscience establishing causal links between voluntary brain activations and cognition and behavior, and as potential novel approaches for clinical applications in severe neuropsychiatric disorders (e.g. schizophrenia, depression, Parkinson´s disease, etc.). Current developments of brain imaging-based neurofeedback include the study of the behavioral modifications and neural reorganization produced by learned regulation of the activity of circumscribed brain regions and neuronal network activations.

In a rapidly developing field, many open questions and controversies have arisen, i.e. choosing the proper experimental design, the adequate use of control conditions and subjects, the mechanism of learning involved in brain self-regulation, and the still unexplored potential long-lasting effect on brain reorganization and clinical alleviation, among others.

This special issue on self-regulation of the brain of emotion and attention using NF approaches interested authors to report technical and methodological advances, scientific investigations in understanding the relation between brain activity and behaviour using NF, and finally studies developing clinical treatment of emotional and attentional disorders. The editors of this special issue anticipate rapid developments in this emerging field.

**Citation:** Ruiz, S., Sitaram, R., Birbaumer, N., Zamorano, F. J., eds. (2016). Learned Brain Self-Regulation for Emotional Processing and Attentional Modulation: From Theory to Clinical Applications. Lausanne: Frontiers Media. doi: 10.3389/978-2-88919-980-8

# Table of Contents


# **I. Methodological advances in Neurofeedback**

*10 FRIEND Engine Framework: a real time neurofeedback client-server system for neuroimaging studies* Rodrigo Basilio, Griselda J. Garrido, João R. Sato, Sebastian Hoefle,

Bruno R. P. Melo, Fabricio A. Pamplona, Roland Zahn and Jorge Moll


Cornelia Pirulli, Anna Fertonani and Carlo Miniussi


Krystyna A. Mathiak, Eliza M. Alawi, Yury Koush, Miriam Dyck, Julia S. Cordes, Tilman J. Gaber, Florian D. Zepf, Nicola Palomero-Gallagher, Pegah Sarkheil, Susanne Bergert, Mikhail Zvyagintsev and Klaus Mathiak

*80 Source-based neurofeedback methods using EEG recordings: training altered brain activity in a functional brain source derived from blind source separation* David J. White, Marco Congedo and Joseph Ciorciari

# **II. Scientific studies**

*89 Self-regulation of frontal-midline theta facilitates memory updating and mental set shifting*

Stefanie Enriquez-Geppert, René J. Huster, Christian Figge and Christoph S. Herrmann

*102 Detaching from the negative by reappraisal: the role of right superior frontal gyrus (BA9/32)*

Rosalux Falquez, Blas Couto, Agustin Ibanez, Martin T. Freitag, Moritz Berger, Elisabeth A. Arens, Simone Lang and Sven Barnow

*118 Corrigendum: Detaching from the negative by reappraisal: the role of right superior frontal gyrus (BA9/32)*

Rosalux Falquez, Blas Couto, Agustin Ibanez, Martin T. Freitag, Moritz Berger, Elisabeth A. Arens, Simone Lang and Sven Barnow

*122 Insula and inferior frontal triangularis activations distinguish between conditioned brain responses using emotional sounds for basic BCI communication*

Linda van der Heiden, Giulia Liberati, Ranganatha Sitaram, Sunjung Kim, Piotr Jas'kowski, Antonino Raffone, Marta Olivetti Belardinelli, Niels Birbaumer and Ralf Veit

*129 Down-regulation of amygdala activation with real-time fMRI neurofeedback in a healthy female sample*

Christian Paret, Rosemarie Kluetsch, Matthias Ruf, Traute Demirakca, Steffen Hoesterey, Gabriele Ende and Christian Schmahl

*144 Self-regulation of circumscribed brain activity modulates spatially selective and frequency specific connectivity of distributed resting state networks* Mathias Vukelic' and Alireza Gharabaghi

# **III. Clinical Studies**

*154 Comparison of anterior cingulate vs. insular cortex as targets for real-time fMRI regulation during pain stimulation*

Kirsten Emmert, Markus Breimhorst, Thomas Bauermann, Frank Birklein, Dimitri Van De Ville and Sven Haller

*167 Neurofeedback of the difference in activation of the anterior cingulate cortex and posterior insular cortex: two functionally connected areas in the processing of pain*

Mariela Rance, Michaela Ruttorf, Frauke Nees, Lothar R. Schad and Herta Flor


Carlos Escolano, Mayte Navarro-Gil, Javier Garcia-Campayo, Marco Congedo, Dirk De Ridder and Javier Minguez

*201 Volitional control of the anterior insula in criminal psychopaths using real-time fMRI neurofeedback: a pilot study*

Ranganatha Sitaram, Andrea Caria, Ralf Veit, Tilman Gaber, Sergio Ruiz and Niels Birbaumer


# **III. Other studies**

*256 Internet addictive individuals share impulsivity and executive dysfunction with alcohol-dependent patients*

Zhenhe Zhou, Hongmei Zhu, Cui Li and Jun Wang

*264 Winning the game: brain processes in expert, young elite and amateur table tennis players*

Sebastian Wolf, Ellen Brölz, David Scholz, Ander Ramos-Murguialday, Philipp M. Keune, Martin Hautzinger, Niels Birbaumer and Ute Strehl

*276 Lateralization of music processing with noises in the auditory cortex: an fNIRS study*

Hendrik Santosa, Melissa Jiyoun Hong and Keum-Shik Hong

*285 Combined neuromodulatory interventions in acute experimental pain: assessment of melatonin and non-invasive brain stimulation*

Nádia Regina Jardim da Silva, Gabriela Laste, Alícia Deitos, Luciana Cadore Stefani, Gustavo Cambraia-Canto, Iraci L. S. Torres, Andre R. Brunoni, Felipe Fregni and Wolnei Caumo

# Editorial: Learned Brain Self-Regulation for Emotional Processing and Attentional Modulation: From Theory to Clinical Applications

#### Sergio Ruiz 1, 2, 3 \*, Niels Birbaumer 3, 4 and Ranganatha Sitaram1, 2, 3, 5 \*

<sup>1</sup> Psychiatry Department, Interdisciplinary Center for Neuroscience, School of Medicine, Pontificia Universidad Católica de Chile, Santiago, Chile, <sup>2</sup> Laboratory of Brain-Machine Interfaces and Neuromodulation, Pontificia Universidad Católica de Chile, Santiago, Chile, <sup>3</sup> Institute of Medical Psychology and Behavioral Neurobiology, University of Tübingen, Tübingen, Germany, <sup>4</sup> Ospedale San Camillo, Istituto di Ricovero e Cura a Carattere Scientifico, Venezia, Italy, <sup>5</sup> Institute for Biological and Medical Engineering, Pontificia Universidad Católica de Chile, Santiago, Chile

Keywords: neurofeedback, self-regulation, brain-computer interfaces, emotion, attention

**The Editorial on the research topic**

# **Learned Brain Self-Regulation for Emotional Processing and Attentional Modulation: From Theory to Clinical Applications**

Mounting evidence in the last years has demonstrated that self-regulation of brain activity can be achieved by neurofeedback (NF). These methodologies have constituted themselves as new tools for cognitive neuroscience, to establish causal links between volitionally controlled brain activations and cognition and behavior, and as potential clinical applications in severe neuropsychiatric disorders (e.g., schizophrenia, depression, Parkinson's disease, etc.). Current developments of brain imaging-based NF include the study of the behavioral modifications and neural reorganization produced by learned regulation of the activity of circumscribed brain regions and neuronal networks.

In this rapidly developing field, many open questions and controversies have arisen, i.e., choosing the proper experimental design, the adequate use of control conditions and subjects, the mechanism of learning involved in brain self-regulation, and the effects on brain reorganization and clinical alleviation in severe brain disorders, among others. The current research topic includes theoretical, technical and experimental achievements in NF based on EEG/MEG and hemodynamic-based NF.

The first part of the current issue considers several methodological advances in the field.

The growing researcher- and user-base of real-time fMRI-NF applications has prompted a handful of laboratories to build open and free software tools and libraries. Basilio et al. report one such toolbox, named Functional Real-time Interactive Endogeneous Neuromodulation and Decoding (FRIEND). The authors present a client-server cross-platform solution, which provides a number of new features including, customization and integration of user-defined graphical interfaces, devices, and data processing, with novel front-ends and plugin support.

Recent studies in NF are increasingly relying on multivariate classification for decoding brain states in real-time for more effective learning. However, an often unrecognized problem is that the classification accuracy is used as a common measure for evaluating the technical performance of a classifier as well as the performance of participants in learning brain self-regulation; thus

Edited and reviewed by: Nuno Sousa, University of Minho, Portugal

#### \*Correspondence:

Sergio Ruiz sruiz@uc.cl; Ranganatha Sitaram rasitaram@uc.cl

Received: 06 March 2016 Accepted: 14 March 2016 Published: 31 March 2016

#### Citation:

Ruiz S, Birbaumer N and Sitaram R (2016) Editorial: Learned Brain Self-Regulation for Emotional Processing and Attentional Modulation: From Theory to Clinical Applications. Front. Behav. Neurosci. 10:62. doi: 10.3389/fnbeh.2016.00062 confounding methodological and neuropsychological outcomes. Bauer and Gharabaghi, propose a solution to this problem based on item-response theory and cognitive load theory, and arrive at a new metric called the zone of maximum development (ZMD) as a measure of participant's cognitive resource and efficacy of NF. Adoption of this measure in future studies should help consolidate its relevance to NF research.

De Massari et al. test the feasibility of online decoding and monitoring of brain states in virtual and mixed reality environments for guiding explicit and implicit learning using ecologically valid scenarios. They report encouraging classification performance for detecting multiple brain states and differentiating between low and high mental workloads, and anticipate the application of this approach for improving self-regulation learning.

Extant literature suggests that transcranial direct current stimulation (tDCS) improves motor learning and influences emotional and attentional processes. Soekadar et al., investigated whether tDCS induced brain activity interferes with brain selfregulation of the EEG mu-rhythm (8–15 Hz). The results of their study shows that tDCS stimulation near the C4 channel causes a signal power increase only in the lower frequencies (below 9 Hz), and hence future applications can safely combine tDCS and NF above the 9 Hz frequency range.

In another work related to tDCS, Pirulli et al., question and examine the common understanding that cathodal tDCS (c-tDCS) has an inhibitory effect on neural activity. The authors varied some important parameters of stimulation over the primary visual cortex, namely, timing, presence of pauses, duration and intensity, and tested their effect on visual orientation discrimination. In contrast to the common understanding, an improvement in task performance was observed when the c-tDCS stimulation was applied before the task for certain parameter values. The authors hypothesize that c-tDCS causes depression of cortical activity in the stimulated region but the brain reacts to restore equilibrium and this might improve visual sensitivity.

In an innovative method called motivational feedback system, Sokunbi et al., show that participants can increase or decrease BOLD levels in response to visual cues or images related to motivational processes such as hunger or craving. They present an example of visual cues of food items that grow or shrink in size, representing feedback, proportional to the subject's selfregulation of the BOLD signal in a brain region that activates hunger or craving for food.

Mathiak et al. expand the literature exploring the role of "reward" as NF signal. They compare the effect of a standard feedback (moving bars), and a social reward feedback (a smiling human face) in an fMRI-NF study that aimed to control the ACC. The experiment demonstrated a higher effectiveness of the social reward feedback, as reflected by higher ACC activity and rewardrelated areas (i.e., putamen). The findings also support the idea that stronger effects on ACC are achieved by social feedback compared to standard feedback during a behavioral test involving a cognitive interference task.

One major drawback of the widely used EEG-NF approach is that activity in any scalp electrode reflects a mixture of activities from multiple sources in the brain and artifacts, which may confound the actual signal from the region of interest, consequently adversely affecting learning of brain self-regulation. To circumvent this problem, White et al., implemented a realtime adaptation of the Blind Source Separation (BSS) algorithm, and tested the technique for NF training of theta oscillatory activity derived from sources in the medial temporal and parietal lobes. Pilot data demonstrate that two of four volunteers learned theta oscillatory control, suggesting moderate feasibility for the approach but calling for further research on this topic.

The second part of the current issue includes several scientific articles.

Enriquez-Geppert et al., investigate the effects of upregulation of the frontal-midline (fm) theta power on executive function through a battery of tasks, and observed improvement in task performance in a 3-back task and reduced mixing and shifting costs in a letter/number shifting task. However, no change was observed on conflict monitoring and motor inhibition, suggesting a specific effect on proactive but not reactive mechanisms of cognitive control. Potential applications of this approach include treatment of executive dysfunctions.

In a rare investigation of the effect of reappraisal of self on emotional events in brain-damaged individuals of tumor or cyst, Falquez et al., show that legions in the right dorsolateral prefrontal cortex and the right dorsal ACC were associated with patients' impairment in the down-regulation of arousal while the intact reappraisal of healthy controls was related to increased gray matter intensity in the same regions. These results indicate that the neural and structural integrity of the right superior frontal gyrus are related to emotional regulation by reappraisal of self. The use of this approach could prove beneficial in regulation of emotional arousal.

The last part of this special issue includes scientific articles that explore the effect of brain self-regulation in symptom alleviation, a topic of great interest in the field (Ruiz et al., 2013; Buyukturkoglu et al., 2015).

Chronic pain, a condition targeted by real-time fMRI-NF since its first implantations on clinical populations, is explored in two articles of this issue. Emmert et al., investigate modulation of pain perception in a group of healthy individuals by downregulation of anterior cingulate and anterior insula during pain stimulation, suggesting that both regions are suitable targets for reducing pain with fMRI-NF.

In a novel approach, Rance et al. (2014) move away from single-ROI self-regulation and explore the capability of individuals to increase the difference in BOLD activation of ACC and posterior insula, two regions of the pain processing network, separately targeted in previous fMRI-NF experiments and pain modulation. Although no correlation was found with pain perception, the finding that individuals can control the differential activation of two functionally connected areas is of importance for clinical applications in disorders of abnormal neural connectivity.

Regarding psychiatric disorders, Cordes et al., investigated the neural strategies used to achieve control of the BOLD signal in the ACC in schizophrenia. The results suggest that schizophrenia patients use different cognitive and neural strategies for self- regulation compared to healthy individuals. In fact, schizophrenia patients activated the dorsal subdivision of the ACC as compared to a control group of healthy individuals, which activated the rostral subdivision, giving support to the idea of a subdivision dysbalance in ACC as part of the psychopathology of the disease.

Escolano et al., explore whether cognitive deficits in depression can be alleviated by NF training of alpha power of parieto-occipital EEG signals. Patients suffering from major depression improved working memory and processing speed compared with control depression subjects (that did not received NF). An interesting outcome involving the correlation of beta power in the genual ACC correlating with processing speed suggests a role of this area in cognitive processing.

In a rare study on brain self-regulation in severe personality disorders, Sitaram et al., investigate the effect of the modulation of the brain fear circuitry in criminal psychopaths. The subjects displayed a low-success rate in regulating insula (different as compared to previous studied in healthy individuals), and a correlation between the severity of psychopathy traits and the difficulties regulating insula cortex. Interestingly, functional connectivity changes in the emotional network are observed throughout NF training, opening interesting questions on potential brain remodeling in severe personality disorders.

In another work on self-regulation of emotional brain areas, Zilverstand et al., presented individuals suffering from spider phobia with a novel dual visual feedback based on the BOLD activity in the insula pertaining to sustained anxiety and the dorsolateral prefrontal cortex pertaining to engagement in regulation. Participants of the NF group achieved downregulation of insula activation levels by cognitive reappraisal and exhibited lower anxiety levels than the control group.

Gharabaghi et al., evaluated the advantages of EEG-NF from the epidural space in a patient with a large ischemic stroke, showing that this methodology can lead to self-regulation of sensorimotor oscillations, even when standard EEG-NF is unsuccessful.

In an effort to develop novel approaches for symptom alleviation in mood disorders, Ray et al., develop a BCI system

# REFERENCES


based on a pattern classifier of brain's affective states. A subjectindependent classifier was first created based on the method of Common Spatial Patterns (CSPs) from the EEG data of several healthy individuals. The BCI system then used the classifier to provide real-time NF for individuals to learn to "match" the affective states provided by the classifier. The authors anticipate the application of this approach in correcting the abnormal affective states in patients suffering from mood disorders.

Scheinost et al., attempted to identify brain connectivity patterns associated with behavioral changes due to fMRI-NF training in Obsessive Compulsive Disorder patients. In this pilot study, it is demonstrated that whole-brain connectivity in the orbitofrontaland anterior prefrontal cortex, collected from resting state-fMRI before the training, correlates with symptom alleviation following NF.

This special issue on self-regulation of the brain of emotion and attention using NF approaches interested authors to report technical and methodological advance, scientific investigations in understanding the relation between brain activity and behavior using NF, and finally studies developing clinical treatment of emotional and attentional disorders. The editors of this special issue anticipate rapid developments in this emerging field.

# AUTHOR CONTRIBUTIONS

All authors listed, have made substantial, direct and intellectual contribution to the work, and approved it for publication.

# ACKNOWLEDGMENTS

We thank Comisión Nacional de Investigación Científica y Tecnológica de Chile (Conicyt Chile) through Fondo Nacional de Desarrollo Científico y Tecnológico Fondecyt (project n ◦ 11121153) and through Anillos de Investigación (Project ACT 1414); the ERA-Net (European Research Area)—New INDIGO project funded by the BMBF (project n◦ 01DQ13004) and Pontificia Universidad Católica de Chile through the Seed Fund University of Texas, Austin—Universidad Católica, Santiago.

network connectivity in schizophrenia. Hum. Brain Mapp. 34, 200–212. doi: 10.1002/hbm.21427

**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2016 Ruiz, Birbaumer and Sitaram. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

#### *Rodrigo Basilio1, Griselda J. Garrido1, João R. Sato1,2, Sebastian Hoefle1, Bruno R. P. Melo1, Fabricio A. Pamplona1, Roland Zahn3 and Jorge Moll <sup>1</sup> \**

*<sup>1</sup> Cognitive and Behavioral Neuroscience Unit and Neuroinformatics Workgroup, D'Or Institute for Research and Education, Rio de Janeiro, Brazil*

*<sup>2</sup> Center of Mathematics, Computation and Cognition, Universidade Federal do ABC, Santo André, Brazil*

*<sup>3</sup> Department of Psychological Medicine, Institute of Psychiatry, King's College, London, UK*

#### *Edited by:*

*Sergio Ruiz, Pontificia Universidad Catolica de Chile, Chile*

#### *Reviewed by:*

*Mohit Rana, Tübingen University, Germany Francisco Javier Zamorano, Universidad del Desarrollo, Chile*

#### *\*Correspondence:*

*Jorge Moll, Cognitive and Behavioral Neuroscience Unit and Neuroinformatics Workgroup, D'Or Institute for Research and Education, Diniz Cordeiro 30, Rio de Janeiro, 22281-100, Brazil e-mail: jorge.moll@idor.org*

In this methods article, we present a new implementation of a recently reported FSL-integrated neurofeedback tool, the standalone version of "Functional Real-time Interactive Endogenous Neuromodulation and Decoding" (FRIEND). We will refer to this new implementation as the FRIEND Engine Framework. The framework comprises a client-server cross-platform solution for real time fMRI and fMRI/EEG neurofeedback studies, enabling flexible customization or integration of graphical interfaces, devices, and data processing. This implementation allows a fast setup of novel plug-ins and frontends, which can be shared with the user community at large. The FRIEND Engine Framework is freely distributed for non-commercial, research purposes.

**Keywords: brain computer interface (BCI), real-time fMRI, FSL, neurofeedback, EEG**

## **INTRODUCTION**

There is a growing push toward the use of real-time fMRI (rt-fMRI) neurofeedback in experimental and clinical investigation, with solid prospects for therapeutic applications (Sulzer et al., 2013; Stoeckel et al., 2014) through the development of brain computer interface (BCI) software integrated with commonly available devices such as fMRI scanners and EEG devices. Successful scientific exploration and application of rt-fMRI neurofeedback will be facilitated by the existence of freely available, user-friendly, and flexible software implementations. To this end, a few research groups have recently contributed with the development of computational tools, each with their own strengths and limitations (LaConte et al., 2007; Zotev et al., 2011; Sorger et al., 2012; Rana et al., 2013; Sato et al., 2013).

Nevertheless, because of the computational and technical complexities that are intrinsic to the emerging field of rtfMRI neurofeedback and the diversity of approaches, there are still few available options for investigators in the area. More importantly, a number of features that could facilitate and encourage advanced users and developers to build on currently available platforms are still lacking. Here, we introduce a new framework that unleashes users to create frontends and customize pipelines for their own studies using the programming language and platform they are most familiar with, flexibly connecting them to a core processing engine. To attain this goal, we have revamped our recently reported rt-fMRI neurofeedback standalone software, "Functional Real-time Interactive Endogenous Modulation and Decoding system" (FRIEND) (Sato et al., 2013), as described below.

#### **FRIEND ENGINE FRAMEWORK**

The original FRIEND neurofeedback tool comprises three processing pipelines (**Figure 1**): (1) the brain decoding-based feedback using support-vector machines (SVM); (2) the single region-of-interest (ROI) based blood-oxygen level dependent (BOLD) feedback; and (3) functional connectivity feedback based on a sliding window of correlations between ROIs. These pipelines were developed and implemented across time, on the basis of research needs in our lab, including the techniques needed to support these pipelines. As such, the first version of FRIEND included only the SVM pipeline for brain decoding-based neurofeedback for emotion modulation, using the libSVM library (Chang and Lin, 2011) and was recently employed in a study of emotional enhancement (Moll et al., 2014). For the purpose of a new research line on motor control physiology and rehabilitation, an ROI activation pipeline for FRIEND was implemented, allowing the flexible creation of anatomically or functionally defined ROIs. Anatomical ROIs are created by selecting predefined areas from a variety of Montreal Neurological Institute (MNI) templates, which are automatically transformed into the subject space. Functionally-defined ROIs can also be created after running an fMRI functional localizer, and then selecting and thresholding a functional cluster for subsequent neurofeedback runs. The third pipeline, focusing on sliding window-based correlations between two brain regions as a guide for neurofeedback information, was developed to enable an ongoing clinical proofof-concept study on remitted major depression, based on the findings of a recent study (Green et al., 2012).

The provision of the ideal feedback to the participant is crucial for the success of any neurofeedback study. In this vein,

ROIs (Sato et al., 2013).

interoperability with other applications such as well-established stimulus presentation tools, including virtual reality ones, and response devices with the neurofeedback interface would be highly desirable (Renaud et al., 2011; Weiskopf, 2012). In the original implementation of FRIEND (Sato et al., 2013), a few standard strategies were implemented, based on a simple visual feedback interface (e.g., rings changing in shape or a thermometer). In that original implementation, the visual feedback could be modified by simply replacing the bitmap figures. However, this initial approach neither allows for a more sophisticated control of stimuli as specialized stimulus presentation softwares do (e.g., http:// www.neurobs.com/), nor does it offer the possibility of immersive game-like experience, which could be more engaging for participants of neurofeedback studies.

values onto a discriminative hyperplane; (2) BOLD level real-time display from

In order to allow users to use their own stimulus feedback strategies and to develop additional processing strategies (e.g., Matlab® pipelines), we have broken the standalone version of FRIEND into smaller parts, encapsulating the complex calculations involving rt-fMRI neurofeedback processing in one unit and the graphic user interface in another. We expect that this approach will enable researchers with average to advanced programming skills to efficiently implement customized graphical interfaces and data processing functionalities, which can be shared with other users via NITRC (http://www.nitrc.org/projects/friend) and GitHub (https://github.com/InstitutoDOr/FriendENGINE). Toward this aim, FRIEND was restructured based on the wellestablished object oriented programming paradigm (Stroustrup, 1988), facilitating modification and implementation of new functionalities.

In FRIEND Engine, basic processes that are common to any neurofeedback pipeline such as anatomical segmentation, motion correction of functional images and gaussian smoothing (**Figure 1**)—largely based on FSL code (http://fsl.fmrib.ox.ac.uk/ fsl/fslwiki/)—are encapsulated in a single unit, called FRIEND Engine (**Figure 2**), the first part of the FRIEND Engine framework. The engine determines the main processing pipeline and defines time points in this pipeline to execute functionalities not implemented in the engine. These functionalities are geared to attain the goal of the study (e.g., classification of brain states, ROI percent signal change and correlation between two ROIs in the previously presented pipelines) and coded separately in independent units, called plug-ins (**Figure 2**) here, the second component of the framework. By definition, a plug-in is an extension of the engine and must therefore be implemented in the same language and be executed on the same platform for compatibility. As such, the engine provides the basic workings whereas the plug-ins implement the specific functionalities necessary for estimation of feedback parameters on the basis of data analysis. The engine does not know and does not need to know how a specific plug-in handles the information. The plug-in library just needs to expose the necessary functions for the engine to work properly. The coordinator of the engine and the neurofeedback processing is the frontend (**Figure 2**), the third part of the FRIEND Engine framework. As the name intuitively implies, it is the graphical interface for the neurofeedback loop, essentially the *interface* of a BCI. It provides feedback information (e.g., BOLD activation level) to the participant, relevant information to the operator (e.g., motion correction parameters), and handles the necessary steps to synchronize the presentation of the feedback with the scanner acquisition. Ideally, the construction of the frontend should not demand learning new and complex technical skills. The optimal solution is allowing users to customize their applications using the programming language and platform of their choice. A TCP/IP network communication protocol is therefore defined to bridge

the frontend and the engine. In this configuration, the engine listens to a port for incoming messages (requests) from the frontend, allowing the engine and frontend to operate on different platforms.

The first step of a neurofeedback study in the FRIEND Engine framework is the same as for the standalone version of FRIEND: the configuration of the specific parameters for the study, such as the input directory and the number of volumes in the acquisition run. This configuration should be provided by the frontend application, which gets the input from the operator of the study and passes it to FRIEND Engine. By default, FRIEND Engine reads the study\_params.txt configuration file located in the same directory of the engine's executable file. The study\_params.txt file is exactly the same as for the standalone version of FRIEND. The frontend can pass a whole configuration file by the TCP/IP network communication protocol to the engine by way of the command "READCONFIG," as explained in the following section. The next important and vital command the frontend must pass on, is the plug-in configuration, which comprises the plug-in library filename and the name of the functions that the engine should call at predefined time points, like the name of the function that calculates the feedback information for a volume. These two messages prepare the engine to properly handle the experiment. The next message the frontend should send is "PREPROC," which executes the same steps executed in the standalone version of FRIEND after the first configuration window.

Next, the frontend needs to send a message indicating that the engine should start processing the acquisition run. There are four options, "PIPELINE," "NBPIPELINE," "FEEDBACK," "NBFEEDBACK" (**Table 1**), explained in the following section. This is equivalent to clicking the "TRAIN" or "FEEDBACK" button in the standalone version of FRIEND. At specific points of the processing, such as the calculation of the feedback, the engine executes the proper plug-in function, previously assigned to the plug-in configuration phase. There is no direct communication between the frontend and the plug-in components.

During the acquisition run, the frontend sends "TEST" messages (**Table 1**), querying for neurofeedback information for each volume of the acquisition scan. The engine executes the configured feedback function of the plug-in to get the feedback information and returns it to the frontend. The frontend must interpret this value and properly display that information to the participant of the experiment. **Figure 3** depicts this message exchange between the frontend and the engine in the avatar finger tapping virtual scenario. It includes a new command, "NEWSESSION" (**Table 1**), which indicates that the engine should create a new session to work with the frontend. That message is only needed in asynchronous communications as explained in Section TCP/IP Communication Protocol. To illustrate this, excerpts of programming code from a Matlab® frontend and the libROI plug-in are provided in the Supplementary Material. FRIEND Engine expects the volume files in exactly the same way as standalone FRIEND does. This implies that the list of computers that can run the engine is restricted to the list of computers that can receive the volume files from the fMRI scanner (or EEG device) in real-time acquisition.

FRIEND Engine runs on Microsoft Windows® (XP or later), Apple Macintosh (OSX 10.8 and above) and Linux (Debian, CentOS 6.4). A mid/high end workstation is required (e.g., PC: quad-core i7, 8 GB RAM or higher, Macintosh: quad-core Intel Core i5, 8 GB RAM or higher). In the original standalone version of FRIEND, FSL (Jenkinson et al., 2012) toolbox commands were encapsulated in a dynamic link Microsoft Windows® library. In FRIEND Engine, this interrelationship was changed: for non-Microsoft Windows® systems, FSL toolbox installation is a pre-requisite and FSL executables are called using system calls to the operational system. This simplifies FSL upgrades, as they can be executed independently from the engine code. In Microsoft

#### **Table 1 | List of commands expected by FRIEND Engine during the TCP/IP communication protocol with the frontend.**


Windows® FRIEND Engine version, our modified source code of the FSL toolbox functionality is embedded within the executable of FRIEND Engine. This embedding process transforms each needed FSL command in a function statically linked to the software. For this reason, no additional installation of the FSL software is necessary.

#### **TCP/IP COMMUNICATION PROTOCOL**

**Table 1** lists the commands expected by the engine in the TCP/IP communication protocol. There are two types of connections: synchronous (blocked) and asynchronous (non-blocked [NB]) connections. In synchronous connections, the frontend needs to wait until command completion to receive the acknowledge response from the engine. Asynchronous connections need to be established when the frontend is not supposed to freeze the execution to wait for the acknowledge response, as in virtual reality scenario frontends. This situation happens when a timeconsuming command needs to be executed, like "PREPROC" or "TRAIN." In this case, regular queries for command termination need to be issued until the expected response is obtained. To appropriately handle asynchronous connections, a multithread approach with at least two threads is adamant: one, the main thread, that performs all the raw real time processing; and the other, the response thread, which responds to queries of various types of information related to the main thread processing. The two threads can respond properly to the frontend by a shared access to the same session. To make these two threads interoperate, the notion of session is introduced. A session is an independent location in the memory of the computer running the engine, capable of storing all the information that is required to be sent back to the frontend, such as neurofeedback information and motion-corrected volume parameters. It is a role of the frontend to present this information to the participant in a user-friendly manner. The Matlab® frontend provided with the software distribution has some built-in capabilities to display feedback and motion parameters.

#### **PLUG-IN LIBRARY**

A plug-in file is a dynamic library file (a .so file on the Linux system, a .dylib file on the Mac OSX system and a .dll file in Microsoft Windows®) that implements specific functions called internally by FRIEND Engine at specific times during the pipeline. This is a major advantage of the FRIEND Engine, because, when in need of additional features, users can focus on writing just the necessary functionality for their specific research needs. This allows customization of the neurofeedback tool, using encapsulated codes that run additional functionalities from external libraries, leaving the engine code intact. This characteristic favors usability and code maintenance, because errors in a plug-in library are also encapsulated in that library and do not affect other plug-ins. This framework makes it easier to setup pilot experiments and to explore new hypotheses.

#### *Plug-in functions and parameters*

The plug-in library must implement all the computational processes required to calculate the feedback responses. A small subset of variables needs to be defined to be used as parameters of the plug-in functions (**Table 2**).

The engine defines six functions (**Table 3**) that are called at predefined time points during the pipeline execution. Not all of those six functions must be implemented in a plug-in library, just the ones necessary to properly calculate the feedback information. We recommend advanced users to code those functions in C++ because that minimizes compatibility errors during the execution of the plug-in functions by the engine.

#### **AVAILABLE PLUG-INS**

The FRIEND Engine distribution comes with four plug-ins: one for the SVM pipeline (libBrainDecoding), using the libSVM

#### **Table 2 | List of parameters used within plug-in functions.**


library (Chang and Lin, 2011); one for the ROI pipeline (libROI), used in the Matlab® and the first game frontend examples (presented in the following sections); one for the functional connectivity between two ROIs (libConnectivity) and one (libMotor) that extracts ROI information from two ROIs located in the motor cortex area (left and right), used in the avatar finger tapping virtual scenario.

#### *libROI plug-in functions and feedback value*

**Table 4** lists functions implemented in the libROI plug-in. The feedback value calculated by the processROI function (**Table 4**) is given by the equation:

$$\frac{\overline{ROI}\_{curr\\_vol} - \frac{1}{B} \sum\_{k=1}^{B} \overline{ROI}\_k}{\frac{1}{B} \sum\_{k=1}^{B} \overline{ROI}\_k} \tag{1}$$

where *ROIcurr*\_*vol* is the mean of the ROI on the current volume, *B* is the number of volumes in the previous baseline condition and *ROIk* is the mean of the *k*th volume.

A code snippet of this function can be found in the Supplementary Material. The feedback function within the lib-Motor plug-in is similar to the one on libROI, except for the fact that two feedback values are calculated, one for each existing ROI.

#### *libBrainDecoding plug-in functions and feedback value*

**Table 5** lists functions implemented in the libBrainDecoding plug-in. In trainSVM function, the voxels of an fMRI scan are first organized (by concatenation) in an input vector *x*. In this training phase, the vector is labeled according to the

#### **Table 3 | List of functions a plug-in can define.**


#### **Table 4 | Functions implemented in the libROI plug-in.**


#### **Table 5 | Functions implemented in the libBrainDecoding plug-in.**


corresponding experimental condition (LaConte, 2011; Sitaram et al., 2011). This initial data is used to train the classifier (currently, a two-class SVM classifier is implemented) to discriminate between the experimental conditions of interest. The output of this function is the trained SVM model (i.e., the hyperplane coefficients). The trained SVM is then used in the subsequent brain decoding sessions (testing sessions), during which participants engage in the same tasks and conditions of interest.

In testSVM function, the projected value of a new observation is used to define the neurofeedback information (Sato et al., 2008). The projected value of a new image volume on the SVM discriminating hyperplane is given by (*xTw* <sup>+</sup> *<sup>b</sup>*), where *<sup>w</sup>* is a vector containing the hyperplane coefficients and *b* is a constant. The boundary between conditions is represented by the value of zero. This value obtained by projecting the new observation in the SVM discriminating hyperplane is then used to choose the feedback figure to be shown to the participant. Further information can be found in Sato et al. (2013).

#### **Table 6 | Functions implemented in the libConnectivity plug-in.**


#### *libConnectivity plug-in functions and feedback value*

**Table 6** lists functions implemented in the libConnectivity plugin. The buildROIs function transforms a mask with two MNI ROIs into the subject space. A subset of voxels from a GLM analysis of the localizer run is then selected by applying the preceding masks and selecting a user-defined percentage of these voxels. These two voxel populations are employed as new ROIs for the sliding window correlation calculation.

The calculateFeedback function calculates the Pearson correlation coefficient between two ROIs, where *ROI*1 and *ROI*2 are vectors containing the means of the ROIs on the last *L* scans:

$$\rho(ROI1, ROI2) = \frac{\sum\_{i=1}^{L} (ROI1\_i - \overline{ROI1})(ROI2\_i - \overline{ROI2})}{\sqrt{\sum\_{i=1}^{L} (ROI1\_i - \overline{ROI1})^2} \sqrt{\sum\_{i=1}^{L} (ROI2\_i - \overline{ROI2})^2}} \tag{2}$$

Where *L* is the size of the sliding window, i.e., the last *L* scans acquired.

# **FRONTEND EXAMPLES**

In all the "game" examples provided with the distribution, there is, in the engine directory, a pre-configured study\_params.txt file that is read by default by the engine; all the volume files are already placed in the input directory referenced by the study\_params.txt file. To read the volume files in a real-time online setup, users need to configure the arrival of images in the input directory in the same way as for the standalone version of FRIEND. Triggers from the scanner can be used to keep track of the time, e.g., the onset of a given experimental condition. It is the frontend that handles the syncing of the experimental paradigm with the scanner.

#### **MATLAB® FRONTEND**

**Figure 4** is a screenshot of a frontend designed with the Matlab® GUIDE tool. Matlab® is a largely used language in the scientific community so it is important to provide a functional example of the Matlab® FRIEND Engine connection. GUIDE helps users to build graphical user interfaces for their applications in Matlab®.

This frontend application shows the activation of a ROI in the motor cortex during a finger-tapping task. It presents the feedback in the shape of a thermometer-like dynamic bar graph. This example uses the libROI plug-in. In this example and the first Unity example below, only one ROI located in the left motor cortex is used.

# **UNITY FRONTENDS**

The frontends based on the Unity game engine (http://unity3d. com/) employ virtual reality scenarios. These scenarios are composed of objects, like rocks, trees and scripts. The scripts in Unity play an important role in how the virtual scenarios behave, such as the interactions between objects, and how, where and when avatars interact with the environment. This aspect is especially relevant for the interrelationship between Unity and FRIEND Engine. Scripts coordinate how the information returned by the engine will impact on the current state of the virtual scenario. The complexity of this coordination increases exponentially with the number of objects and avatars in the scene. Unity currently offers three options of scripting languages: C#, JavaScript and Boo. All game examples showed here were written in C# language. The initial learning curve of Unity for construction of scenarios and writing the scripts is quite demanding, but this pays off because of the great variety of high quality scenarios that can be produced in a short time. The Unity assets store also helps, because users can find a lot of interesting and complex materials, like characters, objects, and animations. The Unity game engine was employed here given its cross-platform availability (Microsoft Windows®, Linux and Mac OSX, web player, IOS and Android) and ease of use, but other tools, such as Unreal Development Kit (UDK, https://www. unrealengine.com/products/udk) and Cry Engine SDK (http:// www.cryengine.com/), could also be potentially implemented as frontends.

### *Medieval virtual scenario frontend*

**Figure 5** is a screenshot of a frontend made in Unity. This frontend is a medieval virtual reality scenario in which the avatar, i.e., the participant, hovers over a path and stops in predetermined locations, blocked by a massive rock. Using the same finger tapping neurofeedback procedure exemplified in the Matlab® frontend (alternating rest and finger tapping blocks), and the same libROI plug-in, the feedback information to the participant is now given in a different way. As the participant moves across the scenario and stops right before the rock, he/she needs to perform the finger tapping task as instructed (as quickly as possible). If the percentage BOLD signal change returned by the engine reaches a predefined threshold, the rock levitates, thus unblocking the path so that the journey continues. If the threshold is not reached, the player stays at the same location until the next try, i.e., the next activation block. This scenario was constructed using objects from the iTween path editor and the Big Environment pack, available in the Unity Assets store.

#### *Avatar finger tapping frontend*

**Figure 6** shows the screenshots of the avatar finger tapping frontend. It was inspired by the BOLD brain pong (Goebel et al.,

**error) and a thermometer indicating BOLD signal change as a feedback.**

**FIGURE 5 | Snapshots of the game like frontend designed in Unity, showing the path that the participant travels during the experiment. (A)** Shows the situation where the rock is blocking the way; **(B)** shows the rock being "levitated," unblocking the way.

2004). This is a finger tapping experiment with intercalating blocks of rest and finger tapping. Different from the previous examples, here we use two ROIs located in the primary motor cortex area of the left and right cerebral hemispheres. The participant is asked to perform finger tapping with either their left or right hand, alternating with resting blocks. This example employs the libMotor plug-in, which calculates the percent BOLD signal change in the left and right ROIs in the same way as the libROI plug-in in the single ROI example. The frontend compares the feedback values between ROIs, in such a way that the greater one will inform which hand of the avatar, showed by the frontend, will perform the finger tapping animation. If successfully performed, this conveys a clear impression that the participant is controlling the hands of the avatar with his/her own hands. The avatar and hand animations were implemented using the VR Hands Unity asset (https://serrarens.nl/passervr/downloads/ vr-hands/).

**FIGURE 6 | Two situations of the avatar finger tapping scenario. (A)** During a rest block and **(B)** during a finger tapping block.

# **DISCUSSION AND CONCLUSION**

In this paper we introduced the FRIEND Engine framework, a reengineered implementation of our previous work (Sato et al., 2013) to provide a flexible and user-friendly framework that enables users to customize frontends and data processing pipelines for neurofeedback studies. For this aim, the standalone FRIEND software was re-implemented by breaking apart the core engine, which performs the basic data processing, the plug-in functionalities, which implement specifics of the neurofeedback study, and the frontend, which handles all the necessary graphic interfaces, external device inputs and sends commands to the engine.

The separation of the graphical interface from the data processing components provides a major advantage because users can implement virtually any type of feedback visualization strategy, such as images, audio, movie clips and virtual scenarios, by developing new frontends or connecting with standard stimulus presentation softwares and external devices. The use of the TCP/IP communication protocol between the frontend and the engine provides an interesting "weak connection," because the frontend does not need to make any assumptions about data processing other than the ones related to the feedback interpretation. This also enables users to more efficiently share frontends that can be used for different purposes. Computer processing time is always a critical concern, especially when real-time processing and feedback are needed. All frontends herein described were run on a different computer from the one running the FRIEND Engine application. The mean processing time for one volume (single-shot EPI, 64 × 64 to 80 × 80 matrix, 22–37 slices) was about 0.7 s in mid-/high-end workstations (PC, Quad-core i7, 8 GB RAM; Macintosh, quad-core Intel Core i5, 8 GB RAM). The performance of the game frontends was not affected by the communication with the engine, and no lags were noticed for the currently implemented routines and data types.

The FRIEND Engine framework flexibility was illustrated here by the game frontends. This implementation demonstrated the potential use of virtual reality in neurofeedback studies, which may increase engagement and compliance with the tasks. The economic push of commercial games and widely available software development kits facilitates constant updates and improvements that can be quickly embedded into new frontends for enhanced neurofeedback studies. The frontend implementation examples herein provided employed the Unity game engine (https://unity3d.com/), because of its user-friendliness, flexibility and availability for the three main operational systems (Windows®, Linux and Mac OSX).

Source code is available for all the software provided within the FRIEND Engine distribution, so that seamless customization is possible. From the user point of view, there is no apparent need to modify the exhaustively tested and well-established FRIEND Engine core functionalities, though these are open for improvement. Advanced users can also modify any existing plug-in (e.g., implementing ROI correlations using more than two ROIs) or create new ones (e.g., implementing simultaneous real-time fMRI and EEG neurofeedback). A few frontends and plug-ins are provided with this first distribution, but we expect that this can be substantially expanded whenever users share their developments, with a benefit for the growing scientific community interested in neurofeedback research (Sulzer et al., 2013).

Whereas the standalone FRIEND implementation includes quality controls mechanisms (e.g., a GUI that allows users to monitor details of the ongoing acquisition, such as motion parameters, visualization of ROIs transformed from the MNI space to subject space, the temporal variation of a ROI mean), FRIEND Engine still lacks a quality control module. While some of these functionalities are currently available on the Matlab® frontend (e.g., display of motion parameters), it is not the case for the game-like frontends. Toward this aim, we are currently developing an ancillary frontend for quality control, which will run independently from the main frontend. To access the motion parameters information and possibly feedback information, this frontend will only need the ID of the session workspace created by the main frontend. This ancillary frontend module also contains visualization capabilities for displaying anatomical and functional reference images, source MNI ROIs, ROIs transformed into subject space, visualization of active voxels prior to neurofeedback based on GLM thresholds or SVM feature-selection steps, among some other options. This frontend will operate in a largely generic way, as to be capable of working in conjunction with current or future frontends. In the short term, we are also planning to deliver a Python (https://www.python.org/) and a Presentation® (http://www.neurobs.com/) frontend.

In terms of usage and availability, currently AFNI (Cox, 1996) and Turbo Brain Voyager (Goebel, 2012) appear to be the leading packages for rtMRI neurofeedback. AFNI is a highly developed fMRI package that has pioneered work on real-time neurofeedback experiments. Turbo Brain Voyager is a user-friendly fMRI processing package containing a rtfMRI module that enjoys the benefits from a number of pre- and post-processing routines and an attractive graphic interface. Similarly, our package uses fMRI spatial and temporal processing routines that are largely based on the widely used and validated FSL package. Both AFNI and Brain Voyager allow ROI processing and thermometer-like feedback, as FRIEND Engine does. In a very recent publication, Cohen et al. (2014) employed Brain Voyager and the Unity environment to enable participants to control an avatar by hand and leg motion imagery, similarly as we report here. AFNI and Brain Voyager also provide interesting quality control functionalities, which are available in the standalone FRIEND version and which are currently being implemented and expanded in FRIEND Engine. For developers, Brain Voyager allows development of plug-ins for Windows®, MAC® and Linux platforms in C++ language whereas AFNI allows the addition of run-time functionalities in C language for MAC® and Linux. For FRIEND Engine, we recommend plug-ins to be developed in C++ for compatibility. Brain Voyager is a commercial package, thus source codes are not available, whereas AFNI and FRIEND Engine Framework are open source. An optimized FSL embedded functionality allows FRIEND to run seamlessly on Windows® (the official FSL package currently does not run on this platform), so there is no need for a virtual machine. AFNI requires Cygwin (https://www. cygwin.com) to run on Windows®, although with reduced functionality and possibly reduced performance. FRIEND Engine has the added advantage of providing full platform and language freedom for the development of frontends. A recent, freely available toolbox for rt-fMRI, implements similar capabilities as those described in FRIEND Engine, Turbo Brain Voyager and AFNI, and and adds the interesting feature of "subject-independent" multivariate pattern classification (Rana et al., 2013). However, so far it has not been made widely available for download, and we have not been able to evaluate and compare it to existing ones.

With respect to future developments in FRIEND Engine, one of our main goals is to encourage clinical applications. For this aim, streamlining routines for blind randomization procedures are under way. These will allow experimenters to run doubleblind randomized controlled fMRI neurofeedback trials in a rigorous and straightforward manner.

The importance of building a repository for frontends and plug-ins is clear. A publicly accessible repository, with a discussion forum for implementations and strategies can be of great help for the development of new neurofeedback projects. To this aim, sharing of plug-ins and frontends will be possible through the NITRC repository (http://www.nitrc.org/projects/friend) and GitHub (https://github.com/InstitutoDOr/FriendENGINE). In summary, we believe that FRIEND Engine can be a valuable contribution to the thriving fMRI neurofeedback community, by providing an open and flexible collaborative platform for developing new solutions for fMRI neurofeedback research and clinical applications.

## **ACKNOWLEDGMENTS**

The authors are thankful to colleagues from IDOR for helpful discussions and support in different stages of this work, in special to Ivanei E. Bramati, Fernando F. Paiva, Theo Marins, Patricia P. Bado, Julie Weingartner, Fernanda Tovar-Moll, Fernanda Meirelles, Luiz Felipe Costa, and Debora O. Lima. This work was supported from internal grants from the Cognitive and Behavioral Neuroscience Unit at IDOR, FAPERJ and MCT/INNT. Roland Zahn was funded by MRC fellowship (G0902304).

# **SUPPLEMENTARY MATERIAL**

The Supplementary Material for this article can be found online at: http://www.frontiersin.org/journal/10.3389/fnbeh.2015. 00003/abstract

# **REFERENCES**


using FMRI neurofeedback. *PLoS ONE* 9:e97343. doi: 10.1371/journal.pone. 0097343


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 08 August 2014; accepted: 05 January 2015; published online: 30 January 2015.*

*Citation: Basilio R, Garrido GJ, Sato JR, Hoefle S, Melo BRP, Pamplona FA, Zahn R and Moll J (2015) FRIEND Engine Framework: a real time neurofeedback clientserver system for neuroimaging studies. Front. Behav. Neurosci. 9:3. doi: 10.3389/ fnbeh.2015.00003*

*This article was submitted to the journal Frontiers in Behavioral Neuroscience.*

*Copyright © 2015 Basilio, Garrido, Sato, Hoefle, Melo, Pamplona, Zahn and Moll. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Estimating cognitive load during self-regulation of brain activity and neurofeedback with therapeutic brain-computer interfaces

# **Robert Bauer 1,2\* and Alireza Gharabaghi 1,2\***

<sup>1</sup> Division of Functional and Restorative Neurosurgery and Division of Translational Neurosurgery, Department of Neurosurgery, Eberhard Karls University Tuebingen, Tuebingen, Germany

<sup>2</sup> Neuroprosthetics Research Group, Werner Reichardt Centre for Integrative Neuroscience, Eberhard Karls University Tuebingen, Tuebingen, Germany

#### **Edited by:**

Francisco Javier Zamorano, Universidad del Desarrollo, Chile

#### **Reviewed by:**

Remko Van Lutterveld, University Medical Center Utrecht, Netherlands Gert Pfurtscheller, Graz University of Technology, Austria

#### **\*Correspondence:**

Robert Bauer and Alireza Gharabaghi, Division of Functional and Restorative Neurosurgery and Division of Translational Neurosurgery, Department of Neurosurgery, Eberhard Karls University Tuebingen, Otfried-Mueller-Str.45, 72076 Tuebingen, Germany e-mail: robert.bauer@ cin.uni-tuebingen.de; e-mail: alireza.gharabaghi@ uni-tuebingen.de

Neurofeedback (NFB) training with brain-computer interfaces (BCIs) is currently being studied in a variety of neurological and neuropsychiatric conditions in an aim to reduce disorder-specific symptoms. For this purpose, a range of classification algorithms has been explored to identify different brain states. These neural states, e.g., self-regulated brain activity vs. rest, are separated by setting a threshold parameter. Measures such as the maximum classification accuracy (CA) have been introduced to evaluate the performance of these algorithms. Interestingly enough, precisely these measures are often used to estimate the subject's ability to perform brain self-regulation. This is surprising, given that the goal of improving the tool that differentiates between brain states is different from the aim of optimizing NFB for the subject performing brain self-regulation. For the latter, knowledge about mental resources and work load is essential in order to adapt the difficulty of the intervention accordingly. In this context, we apply an analytical method and provide empirical data to determine the zone of proximal development (ZPD) as a measure of a subject's cognitive resources and the instructional efficacy of NFB. This approach is based on a reconsideration of item-response theory (IRT) and cognitive load theory for instructional design, and combines them with the CA curve to provide a measure of BCI performance.

**Keywords: neurofeedback, cognitive load theory, zone of proximal development, workload, instructional design, brain-computer interface**

## **INTRODUCTION**

Brain-computer interfaces (BCIs) support reinforcement learning of brain self-regulation by feedback and reward. While assistive BCIs aim to replace lost functions by controlling external devices (Yanagisawa et al., 2011; Hochberg et al., 2012; Collinger et al., 2013; Wang et al., 2013), the ultimate goal of restorative or therapeutic approaches is to improve specific functions by neurofeedback (NFB) training, e.g., hand and arm control following a stroke (Ang et al., 2010; Shindo et al., 2011; Buch et al., 2012; Ramos-Murguialday et al., 2013; Gharabaghi et al., 2014a,b). The fundamental approach of NFB is based on the idea that physiological signals during restful waking (Mantini et al., 2007; Albert et al., 2009; De Vico Fallani et al., 2011) are contrasted to the signals during the task condition, using classification algorithms to weight the respective features (Theodoridis and Koutroumbas, 2009). In this regard, restorative BCI is similar to assistive BCI. However, unlike assistive BCI approaches, which select features on the basis of their ability to maximally contrast the two states, the feature space for restorative BCI and NFB training is constrained in accordance with the specific treatment rationale. In stroke rehabilitation, for example, the feature space might be restricted to power in the β-range (15–30 Hz), since decreased movementrelated desynchronization in this frequency range is related to the amount of motor impairment after the insult (Rossiter et al., 2014). During restorative BCI training, the power of the frequency band is therefore estimated, and the desired modulation of this feature space is reinforced using appropriate visual, auditory or haptic feedback (Gharabaghi et al., 2014a,b). The feature weights are deliberately constrained during these interventions and the modality of feedback is designed to maximize the reinforcing effect of NFB (Sherlin et al., 2011; Vukeli´c et al., 2014). By contrast, during assistive BCI, the feature weights are calculated so as to allow maximal separation (Blankertz et al., 2008; Theodoridis and Koutroumbas, 2009) and the classification output is used for communication or robotic control (Wolpaw et al., 2002). While the primary goal for assistive BCI is accuracy and speed, the main goal for restorative BCI is reinforcement and learning. A theoretical difference therefore exists between the self-regulation of brain activity and the classification algorithm (Wood et al., 2014). However, although there are several ways of measuring the performance of an assistive BCI (Thomas et al., 2013; Thompson et al., 2013), similar appropriate measures for restorative BCI and NFB are currently not available. The most common measure for BCI is classification accuracy (CA). While the magnitude of CA has been used to estimate subject's ability to perform brain self-regulation, this interpretation currently lacks theoretical foundation (Blankertz et al., 2010; Buch et al., 2012; Hammer et al., 2012). What is more, there is no consensus regarding what approaches are appropriate for disentangling the performance of subject and classifier from each other, nor is there any theory as to how they are connected with each other.

By integrating classification theory (Theodoridis and Koutroumbas, 2009) with item response theory (Safrit et al., 1989), we describe how the relationship between the classification algorithm and the ability for self-regulation can be understood. In addition, on the basis of the theory of cognitive load for instructional design (Sweller, 1994; Schnotz and Kürschner, 2007), we will describe how the CA can be interpreted within the framework of NFB training. Our argument is based on the fact that it is possible to make an off-line calculation of the positive rates for different classifiers and thresholds. We will argue that the true positive and the false positive curve provide information about the subject's ability and his/her performance when support is provided. Moreover, since the shape of CA depends on the difference between true and false positive rate (FPR), we propose that it contains information about the subject's zone of proximal development (ZPD). Therefore, on the basis of the theory of cognitive learning, the ZPD may serve as an indirect measure of the subject's cognitive resources (Allal and Ducrey, 2000; Schnotz and Kürschner, 2007).

In this respect, the goal of this paper is to provide a measurement theory for subjects' abilities and ZPD during NFB training. We support this theory by mathematical models and by evidence from empirical data.

#### **EMPIRICAL DATASET**

Exemplary data is based on two right-handed, healthy subjects, one female (age 19) and one male (age 31), who presented different abilities for brain self-regulation. They each performed 75 trials of cued motor imagery. The trial structure consisted of consecutive preparatory (2 s), motor imagery (6 s) and rest (6 s) phases, each of which was initiated by a specific auditory cue. Electroencephalography (EEG) was recorded at 64 channels in accordance with the 10–20 system with Brain Products amplifiers and analyzed offline with custom-written scripts and Fieldtrip in Matlab (Oostenveld et al., 2011) according to the following steps. Data was down-sampled to 200 Hz and band-pass filtered between 14 and 26 Hz using a Butterworth filter. Wavelet transformation was used to perform a timefrequency analysis for time steps of 50 ms for the power in the β-range (15–25 Hz) over sensorimotor regions (FC3, C3 and CP3). For each trial, the power at each time point was normalized by z-scoring based on the mean and standard deviation of the power distribution in the rest and preparatory phase. Both subjects gave written, informed consent prior to participation. The study was approved by the local ethics committee.

# **LINKING SUBJECT'S ABILITY FOR BRAIN SELF-REGULATION WITH THE CLASSIFICATION PERFORMANCE**

In the following section, we will propose a link between the ability for brain self-regulation, as estimated by the item function, with the classifier performance, as estimated by the rate function. This integration will enable us to apply off-line analysis of the positive rate across different thresholds to determine the subjects' ability for brain self-regulation.

### **RATE FUNCTIONS**

When applying NFB for therapeutic purposes, a two-class separation of brain states is usually performed, i.e., rest vs. learned self-regulation of brain activity. Due to the fact that most of the classifiers used in NFB are based on supervised learning algorithms employing linear discriminant analysis, the sensitivity and specificity of the classifier can be calculated relatively easily. The sensitivity informs us how often the classifier detects sufficient self-regulation while the subject is performing brain self-regulation (true positive rate or TPR). The specificity informs us how often the classifier detects rest while the subject is performing insufficient brain self-regulation (true negative rate or TNR). Since the probabilities of each conditional classification must add up to 1 within each class, the false negative rate (FNR) is equal to 1-TPR, and the FPR is equal to 1-TNR. CA is based on the average of TPR and TNR.

$$CA = \frac{\text{(TPR} + \text{TNR)}}{2}$$

# **THRESHOLD-BASED RATE FUNCTIONS**

These rates are functions of the threshold θ (Theodoridis and Koutroumbas, 2009). During the training, the threshold θ usually remains fixed, and the rates therefore also remain fixed. However, provided that the electrophysiological signals have been recorded and stored, the positive rate can be calculated offline after the training for any threshold. We exemplify this by the empirical dataset (see **Figure 1**): the higher—i.e., the more challenging the threshold, the stronger the desynchronization must be if it is to be classified as positive (see **Figure 1A**). The examples also reveal how subjects vary in their ability for brain selfregulation, e.g., sensorimotor beta-band desynchronization. The first subject shows stronger desynchronization and is thus able to reach more challenging thresholds (see **Figure 1A**) than the second subject, who has less pronounced brain self-regulation (see **Figure 1C**).

In parallel, this data enables us to characterize the classifier performance: detecting "positive" during the motor imagery phase (second 0 to 6) is a true positive, whereas "positive" during the preparatory phase (second −2 to 0) is a false positive. The average rate of positives for each phase is a suitable measure for characterizing a classifier's performance, i.e., true and FPRs are expressed as probabilities in the range between 0 and 1 (see **Figures 1B/D**). The first subject has higher TPRs for all thresholds (see **Figure 1B**) than the second subject (see **Figure 1D**).

For most classifiers, the rate functions result in sigmoidal rate curves. We show this sigmoidal shape for the TPR

of the empirical dataset (see **Figure 2A**). The higher the thresholds, the more the probability of success decreases in a logistic fashion. Accordingly, the location of the first subject's true positive curve is further to the right, indicating a generally higher success rate. However, the shape of the respective curves for the first and the second subject are highly similar.

#### **ITEM FUNCTIONS**

The *rate functions* described resemble the *item functions* used in the psychometric item-response theory (IRT). In various fields of research, such as in assessment psychology (De Champlain, 2010) or motor behavior research (Safrit et al., 1989), the parameters of the item functions are usually estimated across datasets of several subjects and items. This enables us to quantify the respective variability of subject's ability and task difficulty necessary for fitting algorithms. If, for example a mathematical test battery is distributed to a school class, the marginal success rates enable us to estimate the difficulty of

a specific test and the ability of a single subject. Generally speaking, students with a higher mathematical ability achieve a higher success rate, and easier tests should result in higher average success rates. This information about the relative ranks based on success rates can be used for parameter estimation. More specifically, the shape of these success curves can be approximated by a two-parameter logistic model (2PLM) using the following function (Safrit et al., 1989; De Champlain, 2010):

$$P = \frac{1}{1 + e^{-D(a - \vartheta)}}$$

In this function, *e* is Euler's number and D is the slope of the curve. The parameter θ, generally known as the threshold parameter in rate functions, now represents the difficulty of the task. Ability α and difficulty θ are located on the same axis. They can therefore be measured in one dimension. The location of the curve depends on the difference between the difficulty of a task and the subject's ability α. The shape of the curve depends on the slope parameter D, and a value of ∼1.7 would result in

an approximate fit for a normal distribution (De Champlain, 2010). If the slope is identical for all curves, all item function are parallel and the difficulty level θ is the only changeable parameter.

#### **SIMILARITY OF RATE AND ITEM FUNCTIONS**

For dichotomous items, the probability of success is modeled as a function of the difficulty of the item and the ability of the subject. Assuming that the latter is stable, the difficulty of an item then depends on the design of the task. The combination of NFB task, i.e., the classification algorithm, the trial structure, the cues and any instructions or extraneous aspects constitute the phrasing of such an item. By way of example: in NFB training, our aim is to differentiate between sufficient and insufficient modulation of brain activity. If we were to use a questionnaire instead of EEG and BCI, we might phrase an item: "Are you currently performing sufficient brain selfregulation?" to which the possible answers would be "yes" and "no". However, since the decision about "yes" or "no" is based on physiological recordings during the task, a *post hoc* reassessment for different thresholds is possible. This recalculation enables us to apply virtually the same items over a range of difficulties. In addition, threshold-based recalculation modifies only one aspect of the "item", namely the difficulty parameter; an aspect that lies in one clearly defined dimension. These properties (unidimensionality, off-line analysis) allow for an interpretation of the parameters that describe the positive rate curves on the threshold dimension within the framework of the item response theory.

#### **INTERPRETATION OF CURVE PARAMETERS TRUE POSITIVE CURVE REPORTS ON ABILITY**

Due to the fact that the measurement of the subject's ability for brain self-regulation can be performed *post hoc*, NFB training is comparable to an action like videotaping a sports exercise such as long jump. The data acquired during the task can then be used later to estimate several aspects of the performance. As in sports training, the compound ability can be divided into subsets (e.g., sprinting, take-off, and landing in long jump). In this example, the coach would be ill-advised to reward any jump independent of the actual performance. Therefore, specificity matters.

NFB training has the ability to provide this specificity by selecting the appropriate features and classification algorithms. In addition to determining the threshold to be passed by the event-related desynchronisation (ERD), additional features might include the speed of the power dip in the first two seconds or continuity of desynchronization. This highlights the fact that the reinforced features must be carefully selected for their respective clinical or rehabilitative purpose.

In this respect, it is also important to note that functional improvement is a combination of several abilities and preconditions, of which for instance, brain self-regulation of sensorimotor beta-rhythms is only one example. Others, such as reaching out and holding a certain position with the upper limb, as assessed in the Fugl-Meyer assessment (Deakin et al., 2003), or interacting with an object, as assessed in the Broetz hand assessment (Brötz et al., 2014), necessitate the involvement of parieto-frontal circuits for motor planning (Andersen and Cui, 2009) and sensorimotor circuits for execution (Chouinard and Paus, 2006). Along these lines, stroke survivors who train to modulate the activity of the primary motor cortex (Kaiser et al., 2011; Kilavik et al., 2012) show improvements in this ability only if the fronto-parietal integrity is preserved (Buch et al., 2012). Training such ability of brain-self-regulation may therefore be related both directly and indirectly to the respective function, e.g., moving the upper extremity. However, improving brain self-regulation does not necessarily lead to functional improvements, since these may also depend on abilities and preconditions that are not influenced by the NFB training. Improvements are therefore required with regard to the clinical efficacy of such a training (Ramos-Murguialday et al., 2013; Ang et al., 2014) by researching the feedback modality (Gomez-Rodriguez et al., 2011) or using it in combination with simultaneous cortical stimulation (Gharabaghi et al., 2014a). Screening examinations might also be necessary to determine the eligibility of subjects for a specific intervention (Stinear et al., 2012; Burke Quinlan et al., 2014; Vukeli´c et al., 2014). In addition, the validity of functional assessment scores requires re-evaluation in the light of biomarkers of sub-clinical improvement.

We therefore conclude that the location of the TPR can be interpreted as a subject's ability for brain self-regulation, regardless of the potential influence of this ability on a specific function. In this respect, the location of the true positive curve mathematically speaking, the point of maximal slope and halfway between success and failure—provides information about the subject's ability to perform the task which is defined by the features and the classifier.

### **FALSE POSITIVE CURVE REPORTS ON ATTEMPT**

According to our previous example, a long jump coach would be ill-advised not to reward any jump. To be more precise, for reasons of motivation, even attempts should sometimes be rewarded, or support is required to transform an attempt into a success. In the case of NFB, specificity and sensitivity also have to be balanced according to their importance for learning. If the task remains identical, such a balance can only be achieved by changing the threshold. Decreasing the threshold increases the number of false positives (see **Figure 2B**). Since the classifier normalizes to rest, there is no apparent difference in the location of the FPR between the two subjects (see **Figure 2B**). This indicates that the subjects have the same opportunity to try to perform the task. This theory is supported by the following line of argument. In this context, "support" or "help" can be formalized by assuming that the subject with the current level of ability α is unable to perform the task at the given level of difficulty θ, whereas providing help will lead to success. If we then detect a success, this will be a "false positive" result, since the subject's current ability is too low for him/her to actually succeed. If no help is provided, the success achieved will be a "true positive" result. This approach will lead to a range of thresholds which are defined by two limits. The lower limit will be marked by the most difficult task that the subject can perform when help is provided. The upper limit is defined by the most difficult task that can be performed by the subject without help (see **Figure 3A**). Once the subject no longer benefits from help, e.g., due to overly high intrinsic or extrinsic load, he can no longer benefit from the training.

# **SHAPE OF CLASSIFICATION ACCURACY SHEDS LIGHT ON THE ZONE OF PROXIMAL DEVELOPMENT**

The range between the most difficult tasks that can be achieved by the subject with and without help, respectively, can be defined as the ZPD. Cognitive load theory argues that mental load can be divided into three categories: intrinsic load, extrinsic load and germane load (Jong, 2009). Intrinsic load resembles the difficulty of the task and is mainly caused by the element interactivity of the task. Extraneous load is mainly caused by irrelevant information. Germane load is caused by the construction and subsequent automation of schemas, i.e., learning. Lower difficulty results in a reduction of intrinsic load while extrinsic load will increase, since the instructional material now contains irrelevant information. If a task is too easy (i.e., θ α) or too difficult (i.e., θ α) for a given ability, the extrinsic or the intrinsic load of the task would be too high.

For every given level of difficulty and ability for a task, there is therefore a ZPD, where learning is possible (Schnotz and Kürschner, 2007). The cognitive load theory thus provides a feasible explanation as to why the boundaries of ZPD are characterized by TPR and FPR (Allal and Ducrey, 2000; see **Figure 3B**). A second line of argumentation considers the likelihood of reward. Since a subject cannot discern a reward with identical qualities, the only way of differentiating between a true and a false reward is to determine the relative probability of their occurrence. The difference between the true and FPR might therefore be a good approximation of the difference with respect to the informational content of the two reward rates. Although more elaborate measures might be better suited to this divergence (MacKay, 2003), the most straightforward approach would consider the magnitude and the shape of ZPD as estimated by a linear transformation of CA in accordance with the following equations:

$$\begin{array}{rcl} CA & = & \frac{TPR + TNR}{2} \\ ZPD & = & TPR - FPR \\ TNR & = & 1 - FPR \\ CA & = & \frac{TPR + (1 - FPR)}{2} = \frac{TPR - FPR + 1}{2} \\ (2 \ast CA) - 1 & = & TPR - FPR \\ ZPD & = & (2 \ast CA) - 1 \end{array}$$

# **CONCLUSION AND OUTLOOK**

In the sections above, we have shown how the true and false positives rates of brain self-regulation can be interpreted within the framework of NFB. We have demonstrated that there is a natural relationship between classification of rate functions and item response functions. We have revealed the connection between applying a threshold to ERD signals and estimating the ability to perform ERD. In this respect, the true positive curve provides information as to the brain's ability to perform brain self-regulation in a NFB task. In addition, we showed that not only can the false positive curve provide information about attempts to perform the task but it can also set the lower limit of the ZPD. Below, we will illustrate how the ZPD, in its capacity as a transformation of CA, can support the instructional design of NFB interventions.

#### **CONCLUSION REGARDING CLASSIFICATION ACCURACY**

The ZPD can be used to compare different classification algorithms. In BCI approaches, classification algorithms are often trained to maximize CA. This can result in a peaky but narrow ZPD (see **Figure 4A**) instead of a flat but broad ZPD (see **Figure 4B**), although the area of ZPD is equal in both cases. A

the location of the ZPD from the absolute difficulty and the absolute ability. The ZPD width is based on the between the true and false positive rate. The blue line indicates the success rate of the task when performed without help

(false positive rate). The dotted black line indicates the equality of difficulty and ability. The area of ZPD is shown in gray. **(B)** shows the ZPD based on FPR and TPR over different thresholds for the first subject.

more broadly shaped ZPD indicates that learning can occur over a larger range of thresholds, whereas a peaky shape means that maximal help is available only for a narrow range of thresholds. This being the case, slight misalignments might have significant adverse effects. The shape of CA may therefore serve as a measure to evaluate whether or not a NFB task is instructionally effective. While the best general instructional efficacy is obviously achieved by NFB training with a high *and* broad ZPD, interpreting the shape enables us to apply tailored approaches. These might be more effective with regard to instructional needs for specific subjects and environments. A broad ZPD might be more robust for home-based training with low availability of supervision and the possibility of noisy measurements. A peaky ZPD might be more suitable for environments where professionals can perform alignments, i.e., adapt the classification algorithm or correct noisy measurements. Shaping the ZPD can thus support instructional design of NFB interventions. The approach presented here will also be applicable to classification algorithms resulting in non-normal distributions where TPR and TNR are calculated numerically (see **Figure 3B**), since the interpretation is also supported by non-parametric IRT-models provided that TPR and FPR are monotonic functions (Mokken and Lewis, 1982; Rost, 2004).

# **RELATIONSHIP TO ALTERNATIVE MEASURES OF PERFORMANCE AND FEEDBACK**

CA is by far the most widely reported measure of performance for BCIs and can be used for both synchronized (e.g., cued) and self-paced interventions (Thomas et al., 2013). However, for some clinical applications, additional measures of performance that are regarded as relevant for the treatment goal have been developed. These include the latency to movement onset or the maximum consecutive movement time in stroke rehabilitation (Ramos-Murguialday et al., 2013) and the path efficiency for a high degree of freedom prosthetic control in tetraplegia (Collinger et al., 2013). How does the ZPD now relate to these alternative performance measures?

These alternative measures may sometimes be translated into one of the basic measures used for calculating the ZPD, e.g., the average movement rate could also be understood as a TPR. However, such measures are much more liable to contain unique information about additional abilities that are required for the given task to be performed, such as already mentioned in the paragraph on the interpretation of the true positive curve.

Since learning is conceptually linked to the accuracy of feedback, we propose that a NFB task should be characterized by its instructional efficacy with regard to the action to be trained. This instructional efficacy is characterized by the feedback curves. In this context, the CA changes as soon as the coupling of the feedback to the action changes. The shape of the ZPD will therefore be useful for the instructional design of the intervention and tends to be independent of other task-specific measures. If, for example, the subject receives feedback to alternative actions, any improvement in these actions will be caused by the task's instructional design. In this respect, a ZPD, e.g., for latency of movement onset or path efficiency, may also exist. Estimating the ZPD for these measures would be similar to the approach illustrated above.

It should be borne in mind that the theory presented here is based on the classical binary feedback, the distance between feedback and no feedback being one bit of information, i.e., reward (Ortega and Braun, 2010). Alternative approaches such as

continuous feedback (e.g., the frequency of an auditory signal) or psychophysical perception rules (e.g., the perception of the duration of binary feedback in a log-linear fashion) do, of course, affect the bit-rate and may thus increase the achievable speed of learning. However, since the ZPD is based on a single bit as a distance metric, adequate mapping of the ZPD for alternative feedback approaches will probably be mathematically demanding. In order to interpret the curves under such conditions, further research and specific transformations might be necessary. A system analytical perspective, where continuous feedback can be understood as a pattern of step functions, and a complexvalued ZPD might help to solve such aspects. Nonetheless, the real-valued ZPD based on the single bit of feedback will also remain a fundamental building block of such advanced approaches.

# **CONCLUSION WITH REGARD TO COGNITIVE RESOURCES**

The ZPD may also act as a measure to compare different subjects with regard to their cognitive resources for a NFB task. If two subjects perform the very same NFB training, one might show a peaky and narrow ZPD (see **Figure 4A**) while the other has a broad and flat ZPD (see **Figure 4B**). Since in this case both curves indicate equal abilities, this difference in the respective shapes requires an alternative explanation. On the basis of the relevance of cognitive resources for the ZPD (Schnotz and Kürschner, 2007), we postulate that the shape of ZPD can also be applied to measure a subject's cognitive resources for coping with the mental load that occurs during a misalignment between ability and difficulty. Such an interpretation would, furthermore, permit a different view on the discussion about BCI illiteracy (Vidaurre and Blankertz, 2010). In particular, when the curves of TPR and FPR cross, they provide information about the specific breakpoints of that task. At this point, any support provided by the instructional design of the training will cease to be beneficial and will begin to be detrimental for the performance (see **Figure 4C**). This would be indicated by a negative value for the ZPD.

However, these concepts require validation by future research. Measurements of cognitive resources are currently based on psychophysiological recordings (e.g., heart rate variability, blink rate, electrodermal response), which are highly variable and very difficult to generalize across task conditions (Cegarra and Chevalier, 2008; Novak et al., 2010). Motor imagery itself can also cause vegetative effects related to the imagined movement, e.g., subjects imagining running at 12 km/h had an increased heart rate and pulmonary ventilation as compared to walking at 5 km/h (Decety et al., 1991). Mental imagery might therefore affect psychophysiological biomarkers, masking the measurement of the mental effort unrelated to the imagery content. One alternative to psychophysiological measures is the application of self-rating questionnaires. However, from the subject's point of view, it is often not possible to distinguish between the intrinsic, extrinsic and germane load (Cegarra and Chevalier, 2008). What is more, many psychophysiological measures and questionnaires can be sampled only at a very low rate. For example, the low frequency part of the heart rate commences at 0.04 Hz (Malik et al., 1996), meaning that at least 25 s of clean data have to be recorded for adequate frequency resolution of the Fourier transformation. Furthermore, slow frequency fluctuations in the EEG (<0.1 Hz) can correlate with psychophysiological performance, but they require similarly large time windows. In addition, slow fluctuations in the EEG measurements appear to be highly masked by imagery-related fluctuations, e.g., movement-related cortical potentials (Shibasaki and Hallett, 2006). This is also an issue if higher frequency components of the EEG are used to estimate cognitive resources, e.g., in the gamma range (Grosse-Wentrup et al., 2011), since they need to be disentangled from pure motor-related fluctuations in the same frequency band (de Lange et al., 2008; Miller et al., 2012).

In this context, the shape of ZPD might prove useful for disentangling the multitude of interacting and complex psychophysiological measurements in challenging tasks. This perspective is in agreement with the understanding that a proper alignment of ability and difficulty will reduce mental effort (Schnotz and Kürschner, 2007). Future studies might focus on psychophysiological correlates of the shape of the ZPD. Furthermore, improving the instructional material should help to reduce extrinsic load. NFB training could similarly be supported by "instructions", e.g., by providing haptic feedback (Gomez-Rodriguez et al., 2011) or visual and auditory cueing (Heremans et al., 2009). Systematic research on the impact of these feedback modalities on the ZPD might provide insight on their utility in guiding instructional design.

#### **ACKNOWLEDGMENTS**

Robert Bauer was supported by the Graduate Training Centre of Neuroscience, International Max Planck Research School, Tuebingen, Germany. Alireza Gharabaghi was supported by grants from the German Research Council and from the Federal Ministry for Education and Research [BFNT 01GQ0761, BMBF 16SV3783, BMBF 03160064B, BMBF V4UKF014]. We wish to thank Valerio Raco for fruitful discussions regarding the concept of cognitive load theory.

#### **REFERENCES**


**Conflict of Interest Statement**: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 30 June 2014; accepted: 20 January 2015; published online: 16 February 2015*.

*Citation: Bauer R and Gharabaghi A (2015) Estimating cognitive load during selfregulation of brain activity and neurofeedback with therapeutic brain-computer interfaces. Front. Behav. Neurosci. 9:21. doi: 10.3389/fnbeh.2015.00021*

*This article was submitted to the journal Frontiers in Behavioral Neuroscience*.

*Copyright © 2015 Bauer and Gharabaghi. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution and reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms*.

# Fast mental states decoding in mixed reality

*Daniele De Massari 1,2 †, Daniel Pacheco3 †, Rahim Malekshahi 1,4, Alberto Betella3, Paul F. M. J. Verschure3,5, Niels Birbaumer 1,2 and Andrea Caria1,2\**

*<sup>1</sup> Institut für Medizinische Psychologie und Verhaltensneurobiologie, Universität Tübingen, Tübingen, Germany*

*<sup>2</sup> Fondazione Ospedale San Camillo, Istituto di Ricovero e Cura a Carattere Scientifico, Venezia, Italy*

*<sup>3</sup> SPECS - Laboratory of Synthetic Perceptive, Emotive and Cognitive Systems, Department of Technology, Center of Autonomous Systems and Neurorobotics,*

*Universitat Pompeu Fabra, Barcelona, Spain*

*<sup>4</sup> Graduate School of Neural & Behavioural Sciences, International Max Planck Research School, Tübingen, Germany*

*<sup>5</sup> Institució Catalana de Recerca i Estudis Avançats, Barcelona, Spain*

#### *Edited by:*

*Nuno Sousa, University of Minho, Portugal*

#### *Reviewed by:*

*Mamiko Koshiba, Saitama Medical University, Japan Emanuele Pasqualotto, Université Catholique de Louvain, Belgium*

#### *\*Correspondence:*

*Andrea Caria, Institute of Medical Psychology and Behavioural Neurobiology, Eberhard-Karls-University of Tübingen, Silcherstrasse 5, D-72076 Tübingen, Germany e-mail: andrea.caria@ uni-tuebingen.de*

*†These authors have contributed equally to this work.*

The combination of Brain-Computer Interface (BCI) technology, allowing online monitoring and decoding of brain activity, with virtual and mixed reality (MR) systems may help to shape and guide implicit and explicit learning using ecological scenarios. Real-time information of ongoing brain states acquired through BCI might be exploited for controlling data presentation in virtual environments. Brain states discrimination during mixed reality experience is thus critical for adapting specific data features to contingent brain activity. In this study we recorded electroencephalographic (EEG) data while participants experienced MR scenarios implemented through the eXperience Induction Machine (XIM). The XIM is a novel framework modeling the integration of a sensing system that evaluates and measures physiological and psychological states with a number of actuators and effectors that coherently reacts to the user's actions. We then assessed continuous EEG-based discrimination of spatial navigation, reading and calculation performed in MR, using linear discriminant analysis (LDA) and support vector machine (SVM) classifiers. Dynamic single trial classification showed high accuracy of LDA and SVM classifiers in detecting multiple brain states as well as in differentiating between high and low mental workload, using a 5 s time-window shifting every 200 ms. Our results indicate overall better performance of LDA with respect to SVM and suggest applicability of our approach in a BCI-controlled MR scenario. Ultimately, successful prediction of brain states might be used to drive adaptation of data representation in order to boost information processing in MR.

**Keywords: mental states decoding, EEG, mixed reality, XIM**

# **INTRODUCTION**

Mixed Reality (MR) is a type of virtual reality-related technology where real and virtual worlds are merged so that real-time interaction with both physical and digital objects (Milgram, 1994; Bohil et al., 2011) is achievable. A particularly promising MR system is the eXperience Induction Machine (XIM) (Bernardet et al., 2011; Omedas et al., 2014). This technology permits to model representational elements analog to real phenomena as well as highly abstract non-representation forms describing complex high-dimensional data in a controlled environment. The exploration of data in XIM is conceptualized as an integrative narrative of varying forms where implicit and explicit responses as well as neurophysiological signals from the user can be utilized to modulate data representation (Bernardet et al., 2011; Lessiter et al., 2011; Verschure, 2011; Omedas et al., 2014).

It has been proposed that the combination of Brain-Computer Interface (BCI) technology, allowing online monitoring and decoding of mental states (Muller et al., 2008; Blankertz et al., 2010), with virtual reality systems may help to shape and guide implicit and explicit learning using ecological scenarios (Lécuyer et al., 2008; Lotte et al., 2013). Online analysis of specific brain activity has mainly been used in BCI applications for communication and control of external devices (Birbaumer and Cohen, 2007; Daly and Wolpaw, 2008), as well as for shaping behavior through neurofeedback paradigms (Delorme and Makeig, 2003; Shibata et al., 2011; Yoo et al., 2011; Caria et al., 2012; Scharnowski et al., 2012). Alternatively, the information acquired with BCI can be used to support human-computer interaction, in applications where its content is adapted to user's implicit interest, as well as for adaptive automation, affective computing, or video games (George, 2010). BCI-based real-time analysis of brain signals, with no need of participants to learn their control (sometimes referred to as "passive" BCI), can additionally be utilized to manipulate behavioral response by delivering information according to specific mental states.

Using this approach, enhancing and depressing learning and memory formation was demonstrated by triggering stimuli presentation during brain states favoring or reducing learning, which were assessed through online detection of activity in the bilateral parahippocampal areas with real-time fMRI (Yoo et al., 2011). In a similar fashion, presentation of external inputs during specific phases of neuroelectric activity might enhance or reduce participant's response.

Successful electroencephalographic (EEG)-based detection of brain states predicting participants' errors during complex cognitive decision tasks has been shown (Eichele et al., 2010). Several studies also demonstrated that brain states preceding stimulus presentation significantly affect perception (Arieli et al., 1996; Boly et al., 2007; Fox and Raichle, 2007; Fox et al., 2007; Busch et al., 2009; Mathewson et al., 2009). Building on these results, real-time information of ongoing brain states might be exploited for controlling data presentation in MR in order to boost information processing. For instance, detection of specific brain states might be used to drive changes in the level of complexity of presented information to facilitate participants' perception.

Toward this aim, we assessed to what extent multiple brain states can be discriminated during MR experience. Previous studies in the visual domain showed that real-time analysis of visual evoked potentials can detect fluctuations of perceptual dominance of each eye during binocular rivalry (Brown and Norcia, 1997). In the field of motor imagery-based BCI Millán and colleagues proposed a simple local neural classifier for the recognition of multiple mental tasks from on-line spontaneous EEG signal that achieved a recognition rate of 70% in distinguishing between relaxation, left and right hand movement imagination (Millán et al., 2002).

However, to date, clear evidence of classification of brain states during different cognitive tasks for BCI control of virtual and MR environments is still lacking. Most of studies on the integration of BCI with virtual reality focused on motor imagery, P300 and steady-state visual evoked potentials (SSVEPs) (Lotte et al., 2013). In particular, SSVEPs, permitting high information transfer rates and minimal training, seem to be suitable for BCI in virtual and MR (Martinez et al., 2007; Faller et al., 2010). Though, BCI based on continuous EEG decoding might be more flexible for monitoring brain activity during natural behavior in virtual and MR applications.

In our study, we tested classification of continuous EEG signal during spatial navigation, calculation and reading toward the implementation of BCI-controlled XIM-based MR. Spatial navigation represents a typical category of actions in virtual reality (Lécuyer et al., 2008; Lotte et al., 2013), while calculation and reading are fundamental tasks performed during information processing and data mining, and are also common cognitive processes used for mental workload assessment (Kohlmorgen et al., 2007). In particular, we performed EEG data classification using supervised classifiers based on linear discriminant analysis (LDA) and support vector machine (SVM). Furthermore, we examined predictive accuracy of our classifiers of mental workload in XIM. Increased mental workload was expected during calculation and reading as compared to spatial navigation because of larger involvement of working memory (Mayes and Koonce, 2001; Destefano, 2004; Imbo et al., 2007).

Based on previous studies showing large inter-individual differences in single-trial EEG classification of mental states in real operational environments (Kohlmorgen et al., 2007), we have used a flexible approach and calibrated our classifiers to each participant. Dynamic single trial classification was conducted using a sliding time-window shifting every 200 ms to permit applicability in a BCI-controlled MR scenario.

# **MATERIALS AND METHODS**

Five participants (29.60 ± 6.73 mean age ± *SD*, 1 female) underwent two consecutive sessions in a MR environment during which EEG signal was acquired continuously. The experiment was performed in the XIM (Bernardet et al., 2011; Omedas et al., 2014). The XIM architecture is an integrated framework that combines a sensing system to evaluate and measure complex psychological states with a number of actuators and effectors to coherently react to the user's actions (**Figure 1**). The internal processing of XIM is based on three main components. First, adaptive data mining that defines what data is presented to the user. Second, spatio-temporal structuring of the presented content in the form of narratives generated by the composition engines, and third an intentional, sentient agent, who controls the XIM interface and guides data exploration. XIM covers a surface area of 5.5 × 5.5 m, with a height of 4 m. Eight video projectors display the scenarios into four projection screens (2.25 × 5 m) surrounding the MR room. During each session participants experienced three different conditions, all involving the visual system: spatial navigation (SPN), reading (MER), and calculation (MEC). The user, sitting on a chair positioned in the middle of the XIM room, could navigate the virtual space by pressing the arrow keys of a keyboard. Participants were required to navigate a squared spiral labyrinth until the central point (indicated by a yellow sphere) (**Figure 2**). Nine different targets represented by red spheres were placed in alternating corners of the path. Proximity to red spheres triggered the beginning of a different condition. In the first session the condition consisted of a 30 s calculation task. When the participant reached the red sphere, screen went black and a random 3-digit

**FIGURE 1 | The eXperience Induction Machine (XIM) architecture is a mixed reality integrated framework that combines a sensing system to evaluate and measure complex physiological and psychological states with a number of actuators and effectors to coherently react to the user's actions.** It is mainly constituted of an immersive room that covers a surface area of 5.5 × 5.5 m, with a height of 4 m. Eight video projectors display the scenarios into four projection screens (2.25 × 5 m) surrounding the MR room. Reprinted with permission from Betella et al. (2014).

**FIGURE 2 | Top:** The immersive XIM modeling the virtual maze used in the experiment (left). **Center:** View from the top of the labyrinth and the nine different targets (red spheres) that were placed in alternating corners of the path. The labyrinth size was 10 × 10 VR units (meters). Participants were required to navigate the squared spiral labyrinth until the central point (yellow sphere). Proximity to red spheres triggered the beginning of a different condition. In the first session the condition consisted of a 30 s calculation task. When the participant reached the red sphere, screen went black and a random 3-digit number was displayed in the graphical interface. The participant was asked to iteratively subtract 17 from a given number. After 30 s, the black screen faded out and the participant was asked to continue spatial navigation. In the second session, the condition consisted of a 30 s reading. The introduction of a scientific article was displayed and the participant was required to read it and press the space keyboard command when finished. **Bottom:** First person perspective of the labyrinth and a red sphere.

number was displayed in the graphical interface. The participant was asked to iteratively subtract 17 from a given number. After 30 s, the black screen faded out and the participant was asked to continue spatial navigation. In the second session, the condition consisted of a 30 reading. The introduction of a scientific article was displayed and the participant was required to read it and press the space keyboard command when finished. In each session 9 SPN conditions were alternated to 8 MER or MEC conditions. An immersive 180◦ virtual reality application was developed and projected into the back screens of the XIM room. The VR application was developed using the Unity3D Game Engine, and adapted to fit the displays of XIM (Bernardet et al., 2011; Omedas et al., 2014). A virtual maze was modeled using Autodesk Maya (Autodesk Inc., San Rafael, CA, USA). The labyrinth size was 10 × 10 VR units (VR units permit to assign any type of units to objects' properties, e.g., weight, distance, etc., in our case they are defined as meters). The environment was constructed as an extension of the real physical space of the XIM—wall, floor, and other virtual objects were modeled so that they were perceived of real size (i.e., the point of view of the participant experiencing MR was equivalent to that of a person with average height). All participants were appropriately instructed about the experimental procedure. This study was approved by the ethics committee of the University of Tübingen.

### **EEG DATA ACQUISITION AND PREPROCESSING**

EEG data were recorded using a 64-channels BrainAmp amplifier (Brain Products GmbH, Munich Germany). An actiCap 64-channels EEG cap (modified 10–20 system, Brain Products GmbH, Munich Germany) was used for data acquisition, referenced to the FCz, and grounded anteriorly to Fz. Only 28 surface active electrodes at the following locations were used: Fp1, Fp2, F7, F3, Fz, F4, F8, Fc5, Fc1, Fc2, Fc6, T7, C3, Cz, C4, T8, Cp5, Cp1, Cp2, Cp6, P7, P3, Pz, P4, P8, O1, Oz, O2. Electrodes impedance was reduced to 15 k before data recording. EEG signals were sampled at 250 Hz.

EEG signal was first visually inspected to exclude channels affected by artifacts. Spectral analysis was then conducted on each channel to prevent our classifiers from being affected by large muscle and eye artifacts. To this aim we explored differences between SPN (low cognitive load) and MER + MEC (high cognitive load) conditions focusing on the frequencies above 20 Hz (typical of muscles artifacts) and below 6 Hz (typical of eye artifacts) (Kohlmorgen et al., 2007). The channels showing a significant difference between different workload conditions in the selected frequency bands were discarded (Kohlmorgen et al., 2007). As in Kohlmorgen et al. (2007), for each subject a customized feature selection—channel subsets, spatial filtering, frequency bands, and window lengths—was performed based on the following set of parameters: EEG channels subset {FC#, C#, P#, CP#}, {F#, FC#, C#, P#, CP#, O#}, {F#, FC#, C#, P#, CP#, O#, T7, T8}, {FC#, C#, P#, CP#, T7, T8}; spatial filter: common median reference or none; frequency band for spectral estimation: 3–15, 7–15, 10–15, 3–10 Hz; window length: 2 or 5 s. Feature extraction was performed by computing a spectral estimation within a dynamic sliding window approach shifting every 200 ms. EEG data analysis and classification were performed using MATLAB (The Mathworks, Natick, MA).

# **CLASSIFICATION AND PHYSIOLOGICAL ANALYSES**

A SVM (from LIBSVM library, http://www.csie.ntu.edu.tw/∼ cjlin/libsvm/faq.html#f203) with non-linear kernel (radial basis function, rbf) and a LDA classifier were tested on each subject. For SVM classification, the regularization parameter C was set to 0.6 in order to prevent over-fitting, (Cherkassky and Ma, 2004). Two different classification schemes were used. In the first scheme, the classifiers aimed to distinguish between three different classes: spatial navigation, reading and calculation. The most common decomposing strategies for multiclass SVM are the "one against one" and "one against all" binary classification approaches. The former implies the training of k∗(k–1)/2 different binary classifiers (where k is the number of classes); each new test sample is then labeled according to the class selected by the majority of classifiers. The latter implies to build a classifier per each class to distinguish the samples of the selected class from the samples of all remaining classes; a new test sample is labeled according to the maximum outcome of all trained SVMs. A comparison of several multi-class SVM methods showed similar performance for the "one-against-all," "one-against-one" and directed acyclic graph SVM (DAGSVM) (Hsu and Lin, 2002). However, the authors suggested that one-against-one and DAG approaches are more suitable for practical use due to reduced training time. Accordingly, we adopted the "one against one" strategy. For each subject 8 MEC, 8 MER, and 18 SPN blocks were recorded in the two sessions. A leave-one-out cross validation (CV) was used for performing feature selection and for measuring classifiers performance (see **Figure 3**). One MEC, one MER, and two SPN blocks were pseudo-randomly selected and retained for testing classifiers performance, while the remaining blocks were fed into a 7-fold cross-validation scheme for performing parameter selection. This procedure satisfies the requirement of independency between parameter selection samples and testing samples for classifier performance assessment. The 7-fold CV scheme was performed for each combination of parameters (i.e., channel subset, spatial filter, frequency band, and window length) to select the parameters' combination providing the best 7-fold CV accuracy (see **Table 1**). In each fold one MEC, one MER, and two SPN blocks were retained to compute the accuracy of the model trained on the other 6 MEC, 6 MER, and 14 SPN blocks. As duration of SPN blocks was approximately half the duration of MEC, a comparable number of samples was obtained by using for SPN double the number of MEC or MER blocks. For measuring classifiers performance the CV procedure was repeated 7 times with different training subset for each class. The 7 accuracy values were then averaged to provide the final CV accuracy.

A second classification scheme aimed to test generalization capability of the classifier in discriminating between high (MER and MEC) and low (SPN) workload (LW and HW) independently of the type of workload. This additional scheme consisted in the application of a 11-fold CV on the merged MEC and MER datasets. The merged dataset was divided into two parts: four blocks (1 LW and 1 HW block from the MEC dataset and 1 LW and 1 HW block from the MER dataset) were pseudo-randomly selected for estimating classifier performance; the remaining 16 LW and 14 HW were fed into a 11-fold CV scheme aiming to select the best parameter setting for each subject. During each fold 14 LW and 12 HW blocks were used for training and the remaining 2 LW and 2 HW blocks for testing. The 11-fold CV scheme was performed for each combination of parameters (i.e., channel subset, spatial filter, frequency band, and window length). The

**Table 1 | Selected parameters in scheme 1 (top) and scheme 2 for each subject (bottom).**


parameters' combination providing the best 11-fold CV accuracy was then employed to test classifier performance on the retained four pseudo-randomly blocks (see **Table 1**). Features selection for the first and second classification scheme included a 5 s window length and the common median reference (in 15 out of 20 cases).

Feature selection process allowed to determine two most discriminant channels' subsets out of the four considered: {F#, FC#, C#, P#, CP#, O#} and {F#, FC#, C#, P#, CP#, O#, T7, T8} (see **Table 1**).

The Matthews correlation coefficient (MCC) was additionally computed, as it guarantees more robustness to performance variability in binary classification accuracy by taking into account differences in data dimensionality (Baldi et al., 2000). *MCC* ranges between -1 and 1, from total disagreement to agreement between prediction and observation, respectively, and with 0 indicating completely random prediction.

EEG spectral differences among the three conditions were inspected considering 3–7 and 8–12 Hz bands. Spectral differences between HW and LW conditions were assessed using a non-parametric Wilcoxon signed-rank test considering each of the 26 channels of all participants for 3–7 and 8–12 Hz bands separately.

# **RESULTS**

Classification performance of continuous EEG data during the three mental states SPN, MEC and MER is reported in **Table 2**. The LDA based classifier generated on average the highest accuracy (83.30%, *MCC* = 0.72) across all subjects, with peaks of 89.72% for accuracy and 0.84 for *MCC* in subject 2. The SVM based classifier generated on average the lower accuracy (65.68%, *MCC* = 0.45). The results of the classification between HW and LW are reported in **Table 3**. As for the first scheme, LDA yielded on average the highest accuracy (88.56%, *MCC* = 0.74) across all subjects, with peaks of 96.92% for accuracy and 0.92 for *MCC* in subject 1. SVM yielded on average a lower accuracy (86.59%) and a lower *MCC* value (0.63). Visual inspection of EEG power spectrum at representative discriminative channels (Fz and Pz, as they are usually less affected by muscle artifacts) showed power

#### **Table 2 | Results of the classification of spatial navigation, reading, and calculation.**


*The accuracy, MCC, and average values for LDA and SVM classifiers are reported for each participant.*

#### **Table 3 | Results of the classification between LW and HW.**


*The accuracy, MCC, and average values for LDA and SVM classifiers are reported for each participant.*

changes at theta frequencies (3–7 Hz) and at alpha band (8– 12 Hz) among all three conditions (see **Figure 4A**). Moreover, increased alpha band power in frontal regions and theta band power in the frontal and parietal regions was measured during reading and calculation with respect to spatial navigation (see **Figure 4B**). Topographic distribution of the 8–12 Hz band power

**FIGURE 4 | (A)** Power spectra (grand average across all subjects) of two discriminative channels of all conditions in frontal and parietal areas (Fz and Pz). Solid, dashed, and dotted lines represent the grand average power spectrum for SPN, MER, and MEC tasks, respectively. Fz shows a clear power difference

icant positive peak in the left frontal area (F7) and several negative peaks in central and parietal areas (*p* < 0.001, see larger dots in **Figure 5**, left), except channels F8, Fc6, T7, C3, C4, T8, Cp1, Pz.

difference comparing HW to LW conditions showed positive peaks of neuroelectric activity in the left frontal area, F7, and bilateral parietal regions, Cp1 and Cp2 (*p* < 0.001, see larger dots in **Figure 5**, right). Topographic distribution of the 3–7 Hz band power difference comparing HW to LW conditions shows a signif-

# **DISCUSSION**

Our study investigated brain states classification during the execution of multiple cognitive processes in MR. To this end we explored continuous EEG data decoding during performance of MR relevant tasks such as spatial navigation, calculation and reading in XIM, using LDA and SVM based classifiers. Results of our first classification scheme, aiming to discriminate spatial navigation, calculation and reading, showed high performance of both LDA and SVM classifiers, with an average accuracy of around 83% for LDA and of around 65% for SVM. Results of our second classification scheme, aiming to decode mental workload independently on the workload task, showed high classification performance of both SVM and LDA (on average both above 86%). Successful decoding of all mental states was achieved considering a 5 s time-window shifting every 200 ms, permitting online applications with a bit-rate of about 5 bits/s.

Previous EEG-BCI mainly investigated mental states decoding considering different states separately, for instance motor

among all conditions in the 3–7 Hz band, whereas Pz shows a clear power difference in the 8–12 Hz band. **(B)** Power spectra (grand average across all subjects) of channels showing positive peaks during high (HW) as compared to low (LW) mental workload (left frontal area, F7, and central parietal region, Cp2).

imagery, attention, performance capability, emotional arousal, or brain activity prefiguring behavioral errors (Millán et al., 2002; Muller et al., 2008; Schubert et al., 2009; Eichele et al., 2010). However, the capability of decoding multiple brain states is particularly important for BCI-controlled MR as it would allow the implementation of more flexible scenarios with less behavioral constraints. A previous study aiming to implement a BCI for the recognition of multiple mental tasks from on-line spontaneous EEG signal reported a recognition rate of 70% in distinguishing between relaxation, left and right hand movement imagination using a simple local neural classifier (Millán et al., 2002). In addition, the authors performed a preliminary analysis in one subject to test generalization of two local neural classifiers in discriminating between three tasks—relaxation, arithmetic subtraction, and left hand movement imagination, as well as relaxation, cube rotation and left hand movement imagination; performance accuracy reached over 90% of correct prediction on the combined task (Millán et al., 2002). A more recent EEG classification study reported successful offline multi-class discrimination of several conditions, such as resting, mental calculation, mental writing and rotation, by combining wavelet transform decomposition for feature selection and a feed-forward neural network with one-step secant algorithm (Upadhyay, 2013).

Here, we tested a simpler approach using LDA and SVM, also on the basis of the results of a comparative analysis of multi-class EEG classifiers for BCI, such as LDA, Nearest Neighbor Classifier (NNC) and linear SVM, indicating that LDA provides the highest classification accuracy with low dimensional feature space (Lee et al., 2005). In line with these results both our classification schemes showed better performance of LDA with respect to SVM.

Increased accuracy of SVM classifier would also be achievable through additional optimization procedure applied to its parameters (i.e., C and γ), typically via cross-validation, but this would result in a substantial increase of the computational time. On the other hand, SVM can achieve higher performance as compared to LDA with high dimensional feature vectors (Lee et al., 2005).

Because of large intersubject variability of EEG data, subjectdependent classifiers, as those used here, guarantee better performance than subject-independent classifiers (Lotte and Ang, 2009) but they require an initial offline calibration during which the participants need to evoke specific mental state by performing appropriate tasks (supervised classifier). However, a subjectindependent BCI system with no need of training, implemented using a combination of large datasets of subject-dependent classifiers into a single subject-independent classifier, demonstrated performance similar to that of subject-dependent methods (Fazli et al., 2009). In addition, Vidaurre and colleagues investigating co-adaptive learning using machine learning techniques implemented a subject-independent supervised classifier with no need of offline calibration procedure that showed good performance even in participants that are not able to control conventional BCI (Vidaurre et al., 2010). These promising findings suggest the possibility to extend to use of subject-independent classifiers in BCI applications.

Our subject-dependent LDA-based classifier provided the highest accuracy mostly using 3–10 and 3–15 Hz frequency power bands. Spectral analysis indicated changes at theta band (3–6 Hz) as well as at alpha band (8–12 Hz) between different mental states in frontoparietal regions. Alpha band increase in the frontal area was observed for all conditions, whereas both theta and alpha bands increase in the parietal regions was larger for reading and calculation with respect to spatial navigation. Accordingly, high workload, that included both reading and calculation, compared to low workload—spatial navigation—showed significant power differences at 3–6 and 8–12 Hz bands. In particular theta band maximum was observed in the left frontal area, while alpha band peaked in the left frontal and bilateral parietal regions.

Despite intersubject variability and our small sample size, the observed alpha band increase in the bilateral parietal areas is in line with previous results reporting alpha changes in bilateral parietal and occipital brain regions associated with mental workload, task engagement or attention (Humphrey and Kramer, 1994; Pope et al., 1995; Kohlmorgen et al., 2007). The measured changes in the theta band power are also in line with previous studies indicating that theta oscillations are related to spatial navigation as well as encoding and retrieval of spatial information (Kahana et al., 1999; Bischof and Boulanger, 2003). In particular, high amplitude theta activity, mainly in the left frontal and right temporal cortices, has been measured during navigation in a virtual maze (Kahana et al., 1999). Other studies corroborated these results and showed that the frequency of theta episodes is directly associated with the difficulty of maze navigation (Bischof and Boulanger, 2003). In light of previous studies our results indicate that indeed the here adopted reading and calculation tasks required increased allocation of mental resources with respect to spatial navigation.

In addition, we observed delta frequencies (3 Hz) power changes during all conditions in frontal regions. These results are in line with previous studies indicating increased EEG oscillations in the range 1–3.5 Hz in frontal regions associated with different cognitive processes (Harmony, 2013), in particular during internal concentration and calculation (Fernandez et al., 1995).

Our mental states classifiers can equally be employed for real-time analysis of frequency bands. Online monitoring of alpha and theta bands power would be important for assessing participants' performance as these bands reflect cognitive and memory processing (Klimesch, 1999). In online MR-combined BCI applications the information stream provided to the user could be adapted to the current workload as indicated by alpha and theta oscillations. Kohlmorgen et al. (2007) proposed the use of ratios of activity in alpha (8–12 Hz) or theta (3–7 Hz) bands to compute an index of the user task engagement.

Alpha band is also critical for visual perception, in particular medium and lower amplitudes can reflect improved performance in somatosensory and visual discrimination tasks (Pfurtscheller and Lopes Da Silva, 1999; Hanslmayr et al., 2005; Palva and Palva, 2007; Van Dijk et al., 2008). Moreover, decrease in the alpha frequencies (8–12 Hz) before target onset was associated with augmented visual target detection (Ergenoglu et al., 2004). Further studies confirmed this observation by showing that the amplitude of prestimulus ∼10 Hz oscillations correlated with the detection of the upcoming target: the smaller the amplitude, the more likely the target would be detected (Van Dijk et al., 2008; Busch et al., 2009; Mathewson et al., 2009). On the contrary, stronger pre-stimulus alpha frequency band amplitude has been linked to increased cognitive performance (Neubauer and Freudenthaler, 1995; Klimesch, 1999). Thus, online inspection of alpha wave oscillations might be used for triggering stimuli presentation so as to optimize stimulus detection, as well as for improving interpretation of novel information and data mining since alpha band activity has also been associated with learning (Klimesch, 1999).

In conclusion, our LDA classifier is sufficiently flexible and powerful for the implementation of a MR-combined BCI system. Successful classification of mental states based on subject-specific single trials EEG indicates the possibility to combine BCI technology with the XIM so that brain activity could drive the adaptation of data representation. A possible way to refine our BCI in MR is the use of an asynchronous modality where participants do not need to follow a fixed repetitive scheme to switch from one mental task to another one. Asynchronous BCI, allowing individuals to decide when to perform a mental task and when to stop it and switch to another one, are more flexible and adaptive to different scenarios (Millán et al., 2002; Millan Jdel and Mourino, 2003). Ultimately, this approach would permit to model the user experience in the XIM as common product between the initial data representations and the changes made interactively as consequences of users' neurophysiological signals associated with spontaneous behavior.

# **ACKNOWLEDGMENTS**

The present study was supported by EU grants: FP7-ICT-2009- 258749 CEEDs: The Collective Experience of Empathic Data Systems; FP7-ICT-2013- 609593 BNCI Horizon 2020. The Future of Brain/Neural Computer Interaction: Horizon 2020; Italian Ministry of Health, GR-2009-1591908.

# **REFERENCES**


**Conflict of Interest Statement:** The Review Editor Dr. Emanuele Pasqualotto declares that, despite having collaborated with some of the authors, the review process was handled objectively. The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 07 June 2014; accepted: 12 November 2014; published online: 27 November 2014.*

*Citation: De Massari D, Pacheco D, Malekshahi R, Betella A, Verschure PFMJ, Birbaumer N and Caria A (2014) Fast mental states decoding in mixed reality. Front. Behav. Neurosci. 8:415. doi: 10.3389/fnbeh.2014.00415*

*This article was submitted to the journal Frontiers in Behavioral Neuroscience.*

*Copyright © 2014 De Massari, Pacheco, Malekshahi, Betella, Verschure, Birbaumer and Caria. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Learned EEG-based brain self-regulation of motor-related oscillations during application of transcranial electric brain stimulation: feasibility and limitations

# *Surjo R. Soekadar 1,2,3\*, Matthias Witkowski 1,2,3, Eliana G. Cossio2,3, Niels Birbaumer 3,4 and Leonardo G. Cohen1*

*<sup>1</sup> Human Cortical Physiology and Neurorehabilitation Section, NINDS, NIH, Bethesda, MD, USA*

*<sup>2</sup> Applied Neurotechnology Lab, Department of Psychiatry and Psychotherapy, University Hospital Tübingen, Tübingen, Germany*

*<sup>3</sup> Institute of Medical Psychology and Behavioral Neurobiology, University of Tübingen, Tübingen, Germany*

*<sup>4</sup> Ospedale san Camillo, IRCCS, Venice, Italy*

#### *Edited by:*

*Andreas Meyer-Lindenberg, Central Institute of Mental Health, Germany*

#### *Reviewed by:*

*Alexander Fingelkurts, BM-Science - Brain and Mind Technologies Research Centre, Finland Andrew A. Fingelkurts, BM-Science - Brain and Mind Technologies Research Centre, Finland*

#### *\*Correspondence:*

*Surjo R. Soekadar, Applied Neurotechnology Lab, Department of Psychiatry and Psychotherapy, University Hospital of Tübingen, Calwerstr. 14, 72076 Tübingen, Germany e-mail: surjo.soekadar@ uni-tuebingen.de*

**Objective:** Transcranial direct current stimulation (tDCS) improves motor learning and can affect emotional processing and attention. However, it is unclear whether learned electroencephalography (EEG)-based brain-machine interface (BMI) control during tDCS is feasible, how application of transcranial electric currents during BMI control would interfere with feature-extraction of physiological brain signals and how it affects brain control performance. Here we tested this combination and evaluated stimulation-dependent artifacts across different EEG frequencies and stability of motor imagery-based BMI control.

**Approach:** Ten healthy volunteers were invited to two BMI-sessions, each comprising two 60-trial blocks. During the trials, learned desynchronization of mu-rhythms (8–15 Hz) associated with motor imagery (MI) recorded over C4 was translated into online cursor movements on a computer screen. During block 2, either sham (session A) or anodal tDCS (session B) was applied at 1 mA with the stimulation electrode placed 1 cm anterior of C4.

**Main results:** tDCS was associated with a significant signal power increase in the lower frequencies most evident in the signal spectrum of the EEG channel closest to the stimulation electrode. Stimulation-dependent signal power increase exhibited a decay of 12 dB per decade, leaving frequencies above 9 Hz unaffected. Analysis of BMI control performance did not indicate a difference between blocks and tDCS conditions.

**Conclusion:** Application of tDCS during learned EEG-based self-regulation of brain oscillations above 9 Hz is feasible and safe, and might improve applicability of BMI systems.

**Keywords: brain-machine interface (BMI) control, motor imagery, EEG, transcranial electric stimulation (TES), stimulation artifacts**

# **INTRODUCTION**

Brain-machine interfaces (BMI) translate physiological features of brain activity associated with the user's intention or state into control signals of a particular device or computer (Birbaumer and Cohen, 2007). BMIs are increasingly used in the context of neurorehabilitation, e.g., after stroke (Ang et al., 2011; Ramos-Murguialday et al., 2013) with two main purposes: (1) to drive assistive devices or computers that surrogate a lost or impeded function, or (2) to facilitate motor recovery related to BMIbased neurofeedback training (Wang et al., 2010; Soekadar et al., 2011). The best-established BMI in stroke is based on volitional modulation of mu-rhythm (8–15 Hz, also called sensorimotor rhythm, SMR) recorded over the sensori-motor cortex using electro- or magnetoencephalography (EEG/MEG) (Buch et al., 2008; Ramos-Murguialday et al., 2013). Motor planning, imagery or execution is associated with a decrease in amplitude of mu-rhythm (Pfurtscheller and Neuper, 1997; Llanos et al., 2013). This decrease can be quantified as event-related desynchronization (ERD) (Pfurtscheller and Aranibar, 1979) and used by a BMI system to control an exoskeleton moving a patient's paralyzed hand (Soekadar et al., 2011).

Depending on the physiological features, several sessions of BMI training are typically required to achieve stable and reliable BMI control (Soekadar et al., 2011). Training-related BMI learning, however, is often substantially impeded in patients with brain lesions, such as stroke, traumatic brain injury or other brain disorders resulting in compromised learning capacity (Buch et al., 2008). Thus, it would be desirable to identify strategies to improve BMI learning and performance in these populations.

Several studies suggest that the application of weak electric currents in the form of transcranial direct current stimulation (tDCS) can improve motor learning (Zimerman et al., 2012), cognition (Metuki et al., 2012), memory consolidation (Dockery et al., 2009; Reis et al., 2009) and emotional processing (Nitsche et al., 2012). While the underlying physiological mechanisms of these effects are still unknown, it has been shown that tDCS can result in polarity-dependent shifts of membrane potentials and modulate cortical excitability (Nitsche and Paulus, 2000; Pellicciari et al., 2013). Recent work indicates that anodal tDCS applied before a motor imagery-based BMI session might improve BMI control (Wei et al., 2013) and modulate motor imagery-related mu-ERD (Matsumoto et al., 2010). A series of experiments suggests that learning is faster when tDCS is applied during a task compared to when it is applied prior to the task (Stagg et al., 2011). It was concluded that such timingdependency might be of particular importance for the development of plasticity-inducing stimulation protocols (Nitsche et al., 2007). Furthermore, in addition to being time-saving in the clinical context when applied during motor or BMI training, direct effects of tDCS on brain oscillatory activity in the context of such training can be assessed, allowing for a better understanding of how transcranial electric currents modulate task-related brain activity.

While a novel strategy using magnetoencephalography (MEG) for *in vivo* assessment of neuromagnetic brain oscillations during transcranial electric brain stimulation was recently introduced (Soekadar et al., 2013a), the immobility and costs of MEG constrains broader clinical applications, e.g., for BMI training in the context of neurorehabilitation. EEG, in contrast, is inexpensive and widely used in clinical environments. However, the application of electric currents at voltages of up to 20 volts to the human head while recording EEG in the range of millivolts is particularly challenging. This is due to two main reasons: (1) Most EEG systems use wet electrodes to improve conduction between electrodes and the skin. Likewise, the application of electric currents to the head also requires good conduction, mostly provided by using electrolyte gels or pastes. If the conductive agents used for the EEG electrodes and stimulating electrodes are in direct contact, the stimulation currents will saturate the EEG amplifier and impede any physiological recordings. (2) Electric stimulation of the head can be associated with additional stimulation-depended signals picked up by the EEG amplifiers that might reduce classification accuracy in BMI control. This would be disadvantageous for both assistive and neurofeedback BMI applications. While denoting any stimulation-dependent signal during neurophysiological recordings of brain activity as artifacts or noise (i.e., an irregular fluctuation of the measured signal that does not contain meaningful information or obscures the information of interest) is somewhat appealing, it should be emphasized that the separation between the physiological responses or effects of the stimulation and stimulator-dependent noise is difficult. While recently systems were introduced that integrate electric stimulation and recording of EEG (e.g., Starstim® by Neuroelectrics, Barcelona, Spain) allowing for simultaneous EEG monitoring during tDCS (Schestatsky et al., 2013), it remained unclear how EEG signals across different frequency bands recorded at different distances from the stimulating electrode are affected by tDCS, and whether learned electroencephalography (EEG)-based brain-machine interface (BMI) control e.g., using motor imagery (MI) during tDCS is feasible, reliable, and safe.

Here, we investigated such a combination choosing an electrode montage that places the stimulation electrode as close as 1 cm to the EEG electrode recording brain oscillations used for BMI control (see **Figure 1A**). To allow generalization to other BMI paradigms, we first characterized stimulation-dependent signals at different EEG locations. We then compared the signal power in different frequencies (delta: 0.1–4 Hz; theta: 4–9 Hz; alpha: 9–15 Hz; and beta: 15–30 Hz) before stimulation and during stimulation, and identified those frequencies significantly influenced by stimulation-dependent signals. Finally, we calculated the effect size of the stimulation conditions for each frequency band. Potential side effects of stimulation, like tingling, itching or burning (Brunoni et al., 2011) were assessed throughout the sessions.

# **MATERIALS AND METHODS**

# **PARTICIPANTS AND EXPERIMENTAL DESIGN**

Ten healthy volunteers (5 males, 5 females, mean age: 26.7 years ± 4.3) exhibiting reliable and stable motor imagery-based BMI control in a previous investigation (Soekadar et al., 2011) were invited for two BMI sessions on consecutive days. All participants were right handed according to the Edinburgh Handedness Inventory (Oldfield, 1971). Each session was divided into two blocks of 60 trials (block 1, block 2). While no stimulation was delivered during block 1 of both sessions, either sham stimulation (session A) or anodal stimulation (session B) was applied during block 2. Sessions were conducted in random order. All participants gave written informed consent before entering the study. The study protocol was approved by the National Institute of Neurologic Disorders and Stroke Institutional Review Board (NINDS IRB).

### **ELECTROENCEPHALOGRAPHIC (EEG)-RECORDINGS AND FEEDBACK OF mu-EVENT-RELATED DESYNCHRONIZATION (mu-ERD)**

Participants were seated comfortably in an armchair facing a computer monitor. EEG was recorded from the following conventional EEG-recording sites (F3, FC5, C3, P3, Fz, AFz, Cz, FCz, F4, FC6, C4, P4 according to the international 10/20 system) using a 12-channel active electrode EEG system (Acti-cap® and BrainAmp MRplus®, BrainProducts, Gilching, Germany) with the reference electrode placed at FCz and the ground electrode at AFz. For translation of neurophysiological signals into visual feedback, BCI2000, a multipurpose standard BCI platform, was used (Schalk et al., 2004).

All participants were instructed to use visuo-kinesthetic MI of moving their left hand modulating right-hemispheric murhythms when they saw the visual cue indicating the initialization of each trial.

To rule out overt movements during MI, electromyography (EMG) was recorded from the first dorsal interosseus muscle (FDI), extensor digitorum communis (EDC), extensor carpi ulnaris (ECU) and flexor carpi radialis (FCR) during the sessions. Skin/electrode resistance was kept below 10 k-. EMG signals were recorded at a sampling rate of 1 kHz and high-pass filtered at 2 Hz (BrainAmp ExG®, Brainproducts, Gilching, Germany). Trials in which EMG activity exceeded that recorded during rest by two standard deviations were interrupted and excluded from further analysis. The number of excluded trials in which

EMG activity was present was comparable across participants and ranged between 5 and 10%.

yellow circle). The reference stimulation electrode was placed over the left supraorbital region (blue). **(B)** BMI paradigm. Electric brain activity recorded

Amplitude of mu-rhythm event-related desynchronization (mu-ERD) and synchronization (mu-ERS) was visually fed back to the participants during each trial (see **Figure 1B**). While increasing mu-ERD was indicated by up-movements of a ball above the horizontal midline of the screen, mu-ERS resulted in down-movements below the midline. Visual feedback was continuously updated every 100 ms. Computation of mu-ERD/mu-ERS included the power spectrum estimation (an autoregressive model of order 16 using the Yule–Walker algorithm) of each incoming sample at the optimal frequency for mu-ERD detection. The optimal frequency was identified in a screening run before the first session (11 Hz in all participants). Resulting values were compared with mean power values of the preceding inter-trial-intervals (ITI) that were continuously updated during BMI control according to the method of Pfurtscheller and Aranibar (1979) and as previously implemented into a mu-ERD BMI system (Soekadar et al., 2011):

$$\overline{R} = \frac{1}{N} \sum\_{t=1}^{N} R\_t \tag{1}$$

$$ERD\left(t\right) = \frac{T\_t}{\overline{R}} - 1\tag{2}$$

Where *t* represents the recorded sample block, *Tt* the eventrelated task condition period and *Rt* the power estimate in a given frequency band of *t*. *R* (reference value) represents power estimates during the rest (task-free) condition.

Each session consisted of 2 blocks with 60 trials. After the first block either sham stimulation (session A) or anodal stimulation (session B) was applied during the following 60 trials in a

instructed to keep the ball above the dotted horizontal line during the task

# **TRANSCRANIAL DIRECT CURRENT STIMULATION (tDCS)**

to hit the target (indicated by red bar).

randomized order.

tDCS was applied via two conducting 4 × 6 cm rubber electrodes and attached to the participant's head using a conductive paste (Ten20®, D.O. Deaver, Aurora, CO, USA). The adhesive features of this paste prevented any sliding or dislocation of the electrodes during the attachment of the EEG cap. A bipolar electrode montage (right M1 and left supraorbital area) was used (see **Figure 1A**) to deliver a current of 1 mA (current density 0.04 mA/cm2; total charge 0.048 C/cm2 using the DC-STIMULATOR PLUS, neuroConn GmbH, Germany). During sham stimulation, the DC stimulator was set up to apply an anodal current for 15 s and—at the offset—decrease stimulation intensity in a ramp-like fashion, a method shown to achieve a good level of blinding (Gandiga et al., 2006). Prior to the first block of the first session, the M1 hand-area was localized in all participants based on the motor evoked potential (MEP) hotspot of the first digit's interosseus muscle (FDI) using transcranial magnetic stimulation (TMS). After the end of block 2 of sessions A and B, all participants rated possible discomfort, pain, tingling, itching or burning associated with the stimulation on visual analog scales (VAS) to assess safety and tolerability of tDCS during BMI control.

#### **OFFLINE ANALYSIS**

For all outcome measures, assumption of a normal distribution (Shapiro–Wilk test of normality) was tested. Parametric tests were corrected by Greenhouse-Geisser estimates if Mauchly's sphericity test indicated significance. To compare signal power across conditions, fast Fourier transformations were performed for all EEG data collected during the first and second blocks of session A and session B (see **Figure 2**). A repeated-measures ANOVA (rmANOVA) with factors "block" (block 1, block 2) and "frequency" (delta, theta, alpha, beta) was performed based on the raw EEG signal power recorded from electrode C4 (in immediate proximity to the stimulation electrode) and P3 (at ∼8–10 cm distance from the stimulation electrode) to investigate stimulation-dependent changes in different EEG channels. *Posthoc* paired-samples Students *t*-tests were used when applicable and corrected for multiple comparisons (Bonferroni). Timefrequency representations (TFR) were plotted for both conditions and tested for statistical differences at two different electrode positions (C4 in immediate proximity of the stimulation electrode, and P3) using a cluster-based permutation test (Maris and Oostenveld, 2007). Effect size of stimulation-dependent signal differences in each frequency band was calculated using Cohen's *d* transformed into a regression coefficient *r* where *r* < 0.3 is considered a small, *r* < 0.5 is considered a medium, and *r* > 0.5 is considered a large effect (Cohen, 1988). A rmANOVA with factors "session" (session A, session B) and "block" (block 1, block 2) was used to evaluate changes of BMI control across sessions in the absence (block 1) and presence (block 2) of anodal tDCS. BMI control was defined as the time during each trial in which the ball was above midline (indicating mu-ERD). All analyses were performed in SPSS 17.0. Significance level was set to *p* < 0.05. Variance is defined as the standard error of the mean.

# **RESULTS**

### **STIMULATION-DEPENDENT CHANGES OF SIGNAL POWER ACROSS DIFFERENT FREQUENCY BANDS AND CONDITIONS**

#### *Stimulation electrode in close proximity (***∼***1 cm) to the recording EEG channel (C4)*

While rmANOVA of data recorded during session A (sham stimulation during block 2) from electrode C4 (**Figure 2A**) showed a main effect for "frequency" [*F*(1, <sup>27</sup>) = 37.156, *p* < 0.0001], indicating an expected difference between power values across the investigated frequency bands, there was no effect for "block" [*F*(1, <sup>27</sup>) = 0.233, *p* = 0.641] and no interaction between the factors (*p* = 0.188). *Post-hoc t*-tests showed no significant differences between power values of block 1 and 2 in any frequency band (delta: *p* = 0.729; theta: *p* = 0.963; alpha: *p* = 0.946; beta: *p* = 0.784).

significant during sham stimulation (left column), anodal stimulation

right panel), while alpha (9–15 Hz) and beta (15–30 Hz) frequencies showed no difference between conditions.

When analyzing data of session B (anodal stimulation during block 2) from the same location, we found a main effect for both "frequency" [*F*(1, <sup>27</sup>) = 77.536, *p* < 0.0001] and "block" [*F*(1, <sup>27</sup>) = 33.16, *p* < 0.0001] and an interaction between the two [*F*(1, <sup>3</sup>) = 39.584, *p* < 0.0001]. *Post-hoc t*-tests showed a significant difference between power values of block 1 and 2 in the delta band (*p* < 0.0001) and a trend in the theta band (*p* = 0.068), but no significant differences in the alpha (*p* = 0.482) or beta (beta: *p* = 0.336) bands.

## *Stimulation electrode at* **∼***8–10 cm distance to the recording EEG channel (P3)*

Analysis of EEG data recorded ∼8–10 cm away from the stimulation electrode (P3) (**Figure 2B**) during session A, revealed a main effect for "frequency" [*F*(1, <sup>27</sup>) = 28.28, *p* < 0.0001], but no effect for "block" [*F*(1, <sup>27</sup>) = 1.202, *p* = 0.301] and no interaction between the factors (*p* = 0.056). We found no significant differences between power values of block 1 and 2 in any frequency band (delta: *p* = 0.655; theta: *p* = 0.984; alpha: *p* = 0.937; beta: *p* = 0.923).

The same analysis performed for data acquired during session B (anodal stimulation during block 2) at electrode position P3, showed a main effect for both, "frequency" [*F*(1, <sup>27</sup>) = 34.39, *p* < 0.0001] and "block" [*F*(1, <sup>27</sup>) = 11.34, *p* < 0.01] and a significant interaction between the two [*F*(1, <sup>3</sup>) = 14.152, *p* < 0.0001]. *Posthoc t*-tests indicated a significant difference between power values of block 1 and 2 in the delta band (*p* < 0.05), but not in the theta (*p* = 0.217), alpha (*p* = 0.445) or beta (beta: *p* = 0.482) bands.

Fourier transformations and TFR calculated for both sessions and blocks separately showed an increase in signal power during the second block of session B (see **Figures 2**, **3**), which was highest in the slow frequency bands showing a decay of ∼12 dB per decade. While not significant, we found a slight increase in broadband noise across all frequencies during anodal stimulation (second block, session B; see **Figure 3B**, right panel). Statistical analysis using a non-parametric cluster-based permutation test indicated significant stimulation-dependent changes of signal power in frequencies below 8 Hz (**Figure 3C**, right panel), but not in frequencies above 9 Hz. Calculation of the stimulationdependent signal difference's effect size on each frequency band indicated a large effect on the delta band (*d* = 0.9893, *r* = 0.443) which was weaker in the theta band (*d* = 0.5721, *r* = 0.275) and small in the alpha (*d* = 0.2988, *r* = 0.122) as well as beta band (*d* = 0.2791, *r* = 0.115).

#### **BRAIN-MACHINE INTERFACE (BMI) CONTROL ACROSS SESSIONS AND CONDITIONS**

RmANOVA indicated no main effects for "session" [*F*(1, <sup>9</sup>) = 0.30, *p* = 0.597] or "block" [*F*(1, <sup>9</sup>) = 1.097, *p* = 0.322] and no interaction between the factors [*F*(1, <sup>1</sup>) = 0.349, *p* = 0.569]. There also was no difference between block 1 of session A and session B (*p* = 0.541), nor a difference between block 1 and block 2 of session A (*p* = 0.880) or session B (*p* = 0.470) (**Figure 4**).

#### **SAFETY AND TOLERABILITY OF SIMULTANEOUS tDCS DURING EEG-BASED BRAIN-MACHINE INTERFACE (BMI) CONTROL**

Six of ten participants reported light tingling (rated at 2–3 on a VAS, mean value across all participants 1.6 ± 0.4 with 0 = none and 10 = unbearable, extreme tingling) in at least one of the sessions, and five reported light itching (mean value 1.4 ± 0.8; 0 = none, 10 = unbearable, extreme itching), mainly at the beginning of block 2. None of the participants reported any form of severe discomfort or pain. Participants were unable to distinguish sensations between session A and session B.

#### **DISCUSSION**

Our study investigated the influence of simultaneous tDCS on EEG recordings across different frequency bands and shows that online extraction of physiological signals during learned selfregulation of brain oscillations for online MI-based BMI control is feasible and safe. We found that application of tDCS is associated with a significant signal increase across slower frequency bands below 9 Hz (delta and theta) in direct proximity of the stimulation electrode, and delta band (<4 Hz) recorded at larger distance (>∼8 cm). However, signals oscillating above 9 Hz (e.g., 11 Hz) were not influenced by stimulation and could be successfully used for reliable, motor imagery-based BMI control. Our results indicate that any BMI paradigm driven by modulation of brain oscillations in the alpha (9–15 Hz) or beta range (15–30 Hz) or even higher frequencies (>30 Hz) is possible, given that the signal/noise ratio allows proper linear or non-linear classification of the neurophysiological features used for BMI control.

An important aspect in the combination of tDCS and BMI control is the stimulation montage. Due to the conduction properties of the human head, most electric currents pass through the

**mu-ERD)-based brain-machine interface (BMI) performance was defined as the percentage of time mu-ERD was detected during trials.** BMI performance was comparable between both sessions and did not exhibit differences between block 1 and block 2. There was neither a difference between block 1 of session A and session B (*p* = 0.541), nor a difference between block 1 and block 2 of session A (*p* = 0.880) or session B (*p* = 0.470).

skin and cerebro-spinal fluid (CSF), while only a fraction enters the gray or white matter (Sadleir et al., 2012). The path of the electric currents in any given case depends on many individual characteristics, such as bone thickness, shape of the skull, density of bone-passing veins or volume of the outer CSF space (which is increased, for instance, in brain atrophy). Various computational models were developed to calculate intracranial current flow and to identify brain areas with the highest magnitude of cortical electric fields (Bikson et al., 2012; Sadleir et al., 2012). Depending on the montage, the area with the highest magnitude of the cortical electrical field might be in larger distance from the stimulating electrode not directly underlying the current source (Edwards et al., 2013). Thus, the technique described here which allows for the application of electric currents as close as 1 cm to the EEG recording channel might be used for modulating cortical activity of brain areas functionally related to BMI control.

When using a different stimulator than the one used in this investigation, corresponding characteristics of this device should be tested first before combining it with EEG-based BMI systems to rule out other stimulation-dependent contaminations of the EEG signal.

While some previous studies suggested that tDCS can have immediate effects on motor imagery-related ERD (Matsumoto et al., 2010) and BMI control (Wei et al., 2013), we did not find such immediate effects. This might be due to the fact that participants were not BMI-naïve and exhibited already high and stable BMI control before admission to the study. tDCS, here applied as anodal tDCS, might not have improved BMI performance further, as a ceiling in BMI control may have been reached in these participants. Another reason could be the montage of the stimulation electrode placed 1 cm anterior of the C4, which might have resulted in the highest magnitude of cortical electrical fields in brain areas not related to motor imagery-based BMI control. A different placement of the stimulation electrode might have led to other results.

The immediate effects of electric currents on oscillatory activity in the human brain and their relatedness to behavior are still poorly understood (Dayan et al., 2013). As noted previously, a sensor-space based approach does not allow unambiguous differentiation between signals with a physiological origin opposed to stimulation-dependent signals deriving from the electric circuit of the stimulator. It is conceivable that the observed slow frequency signal power elevation during tDCS is in part of physiological origin and might reflect mechanisms that also underlie the previously well-described after-effects of tDCS, for instance polarityspecific modulation of cortical excitability and improvements of cognition and learning. Due to the design of the paradigm, the effect of tDCS on slow cortical potentials (SCP) could not be investigated here, but might be of interest in future studies to improve general understanding of the physiological effects of tDCS in the context of learned brain self-regulation during motor imagery-based BMI control. Implementation of algorithms with noise-canceling features, e.g., online source-reconstruction using beamformers (Soekadar et al., 2013b) might help to further investigate these mechanisms. The combination of electrical stimulation during multimodal EEG-MEG recordings in the context of brain self-regulation might help to shed light on these issues and further improve understanding of the exact neural substrates and mechanisms underlying the learning of abstract skills, like volitional modulation of brain oscillatory activity in the context of BMI control (Koralek et al., 2012).

## **CONCLUSION**

tDCS delivered at 1 mA in close proximity (1 cm) to an EEG channel used for learned self-regulation of brain oscillatory activity above 9 Hz is feasible and safe. While associated with a signal power elevation across slower frequencies, brain signals above 9 Hz were unaffected by the stimulation allowing simultaneous application of electric currents during motor imagery-based online BMI control. Such combination might substantially improve the applicability and practicality of BMI use in patient populations, for instance, in the context of neurorehabilitation, and allow systematic investigation of the relatedness between learned brain self-regulation, brain oscillatory activity and behavior.

#### **ACKNOWLEDGMENTS**

This work was supported by the Intramural Research Program (IRP) of the National Institute of Neurological Disorders and Stroke (NINDS), USA; the Center for Neuroscience and Regenerative Medicine (CNRM), Uniformed Services University of Health Sciences, USA; the Advanced Convergence Research Center at the Daegu Gyeongbuk Institute of Science and Technology, Korea; the German Federal Ministry of Education and Research (BMBF, grant number 01GQ0831, 16SV5838K to Surjo R. Soekadar and Niels Birbaumer); the Deutsche Forschungsgemeinschaft (DFG, grant number SO932-1 to Surjo R. Soekadar and Reinhart Koselleck Project support to Niels Birbaumer); the European Commission under the project WAY (grant number 288551 to Surjo R. Soekadar and Niels Birbaumer); the Volkswagenstiftung and the Baden-Württemberg Stiftung, Germany. We thank Sook-Lei Liew and Birgit Teufel for their assistance in preparing the manuscript.

# **REFERENCES**


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 12 January 2014; accepted: 03 March 2014; published online: 18 March 2014. Citation: Soekadar SR, Witkowski M, Cossio EG, Birbaumer N and Cohen LG (2014) Learned EEG-based brain self-regulation of motor-related oscillations during application of transcranial electric brain stimulation: feasibility and limitations. Front. Behav. Neurosci. 8:93. doi: 10.3389/fnbeh.2014.00093*

*This article was submitted to the journal Frontiers in Behavioral Neuroscience. Copyright © 2014 Soekadar, Witkowski, Cossio, Birbaumer and Cohen. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Is neural hyperpolarization by cathodal stimulation always detrimental at the behavioral level?

# *Cornelia Pirulli 1†, Anna Fertonani 1† and Carlo Miniussi 1,2\**

*<sup>1</sup> Cognitive Neuroscience Section, IRCCS Istituto Centro San Giovanni di Dio Fatebenefratelli, Brescia, Italy*

*<sup>2</sup> Neuroscience Section, Department of Clinical and Experimental Sciences, University of Brescia, Brescia, Italy*

#### *Edited by:*

*Niels Birbaumer, University of Tuebingen, Germany*

#### *Reviewed by:*

*Andrea Brovelli, Centre National de la Recherche Scientifique, France Niels Birbaumer, University of Tuebingen, Germany*

#### *\*Correspondence:*

*Carlo Miniussi, Department of Clinical and Experimental Sciences, University of Brescia, Viale Europa 11, 25123 Brescia, Italy e-mail: carlo.miniussi@ cognitiveneuroscience.it*

*†These authors have contributed equally to this work.*

Cathodal transcranial direct current stimulation (c-tDCS) is usually considered an inhibitory stimulation. From a physiological perspective, c-tDCS induces hyperpolarization at the neural level. However, from a behavioral perspective, c-tDCS application does not always result in performance deterioration. In this work, we investigated the role of several important stimulation parameters (i.e., timing, presence of pauses, duration, and intensity) in shaping the behavioral effects of c-tDCS over the primary visual cortex. In Experiment 1, we applied c-tDCS at two different times (before or during an orientation discrimination task). We also studied the effects of pauses during the stimulation. In Experiments 2 and 3, we compared different durations (9 vs. 22 min) and intensities (0.75 vs. 1.5 mA) of stimulation. c-tDCS applied before task execution induced an improvement of performance, highlighting the importance of the activation state of the cortex. However, this result depended on the duration and intensity of stimulation. We suggest that the application of c-tDCS induces depression of cortical activity over a specific stimulated area; but to keep reactivity within given limits, the brain react in order to restore the equilibrium and this might result in increased sensitivity in visual performance. This is a further example of how the nervous system dynamically maintains a condition that permits adequate performance in different environments.

**Keywords: transcranial direct current stimulation, perceptual learning, metaplasticity, facilitation, cathodal tDCS, homeostasis, NIBS, neural noise**

# **INTRODUCTION**

Transcranial direct current stimulation (tDCS) is a technique that allows the modulation of cortical excitability. A direct current of low-level intensity (∼2 mA) is applied to electrodes that sit on the subject's scalp. This current passes through the scalp and crosses the extra cortical layers to reach the cortex, modulating the membrane polarity of neurons within a region of underlying neural tissue. The low strength of the current is not able to induce depolarization at a threshold level of "inactive" neurons (i.e., inducing an action potential). However, if there is ongoing activity (i.e., background activity determined by state or taskinduced activity), the change in membrane potential induced by tDCS can promote more effective "excitation" or "inhibition" in a polarity-specific manner (Creutzfeldt et al., 1962; Bindman et al., 1964; Stagg and Nitsche, 2011). These tDCS-induced changes in the neuronal threshold during stimulation result from changes in membrane permeability, including depolarization of the soma by anodal stimulation (a-tDCS) and hyperpolarization by cathodal stimulation (c-tDCS) (Liebetanz et al., 2002; Nitsche et al., 2003b, 2004a,b). Polarization effects outlast the tDCS period (Nitsche and Paulus, 2000, 2001), and these after-effects are due to changes in receptor activity at the synaptic level, in addition to membrane polarity shifts (Nitsche et al., 2003a, 2004a,b).

From a behavioral standpoint, strong *a priori* assumptions are often made in neuromodulation studies using tDCS where physiological effects are directly mapped on to behavioral effects. The application of a-tDCS during a task is thought to induce facilitation, while c-tDCS is assumed to induce inhibition of performance. While the concept of a-tDCS facilitating and ctDCS worsening performance seems well established, these effects are mainly valid for tDCS in the motor system (Nitsche et al., 2008, but see Wiethoff et al., 2014). Behavioral effects of tDCS have been identified in several functional areas, but the relationship between facilitation and inhibition is often quite complex (Jacobson et al., 2012; Miniussi et al., 2013). tDCS is thought to prime the behavioral system by increasing/decreasing cortical excitability and producing behavioral effects in the cognitive system. Nevertheless, the final effect depends on the stimulated area and on its involvement in a given task (i.e., the type of process and the final goal of the task) (Dockery et al., 2009; Nitsche and Paulus, 2011; Vallar and Bolognini, 2011; Berryhill and Jones, 2012; Jacobson et al., 2012; Tseng et al., 2012; Weiss and Lavidor, 2012; Filmer et al., 2013; Hsu et al., 2014; Nozari et al., 2014).

Several examples of contradictory final behavioral outcomes for c-tDCS can be found at different processing levels (Jacobson et al., 2012). Monti et al. (2008) found facilitation in a naming task using c-tDCS over Broca's area. Antal et al. (2004b) found that c-tDCS applied to the visual middle temporal area improved performance in a visuomotor coordination task. In a different study from the same group (Antal et al., 2001) using a Gabor patch stimulus, only c-tDCS showed a significant change in static and dynamic contrast sensitivities. Vicario et al. (2013) showed that c-tDCS applied to the posterior parietal cortex enhanced temporal accuracy in a time reproduction task compared to sham stimulation (see also Dockery et al., 2009; Moos et al., 2012; Weiss and Lavidor, 2012; Filmer et al., 2013). Moreover, several studies reported no effect of c-tDCS on task performance (e.g., Kincses et al., 2004; Iyer et al., 2005; Sparing et al., 2008; Cerruti and Schlaug, 2009; Fertonani et al., 2010; Kraft et al., 2010). Contrasting effects of c-DCS have also been found at the neurophysiological level (see Matsunaga, 2004; Pellicciari et al., 2013). A magnetoencephalography study has reported that the two tDCS polarities induce the same cortical EEG power density (Venkatakrishnan et al., 2011). Antal et al. (2004a) showed that c-tDCS over the primary visual cortex (V1) decreased the amplitude of an early visual evoked potential (N70), whereas Accornero et al. (2007) reported the opposite result with an increased amplitude of the early P100 potential.

This complex pattern of results may be explained by the fact that the ability or the efficacy of tDCS to induce modifications of membrane polarity—and consequently behavioral performance—depends on several methodological and technical parameters: current density, duration, timing of application, and pauses between stimulations (for a review, see Brunoni et al., 2012). Clearly, all these methodological aspects will determine the "reaction of the brain" to the DC stimulation in relation to the subject's state and task/protocol. As an example, the importance of duration and of the presence of pauses during the period of stimulation has been recently demonstrated by Fricke et al. (2011). These authors measured the motor evoked potential (MEP) amplitude after repeated tDCS. The authors found that 5 or 10 min of c-tDCS decreased excitability and suppressed MEP amplitude for 5 and 30 min, respectively, indicating the importance of tDCS duration. However, if a pause of 3 min was inserted between two stimulations of 5 min, there was an inversion of the effect, resulting in MEP amplitude enhancement (see also Monte-Silva et al., 2010). The stimulation intensity is also an important factor. Batsikadze et al. (2013) showed that 1 mA c-tDCS decreased MEP amplitude, but the application of 2 mA resulted in increased cortical excitability (see also Teo et al., 2011; Moos et al., 2012; Hoy et al., 2013). In addition, behavioral performance induced by tDCS depends on the timing of application in relation to task execution (Stagg et al., 2011; Pirulli et al., 2013; Fertonani et al., 2014). Several studies have shown that the same type of stimulation may have different behavioral effects (facilitation vs. inhibition vs. null-effect) depending on whether it is applied before or during the task execution.

Given the heterogeneity of the effects induced by c-tDCS, the aim of this work was to explore the outcome of applying c-tDCS on V1. We chose an orientation discrimination task (ODT) because it is a well established task to study visual perceptual learning (Vogels and Orban, 1985; Shiu and Pashler, 1992), and it has been showed to involves primarily V1 (Schoups et al., 2001; Li et al., 2004). Nevertheless it should be considered that we cannot be totally sure that stimulation delivered by tDCS is focalized only under the stimulation electrode (Miranda et al., 2006; Wagner et al., 2007). We applied c-tDCS before or during the execution of ODT, with or without pauses during stimulation, and at different intensities and durations. Based on the previous reports, our expectation was that the final outcome of the tDCS in terms of response facilitation or inhibition would depend on the interaction between the brain state and when and how c-tDCS was applied. Therefore, c-tDCS would not necessarily induce a univocal behavioral inhibition. Given that the brain is constructed to keep certain important parameters within given limits, the brain would react proportionally when it is shifted from these limits in order to restore the equilibrium, as suggested by the concept of homeostasis (see Bernard, 1878 and Cannon, 1929 in Cooper, 2008). Therefore, an initial down-regulation induced by an inhibitory stimulation, given for a longer time, at higher intensity can be reverted, rendering the involved neurons more easily responsive. Nevertheless to explain the effects of tDCS during a behavioral task it has been proposed that a neural noise framework should be consider (see non-linear systems and stochastic resonance; Miniussi et al., 2013), suggesting that the outcome of applying tDCS depends on the noise present in the system and the level of tDCS and task -induced activity, rather than solely on the stimulation polarity.

# **MATERIALS AND METHODS**

#### **ORIENTATION DISCRIMINATION TASK**

We chose an ODT that is a widely studied VPL task and involves V1 neurons (Vogels and Orban, 1985; Shiu and Pashler, 1992). This task has been previously described in detail by our group (Fertonani et al., 2011; see Pirulli et al., 2013). Given the ODT characteristics, is likely that a local (i.e., V1) circuit of neuronal populations is dedicated to execute the task, nevertheless we cannot exclude that tDCS effects are due to a more complex neuronal network, involving others parietal areas. Briefly, throughout the experiment, participants were comfortably seated in an armchair in a quiet, dimly illuminated room. The subjects had to decide as quickly and accurately as possible whether the presented stimulus (a target line) was tilted clockwise or counter clockwise relative to the previously presented stimulus (reference line) (see trial structure in **Figure 1**). After each response an auditory feedback informed the subjects about the correctness of their responses (an high tone indicated the right response while a low tone the wrong response).

Each block of the ODT consisted of 64 trials and lasted approximately 4 min. The ODT consisted of 5 experimental blocks plus a training block. The training block contained 8 trials and an increased rotation angle between the two stimuli (10◦ clockwise or counter clockwise). In Experiments 1 and 2, the angular difference between the reference and the target was ±1.10, 1.21, 1.33, and 1.46◦. All of the experimental parameters were balanced and randomized between blocks. In Experiment 3 (control experiment), the task was made easier by replacing the smallest degree of rotation (1.10◦) with 1.60◦ (1.21, 1.33, 1.46, and 1.60◦). All of the other task characteristics were unchanged, except for the presence of a baseline block before the stimulation.

#### **TRANSCRANIAL DIRECT CURRENT STIMULATION**

tDCS was delivered by a battery-driven current stimulator (Eldith-Plus, NeuroConn GmbH, Ilmenau, Germany) through

a pair of saline-soaked surface sponge electrodes. The "active" electrode (16 cm2) was placed over the occipital cortex in the area corresponding to V1, which was defined as 10% of the nasion-inion distance above the inion (mean position = 3.5 ± 0.2 cm above the inion). The reference electrode (60 cm2) was fixed extra-cephalically on the right arm. The electrodes were kept in place with elastic bands, and an electro-conductive gel was applied under the electrodes to help reduce impedance to the electrical current. When tDCS was applied, the polarity of the active electrode over V1 was always cathodal. For active tDCS, the current was ramped up over 8 s (fade-in phase), held constant for the experimental time, and then ramped down over 8 s (fade-out phase). In the sham c-tDCS, the current was ramped up (8 s) and down (8 s) and stayed at level for 15 s.

#### *Experiment 1—"Timing and pauses"*

In this experiment, we investigated the effect of the timing of stimulation during task execution, either online or offline (before task execution), and the presence of pauses during the stimulation (intervals of 2 min between blocks). In all conditions, we applied c-tDCS for 22 min at 1.5 mA (current density of the active electrode 0.094 mA/cm<sup>2</sup> of the reference 0.025 mA/cm2) as shown in **Figure 2A**. In the continuous stimulation, current was administered without pauses, while in the paused stimulation condition,

the stimulation was turned on at the beginning of each experimental block and maintained until the end of the block. c-tDCS was applied for approximately 4 min during each of the 5 experimental blocks, with 2 min of pauses between blocks (i.e., 4 min of stimulation—2 min of pause—4 min of stimulation, and so on). In the online condition, the stimulation was applied during the task execution. In the offline condition, stimulation was applied before task execution while the subjects were listening to an audio book played on an audio device, maintaining the same time intervals used in the online condition. The duration of the entire experimental session was approximately 30 min for the online conditions and approximately 60 min for the offline conditions. The procedure is described in **Figure 2A**.

#### *Experiment 2—"Duration and intensity"*

In this experiment, we investigated the effect of the duration and intensity of stimulation for the offline condition, as shown in **Figure 2B**. Subjects were stimulated for 9 min at an intensity of 1.5 mA or for 22 min at an intensity of 0.75 mA (current density of the active electrode 0.047 mA/cm<sup>2</sup> of the reference 0.013 mA/cm2). Data were compared with those for 22 min of stimulation at an intensity of 1.5 mA in the offline condition collected in Experiment 1 (see **Figure 2B**).

#### *Experiment 3—Control experiment*

In Experiment 3, we corroborated the effects of the stimulation intensity for the offline protocol with a modified experimental design (see the description of the ODT task, above). We facilitated the ODT and added a baseline block before the beginning of the offline stimulation. The stimulation was applied continuously for 22 min with an intensity of 1.5 or 0.75 mA (see **Figure 2C**).

### *Sensation questionnaire*

In all of the experiments, at the end of the experimental session, we asked all subjects to complete a questionnaire (Fertonani et al., 2010) about the tDCS-induced sensations that they experienced during the different conditions so that we could evaluate if different stimulation protocols (e.g., active vs. sham) induced different sensations.

### **SUBJECTS**

A total of 139 healthy subjects participated in the three experiments. All of the participants were right-handed except for 6 subjects tested in Experiment 3, who were equally distributed in the experimental groups. All participants had normal or corrected-to-normal vision. Subjects with a history of seizures, implanted metal objects, heart problems or any neurological disease were not recruited. Moreover, subjects who had a task performance below chance (no learning) were excluded from the study. Based on these criteria, 17 participants were excluded. The remaining 122 subjects (61 males, mean age ± standard deviation 22.0 ± 2.9 years; range 19–33 years) participated in the experiments.

### *Experiment 1—"Timing and pauses"*

Seventy-two subjects were assigned to one of the five groups stimulated for 22 min at 1.5 mA: online paused (14 subjects, 7 males; 21.7 ± 2.6 years), offline paused (14 subjects, 7 males; 21.6 ± 2.6 years), online continuous (10 subjects, 5 males; 21.7 ± 0.8 years), offline continuous (10 subjects, 5 males; 23.0 ± 3.6 years) and placebo stimulation (sham, 24 subjects, 12 males; 21.7 ± 3.6 years). In the sham group, 14 subjects were stimulated online and 10 subjects were stimulated offline. The data for the online conditions were collected in a previous experiment (Fertonani et al., 2011; for details see Pirulli et al., 2013).

# *Experiment 2—"Duration and intensity"*

We tested two additional groups (20 Subjects): 10 subjects (5 males; 22.1 ± 2.0 years) stimulated for 9 min at 1.5 mA (offline-9 min) and 10 subjects (5 males; 21.4 ± 2.0 years) stimulated for 22 min at 0.75 mA (offline-0.75 mA). We compared these two groups with the offline and sham groups of Experiment 1.

# *Experiment 3—Control experiment*

We tested three new groups (30 Subjects) stimulated for 22 min at 0.75 mA (10 subjects, 5 males, 23.0 ± 4.4 years), 1.5 mA (10 subjects, 5 males, mean age 20.8 ± 1.8 years) and sham (10 subjects, 5 males; 23.6 ± 4.1 years).

The study was approved by the Ethics Committee of the IRCCS Centro San Giovanni di Dio Fatebenefratelli, Brescia, Italy. Safety procedures were used in accordance with non-invasive brain stimulation indications (Iyer et al., 2005; Poreisz et al., 2007; Rossi et al., 2009), and written informed consent was obtained from all participants prior to the beginning of the experiments.

# **DATA ANALYSIS**

The average orientation sensitivity was calculated as a d prime value (d ) from measurements of the hit rate and false-alarm rate, for each subject and each block separately for each stimulation condition. We have chosen the d as a measure of accuracy because it is roughly invariant when response bias is manipulate, whereas simple indexes such as proportion correct don't have this property. As a first index of learning rate, we analyzed the relationship between d values and block numbers using linear regression analysis. This analysis allowed us to associate a slope value with each subject. A second index called the "learning index" was calculated, for Experiment 3, by subtracting the mean baseline d value from the mean d value of block 5 for each subject.

The Kolmogorov-Smirnov test confirmed the normality of the distribution of all data (d values, slope, learning index), and subsequently data were analyzed using a repeated-measures analysis of variance (ANOVA). The data sphericity was tested using the Mauchly test, where appropriate. When the test results were statistically significant, the data were corrected using the Huynh-Feldt correction. The effect size is reported using the partial Eta squared value. A *p*-value *<* 0.05 was considered significant for all statistical analyses. For multiple comparisons, we used Fisher's Least Significant Difference (LSD) method to test our specific "a priori" hypotheses (i.e., to compare different timings of application, intensities and durations). For all other comparisons, the *p*-values were corrected using a Tukey correction.

Data from the sensations induced by c-tDCS were analyzed using the Kruskal–Wallis one-way analysis of variance and, subsequently, with multiple comparisons.

# **RESULTS**

# **ORIENTATION SENSITIVITY—d-**

### *Experiment 1—"Timing and pauses"*

We performed a repeated-measure ANOVA with *block* (from 1 to 5) as a within-subjects factor and *stimulation* (online paused, online continuous, offline paused, offline continuous, and sham) as a between-subjects factor. We observed a significant main effect for *block* [*F*(4*,* 268) <sup>=</sup> <sup>24</sup>*.*105; *<sup>p</sup> <sup>&</sup>lt;* <sup>0</sup>*.*001; <sup>η</sup><sup>2</sup> *<sup>P</sup>* = 0*.*265] and *stimulation* [*F*(4*,* 67) <sup>=</sup> <sup>3</sup>*.*272; *<sup>p</sup>* <sup>=</sup> <sup>0</sup>*.*016; <sup>η</sup><sup>2</sup> *<sup>P</sup>* = 0*.*163]. The interaction between *block* and *stimulation* was not statistically significant [*F*(16*,* 268) = 1*.*005; *p* = 0*.*452].

For *block*, multiple *post-hoc* comparisons revealed a statistically significant difference between block 1 and blocks 2, 3, 4, and 5 (all *p <* 0*.*01), between block 2 and blocks 4, and 5 (all *p <* 0*.*001), and between block 3 and block 5 (*p* = 0*.*043).

For *stimulation*, multiple *post-hoc* comparisons revealed that the offline paused (mean d ± standard error of the mean s.e.m. = 0.615 ± 0.122) and offline continuous (0.623 ± 0.114) conditions were significantly different from the online paused (0.304 ± 0.121), online continuous (0.346 ± 0.118), and sham (0.368 ± 0.083) (all *p <* 0*.*05) conditions (see **Figure 3**).

These results support the initial hypothesis that c-tDCS, applied before the ODT, modulates behavior, while c-tDCS applied during the task does not modify the final outcome. We found an improvement in the subject's accuracy when c-tDCS was applied offline.

These data highlight the absence of a difference between the conditions with or without pauses. Confirmation was obtained with an ANOVA with *block* (from 1 to 5) as a within-subjects factor and *timing* (online vs. offline) and presence of *pauses* (paused vs. continuous) as between-subjects factors. We observed a significant main effect of *block* [*F*(4*,* 176) <sup>=</sup> <sup>18</sup>*.*438; *<sup>p</sup> <sup>&</sup>lt;* <sup>0</sup>*.*001; <sup>η</sup><sup>2</sup> *P* = <sup>0</sup>*.*295] and *timing* [*F*(1*,* 44) <sup>=</sup> <sup>11</sup>*.*026; *<sup>p</sup>* <sup>=</sup> <sup>0</sup>*.*002; <sup>η</sup><sup>2</sup> *<sup>P</sup>* = 0*.*200]. The factor *pauses* was not statistically significant [*F*(1*,* 44) = 0*.*077; *p* = 0*.*782]. No interaction was statistically significant. For *block*, multiple *post-hoc* comparisons revealed a statistically significant difference between block 1 and blocks 3, 4, and 5 (all *p <* 0*.*001) and between block 2 and block 3 (*p* = 0*.*042), 4 (*p <* 0*.*001), 5 (*p <* 0*.*001) (see **Figure 3**).

Having verified that the presence of pauses during stimulation does not influence the effect of stimulation, we collapsed the two online conditions (continuous and paused) and the two offline conditions (continuous and paused) into one online and one offline condition (hereafter, all conditions with the initial parameters, i.e., 22 min duration and 1.5 mA intensity, will be called "online" and "offline").

#### *Experiment 2—"Duration and intensity"*

We performed a repeated-measure ANOVA with *block* (from 1 to 5) as a within-subjects factor and *stimulation* (online, offline, offline-9 min, offline-0.75 mA, and sham) as a between-subjects factor. We observed a significant main effect for *block* [*F*(4*,* 348) = 20*.*286; *p <* 0*.*001; η<sup>2</sup> *<sup>P</sup>* = 0*.*189] and *stimulation* [*F*(4*,* 87) = 4*.*727; *<sup>p</sup>* <sup>=</sup> <sup>0</sup>*.*002; <sup>η</sup><sup>2</sup> *<sup>P</sup>* = 0*.*163]. The interaction between *block* and *stimulation* was not statistically significant [*F*(16*,* 348) = 1*.*512; *p* = 0*.*092].

For *block*, multiple *post-hoc* comparisons revealed a statistically significant difference between block 1 and blocks 2, 3, 4, and 5 (all *p <* 0*.*001), between block 2 and blocks 4, and 5 (all *p <* 0*.*001), and between block 3 and block 5 (*p* = 0*.*003).

**FIGURE 4 | Results of Experiment 2.** Data are presented as the mean d values. The lines represent the fit of each condition. The blue line represents the offline 1.5 mA for the 22 min condition, the green line represents the online 1.5 mA for the 22 min condition, the turquoise line represents offline 1.5 mA for the 9 min condition, and the double blue line represents the offline-0.75 mA for the 22 min condition. The sham condition is shown in black.

For *stimulation*, multiple *post-hoc* comparisons revealed that offline (0.618 ± 0.085) was different from sham (0.368 ± 0.083), online (0.321 ± 0.086) and offline-0.75 mA (0.264 ± 0.109). Moreover, offline-9 min (0.577 ± 0.123) was different from online and offline-0.75 mA and showed a marginally significant difference from sham (*p* = 0*.*069, all other *p <* 0*.*05) (see **Figure 4**).

#### *Experiment 3—Control experiment*

We performed a repeated-measure ANOVA with *block* (from baseline to 5) as a within-subjects factor and *stimulation* (offline, offline-0.75 mA, and sham) as a between-subjects factor. We observed a significant main effect for *block* [*F*(5*,* 135) = 7*.*843; *p <* 0*.*001; η<sup>2</sup> *<sup>P</sup>* = 0*.*225] and an interaction between *block* and *stimulation* [*F*(10*,* 135) = 1*.*980; *p* = 0*.*128]. The factor *stimulation* was not statistically significant [*F*(2*,* 27) = 1*.*434; *p* = 0*.*256].

For *block*, multiple *post-hoc* comparisons revealed a statistically significant difference between block baseline and blocks 1, 2, 3, 4, and 5 (all *p <* 0*.*01). The interaction between *block* and *stimulation* revealed that stimulation influences the block trend. In the offline condition, block baseline was different from blocks 1, 2, 3, 4, and 5; block 1 was different from blocks 4 and 5; and block 2 was different from block 5. In the offline-0.75 mA condition, block baseline was different from block 4. In the sham condition, block baseline was different from blocks 1 and 5 and block 1 was different from block 2.

A One-Way ANOVA on block baseline showed that in this block, d was not different between the stimulation conditions [*F*(2*,* 27) = 0*.*064; *p* = 0*.*938].

In Experiment 3, the presence of a block of baseline allowed us to show a different rate of learning between the stimulation conditions. For this purpose, we executed two different analyses. A One-Way ANOVA on slope [*F*(2*,* 27) <sup>=</sup> <sup>4</sup>*.*630; *<sup>p</sup>* <sup>=</sup> <sup>0</sup>*.*019; <sup>η</sup><sup>2</sup> *P* = 0*.*255] demonstrated that offline was different from sham (*p* = 0*.*008) and 0.75 mA (*p* = 0*.*027). A One-Way ANOVA on the learning index [*F*(2*,* 27) <sup>=</sup> <sup>3</sup>*.*658; *<sup>p</sup>* <sup>=</sup> <sup>0</sup>*.*039; <sup>η</sup><sup>2</sup> *<sup>P</sup>* = 0*.*213] showed that offline was different from sham (*p* = 0*.*034) and 0.75 mA (*p* = 0*.*022) (see **Figure 5**).

#### *Sensations induced by different conditions*

In the tDCS sensation questionnaire (Fertonani et al., 2010), each participant reported having tolerated the stimulation without discomfort. The results of the questionnaire are reported in **Table 1** (Experiments 1 and 2) and **Table 2** (Experiment 3). In Experiments 1 and 2, the analysis did not reveal any differences

between the stimulations for pain, burning, heat, iron taste, and fatigue sensations. The analysis demonstrated a statistically significant difference between stimulations with respect to itchiness [*H*(6*, <sup>N</sup>* <sup>=</sup> 92) = 17*.*382, *p* = 0*.*008] and pinching [*H*(6*, <sup>N</sup>* <sup>=</sup> 92) = 18*.*974, *p* = 0*.*004]. Subsequently, multiple comparisons were performed for these two sensations. For itchiness, offline continuous was significantly different (*p* = 0*.*03) from sham, whereas for pinching, online paused and offline paused were significantly different from sham (respectively *p* = 0*.*03 and *p* = 0*.*01). The subjective influence on performance was equal for all the stimulations. In Experiment 3, the analysis did not reveal any difference between the stimulation conditions in the perceived sensations.

#### **DISCUSSION**

In this work, we demonstrated that c-tDCS, which is often considered to be inhibitory at the behavioral level, can actually induce facilitatory effects, enhancing subjects' performance. In the Experiment 1, we showed that the effects of c-tDCS are dependent on the timing of stimulation. Only c-tDCS before the task induced an improvement of performance. Moreover, in this experiment, we applied 22 min of stimulation interspersed by pauses of 2 min and compared this protocol to a continuous protocol. We found that short pauses do not play any role in shaping the final outcome. In Experiment 2, we applied offline stimulation and found that 9 min of c-tDCS facilitated the performance only slightly less. Instead, current density induced a facilitatory effect in our protocol. Finally, Experiment 3 confirmed the importance of current density and showed that the facilitatory effect is not due to skill differences between subjects (i.e., differences in accuracy) because all the groups had the same performance at baseline.

The presence of short pauses (2 min) during c-tDCS, both in the online and in the offline conditions, did not influence the results. In previous works, it has been shown that interstimulation intervals determine the effects of c-tDCS on the

**Table 1 | Transcranial direct current stimulation (tDCS)-induced sensations—Experiments 1 and 2: mean intensity of the sensations reported by subjects after tDCS and the percentage of subjects who reported each sensation.**


*Sensation intensity was evaluated on a 5-point scale: 0 none, 1 mild, 2 moderate, 3 considerable, and 4 strong. The column "Effect on performance" indicates the subjective feelings of the participants relative to the effect of the tDCS-induced sensation on performance (valid only for the online condition).*


**Table 2 | Transcranial direct current stimulation (tDCS)-induced sensations—Experiment 3 (See Table 1 description).**

motor cortex (Monte-Silva et al., 2010; Fricke et al., 2011). These previous studies found that varying the length of the pause between two stimulations modified the effects on cortex excitability. Fricke et al. (2011) demonstrated that short inter-stimulation intervals (3 min) induced an inversion of the inhibitory effect of c-tDCS, resulting in MEP amplitude enhancement. Monte-Silva et al. (2010) showed that the inhibitory effects of c-tDCS were more efficacious if a second period of stimulation was applied during the after-effects of the first stimulation with an interval of 3 or 20 min. However, with long inter-stimulation intervals, the c-tDCS-induced inhibitory after-effects were diminished. In Experiment 2 of our study, we confirmed that c-tDCS applied before the task induces a significant improvement in performance, regardless of the presence of pauses during stimulation. A possible explanation for these partially contrasting data could be the differences in stimulation parameters (i.e., the intensity and number of pauses\blocks of stimulation). A higher intensity of stimulation (1.5 mA) may induce a stable excitability shifts in relation to the execution of the task, and therefore, the additional presence of pauses might not further affect the cortical state (see Batsikadze et al., 2013). Similarly the presence of successive multiple pauses\blocks of stimulation might decrease further changes of the cortical state through adaptation.

During the online application of c-tDCS, while the subjects were performing the ODT, we expected a worsening of performance. However, it was difficult to observe a decline in subject accuracy in our learning task because subjects could not have a performance level lower than chance ("floor effect"). This result suggests that the behavioral level of task performance at baseline is a key factor in determining a null effect of c-tDCS. Additionally, compensatory networks can be activated during stimulation (Sack et al., 2005), and therefore, a functional compensation might intervene, maintaining behavior after neuronal challenge (O'Shea et al., 2007). Nevertheless, the absence of inhibition by online c-tDCS is in line with previous data (Jacobson et al., 2012).

Our most important result involves the fact that c-tDCS applied before the task at 1.5 mA induced a facilitatory effect on subjects' accuracy during the ODT (see also Dockery et al., 2009; Moos et al., 2012; Weiss and Lavidor, 2012; Filmer et al., 2013). Most studies that have applied 1 mA c-tDCS in the motor system found decreased cortical excitability; this has also been shown with a decrease in MEP amplitude (see Nitsche and Paulus, 2011; Medeiros et al., 2012). However, these physiological effects are not linear, but they seem to depend on several parameters. In a recently published work, Batsikadze et al. (2013) showed that 20 min of c-tDCS at 2 mA applied to the motor cortex significantly increased MEP amplitudes, while 1 mA of the same stimulation decreased cortico-spinal excitability. This result highlights that an increase in intensity (2 mA) and duration (20 min) of stimulation induces an opposite outcome compared to standard parameters. Batsikadze et al. (2013) suggested that this result might be due to the direction of plasticity from the amount of neuronal calcium influx caused by the stimulation protocol: whereas low postsynaptic calcium enhancement induced by lowintensity c-tDCS causes long-term depression, higher intensity c-tDCS induces a large calcium increase, resulting in long-term potentiation (LTP) (Cho et al., 2001; Lisman, 2001; Batsikadze et al., 2013). Our data are in line with this result; nevertheless, at a systems level, different intensities of stimulation might induce different adaptive responses by the brain, as discussed below.

Importantly, our stimulation was applied before the task execution. The time at which the stimulation is applied has been investigated at the behavioral level in the motor (Stagg et al., 2011) and visual domains (Pirulli et al., 2013). Stagg et al. (2011) showed that a-tDCS on the motor cortex has opposite effects if applied during or before a sequence—a motor learning task. However, the effects of c-tDCS do not seem to be timing dependent. c-tDCS application during or before an explicit motor learning task induces a slowing in reaction time (Stagg et al., 2011). Nevertheless, in a previous work (Pirulli et al., 2013), we demonstrated that the same type of current can induce different effects on visual performance depending on the state of activation of the cortex.

Here, we demonstrated that the application of c-tDCS before the VPL induces an improvement in performance that is not present if c-tDCS is applied during the execution of the task. Therefore, c-tDCS causes different effects depending on the state level of the neurons at the moment of stimulation (Dockery et al., 2009; Tseng et al., 2012; Weiss and Lavidor, 2012; Filmer et al., 2013; Hsu et al., 2014; Nozari et al., 2014). While an online effect would rely mainly on the depolarization or hyperpolarization induced by tDCS interacting with task execution (see Miniussi et al., 2013), an offline protocol would rely more on a change in the state of the stimulated area induced by the stimulation i.e., a shift in the input strength needed for the final response (see Figure 5 in Miniussi et al., 2013). Thus, all these aspects, which can be defined as "relatively simple" technical parameters, influence brain activity in response to exogenous stimulation.

Artificially altering neuronal function can trigger homeostatic changes at the synaptic level (Turrigiano and Nelson, 2004). For example, if a synapse is constantly over-inhibited, there can be a compensatory increase in receptor activity at the postsynaptic membrane, termed up-regulation. Homeostatic plasticity is a fundamental physiological mechanism that maintains neural functions within predefined optimal ranges (Bienenstock et al., 1982; Turrigiano and Nelson, 2004; Abraham, 2008). The basis of homeostatic plasticity is that the threshold for LTP induction is not stable but varies depending on previous neuronal activity induced by non-invasive stimulation (Ziemann and Siebner, 2008; Siebner, 2010). Therefore, the application of tDCS before and during the task of interest may result in different functional states. The application of c-tDCS before the execution of the task would lower the level of postsynaptic neural activity, causing a decrease in the threshold for the induction of successive facilitatory mechanisms. The induced neural modification could then facilitate LTP-like mechanisms and consequently induce an improvement in behavioral performance (Bienenstock et al., 1982; Abraham, 2008). Homeostatic mechanisms can stabilize cortical excitability within a range (Siebner et al., 2004). Thus, an initial down-regulation induced by c-tDCS can be reverted. Applying this theory to our data, pre-conditioning the V1 cortex with 22 min of inhibitory stimulation could render neurons that are involved in task execution more easily excitable. In classical studies, a release phenomenon (rebound) has been found at the end of the cathodal DC (Creutzfeldt et al., 1962). However, the timing of this inversion might depend, at least in part, on stimulation parameters like intensity and duration. Indeed, stimulating offline with a different duration or intensity changes the behavioral effects. Our data highlighted the fact that the final behavioral response, obtained by applying tDCS, depends on the history of the stimulated area (i.e., metaplasticity). This has been previously demonstrated in animal studies in which the direction and magnitude of synaptic plasticity depends on the previous history of postsynaptic activity (Huang et al., 1992; Wang and Wagner, 1999).

From a physiological perspective, c-tDCS over the scalp can induce a hyperpolarization at the soma of perpendicularly oriented neurons (Jefferys, 1981; Bikson et al., 2004). However, we are measuring behavior and therefore should consider a network level of reasoning rather than just a cellular one and present data testify the complexity of a neural network response. Indeed, changes in hyperpolarization alter the sensitivity of the entire system and therefore its response threshold, but these changes are ultimately expressed on subject performance. The final response to the task depends on the strength of the signal and on the signal-to-noise ratio, where the signal is the neural activity operational to the task and the noise is random neural activity. Clearly, in this case, the signal-to-noise ratio relies on a system that has changed its state after c-tDCS, a system that has adapted to exogenous stimulation. Adaptation occurs when receptors\neurons change their sensitivity to a stimulus. In the visual system, adaptation to reduced light intensity increases the visual system's ability to detect a stimulus. In the same way, the reduced cortical activity induced by c-tDCS causes an improved threshold for orientation sensitivity (d ) by reducing background noise. Therefore, an inhibitory stimulation may increase the signal-to-noise ratio in the system and facilitate perceptual learning (Miniussi et al., 2013).

In conclusion, the present data show that an inhibitory stimulation does not always induce a deterioration in performance. The effects of c-tDCS should be considered in relation to the timing and the application parameters that will alter the state of the cortical network carrying out a task. We suggest that when applying c-tDCS before a task, it is necessary to consider the involvement of cognitive and non-cognitive adaptation mechanisms. The application of a tDCS protocol that induces a depression in cortical activity over a specific stimulated area might result in increased sensitivity in visual performance. This is a further example of how the nervous system maintains a dynamic state to maintain performance in different environments.

# **REFERENCES**


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 02 May 2014; accepted: 05 June 2014; published online: 27 June 2014. Citation: Pirulli C, Fertonani A and Miniussi C (2014) Is neural hyperpolarization by cathodal stimulation always detrimental at the behavioral level? Front. Behav. Neurosci. 8:226. doi: 10.3389/fnbeh.2014.00226*

*This article was submitted to the journal Frontiers in Behavioral Neuroscience. Copyright © 2014 Pirulli, Fertonani and Miniussi. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Real-time fMRI brain-computer interface: development of a "motivational feedback" subsystem for the regulation of visual cue reactivity

#### **Moses O. Sokunbi 1,2\*, David E. J. Linden1,2 , Isabelle Habes 1,2 , Stephen Johnston<sup>3</sup> and Niklas Ihssen1,2**

<sup>1</sup> MRC Centre for Neuropsychiatric Genetics and Genomics, Institute of Psychological Medicine and Clinical Neurosciences, Cardiff School of Medicine, Cardiff University, Cardiff, UK

<sup>2</sup> Cardiff University Brain Research Imaging Centre (CUBRIC), Cardiff University, Cardiff, UK

<sup>3</sup> Department of Psychology, Swansea University, Swansea, UK

#### **Edited by:**

Sergio Ruiz, Pontificia Universidad Catolica de Chile, Chile

#### **Reviewed by:**

Corinde Wiers, Berlin School of Mind and Brain, Germany Ralf Veit, Institute of Medical Psychology, Germany

#### **\*Correspondence:**

Moses O. Sokunbi, MRC Centre for Neuropsychiatric Genetics and Genomics, Institute of Psychological Medicine and Clinical Neurosciences, Cardiff School of Medicine, Cardiff University, Hadyn Ellis Building, Maindy Road, Cathays, Cardiff, CF24 4HQ, Wales, UK

e-mail: SokunbiMO@cardiff.ac.uk

Here we present a novel neurofeedback subsystem for the presentation of motivationally relevant visual feedback during the self-regulation of functional brain activation. Our "motivational neurofeedback" approach uses functional magnetic resonance imaging (fMRI) signals elicited by visual cues (pictures) and related to motivational processes such as craving or hunger. The visual feedback subsystem provides simultaneous feedback through these images as their size corresponds to the magnitude of fMRI signal change from a target brain area. During self-regulation of cue-evoked brain responses, decreases and increases in picture size thus provide real motivational consequences in terms of cue approach vs. cue avoidance, which increases face validity of the approach in applied settings. Further, the outlined approach comprises of neurofeedback (regulation) and "mirror" runs that allow to control for non-specific and task-unrelated effects, such as habituation or neural adaptation. The approach was implemented in the Python programming language. Pilot data from 10 volunteers showed that participants were able to successfully down-regulate individually defined target areas, demonstrating feasibility of the approach. The newly developed visual feedback subsystem can be integrated into protocols for imaging-based brain-computer interfaces (BCI) and may facilitate neurofeedback research and applications into healthy and dysfunctional motivational processes, such as food craving or addiction.

**Keywords: brain-computer interface (BCI), hunger, visual cue reactivity, food craving, functional magnetic resonance imaging (fMRI), neurofeedback, self-regulation**

# **INTRODUCTION**

Recent advancements in real-time functional magnetic resonance imaging (fMRI) have been based on the increased availability of high-field (in excess of 1.5 tesla) MRI scanners, fast data acquisition sequences, improved real-time pre-processing, improved statistical analysis techniques and methods of data visualization and feedback. These developments have made the implementation of fMRI brain-computer interfaces (fMRI-BCI) and neurofeedback possible (Weiskopf, 2012). In fMRI-BCI, the self-regulation of brain activity in a target region can be achieved by providing real-time feedback of brain activation levels in that target region. While paradigms more commonly utilize feedback signals based on overall activity levels in a given region, more complex metrics of brain activation are being used based on multivariate statistical techniques (LaConte et al., 2007; Sitaram et al., 2008) or correlation of activity in several brain areas (Zilverstand et al., 2014).

#### **NEUROFEEDBACK AND VISUAL CUE REACTIVITY**

In recent years an emerging but technically challenging field in neurofeedback research has been the pairing of feedback signals (and regulation instructions) with simultaneous visual stimulus presentation. Such approaches are especially interesting for research into neural and behavioral responses that are passively triggered by salient emotional or motivational cues, such as fear responses evoked by emotional pictures or craving elicited by drug-related pictures. The degree of drug cue-elicited neural activation within circuits of the orbitofrontal cortex (Ernst et al., 2014), the nucleus accumbens and medial prefrontal cortex (PFC; Wiers et al., 2014) has been shown to determine the degree of cue approach vs. avoidance behavior in alcohol dependent patients. Moreover, alcohol cues automatically evoke cognitive processes that are biased towards approaching the cue and correlate with drinking behavior in adolescents (Peeters et al., 2012). Neurofeedback training to control these emotional/motivational responses should thus aim at the down-regulation of blood oxygen level dependent (BOLD) responses, especially when the targeted neurocognitive process is considered to be dysfunctional. To date, however, most fMRI neurofeedback studies have trained participants to up-regulate activation of a target area. Further, a direct comparison between up- and down-regulation training of the same brain region (anterior insula) showed that down-regulation was more difficult to learn (Veit et al., 2012). In light of its promising clinical applications, this underscores the need for methodological development and research into the area of neurofeedback related to visual cue reactivity. Here we present a novel neurofeedback subsystem for the presentation of motivationally relevant feedback during exposure to appetitive pictures.

### **THE CHOICE OF FEEDBACK TYPE**

When setting up visual cue reactivity neurofeedback paradigms, researchers are confronted with several challenges. One relates to the choice of feedback type: With the simultaneous visual presentation of feedback information and task stimulus, there is a potential risk that monitoring the feedback stimulus distracts attention from the emotional/motivational cue which will confound any effortful down-regulation of the targeted neural response. Since the pioneering neurofeedback studies at the beginning of the last decade, various techniques have been employed to present visual feedback for the self-regulation of BOLD activation over time. Weiskopf et al. (2003, 2004) and deCharms et al. (2004) have used scrolling time series graphs and curves of BOLD activation in the region of interest (ROI) to provide immediate information to the participant. Sitaram et al. (2005) introduced the "thermometer" as a type of visual feedback for the display of brain activity. Positive BOLD activity, compared to a baseline period is shown as an increasing number of red "hot" bars, whereas negative BOLD activity is similarly shown in blue. The authors also introduced a virtual reality feedback system (Sitaram et al., 2005) where volunteers controlled a 3D animated character (a fish in water). Other visual feedback approaches include the presentation of functional maps of the brain (Yoo and Jolesz, 2002) or video-based feedback that has been used to train stroke patients to self-regulate ventromedial premotor cortex (Sitaram, 2007a).

Most studies using simultaneous visual stimulation during feedback-guided regulation have adopted the thermometer approach as described above but temporally separated the feedback presentation from the regulation/visual stimulation period (Posse et al., 2003; Li et al., 2013). In contrast to such delayed feedback, continuous/on-line feedback during regulation has the advantage that participants can adaptively test different mental strategies to optimize regulation. Only few studies have paired the presentation of pictorial cues with BOLD signal feedback in a simultaneous manner. Brühl et al. (2014) trained participants to down-regulate activity of the right amygdala while being exposed to negative emotional faces. During the 20-sec down-regulation blocks, a feedback stimulus reflecting activation levels of the amygdala was presented to the participants, showing two colored rectangles at the picture edges which color-coded activity levels in a similar way as the thermometer approach described above. Rather than using a peripheral feedback stimulus, Veit et al. (2012) embedded the thermometer inside the simultaneously presented emotional cues that consisted of blocks of aversive pictures. Also, Mathiak et al. (2010) provided a positive feedback through facial expression (smiling) when activity in the anterior cingulate cortex (ACC) increased and gradually returned to a neutral expression when the activity dropped.

Critically, such spatial or feature-based distinction between the feedback and the task stimulus can induce distraction or interference effects as seen in dual tasks. As a consequence, responses to the task stimulus (e.g., emotional picture) may be reduced, confounding the effects of effortful down-regulation. Here, we delineate a novel approach in which feedback-guided self-regulation is based on visual changes in the stimulus responsible for the targeted brain responses itself. Specifically, the present paradigm uses appetitive food pictures to evoke responses in brain circuits related to hunger and food craving. The success of regulating those responses during picture exposure in the neurofeedback session is represented as gradual changes in image size. Compared to the feedback stimuli used by Veit et al. and Brühl et al. our paradigm has the advantage that distracting/dual-task effects associated with monitoring the feedback stimulus during cue exposure are minimized. At the same time, by linking decreases in image size to successful downregulation and image size increases to failure, respectively, the paradigm provides real motivational consequences of the participant's regulation effort, i.e., the stimuli mimic avoidance behavior during successful down-regulation and approach behavior during unsuccessful down-regulation. Such behavioral/motivational relevance of the task can increase face validity of the neurofeedback training and may thus facilitate its therapeutic use in pathologies of motivational systems, such as obesity or addiction.

#### **THE ISSUE OF HABITUATION**

Another challenge of visual cue reactivity neurofeedback is that repeated exposure to the same or similar visual stimulus will lead to a decrease of neural responses. Such effects have been well documented in brain imaging tasks using fMRI adaption or "repetition suppression" techniques, in which the reduction of BOLD responses associated with the repeated presentation of identical stimuli is used as a tool to characterize the neural representation of visual objects (Grill-Spector et al., 2006). Similar effects of neural adaptation can also occur in brain circuits involved in emotional processes. For instance, multiple presentations of threatening pictures will lead to a gradually weaker activation of the amygdala (Wright et al., 2001). In the present context of down-regulation of cue-induced BOLD activation patterns, it is thus crucial to control for such effects, as a gradual habituation effect over successive regulation blocks may be misinterpreted as reflecting successful regulation. In previous research such control measures have been implemented, for instance, by including a passive viewing condition (either within- or between-runs) where the same set of pictures and the feedback stimulus shown during regulation is repeated but participants are instructed not to regulate their brain responses (Brühl et al., 2014). Here, we took a similar approach by including "mirror" runs in the paradigm, in which participants were exposed to the same size sequence that was "produced" during a previous regulation block but instructed not to regulate target area activation.

# **COMPUTATIONAL TRANSLATION AND RESPONSE RANGE ADAPTATION OF THE BOLD SIGNAL**

All BCIs include a series of steps for converting the measured brain signal, such as percentage signal change, into commands (Linden, 2014). First, the relevant feature is extracted from the wealth of information that the measurement device picks up, ideally in real time. Secondly, the extracted feature needs to be converted into an output signal for the participant to use through a translation algorithm. Finally, the translation from the extracted feature of the brain signal to the output needs to be adaptive. Peoples' individual neurophysiological responses vary widely yet all must ultimately appear in an identical summary form, for their use, projected onto a computer screen. Thus the conversion has to be adapted to an individual's response range. This adaptation to the original signal also has to take into account the fluctuation of the signal over time and the improvement with training as well as an individual's training capacity. For example the duration of the training has to be adjusted to psychological factors like motivation and fatigue, which again corresponds to well-known general principles from teaching and training.

A well designed feedback system is an important criterion in successfully training participants to self-regulate their BOLD response. Contingent feedback following the participant's response with minimum lag and with reliable information content pertaining to task success improves learning (Sitaram et al., 2008). As detailed below, in the current project we developed a solution for steps two (translation algorithm) and three (adaptation) of the motivational BCI approach outlined above.

# **MATERIALS AND METHODS**

# **fMRI BRAIN-COMPUTER NEUROFEEDBACK ARCHITECTURE**

**Figure 1** describes the fMRI-BCI architecture for neurofeedback training at our center. Our architecture is a closed-loop system with the following major subsystems; signal acquisition, signal analysis and signal feedback. In **Figure 1**, the MRI image pool is the end point of the signal acquisition subsystem and is connected to the signal analysis and signal feedback subsystems through a local area network (LAN). The signal analysis subsystem communicates with the signal feedback subsystem and they both reside on the same computer. The signal feedback subsystem is connected to the projector screen, where the feedback signal is presented.

At the signal acquisition subsystem, localized brain activity of the participant while viewing images on the projector screen is measured by fMRI using a BOLD sequence; a contrast is then made between the signal elicited by the target stimuli (food pictures) and neutral control stimuli (household objects in localiser run, see below), or a fixation baseline (regulation runs). A 3 T whole body scanner (General Electric, Milwaukee, USA) with an 8-channel head coil is used. The parameterisation of the BOLD sensitive echo planar imaging (EPI) sequence is as follows: TR = 2 s, TE = 45 ms, flip angle = 80◦ , 30 slices, FOV = 192 mm, image matrix 64 × 64, in-plane voxel size = 3 mm × 3 mm, slice thickness = 4 mm and a gap of 1 mm. Image reconstruction, distortion correction and image averaging are performed on the MRI scanner computer and stored in the MRI image pool.

The signal analysis subsystem is performed using Turbo-Brainvoyager (TBV) version 3.0 (Brain Innovation, Maastricht, The Netherlands; (Goebel, 2001)). TBV retrieves reconstructed fMRI images from the MRI pool via the LAN and performs online 3D motion correction, temporal filtering, spatial smoothing, spatial normalization and online statistical analysis calculating beta parameters from an incremental general linear model (GLM) based on the predictors of interest (e.g., food vs. neutral pictures). A static ROI is selected by drawing an area on the functional map (3D BOLD signal) computed in TBV. All supra-threshold voxels (according to an investigatorchosen statistical threshold between 2 and 3) are included in the target area for signal extraction. Average BOLD signal values (betas) from the target area are extracted by TBV and stored in a continuously updated real-time protocol file (rtp file).

The signal feedback subsystem retrieves the rtp files, processes and analyses the BOLD signal values contained within. Feedback is presented to the participant in real-time as food image sizes corresponding to the percentage change in BOLD signal values of the ROI during food picture presentation relative to fixation baseline using a moving average across three consecutive TRs. Feedback is presented to the participant with a delay that depends on the time for signal acquisition, signal analysis and signal feedback processing. Minimizing the delay is critical for volitional control (Sitaram et al., 2007b). Applying the general experience from operant conditioning experiments, the maximum delay for successful learning of biological responses is 2 min (Yoo et al., 2004). The design and implementation of the visual feedback subsystem for the regulation of hunger or food craving is described in the following sub-sections.

# **DESIGN**

The visual feedback subsystem involves two types of functional imaging run; the neurofeedback and mirror/control runs. The present paradigm involves four runs of each type presented in an alternating order. The neurofeedback runs are the experimental runs, where the participants learn to control their fMRI signal in a block design with periods of rest followed by periods of down-regulation. During the rest blocks, a fixation cross is displayed for 20 s. During the down-regulation block, also 20 s in duration, one food image is presented which varies in size dependent on the percent fMRI signal change, during the block, relative to the preceding fixation block. There are five rest blocks and four down-regulation blocks in the entire display sequence of the neurofeedback run totalling a run length of 180 s. During the down-regulation block, the size of the food image is updated every TR (2 s) leading to a consecutive display of 11 different image sizes in total. Image sizes can vary between 10% and 100% of the original image size (1013 by 760 pixels) that are distributed across the whole size range. The size of the food image displayed at the first TR of each down-regulation block is set to 50% of the maximum image size, which corresponds to the percent signal change (PSC) at the first TR (FPSC) relative to the preceding rest

block. Here, the FPSC is taken as the reference scaling point for the calibration of subsequent PSCs. The increase or decrease in image size calibration of the PSC is described using the following calibration algorithm:

```
def PSCDisplay(PSC):
  if PSC < FPSC - 100:
    Display 10% of the image size
  elif FPSC - 100 <= PSC < FPSC - 75:
    Display 15% of the image size
  elif FPSC - 75 <= PSC < FPSC - 50:
    Display 20% of the image size
  elif FPSC - 50 <= PSC < FPSC - 25:
    Display 30% of the image size
  elif FPSC - 25 <= PSC < FPSC:
    Display 40% of the image size
  elif PSC == FPSC:
    Display 50% of the image size
  elif FPSC + 25 >= PSC > FPSC:
    Display 60% of the image size
  elif FPSC + 50 >= PSC > FPSC + 25:
    Display 70% of the image size
  elif FPSC + 75 >= PSC > FPSC + 50:
    Display 80% of the image size
  elif FPSC + 100 >= PSC > FPSC + 75:
```
Display 90% of the image size elif PSC > FPSC + 100: Display 100% of the image size retun Display

This manner of calibrating the PSCs ensures that the effect of the FPSC is reflected in all the image sizes and adapts the feedback subsystem to peoples' individual neurophysiological response range. Each neurofeedback block uses as its feedback signal a different food image from a pool of 16 pictures selected at random. By default the range of the response is set to a 1% deviation from baseline (upwards or downwards), which implies that a 1% upregulation will result in presentation of the full-size image and 1% down-regulation in presentation of the smallest image. Further up- or down-regulation will then not be reflected in further in- or decreases of the image size. However, the gain of the conversion from the extracted brain signal feature (% BOLD signal change) to image size can be set freely to reflect an individual's response range, and can be adjusted adaptively to reflect dynamic changes in self-regulation ability. The food images were taken from the International Affective Picture System (Lang et al., 2005) and the Internet.

After each neurofeedback run, an associated mirror/control run is presented to the participant. In the mirror run the exact same picture (size) sequence (using the same picture

exemplar) is repeated that was shown in the preceding neurofeedback run. The only difference is that during the mirror run participants are instructed NOT to regulate their brain responses during exposure to the food images but should passively watch the image (size) sequences instead. By this means, the mirror run can serve as a perceptual control: when BOLD responses from the mirror run are (offline) subtracted from BOLD responses in the neurofeedback run, any decrease in brain activation (in the target ROI or elsewhere in the brain) *cannot* be attributed to differences in physical stimulation—which may include diminishing brain activation to motivational cues as a result of habituation over time/successive runs—but will likely reflect the down-regulation effort. Importantly, each mirror run is always presented *after* its corresponding regulation run showing the same picture exemplar and size sequence. Any habituationrelated reduction of brain responses caused by repeating the same stimulation in the mirror run thus *cannot* lead to an erroneous detection of down-regulation success (difference regulation mirror) as it works in the opposite direction. **Figures 2A,B** depicts the temporal structure of the neurofeedback and mirror runs.

#### **IMPLEMENTATION**

**Figures 3**, **4** depict flow-charts of the execution path of the neurofeedback and mirror runs respectively. The flow-charts were implemented by writing the sequence of executed events in the Python programming language and executed using the PsychoPy graphical user interface (Peirce, 2007). During the neurofeedback run, the signal feedback subsystem accesses the average BOLD signal values of the ROI time courses (in the rtp files) computed by TBV and presents the feedback display in **Figure 2A**. The mirror run presents a visually identical feedback display (see **Figure 2B**) which uses the average BOLD signal values of the neurofeedback run stored in a separate directory (termed "mirror files"). The mirror files are processed and displayed in synchronization with the average BOLD signal values of the ROI time courses (rtp files) computed by TBV during the mirror run (scanner files).

# **PILOT STUDY TESTING THE FEASIBILITY OF "MOTIVATIONAL NEUROFEEDBACK"**

Data from 10 female participants (mean age *M* = 21.40 years, Standard Deviation (SD) = 2.27) were acquired to test whether the newly developed motivational neurofeedback paradigm can be successfully used to down-regulate brain activation in response to appetitive food pictures. Participants were asked not to eat for 4 h before the scanning session to increase motivational brain responses related to hunger or food craving. Written informed consent was obtained in accordance with the local ethics committee prior to the start of the study.

The experiment began with the functional localizer run followed by an alternating sequence of four neurofeedback and four mirror runs. During the localizer run the participants passively viewed five blocks of food pictures and five blocks of

pictures showing neutral household objects in alternating order. The picture blocks contained five pictures randomly selected from the relevant category, presented for 2 s each. Blocks were interleaved with a 10 s fixation period (with the exception of an initial fixation period at the start of the experiment which was 12 s in duration), resulting in a total run length of 222 s. The sequence of four neurofeedback and mirror runs were presented using the trial/run structure described above. Due to technical problems, for three participants only three neurofeedback/mirror runs could be acquired. For ROI analyses testing the feasibility of down-regulation of the target area in the neurofeedback runs, we used the pre-processed functional images created by the TBV software, which were co-registered to individual high-resolution T1-weighted anatomical images and spatially normalized to Talairach space. Individual volume time courses were spatially smoothed using a kernel with a full

width at half maximum (FWHM) of 4 mm and temporally filtered with a high pass filter of 2 cycles/time course. For each run and participant separately, mean beta estimates in the target area were then extracted based on individual whole-brain GLMs and three different predictors for BOLD signal changes during (i) mirror, (ii) regulation (neurofeedback); and (iii) rest blocks. **Table 1** shows the associated brain regions, mean Talairach coordinates and size of activation clusters selected as target areas for neurofeedback-guided down-regulation in the pilot sample.

# **RESULTS AND DISCUSSION**

Based on a whole brain GLM analysis for the contrast between food and neutral pictures during the localiser run we visually selected for each participant individually a target area (ROI) encompassing the cluster showing the strongest response in the


**Table 1 | Associated brain region (Left/Right), mean Talairach coordinates and size (1** × **1** × **1 mm<sup>3</sup> voxels) of activation clusters selected as target areas for neurofeedback-guided down-regulation in the pilot sample**.

Target areas were functionally selected using a localizer scan with food and neutral pictures. The table also includes mean beta differences for the regulation/neurofeedback vs. mirror/passive viewing condition across runs. Negative values indicate successful down-regulation of target area activation during neurofeedback runs.

statistical activation maps (see **Table 1**). The feedback signals (and its corresponding picture sizes) in the subsequent neurofeedback runs were computed as the average percent signal change from all significantly activated voxels within a rectangle drawn over the target region across three axial slices. We constrained the selection to regions with a known involvement in emotional-motivational processes and to non-visual brain areas.

During the neurofeedback run participants were asked to reduce the size of the food images displayed on the projector screen by reducing the average fMRI signal strength in the target ROI. During the mirror run the participants were instructed to passively view the food image sequence repeated from the previous neurofeedback run. **Figure 5** shows mean beta estimates for the pilot sample, separated for the four consecutive regulation and mirror runs. Paired *t*-tests showed that the regulation condition led to significantly lower target area activation in run 2, *t*(9) = 2.49, *p* = 0.034, and in run 3, *t*(9) = 2.77, *p* = 0.022. Activation levels were not significantly different between regulation and passive viewing in run 1 and run 4, *t*s < 1.0. However, as indicated in **Figure 5** this was caused by increased activation in the mirror condition in the first and fourth run while target area activation during regulation was low throughout the session, with beta estimates not exceeding 0.1. Across runs, 8 out of 10 participants successfully reduced target area activation during neurofeedback, showing lower mean beta values for the regulation vs. passive viewing condition (see **Table 1**). To summarize, pilot data suggests that our paradigm enables participants to successfully downregulate brain areas involved in processing motivational cues, such as appetitive food pictures. These results may be relevant to the increasing interest in the combination of Pavlovian and instrumental techniques in neurofeedback research (Mendelsohn et al., 2014).

### **MOTIVATIONAL NEUROFEEDBACK—RELEVANCE FOR CLINICAL APPLICATIONS**

Providing motivationally relevant neurofeedback through changes in stimulus size may also provide an avenue to help patients gain control over dysfunctional motivational processes, such as craving elicited by environmental cues in substance dependence. It is well known that visual cue reactivity, and specifically craving responses to drug-related cues (e.g., in the media or during social interactions) are a major determinant of relapse after treatment of addiction (Weiss, 2005). Moreover, maladaptive brain responses to visual drug cues can be identified in early stages of addictive disorders (Ihssen et al., 2011). Identifying a suitable neurofeedback approach that can help to alter these activation patterns directly would thus provide an important clinical tool complementing traditional psychological and pharmacological interventions. A few studies have begun to test the effects of neurofeedback in the context of addiction. For instance, Li et al. (2013) and Hanlon et al. (2013) trained treatment-seeking smokers to regulate brain responses in craving-related brain areas ACC and PFC during exposure to smoking-related pictures. Pictures were presented in blocks of 22 s during which participants were asked to regulate ACC/PFC activity, followed by a thermometer feedback shown for 4 s. However, these studies did not control for habituation effects as described above, presenting regulation runs always after passive viewing control runs. Moreover, feedback was implemented with a delay. The present approach overcomes these limitations. It may also be especially suitable in an applied setting as it has high face validity and presents to the participant visible, quasi-behavioral consequences of his/her mental regulation effort: Visual cues, such as picture showing high-calorie food or alcoholic beverages, can be directly manipulated through the regulation, i.e., "pushed away" through successful down-regulation or "dragged towards oneself " through up-regulation (or non-regulation). Providing such approach and avoidance consequences of neural selfregulation may increase the likelihood that learned strategies are transferred to a natural environment. The importance of such motivational factors for drug addiction is demonstrated by behavioral interventions showing that making avoidance movements (pushing a joystick) in response to alcohol pictures can change the automatic approach bias and improve treatment outcomes of alcohol-dependent patients (Wiers et al., 2011; Eberl et al., 2013).

On the other hand, the present paradigm allows participants to directly manipulate and control the visual cue which can be predicted to increase the participant's sense of agency and perception of self-efficacy (Bandura, 1977)—factors that are central for the maintenance of drug-abstinent behavior (Greenfield et al., 2000). One limitation of our paradigm relates to the effects that visual changes in picture size may themselves have on motivational brain responses. However, changes in picture size have been shown to affect neural responses of emotional brain networks only marginally (De Cesarei and Codispoti, 2006). More importantly, in the present paradigm such effects are also controlled for by the mirror runs, which repeat the size changes of the regulation run. Nonetheless, care should be taken when defining maximal and minimum picture sizes for a specific scanner-projector set-up in order to remain within a visible range.

#### **SUMMARY AND CONCLUSIONS**

The visual feedback subsystem we have developed has been tailored to specifically allow (i) feedback-guided regulation of visual cue reactivity, i.e., to control brain responses to pictorial cues *during* exposure to those cues; and (ii) to increase motivational relevance of visual feedback by using decreases (avoidance) or increases (approach) of stimulus size as an indication of successful or unsuccessful down-regulation. Further, the feedback subsystem is able to adapt to peoples' individual neurophysiological response range. These individual ranges are scaled to fall within a standard range before being mapped to the corresponding image sizes for display. Importantly, the introduction of the mirror run controls for physical/perceptual confounds, allowing separation of BOLD response changes resulting from successful regulation from those related to variations of visual stimulus properties or visual responses, such as habituation. Finally, the present feedback subsystem displays a new food image size every 2 s allowing good performance and providing a minimum delay in line with existing visual feedback subsystems (Weiskopf et al., 2003, 2004; Sitaram et al., 2007b, 2008). To conclude, our approach may facilitate the control of brain activation during neurofeedback training involving simultaneous presentation of visual cues and may thus help the translation of neurofeedback into clinical applications, such as the regulation

of craving responses to substance-related visual cues in addictive disorders.

# **ACKNOWLEDGMENTS**

This study was supported by the BRAINTRAIN grant, funded by the European Union's Seventh Framework Programme for research, technological development and demonstration under grant agreement no. 602186, a seed corn grant of the MRC Centre for Neuropsychiatric Genetics and Genomics, Cardiff University and MRC grant G1100629.

# **REFERENCES**


**Conflict of Interest Statement**: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 31 March 2014; accepted: 20 October 2014; published online: 25 November 2014*.

*Citation: Sokunbi MO, Linden DEJ, Habes I, Johnston S and Ihssen N (2014) Real-time fMRI brain-computer interface: development of a "motivational feedback" subsystem for the regulation of visual cue reactivity. Front. Behav. Neurosci. 8:392. doi: 10.3389/fnbeh.2014.00392*

*This article was submitted to the journal Frontiers in Behavioral Neuroscience*.

*Copyright © 2014 Sokunbi, Linden, Habes, Johnston and Ihssen. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution and reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms*.

# Social reward improves the voluntary control over localized brain activity in fMRI-based neurofeedback training

Krystyna A. Mathiak 1, 2, 3 \* † , Eliza M. Alawi 1, 2 †, Yury Koush1, 2, 4, 5, Miriam Dyck 1, 2 , Julia S. Cordes 1, 2, Tilman J. Gaber 2, 3, Florian D. Zepf 6, 7, Nicola Palomero-Gallagher <sup>8</sup> , Pegah Sarkheil 1, 2, Susanne Bergert 1, 2, Mikhail Zvyagintsev 1, 2 and Klaus Mathiak 1, 2

<sup>1</sup> Department of Psychiatry, Psychotherapy and Psychosomatics, Behavioral Psychobiology, RWTH Aachen University, Aachen, Germany, <sup>2</sup> Translational Brain Medicine, Jülich-Aachen Research Alliance, Jülich, Aachen, Germany, <sup>3</sup> Department of Child and Adolescent Psychiatry, Psychotherapy and Psychosomatics, RWTH Aachen University, Aachen, Germany, <sup>4</sup> Department of Radiology and Medical Informatics, University of Geneva, Geneva, Switzerland, <sup>5</sup> Institute of Bioengineering, Ecole Polytechnique Fédérale de Lausanne, Lausanne, Switzerland, <sup>6</sup> Department of Child and Adolescent Psychiatry, School of Psychiatry and Clinical Neurosciences and School of Paediatrics and Child Health, Faculty of Medicine, Dentistry and Health Sciences, The University of Western Australia (M561), Perth, WA, Australia, <sup>7</sup> Specialised Child and Adolescent Mental Health Services, Department of Health in Western Australia, Perth, WA, Australia, <sup>8</sup> Research Centre Jülich, Institute of Neuroscience and Medicine (INM-1), Jülich, Germany

#### Edited by:

Francisco Javier Zamorano, Universidad del Desarrollo, Chile

#### Reviewed by:

Annette Beatrix Bruehl, University of Zurich, Switzerland Frauke Nees, Central Institute of Mental Health, Germany

#### \*Correspondence:

Krystyna A. Mathiak, Department of Psychiatry, Psychotherapy and Psychosomatics, RWTH Aachen University, Pauwelsstr. 30, 52074 Aachen, Germany kamathiak@ukaachen.de

† These authors have contributed equally to this work.

Received: 14 November 2014 Accepted: 11 May 2015 Published: 03 June 2015

#### Citation:

Mathiak KA, Alawi EM, Koush Y, Dyck M, Cordes JS, Gaber TJ, Zepf FD, Palomero-Gallagher N, Sarkheil P, Bergert S, Zvyagintsev M and Mathiak K (2015) Social reward improves the voluntary control over localized brain activity in fMRI-based neurofeedback training.

> Front. Behav. Neurosci. 9:136. doi: 10.3389/fnbeh.2015.00136

Neurofeedback (NF) based on real-time functional magnetic resonance imaging (rt-fMRI) allows voluntary regulation of the activity in a selected brain region. For the training of this regulation, a well-designed feedback system is required. Social reward may serve as an effective incentive in NF paradigms, but its efficiency has not yet been tested. Therefore, we developed a social reward NF paradigm and assessed it in comparison with a typical visual NF paradigm (moving bar). We trained twenty-four healthy participants, on three consecutive days, to control activation in dorsal anterior cingulate cortex (ACC) with fMRI-based NF. In the social feedback group, an avatar gradually smiled when ACC activity increased, whereas in the standard feedback group, a moving bar indicated the activation level. In order to assess a transfer of the NF training both groups were asked to up-regulate their brain activity without receiving feedback immediately before and after the NF training (pre- and post-test). Finally, the effect of the acquired NF training on ACC function was evaluated in a cognitive interference task (Simon task) during the pre- and post-test. Social reward led to stronger activity in the ACC and reward-related areas during the NF training when compared to standard feedback. After the training, both groups were able to regulate ACC without receiving feedback, with a trend for stronger responses in the social feedback group. Moreover, despite a lack of behavioral differences, significant higher ACC activations emerged in the cognitive interference task, reflecting a stronger generalization of the NF training on cognitive interference processing after social feedback. Social reward can increase self-regulation in fMRI-based NF and strengthen its effects on neural processing in related tasks, such as cognitive interference. A particular advantage of social feedback is that a direct external reward is provided as in natural social interactions, opening perspectives for implicit learning paradigms.

Keywords: neurofeedback, real-time fMRI, social communication, reward, smile, avatar, Simon task, cognitive interference

# Introduction

People constantly control their brain activity by engaging in voluntary actions that are linked to activation of specific brain regions (deCharms et al., 2005; deCharms, 2007). This does not always work well: to excel in difficult skills, to suppress unwanted emotions or to override automatic actions requires a long and difficult learning process or even sometimes fails altogether. Although we do control the brain activity indirectly via our actions, typically we cannot exert direct control over specific brain regions. Brain imaging techniques, such as functional magnetic resonance imaging (fMRI), help to understand the link between the physiological processes taking place within the brain and our subjective awareness (deCharms, 2007, 2008). Neurofeedback (NF) based on real-time fMRI (rt-fMRI) takes us even a step further: Subjects can voluntarily change the activity in a selected brain region and directly see the effect of the evoked brain activation. Although this particular method is still relatively new and subject to certain limitations, its potential implications are vast.

NF based on electroencephalography (EEG) is wellestablished for the treatment of attention-deficit/hyperactivity disorder (Monastra, 2005) and epilepsy (Sterman and Egner, 2006) for over 4 decades now. fMRI-based NF, enabling the regulation of a precisely selected brain region, became available much later with the development of brain-computer interfaces (BCIs) based on rt-fMRI (for a review, see Weiskopf et al., 2004b, 2007). rt-fMRI NF proved to have a good anatomical resolution and to elicit behavioral changes (for a review, see Birbaumer et al., 2009). However, its use as a clinical method is still limited. One of the limitations is that only about two thirds of the people subjected to this particular method succeed in controlling computerized devices with brain signals, while the remaining one third fails to do so (Friedrich et al., 2014). A number of factors determine how well this control can be achieved, including the training protocol, instructions, tasks, mode of feedback as well as psychological traits such as motivation and expected reward, mood, locus of control, and empathy. Indeed, according to Goebel et al. (2004), the social component of the study paradigm, that is the willingness to compete and win against a real opponent, may lead to very fast and effective learning. In accordance with these observations, deCharms (2008) proposed to develop new types of task paradigms for rt-fMRI NF, where participants would be trained without engaging in a deliberate cognitive process.

In a standard fMRI NF paradigm, participants are presented with a visual display of a color bar moving up-and-down or a fluctuating thermometer that reflect brain activity in a region of interest (ROI). Their task is to raise the level on such a bar display by regulating the brain activity in the selected ROI. It is proposed that successful learning follows the principles of operant conditioning, involving a reward when the required threshold is achieved (McCarthy-Jones, 2012).

**Abbreviations:** ACC, anterior cingulate cortex; BCI, brain-computer interface; EEG, electroencephalography; EPI, echo planar imaging; FWE, family-wise error (correction); NF, neurofeedback; ROI, region of interest; rt-fMRI, real-time functional magnetic resonance imaging; SPM, Statistical Parametric Mapping.

In EEG-based NF, the reward is often explicit, involving appointing points or making a game character move on the screen (Egner and Sterman, 2006) or make a Lego robot move forward (Mirkovic et al., 2013 ´ ). In published rt-fMRI NF studies, the reward is often less direct such as the subject's own satisfaction with successful control of the bar display (or receiving social reward from an experimenter at the end of the task). deCharms et al. (2005) used task-related feedback stimulus, namely images of a fire changing its size to reflect a successful regulation of a pain-related area (rACC). Sokunbi et al. (2014) extended this particular approach, introducing feedback-guided self-regulation based on changing size of appetitive food pictures to regulate brain circuits related to hunger and food craving. They argued that the stimuli mimic avoidance behaviors during successful down-regulation and approach behaviors during unsuccessful down-regulation, increasing the face validity of the used training. Two remarkable studies applied explicit rewards in fMRI NF: Bray et al. (2007) offered monetary rewards when subjects successfully modified their motor cortex activity, and Goebel et al. (2004) added a social rivalry aspect in a so-called Brain-Pong game, where subjects played virtual ping-pong against each other, using their brain activity to control a racket ("brain-pong"). Although all the above attempts were successful, none was shown to be superior to standard feedback signals.

In daily life, we control our brain activity to change our facial expression, prosody, body posture, and other behavior based on subtle feedback signals that we receive from our partners in social interactions. Social reward, such as smile, can activate reward-related areas of the brain, similarly to other reward types, e.g., money (Izuma et al., 2008). We demonstrated in a pilot study that social reward can directly reinforce localized brain activity (Mathiak et al., 2010). In contrast to monetary reward (Bray et al., 2007), social reward can even be provided in realtime, i.e., by displaying positive facial expressions. Similar to the Brain-Pong setup (Goebel et al., 2004), where the inclusion of motivating (but not directly rewarding) social competition improved performance, social reward in the present approach may improve NF training by enhancing the motivation. Here, we investigated the impact of the feedback mode—smiling avatar face vs. standard bar display—on regulation performance during fMRI NF of the anterior cingulate cortex (ACC).

The ACC has been the focus of many studies due to its key role in regulating emotions, goal-directed behaviors, attentional processes, response selection, motor functions (Bush et al., 2000; Carter and van Veen, 2007), and above all in conflict monitoring and error perception (Botvinick et al., 2004; Kerns et al., 2004). The ACC can be successfully controlled using fMRI-based NF with standard paradigms (Weiskopf et al., 2003; deCharms et al., 2005; Emmert et al., 2014; Rance et al., 2014). However, in contrast to some visual and motor areas, no evident strategy emerged that yields activity increases without feedback mechanisms, so that the ACC is a suitable ROI to study learning in rt-fMRI NF. The ACC is reliably activated in both the colorword Stroop and the Simon tasks, which are both based on introducing interfering task-irrelevant stimuli (Peterson et al., 2002). In the Simon task, reactions to a target stimulus are slowed when the location of the target and the response side do not correspond, even though this is task-irrelevant. Both, the Stroop and the Simon task involve the ACC. However, in a direct comparison the Simon task led to significantly stronger ACC activations (Liu et al., 2004). Thus, we applied the Simon task to test for altered activation in the ACC region after NF training (generalization).

Twenty-four healthy participants were randomly allocated to one of two groups: a social and a standard feedback group. As for social reward, an avatar started gradually smiling when ACC activity increased. A bar display indicated the activation levels in the standard feedback group. To control for non-specific effects of the NF procedure, the subjects attempted to up-regulate their brain activity without receiving any feedback directly before (pretest) and after the NF training (post-test). Further during the pre- and post-test, a cognitive interference task (Simon task) investigated change of ACC activity in a novel setting, also without feedback. The ability to voluntarily activate the ACC in an identical paradigm with no feedback served as measure for transfer of the NF training; the impact of the NF training on the ACC activity in the novel setting without feedback assessed generalization of the NF training (after Poppen et al., 1988; Simon and Gluck, 2013).

# Hypotheses

We expected successful NF training in both groups. In comparison with the control group, the social feedback group should demonstrate:


# Materials and Methods

#### Participants

Twenty-four right-handed subjects (13 females; age 25.62 ± 4.79) participated in the study. They were allocated based on the order of their inclusion in the study either to the social (12 subjects, 6 females, age 24.75 ± 2.80) or the standard feedback group (12 subjects, 7 females, χ 2 (1) <sup>=</sup> <sup>0</sup>.168, <sup>p</sup> <sup>&</sup>gt; <sup>0</sup>.682; age 26.5 <sup>±</sup> 6.1, t(22) = −0.891, p > 0.383). The alternating strategy did not exclude selection bias but minimized chronological bias (see Tamm and Hilgers, 2014). All participants were naïve to NF and reported the absence of any acute or history of major neurological or psychiatric disorder, any current use of psychoactive drugs as well as any contraindication for MRI. Written informed consent was obtained prior to participation. Afterwards, demographic information about age, gender, and education was collected. In addition, the participants were asked to complete The Positive and Negative Affect Schedule (PANAS; Watson et al., 1998) before and after NF training on each day. The study protocol was approved by the Ethics Committee of the Medical Faculty of the RWTH Aachen University, Germany, and the study was carried out in accordance with the Declaration of Helsinki.

# Experimental Stimuli and Task Training

All subjects underwent standardized instructions for mental strategies to obtain voluntary control of localized brain activation (based on a written instruction set, see Supplementary Material). The instructions suggested to either recall positive emotional autobiographic memories, to imagine performing their hobby (like engaging in sportive or musical exercise), or to concentrate on a specific perception (like the temperature in one of their feet) in order to increase the activity in the ROI. The NF procedure

was explained in detail, including the delay of the NF signal, and they were instructed to try each regulation strategy for at least 10 s. These instructions were delivered by the experimenter in a personal contact before the first measurement, and on the other days, participants received reminder of the task. Additionally, after each session the participants were asked which strategies they used in order to control their brain activity.

# Design Specification

All participants were trained to control their ACC activity by means of rt-fMRI NF on three separate days within 1 week. On each day, they performed three NF training sessions and two test sessions. We investigated neural correlates of three different conditions: (A) NF, i.e., up-regulation of the localized ACC activation with online feedback from the ROI signal (calculated over all nine NF sessions). The feedback signal here was either the bar or the avatar display; (B) transfer, i.e., up-regulation of the ACC ROI without feedback; and (C) generalization to a cognitive interference task, i.e., activity during the Simon task after NF.

#### Neurofeedback

In the social feedback group, 12 participants received social feedback in which a male avatar (created with Poser Pro, Smith Micro Inc.), with either dark or fair hair (alternating and counterbalanced among subjects) provided a rewarding smile when subjects succeeded to increase ACC activity. The avatar became neutral when ACC activity decreased. The facial expression changed gradually within 100 frames. The second avatar was presented motionless and created a baseline (**Figure 1A**; see also Mathiak et al., 2010).

In the standard feedback group (control group), twelve further participants underwent the same ACC NF training. For this particular group, the change of ACC activity was indicated by either increase or decrease of a green moving bar. A blue motionless bar indicated the baseline condition in this group (**Figure 1B**; see also Gröne et al., 2014). Each NF session consisted of eight NF blocks and nine baseline blocks (30 s each; see exemplary session in **Figure 1**). The feedback was updated every repetition time (TR; 1 s). During baseline blocks, participants' instructions were to count backwards from 100.

#### Transfer

In order to test for the transfer of the NF training, subjects were instructed to regulate their localized brain activity without receiving feedback directly before (pre-test) and after (post-test)

baseline blocks (backwards counting) of 30 s duration each. (A) In the social feedback condition, a dynamic avatar (here the one with dark hair) rewarded successful up-regulation with a smile while the other avatar (blond) indicated the baseline. From the completed datasets, 5

participants the blond one provided the feedback. (B) During the standard feedback condition, green bars moved toward the red one to indicate increase of activity while blue bars provided a cue to count backwards (baseline).

the NF sessions. The static stimuli from the NF training—the avatar or the green bar respectively—were presented in four blocks indicating to use a mental strategy to regulate ACC activity. As in the NF sessions, the baseline stimuli indicated to count-backwards (five blocks).

# Generalization

In order to test for behavioral effects of the ACC NF and for a generalization effect of the training, a cognitive visuospatial interference task, an adapted version of a Simon task, was conducted in the pre- and post-test. A fixation cross was presented in the middle of the screen and accompanied by arrows pointing up or down on either the left or the right side of the cross (**Figure 2**). The participants responded with the button press (right or left) to the direction of the arrow (up or down). Thus, when the subject had to press the button on the opposite side of the arrow, a conflict occurred (incongruent trials). The Simon task was presented in eight blocks of 42 s each. The response buttons were counterbalanced between subjects and the events were presented in a pseudo-randomized order. Reaction time and accuracy of each trial were collected as behavioral measures during the Simon task. The stimulation for the transfer and generalization task was programmed with Presentation software (Version 16.3, www.neurobs.com).

# Data Acquisition and Analyses

fMRI scanning was conducted using a three Tesla whole body scanner (Magnetom TIM TRIO, Siemens, Erlangen, Germany). Echo planar imaging (EPI) covered 16 transverse slices parallel to the AC-PC line at a repetition time TR of 1 s (echo time TE = 28 ms; 64 × 64 matrix with 3 × 3 mm<sup>2</sup> resolution; 3 mm slice thickness plus 0.75 mm gap). We obtained 520 volumes for each NF training run (about 8.5 min) and 760 volumes for each preand post-test (12.5 min). A custom made anatomical template of the ACC defined the ROI (Mathiak et al., 2010).

Online spatial preprocessing of the acquired brain volumes was conducted using a custom toolbox based on standard SPM procedures (Koush et al., 2012). In short, motion correction used spline interpolation with co-registration to the preselected template. The NF signal was extracted from each voxel in the ROI during the NF conditions, averaged for each volume and calculated as percentage of signal change relative to the preceding

baseline block. Low frequency drifts were removed with an exponential moving average algorithm to improve the signal-tonoise ratio. A modified Kalman filter reduced outliers and highfrequency fluctuations. For feedback, the signal was rescaled in a fixed ratio such that about 1% signal change represented the full scale from neutral to maximally smiling face or from lowest bar position to the high target. Real-time analysis was performed on a separate PC using a custom Matlab toolbox for online fMRI preprocessing, analysis, and online feedback (for details on the online processing, see Koush et al., 2012).

Offline analysis of the imaging data comprised standard preprocessing and first level analysis in a block design. For the main effect, all runs and days were averaged since no specific time course of learning could be predicted. Group analysis was implemented as second-level two-sample t-test using the rather conservative family-wise error (FWE) correction for whole brain analysis and confirmative ROI analyses. In detail, the mapping analysis consisted of standard preprocessing steps with realignment, normalization, resampling with 2 mm isometric voxels, and smoothing (8-mm full-width at half-maximum Gaussian kernel) with SPM8 (FIL, http://www.fil.ion.ucl.ac.uk/ spm/). The first 10 volumes of each run were excluded from the analyses to account for T1-saturation effects. For the NF runs, the regulation was modeled in a block design applying a generic hemodynamic response function. Transfer and generalization conditions were modeled in a block design as well. T-maps for contrasts of interest in the second-level group analyses were corrected for multiple comparisons across the volume using FWE correction and are shown at corrected threshold (p < 0.05). For data exploration, interaction of transfer and learning in the social reward condition are presented for a voxel-wise uncorrected threshold (p < 0.001). Threshold for cluster extend was always 15 voxels. Anatomical labeling was conducted in accordance with the Anatomy toolbox for SPM8 (Eickhoff et al., 2005).

In addition to the whole brain analyses, we conducted ROI analyses using small volume correction focusing on the ACC and on the reward system, respectively. Thereby we could specifically address the hypotheses 1–4 and ensure that signal changes encompassed the ACC or reward system ROI. The definition of the ACC was based on three-dimensional probability cytoarchitectonical maps, which offer a precise tool for the localization of brain functions as obtained from functional imaging studies (Amunts et al., 2007; Zilles and Amunts, 2010). The mask for the reward system comprised putamen and caudate nucleus as well as globus pallidus and was created using WFU PickAtlas toolbox for SPM8 Maldjian et al., 2003). Activation clusters were displayed at a threshold according to p < 0.05 FWE-corrected for the small volumes with cluster size bigger than 15 voxels.

For data exploration, we extracted average hemodynamic responses from ROIs for ACC and the reward system and as baseline control—from bilateral parieto-occipital clusters (MarsBaR toolbox; Brett et al., 2002). Correlation between ACC regulation and reward responses were calculated. To study learning effects over the runs and sessions, the baseline-corrected ACC ROI signal entered into a repeated-measures ANOVA using linear predictors for run and day and the inter-subject variable group. All calculations were performed using Matlab 2010b (The Math Works, Natick, MA).

# Results

# Behavioral Data

Social and the standard feedback group did not differ with respect to the demographic variables age [t(22) = −0.891, p > 0.383] or education [t(22) = −0.266, p > 0.792]. For the positive affect subscale of the PANAS, repeated-measures ANOVA revealed significant main effects of days [F(2, 42) = 11.829, p < 0.0001; day 1: 27.2 ± 0.9, d2: 24.0 ± 1.3, d3: 23.7 ± 1.3] and session [before vs. after fMRI measurement; F(1, 21) = 5.801, p < 0.025; before: 26.0 ± 1.0, after: 23.9 ± 1.3]. Neither group [F(2, 42) = 1.709, p > 0.521] nor the interactions between group and days [F(2, 42) = 1.709, p > 0.193] and session [F(1, 21) = 0.329, p > 0.572] yielded a significant effect. The negative affect exhibited the same pattern [days: F(2, 42) = 11.829, p < 0.0001, d1: 11.9.3, d2: 11.0 ± 0.2, d3: 10.7 ± 0.2; session: F(1, 21) = 16.774, p < 0.025; before: 11.7 ± 0.3, after: 10.7 ± 0.2; group or interaction with group: all p > 0.09]. In summary, the random allocation yielded comparable groups and general blunting over time but no effect of the feedback strategy on the reported mood emerged.

Reaction times and accuracies of responses collected during the Simon task were assessed with ANOVAs for repeated measures. One participant was excluded from this analysis due to missing data (from the standard feedback group). Since the sphericity assumption was violated for days [Mauchley's test χ 2 (2) <sup>=</sup> <sup>15</sup>.88, <sup>p</sup> <sup>&</sup>lt; <sup>0</sup>.0001] and for the interaction of days with congruency [χ 2 (2) <sup>=</sup> <sup>6</sup>.2, <sup>p</sup> <sup>&</sup>lt; <sup>0</sup>.045], the Greenhouse-Geisser correction was applied. Days [F(1.29, <sup>27</sup>.13) = 15.065, p < 0.0001], session [pre- vs. post-test, F(1, 21) = 11.731, p < 0.003] and the Simon effect [F(1, 21) = 43.301, p < 0.0001] yielded significant effects on the reaction time. Subjects responded faster over the 3 days (day 1: 535.8 ± 12.4; day 2: 501.4 ± 11.2: day 3: 491.8 ± 14.0 ms) faster during post- than pre-tests (pre: 520.1 ± 13.2; post: 499.2 ± 10.7 ms), and faster during congruent than incongruent trials (congruent: 490.7 ± 10.8, incongruent: 528.6 ± 13.1 ms). Accuracy was only affected by congruency [F(1, 21) = 26.318, p < 0.0001; congruent: 97.4 ± 0.8%, incongruent: 94.6 ± 0.8%]. In summary, a clear effect of stimulus congruency on performance in the Simon task was replicated and training speeded the responses, but no effect of the specific NF training on behavior emerged.

#### Neurofeedback

In the feedback runs, a distributed network was more active during NF as compared to the counting backward baseline (**Figure 3A**). In addition to the ACC, this network comprised bilateral lateral occipital complex, striatum, and dorsolateral prefrontal cortex. In contrast, activation decreased in bilateral posterior insula, postcentral gyrus, and the posterior cingulum (**Table 1A**). Masking with the anatomically defined ACC and reward system confirmed the localization of this activation pattern to encompass the ACC (MNI = [−4, 28, 36], tpeak = 10.02, pFWE < 0.0001) and the reward system with peaks in bilateral caudate nucleus (left: [−12, 6, 14], tpeak = 11.94, pFWE < 0.0001; right: [14, 2, 18], tpeak = 13.13, pFWE < 0.0001).

The group comparison revealed a higher effectiveness of the social NF over the standard feedback, as demonstrated by a significantly higher bilateral ACC activity (tpeak = 10.67, pFWE < 0.0001; **Figure 3B**). Furthermore, an extended activation cluster emerged encompassing bilateral inferior frontal gyrus, the left occipital gyrus, and the left middle temporal gyrus (**Table 1B**). Anatomical ACC and reward system masks confirmed the localization of higher activation during social feedback in the ACC ([−10, 34, 10], tpeak = 9.00, pFWE < 0.0001) and the reward system bilaterally with peaks in bilateral putamen (left:

FIGURE 3 | NF training. (A) Both modes of neurofeedback led to increased activity in ACC and in reward-related brain areas. (B) In the social feedback group, activity was higher in bilateral ACC and in the reward system as compared to the standard feedback

group. Moreover, clusters in prefrontal, occipital, and temporal lobe emerged in this group comparison as well (see Table 1 for details). All maps are displayed at a threshold according to p < 0.05, FWE-corrected.

#### TABLE 1 | Activation clusters during NF training.


\*The cluster sizes for the ACC were calculated in a mask based on three-dimensional probabilistic cytoarchitectonic maps.

[−32, −10, 2], tpeak = 9.51, pFWE < 0.0001; right: [36, 0, −4], tpeak = 9.06, pFWE < 0.0001). Thus, hypotheses 1 and 2 were confirmed with higher ACC and reward system activity during social feedback. Notably, the average responses in the ACC ROI correlated with the one from the reward system [r(24) = 0.535, p = 0.0071], suggesting a direct relationship of reward processing and learning success.

Learning of NF related regulatory control may be associated with increase of signal change over time. After baseline correction for the bilateral parieto-occipital junction clusters, average signal change in the ACC ROIs revealed a complex learning pattern influenced by the repetition over three runs on 3 days each (see **Figure 4**). Learning curves in NF may be complex and highly non-linear (Sarkheil et al., 2015), but frequently are approximated by linear curves. Therefore, repeated-measures ANOVA included runs and days as separate linear predictors and revealed a clear days × group interaction [F(1, 23) = 8.239, p < 0.0089] but no main effect or other interaction [all F(1, 23) < 1.8, p > 0.19, except a trend for days, F(1, 23) = 3.022, p = 0.0961]. Further, the probability that individuals achieved control over the signal was estimated on their run-wise success rate and varied across subjects but not between the groups (mean ± SD: 69.4 ± 32.3%). In summary, the differential signal increase observed in the ACC seemed stronger in the social feedback group across runs as well as days, which was statistically confirmed for a stronger linear increase across days only.

# Transfer

Transfer conditions revealed significantly higher ACC activity during the post-test regulation blocks without feedback compared to baseline blocks; in addition to ACC activity,

distributed activation clusters emerged in bilateral inferior frontal gyrus and occipital gyrus, in the right middle occipital and middle temporal gyrus, left posterior cingulate cortex as well as thalamus (**Figure 5A**, **Table 2A**). ROI masks confirmed localization of activity in the ACC ([−10, 32, 24], Tpeak = 5.23, pFWE < 0.0001).

To test the prediction that transfer may differ between the two learning conditions, the interaction of transfer and learning groups was calculated. Indeed, during regulation blocks higher

transfer condition i.e., regulation without feedback (p < 0.05, FWE-corr.). (B) Social feedback led to higher transfer than standard feedback, although this

social learning group yielded higher activity in the left inferior frontal gyrus and the left inferior parietal cortex (see Table 2).

ACC activity was found in the social feedback group as compared to standard feedback (**Figure 5B**, **Table 2B**) but this interaction survived only an uncorrected threshold (p < 0.005) with an cluster-extend threshold of 15 voxels. Only peaks at the left inferior frontal gyrus and inferior parietal cortex survived the FWE-correction (**Table 2B**). The ROI analysis indicated higher regulation increase of the ACC in the social feedback group, but the peak did not survive the FWE-correction (MNI = [6, 10, 34], Tpeak = 2.58, puncorr < 0.005). Lacking a higher activation in the social transfer condition after FWE-correction, we could not confirm Hypothesis 3.

# Generalization

Generalization was tested as the effect of the transfer (regulation without feedback) on a subsequent block with the cognitive interference task, i.e., the group-by-task interaction during the Simon task. We found that ACC activation during cognitive interference processing was reduced after social reward compared to standard feedback. Higher ACC activity emerged in the non-social feedback group compared to social feedback group (Tpeak = 5.34, pFWE < 0.001; **Table 3**; **Figure 6**). ROI analysis confirmed the localization in the ACC (MNI = [−8, 34,−6], Tpeak = 4.63, pFWE < 0.001). Hypothesis 4 stated stronger effects on ACC activity during the generalization task after social NF training and this was corroborated by the data.

# Discussion

The present study investigated the effectiveness of social reward in rt-fMRI NF training of the ACC and compared it to a standardtype feedback in form of a moving bar. As predicted, social reward led to stronger ACC activity during NF training. After the training, both groups were able to regulate ACC activity without receiving feedback, with a trend for better performance in the social feedback group. Furthermore, during a cognitive interference task a significant difference for ACC activation emerged suggesting stronger generalization of the social feedback training on cognitive processing.

We extended previous studies using monetary reward (Bray et al., 2007) and created an innovative NF training based on a real-time social reward. In operant conditioning, a desired response is repeatedly paired with reward, resulting in increasing probability that the response occurs again. A conscious process is not necessary for the learning to take place. Although NF is believed to be based on principles of operant conditioning, no reward is delivered for a correct response in typical fMRI NF paradigms. The learning requires instead the explicit knowledge of the task in order to perform it correctly. Although changing the size of the color bar according to instruction can be satisfying as it signals success (the own satisfaction serves as a reward in this case), in a different context, e.g., during watching a movie with a color bar changing, it would not represent a rewarding value. Bray et al. (2007) made a first step in implementing an implicit feedback in a behavioral shaping paradigm; subject's responses were gradually changed by reinforcing small changes leading to a desired target behavior (Dinsmoor, 2004). The subjects did not need to have explicit knowledge of the task, but learned it gradually via receiving or missing a financial reward, depending on their performance. Although monetary reward constitutes a strong reinforcer, it is difficult to deliver in a real-time feedback in order to gradually shape the behavior.

#### TABLE 2 | Activation clusters during transfer.


uc.: uncorrected p-value

TABLE 3 | Group comparison of generalization (Simon task).


Emotional expressions aim to communicate our experiences and to influence the behavior of others (Horstmann, 2003). Social reward offers therefore a more ecologically valid paradigm to shape the behavior of subjects in real-time as compared to monetary reward. This common social learning mechanism can directly influence the level of localized brain activity using a BCI. Indeed, the social reward led to stronger localized brain activity than the standard feedback. Subjects learned to differentially regulate brain activity depending on the avatar faces. The use of differential stimuli to shape behavior opens new perspectives for developing social feedback paradigms with implicit learning, circumventing explicit cognitive control.

The presence of social reward led to bilateral activation of an anatomically-defined ROI in the corpus striatum (putamen, caudate nucleus, and globus pallidus). These structures belong to a network activated by pleasant and rewarding events (Haber and Knutson, 2010). They are involved in driving incentivebased learning and choosing appropriate responses to stimuli, thereby helping to achieve rewards and avoid punishments, and consequently allow the development of goal-directed behavior (Robbins and Everitt, 1996; Delgado, 2007; Liljeholm and O'Doherty, 2012). Social reward was demonstrated to share comparable neural pathways with monetary reward (Izuma et al., 2008). A number of fMRI and neurophysiology studies confirmed that neural activity in the striatum is modulated by social rewards and by learning in a social context (for a review see Báez-Mendoza and Schultz, 2013; Ruff and Fehr, 2014). Our results are compatible with these studies; moreover we demonstrated that the learning of control over the brain activation improves due to the direct reward.

During the generalization condition, activation in the ACC decreased more in the social feedback group. Although cerebral activation typically increases with higher task load, it is well established that in the course of skill training one can observe the decrease of brain activation (Chein and Schneider, 2005). The effects of training on brain plasticity have been studied in the

sensorimotor system, demonstrating a systematic decrease in the motor and somatosensory cortex (Ikegami and Taga, 2008; Kwon et al., 2013; Walz et al., 2014). In trained musicians, gray matter density decreased with expertise in bilateral perirolandic and striatal areas that are related to sensorimotor function, possibly reflecting high automation of motor skills (James et al., 2013). In a similar vein, in a working memory task, the activation in the right inferior frontal gyrus and the right intraparietal sulcus initially increased with improved performance, but decreased when performance consolidated after the prolonged training (Hempel et al., 2004). Moreover, low-performance led to large and loaddependent activation increases in distributed cortical areas when exposed to excessive task requirements, suggesting a recruitment of additional attentional and strategy-related resources by low- as compared to high-performing participants (Jaeggi et al., 2007). In general, the recruitment of a large-scale neural network decreases in the automatic phase, as stimulus-response associations become better and task performance progresses from a consciously controlled manner in the early learning phase to an unconscious form in the late automatic phase (Toni et al., 1998; Müller et al., 2002; Dobbins et al., 2004). Kozasa et al. (2012) compared the performance of trained meditators with non-meditators in a Word-Color-Stroop task, i.e., a cognitive interference task based on a similar principle as the Simon task. Although there were no group differences for the behavioral interference effect, non-meditators activated attention and motor control higher than meditators. The authors suggested that the meditation training improved efficiency via enhanced sustained attention and impulse control. Similarly, in our study, after up to 2 weeks of NF-training, subjects who received social reward could maintain the similar behavioral results in Simon task while engaging less ACC activity than subjects who received standard feedback. The behavioral effects in our study demonstrated an increase of the performance in the Simon task over the training time, reflecting the accompanying decrease in ACC activation due to learning and the corresponding shift from a large network to more specialized regions. In combination with the lack of effects on the behavioral level, we conclude that the social reward led to a reduced neural recruitment to achieve a similar behavioral performance in the Simon task.

Rapid technological advance in fMRI and BCI extends the range of NF applications leading to its increasing popularity. Within the last 2 decades, a number of brain regions were controlled with rt-fMRI NF, including motor areas (deCharms et al., 2004; Yoo et al., 2008), anterior cingulate cortex (Weiskopf et al., 2003; deCharms et al., 2005), supplementary motor and parahippocampal areas (Weiskopf et al., 2004a), anterior insula (Caria et al., 2007; Berman et al., 2013), right inferior frontal gyrus (Rota et al., 2009), amygdala (Zotev et al., 2011; Brühl et al., 2014; Young et al., 2014), nucleus accumbens (Greer et al., 2014), dopaminergic neurons in the substantia nigra/ventral tegmental area complex (Sulzer et al., 2013) or networks of regions, such as individually localized emotion networks (Johnston et al., 2010), the interhemispheric balance between left and right visual cortices (Robineau et al., 2014), or a distributed ensemble of brain regions related to feelings of tenderness/affection (Schoenberg and David, 2014). The first applications of NF in patient groups suggest its potential in the treatment of several disorders, including chronic pain (deCharms et al., 2005), chronic tinnitus (Haller et al., 2010), Parkinson's disease (Subramanian et al., 2011), depression (Linden et al., 2012; Young et al., 2014), obesity (Frank et al., 2012), nicotine addiction (Canterberry et al., 2013; Li et al., 2013), or schizophrenia (Ruiz et al., 2013).

A well-designed feedback system is crucial in order to achieve a successful training of regional brain activation (Sitaram et al., 2008; Sokunbi et al., 2014) and allow its further development into an effective and accurate clinical intervention. Social feedback, offering direct reward for successful regulation, increased the effectiveness of the NF training. We applied a social smile of a changing intensity, which is a very simplified form of social reward. Indeed, more complex social stimulation (including social gestures, prosody, and complex emotional expression) could serve as an even stronger reinforcer and further improve performance. Sokunbi et al. (2014) propose to choose the visual stimuli that relate to the function of the target brain area. In accordance with this view, social feedback could be particularly well fitted to train impaired social interactions in psychiatric patients in implicit learning tasks.

#### Limitations

Although we studied a relatively large group of participants for such a complex paradigm, the group size is a limitation. Possibly due to the small group size, we failed to demonstrate stronger ACC regulation during the transfer sessions (regulation without feedback) and behavioral effects on the Simon task in the social feedback group. Despite the high variability of learning success, subgroup analyses with the focus on learners and non-learners are not feasible at this stage. It would be of particular importance to determine the variability between subjects in learning and reward sensitivity during NF and determine predictors for this (Scheinost et al., 2014). In particular, we did not consider the individual learning processes over the three session in 3 days each. Moreover, the test for difference in transfer effects between the social and standard NF might not be optimally selected. Although the test was identical with the learning procedure, it could have a different meaning for both groups. While in the standard feedback the bar in itself presented no rewarding value, it was not the case with the smiling faces. During social NF, subjects received social reward. In the transfer task, they were presented with slightly smiling facial expressions that might have had negative emotional value relative to smiling faces they viewed while regulating successfully. Showing subjects a neutral stimulus while trying to regulate their ACC activation without feedback might improve those results.

The reward system is typically associated with the basal ganglia, but many other brain regions respond to reward as well, including the ACC, the orbital prefrontal cortex, the midbrain dopamine neurons, the dorsal prefrontal cortex, amygdala, hippocampus, thalamus, lateral habenular nucleus, and specific brainstem structures such as the pedunculopontine nucleus and the raphe nucleus (Haber and Knutson, 2010). The exact role of ACC in reward processing is however not fully understood. It has been hypothesized to play a role in sustaining effective choice behavior based on the previous experience (Chudasama et al., 2013) and particularly in anticipation of loss by risky decisions (for a review, see Liu et al., 2011). A recent metaanalysis of brain imaging studies on social decision making in the ultimatum game suggested that the ACC controls and monitors conflicts between emotional and cognitive motivation, in line with its postulated role in general conflict monitoring (Gabay et al., 2014). In this respect, replacing the moving bar with an explicit social reward should not lead to additional ACC involvement, among others, because both tasks require similar involvement to obtain the desired outcome and only the rewarding value of this outcome differs. Although introducing social reward in the NF paradigm improved learning, we cannot rule out a direct impact on reward on the ACC activation, e.g., via increasing the net value of the expected reward (Apps and Ramnani, 2014). Future research may focus on other brain regions to examine if the effect of social reward is universal for all brain structures, or if it specifically facilitates learning in reward-sensitive regions.

Finally, the sequential group allocation based on the order of inclusion does not preclude observer biases. This should be addressed by using random allocation. This in turn, however, may introduce time effects depending on the block size for random allocation (see Tamm and Hilgers, 2014). Another problem in this study design, like in many other feedback trials, is the limited possibility to blind the conditions to the participant as well as the experimenter. In particular for therapeutic trails, this remains a challenge to blind control conditions in fMRI neurofeedback.

# Conclusions

We suggest that social reinforcers can lead to improved learning of self-regulation and improve effects of fMRI-based NF on underlying neural processes such as cognitive interference processing. The advantage of social feedback over standard visual feedback or over monetary rewards is the online provision of a direct external reward that we can experience every day in social interactions. Further research is needed to evaluate if social feedback training has the potential to make the learning process more implicit (deCharms et al., 2005; Sulzer et al., 2013).

# Author Contributions

KAM: Development of study paradigm and data analysis. Supervision over data analysis and interpretation. Manuscript revision. EA: Data acquisition, data analysis and interpretation. Manuscript writing. YK: Implementation of toolbox for realtime fMRI and technical support by data acquisition. Revising the manuscript. MD: Contribution to design and data analysis. Revising the manuscript. JC: Contributions to data collection and statistical analysis. Revising the manuscript. TG: Contribution to design and data collection. Revising the manuscript. FZ: Contribution to design. Revising the manuscript. NP: Contribution to development of ACC masks and data analysis. Revising the manuscript. PS: Contribution to data acquisition. Revising the manuscript. SB: Contribution to analysis of behavioral data. Revising the manuscript. MZ: MRI support and technical support by data acquisition. Revising the manuscript. KM: Supervision of and conceptual contributions to study. Data analysis and interpretation. Manuscript revision. All the authors read and approved the final version of the manuscript. All the authors agree to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

# Acknowledgments

Krystyna A. Mathiak was supported by a habilitation grant (Habilitationsstipendium) of the Faculty of Medicine, RWTH Aachen. This research project was supported by the German Research Foundation (MA 2631/6-1, BE 5328/2-1, IRTG 1328), the Federal Ministry of Education and Research (APIC: 01EE1405B), and the START-Program of the Faculty of Medicine, RWTH Aachen. Support was provided by the Brain Imaging Facility of the Interdisciplinary Center for Clinical Research.

# Supplementary Material

The Supplementary Material for this article can be found online at: http://journal.frontiersin.org/article/10.3389/fnbeh. 2015.00136/abstract

# References


Zotev, V., Krueger, F., Phillips, R., Alvarez, R. P., Simmons, W. K., Bellgowan, P., et al. (2011). Self-regulation of amygdala activation using real-time fMRI neurofeedback. PLoS ONE 6:e24522. doi: 10.1371/journal.pone.0024522

**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2015 Mathiak, Alawi, Koush, Dyck, Cordes, Gaber, Zepf, Palomero-Gallagher, Sarkheil, Bergert, Zvyagintsev and Mathiak. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Source-based neurofeedback methods using EEG recordings: training altered brain activity in a functional brain source derived from blind source separation

# **David J. White<sup>1</sup>\*, Marco Congedo<sup>2</sup> and Joseph Ciorciari <sup>3</sup>**

<sup>1</sup> Centre for Human Psychopharmacology, School of Health Sciences, Swinburne University of Technology, Hawthorn, VIC, Australia

<sup>2</sup> Grenoble Images Parole Signal Automatique (Gipsa-lab), CNRS and Grenoble University, Grenoble, France

<sup>3</sup> Brain and Psychological Sciences Research Centre, School of Health Sciences, Swinburne University of Technology, Hawthorn, VIC, Australia

#### **Edited by:**

Niels Birbaumer, University of Tuebingen, Germany

#### **Reviewed by:**

David E. Linden, Cardiff University, UK Martijn Arns, Research Institute

Brainclinics, Netherlands

#### **\*Correspondence:**

David J. White, Centre for Human Psychopharmacology, School of Health Sciences, Swinburne University of Technology, Mail H24, PO Box 218, Hawthorn, VIC 3122, Australia e-mail: dawhite@swin.edu.au

A developing literature explores the use of neurofeedback in the treatment of a range of clinical conditions, particularly ADHD and epilepsy, whilst neurofeedback also provides an experimental tool for studying the functional significance of endogenous brain activity. A critical component of any neurofeedback method is the underlying physiological signal which forms the basis for the feedback. While the past decade has seen the emergence of fMRI-based protocols training spatially confined BOLD activity, traditional neurofeedback has utilized a small number of electrode sites on the scalp. As scalp EEG at a given electrode site reflects a linear mixture of activity from multiple brain sources and artifacts, efforts to successfully acquire some level of control over the signal may be confounded by these extraneous sources. Further, in the event of successful training, these traditional neurofeedback methods are likely influencing multiple brain regions and processes. The present work describes the use of source-based signal processing methods in EEG neurofeedback. The feasibility and potential utility of such methods were explored in an experiment training increased theta oscillatory activity in a source derived from Blind Source Separation (BSS) of EEG data obtained during completion of a complex cognitive task (spatial navigation). Learned increases in theta activity were observed in two of the four participants to complete 20 sessions of neurofeedback targeting this individually defined functional brain source. Source-based EEG neurofeedback methods using BSS may offer important advantages over traditional neurofeedback, by targeting the desired physiological signal in a more functionally and spatially specific manner. Having provided preliminary evidence of the feasibility of these methods, future work may study a range of clinically and experimentally relevant brain processes where individual brain sources may be targeted by source-based EEG neurofeedback.

**Keywords: neurofeedback, EEG, blind source separation, BSS, theta**

# **INTRODUCTION**

The activity-dependent nature of neuroplasticity in the brain has highlighted the potential for manipulations of brain activity in enhancing our understanding of brain processes, but also treating clinical conditions (Cramer et al., 2011). A number of methods exist which apply external stimulation or manipulations to alter brain activity, these include pharmacological interventions, electrical stimulation methods (e.g., deep brain stimulation, transcranial direct current and alternating current stimulation) and Transcranial Magnetic Stimulation (TMS). Unlike these external stimulus driven methods, neurofeedback offers a noninvasive technique capable of manipulating endogenous brain activity. A developing literature supports the use of neurofeedback in the treatment of a range of clinical conditions, particularly ADHD (Arns et al., 2013, 2014) and epilepsy (Sterman and Egner, 2006; Tan et al., 2009). In addition, experimental neurofeedback enables the study of brain activity as the independent variable, providing a powerful method for studying the functional significance of endogenous brain activity (Weiskopf, 2012).

Traditional EEG neurofeedback methods typically utilize a small number of active electrodes on the scalp. Scalp EEG at a given electrode site reflects a linear mixture of activity of multiple brain sources and artifacts, with skull and other tissue having a spatial smearing effect (Congedo et al., 2008). With this in mind, sources optimally aligned and in closer proximity to the scalp electrode represent a greater proportion of the observed activity, but far from the entirety of the observed signal. Thus, traditional neurofeedback methods training single electrode sites are likely influencing multiple brain regions and processes. It is therefore not surprising that training methods using a single scalp site influence large scale EEG dynamics beyond the training frequency and site (for example, Egner et al., 2004). Additionally, as the observed signal reflects multiple brain processes, it has also been suggested that this may impede the ability to acquire control over the feedback signal. This point was highlighted by Philippens and Vanwersch (2010), who demonstrated learned sensory-motor rhythm (SMR) enhancement in four sessions of neurofeedback in non-human primates using intracranial recordings. These authors stressed the ability to acquire control in such a short training period may have partially been a result of the increased spatial resolution, and reduced influence of EMG artifact. Given these limitations of traditional EEG neurofeedback methods, a number of more spatially and functionally specific neurofeedback techniques have been explored. The major development in this area is fMRI-based neurofeedback (Yoo and Jolesz, 2002; Weiskopf et al., 2003, 2004), but also includes spatially specific MEG-based neurofeedback (Florin et al., 2014).

In light of these emerging neurofeedback methods, efforts to develop methods which maximize the functional and spatial specificity of EEG-based neurofeedback techniques remain pertinent given the comparative availability and affordability of such technology, and the capacity to directly target endogenous electrophysiological activity (cf. fMRI methods based on the BOLD response). While source-based EEG neurofeedback using source localization methods has been demonstrated (Congedo et al., 2004), offering potential for an improved spatial precision of a training region, these methods remain limited by the susceptibility of source localization methods to artifacts, the inability to isolate neighboring but functionally separate sources, and the spatial precision offered. Perhaps for these reasons, the capacity for learned regulation using these methods has been inconsistently shown (Maurizio et al., 2014). Blind Source Separation (BSS) is a group of processing techniques which seek to identify source activity from a mixed signal. These methods have been employed in a variety of fields including speech processing (Jang et al., 2002), face recognition (Yuen and Lai, 2002), wireless communication (Van Der Veen et al., 1997), radar applications (Fiori, 2003), and with a range of biomedical signals (James and Hesse, 2005). The blind nature of these methods has facilitated such widespread applicability, where no knowledge of the source activity or mixing process is required. Given the properties of scalp EEG, viewed as an instantaneous linear mixture of multiple brain sources and artifacts as a result of volume conduction, BSS was identified as a method suited to EEG signal processing (Makeig et al., 1996), and has subsequently seen widespread use in EEG research both in the identification and removal of artifacts (Vigário, 1997; Delorme et al., 2007; Romero et al., 2008), and the exploration of functionally and spatially distinct brain sources (Makeig et al., 2004; Onton et al., 2005, 2006; Congedo et al., 2008; Kopˇrivová et al., 2011). It has recently been proposed that BSS methods may have important advantages in multi-channel neurofeedback beyond those methods based on source localization. Specifically, BSS-based neurofeedback may address the limitations of previous source-based neurofeedback methods by offering enhanced spatial and functional specificity of the training substrate, while being less susceptible to artifacts and noise, and

being computationally inexpensive (Congedo and Joffe, 2007; Grandchamp and Delorme, 2009).

Neurofeedback based on sources derived from signal processing methods such as BSS may be ideally suited to isolating a spatially and functionally distinct source, which may be less susceptible to common artifacts, representing significant advantages over traditional methods. Further, given the prominence of BSS-based signal processing methods in the field of cognitive neuroscience, particularly with EEG, demonstrating the "trainability" of functional sources derived from these methods with neurofeedback may open future investigations to study the functional significance of identified sources via trained perturbation of this activity. To this end, the present investigation explores the capacity to learn enhanced activity on a BSS-derived source derived from functional brain activity during completion of a complex cognitive task.

# **MATERIALS AND METHODS**

### **PARTICIPANTS AND EXPERIMENTAL DESIGN**

Four healthy right-handed adult volunteers aged 24–38 years old participated in the study (1 female). Written informed consent was obtained from all participants, with all procedures carried out in accordance with the Swinburne University Human Research Ethics Committee. Participants underwent 20 sessions of neurofeedback over the course of 7 weeks. In addition, three assessment sessions were completed across the course of the neurofeedback period, one at baseline, one after 10 sessions, and a final assessment after the 20 sessions. Beyond the data from the baseline assessment session used to isolate individual sources for neurofeedback, these assessment sessions will not be further discussed in the present report.

#### **BSS-BASED NEUROFEEDBACK**

The linear BSS problem can be defined as:

$$\mathbf{x}(t) = A\mathbf{s}(t)$$

where *x*(*t*) is the observed data and *s*(*t*) the underlying source signals, *A* is a time-invariant mixing matrix. Following matrix algebra, estimated source activity is thus given by:

$$
\hat{s}(t) = B\mathbf{x}(t)
$$

where *B*, known as the separating or demixing matrix, is the pseudo-inverse of *A*. In this way, reconstructed source activity is given by multiplying the separating matrix by the observed data. This BSS model assumes that observed signals are an instantaneous linear mixture of underlying sources (Cardoso, 1998). These methods typically seek the separating and mixing matrices through cancellation of second order or higher order statistics, seeking maximally independent sources.

When applied to EEG data, *x*(*t*) above is an *n* (electrodes) by *t* (time points) matrix of observed scalp EEG, the mixing matrix (*A*) describes the relative weights with which each source projects to the scalp, and the separating matrix (*B*) obtains the estimated brain source activity, in the form of arbitrarily scaled reconstructed source time-series, when multiplied by the observed EEG. It follows that the estimated activity, or time-series, of a single source of interest (ˆ*si*(*t*)) is obtained by:

$$
\hat{s}\_i(t) = B\_i \mathbf{x}(t)
$$

that is, by multiplying the observed scalp data by the vector of weights (*Bi*) from the separating matrix which corresponds to the source of interest. The separating matrix can thus be conceived as a spatial filter, used to estimate source activity. In the context of real-time BSS-neurofeedback, online multiplication of scalp EEG by the vector of the separating matrix corresponding to the target training source will obtain the source time-series. The major issue with such an approach is identifying and obtaining a stable estimate of the spatial filter for the training source. The method adopted in order to achieve this in the present experiment was to base the training on a robust task-related source identified in a group BSS analysis, from which the most closely related individual source was sought.

# **Determining individual neurofeedback sources**

The functional brain source selected for neurofeedback training was based on a previously reported BSS-derived source including medial-temporal lobe (MTL) and parietal lobe regions in which spatial memory performance was associated with source theta oscillatory activity (White et al., 2012). As part of this study, EEG data during completion of a spatial navigation task was analyzed using a BSS method known as Approximate Joint Diagonalization of Cospectral matrices (AJDC; Congedo et al., 2008). Using this method, a source was identified which demonstrated significantly increased theta oscillatory activity during navigation. Within a sample of 25 healthy adults, greater theta power within this source, localized to MTL and parietal regions using sLORETA (Pascual-Marqui, 2002), was associated with better task performance.

As part of the neurofeedback protocol, each participant required individually determined weights corresponding to the BSS component showing the strongest correlation with the group MTL—parietal theta source identified in White et al. (2012) during completion of this same task. Using identical EEG acquisition and pre-processing routines as that used in White et al. (2012), individual participants' EEG data during spatial navigation was decomposed using the identical BSS method (AJDC using the same parameters previously reported, using ICoN software, Version 3.1<sup>1</sup> ). Source time-series derived from the individual BSS decomposition were then correlated with the group MTL/parietal theta source time-series described in White et al. (2012), with the individual source showing the strongest correlation selected as the feedback source (for all four participants, *r* ≥ ± 0.550). The weights corresponding to this component in the separating matrix were extracted for use as a spatial filter for neurofeedback, using a subset of electrodes which did not compromise the source signal (39–42 electrodes were retained for neurofeedback sessions, from the original 62). Peak theta for the feedback training band was determined as peak power within

# **Neurofeedback protocol**

All participants underwent 20 neurofeedback sessions across 7 weeks. Each neurofeedback session involved a resting eyes open baseline, then five blocks of training each lasting 4 min. As this study represented a preliminary investigation exploring the feasibility of BSS-based training, no control neurofeedback group was included. Instead, a series of trials were conducted at a follow-up session upon completion of the 20 sessions in which participants were asked to increase or decrease the feedback signal. As the focus of this experiment was the feasibility of learned regulation of BSS-derived source activity, these trials were included to probe for evidence of learned volitional regulation of the target signal. SynAmps<sup>2</sup> amplifiers and Acquire 4.3 software were used to acquire the EEG data as part of the neurofeedback sessions (Neuroscan Inc., Abbotsford, VIC, Australia). Data acquisition for neurofeedback sessions employed a band-pass filter from 1–50 Hz, with a linked mastoid reference. This was done to ensure consistency with the reference used in the off-line analysis of the previous experiment. A second computer running the Open-ViBE software platform (Renard et al., 2010) provided the on-line processing and feedback required by the neurofeedback paradigm. The set-up made use of the built-in client/server operations available in Scan Acquire 4.3 software; whereby the acquisition system acted as the server which sent acquired data on to the client system (Open-ViBE) via a Local Area Network. An acquisition driver written in C++ facilitated this process within the Open-ViBE platform.

A "Scenario" was developed for each participant within the Open-ViBE software which applied a processing chain to generate the feedback signal, before providing visual feedback to the participant with minimal delay. The Scenario for each participant applied the individually defined spatial filter to the incoming EEG data, generated a ratio of peak theta (*p*θ = peak ± 0.5 Hz) to total theta (*tot*θ = 4–8 Hz) for the source time-series, then provided visual feedback. In order to obtain on-line band-power estimates for *p*θ and *tot*θ the time-series was first band-pass filtered in the designated frequency range (Butterworth filter, 0.5 dB band-pass ripple), then segmented into 1 s epochs with a 250 ms moving window, the data were then squared and an average calculated for each 1 s epoch. Finally, each estimate was log transformed (ln(*x +* 1)) to minimize deviations from normality (Kiebel et al., 2005). The feedback signal was simply the ratio of these log transformed band-power estimates for peak and total theta (*p*θ*/tot*θ). Electro-oculogram (EOG) artifact remains an important consideration when dealing with thetaband activity. This motivated the use of a ratio, as opposed to absolute power. Eye blink and movement artifact is generally maximal in the delta range, decreasing in a steep and monotonous manner with increasing frequency (Gasser et al., 1985; Hagemann and Naumann, 2001). We reasoned that in using the ratio of peak theta activity relative to total theta, the potential confounding influence EOG artifact would be minimized. For example, it is highly unlikely that EOG artifact would manifest itself as

the 4–8 Hz band via Fast-Fourier Transform of individual source activity during completion of the navigation task.

<sup>1</sup>http://sites.google.com/site/marcocongedo/software/icon

a frequency-specific power increase coinciding with peak theta activity, and much more likely that the presence of EOG artifact would result in broadband theta power increases, largest at the low end of the bandwidth, resulting in little change or a drop in the *p*θ*/tot*θ ratio.

The feedback received by the participant contained both continuous and discrete elements (see **Figure 1**). A scrolling line graph showing the exact ratio level formed the continuous feedback, whilst a reward box flashed blue and registered a point each time the ratio exceeded a predefined threshold. The continuous feedback has the advantage of being easy to interpret for the participant (Weiskopf et al., 2004), whilst the threshold score and blue flash provided a discrete reward, which utilizes the common conception of neurofeedback learning by means of operant conditioning. The importance of discrete feedback in neurofeedback protocol design has recently been emphasized (Sherlin et al., 2011). This threshold was determined by calculating a percentile during the baseline recording of the first neurofeedback session, where 6–12 discrete rewards would be received per minute at baseline levels.

# **Assessment of neurofeedback learning**

The capacity to regulate the feedback signal, in the form of neurofeedback learning, forms the primary outcome for this feasibility study. A number of methods have been adopted for operationalizing and quantifying relative success at neurofeedback learning, yet there is little agreement on the most appropriate method (Dempster and Vernon, 2009). In order to demonstrate learned control over the feedback signal, evidence that changes occur beyond baseline levels appears a minimum requirement. In order to first assess this, non-learners were identified with an initial paired samples *t*-test, contrasting mean source *p*θ*/tot*θ ratio scores at all 20 feedback sessions for each participant with the corresponding baseline. Only those to demonstrate significantly elevated source *p*θ*/tot*θ ratio during feedback when contrasted with corresponding baseline were analyzed for evidence of neurofeedback learning. Learning within and across sessions involve desired changes emerging over the course of training, and thus can be plotted as a learning curve. In each of these cases, the presence of learning is demonstrated by the desired increase or decrease in the signal of interest across sessions or over blocks of time within sessions. Within and across sessions learning was assessed by ordinary least squares regression, with the time within training as the predictor (session number for across sessions, and minute from beginning of feedback for within sessions; both 1–20). For both learning analyses, data was normalized with respect to the mean and standard deviation of source *p*θ*/tot*θ ratio during a baseline period. For within sessions learning, mean data was extracted for each minute of neurofeedback at each session, and normalized to the corresponding baseline data for that session. Across sessions learning analysis used the mean source *p*θ*/tot*θ ratio for each session normalized to the baseline period at the first neurofeedback session.

Clearest demonstrations of volitional self-regulation use a series of trials in which the participant is instructed to produce the desired change in signal, contrasted with trials where the opposite change or no change are desired. Neurofeedback for Slow Cortical Potentials (SCP; for a review see Birbaumer, 1999) lends itself to this type of analysis, and self-regulation of positive and negative shifts have been demonstrated in this way (e.g., Schneider et al., 1992). Volitional self-regulation has also been demonstrated with the use of specifically conceived experimental design following LORETA neurofeedback training for enhanced low beta activity in the anterior cingulate gyrus (Congedo et al., 2004). As part of the present exploration, participants completed a follow-up session upon completion of the 20 neurofeedback sessions in which they were asked to increase or decrease the feedback signal in eight randomized blocks of 3-min each (total of four "up" and four "down" trials). A randomization *t*-test (Edgington, 1987) comparing the mean *p*θ*/tot*θ ratio feedback signal obtained in each up trial vs. the mean of each down trial exploits the design of the trials, providing an appropriate assessment of volitional self-regulation of the feedback signal post-training.

A designated exploration of the relationship between ocular artifact and the feedback signal was undertaken offline, using data from the first neurofeedback session for each participant. Power in the delta range (1–3 Hz), averaged across frontal electrode sites, formed a surrogate measure of EOG artifact. For each participant, power estimates were calculated in the same way as those used to derive the source peak theta to total theta ratio, with an additional parallel processing stream calculating frontal delta power. The relationship between the two signals was then assessed by correlating the power estimates for the two signals for the first session in each neurofeedback participant.

## **Assessing the impact of neurofeedback training on navigation performance**

Owing to the exploratory nature of the current experiment, the sample size limited the scope for a full exploration of the impact of the neurofeedback intervention, instead focussing on the feasibility of such a protocol. While this prevented the use of traditional statistical analysis of group differences in outcome measures across the intervention period, we briefly describe the trends in behavioral performance on the navigation task from which the neurofeedback source was derived across the neurofeedback training period contrasted with an age and gender matched no-treatment control group. Three assessment points were completed across 7 weeks for both neurofeedback and control participants (pre-treatment baseline, week 4, week 7 (post-treatment)). The difference in these trends, in the form of the gradient of the slope estimated by Ordinary Least Squares from all navigation performance observations for each group, was then contrasted between neurofeedback and control group using a small sample *t*-test for parallelism (Kleinbaum and Kupper, 1978).

# **RESULTS**

### **EXCEEDING RESTING LEVELS**

Paired samples *t*-tests revealed two of the four participants demonstrated significantly elevated source *p*θ*/tot*θ during neurofeedback sessions, when contrasted with baseline at each session (see **Figure 2** below). Analysis of learning trends was pursued for these two participants only. Surprisingly, one participant showed a significant reduction in the feedback ratio during feedback sessions compared to baseline, while one showed slight nonsignificant increases from baseline to feedback.

#### **NEUROFEEDBACK LEARNING WITHIN AND ACROSS SESSIONS**

Both participants to show elevated source *p*θ*/tot*θ during neurofeedback sessions demonstrated evidence of neurofeedback learning. **Figure 3** summarizes the results of linear regression analyses probing neurofeedback learning within and across sessions for these two participants. This learning emerged across sessions for Participant B, but within sessions for Participant C. Participant B appeared to demonstrate a sharp within sessions learning trend in the first half of each session, before significantly dropping away.

#### **VOLITIONAL CONTROL OF SOURCE p**θ**/tot** θ **RATIO**

Results of the follow-up session exploring volitional control of the feedback signal corroborated findings of the neurofeedback learning analyses. **Figure 4** shows Participants B and C again demonstrated significantly greater *p*θ*/tot*θ source activity during

the "up" trials than the "down" trials (Participant B: *t* = 4.19, *p* = 0.0143; Participant C: *t* = 2.75, *p* = 0.0143). In addition, the findings from these trials suggested Participant D obtained some level of volitional control over the signal that did not translate into neurofeedback learning during the training period (Participant D: *t* = 2.15, *p* = 0.0429).

#### **OCULAR ARTIFACT**

The selection of a ratio feedback signal (*p*θ*/tot*θ) was motivated by a desire to minimize the influence of artifacts in neurofeedback sessions. However, the potential influence of ocular artifact was explored offline, correlating delta power estimates at frontal electrode sites with the feedback signal during neurofeedback. Results of this analysis suggested minimal relationship between the source feedback ratio and frontal delta power for each participant (Participant A: *r* = 0.004; Participant B: *r* = −0.002; Participant C: *r* = 0.115; Participant D: *r* = 0.043). As the maximum value, observed in Participant C corresponds to approximately 1% of variance explained, it appears unlikely that any participant experienced the capacity to achieve the desired signal increases through increasing the presence of ocular artifact.

#### **NAVIGATION PERFORMANCE**

Trends in behavioral performance on the navigation task assessed across the training period showed evidence of improvement in both neurofeedback and control groups. The two groups showed large baseline differences in performance, and evidence of practice effects across repeated assessments, but the trend across the training period did not significantly differ between

the groups (*t*(20) = −0.27, *p* > 0.05). These trends, in the form of the gradient of the slope estimated by Ordinary Least Squares (β), are plotted for performance on the navigation task in **Figure 5**.

**and (B) and Participant C, see panels (C) and (D)**. Participant B did not

# **DISCUSSION**

The current study described the implementation of a neurofeedback paradigm using spatial filtering of scalp EEG to obtain ongoing activity of a BSS component derived from functional brain activity. The study aimed to explore the feasibility of training enhanced theta using this BSS-based neurofeedback. Results showed evidence of learned augmentation of source peak theta activity in 50% of the neurofeedback sample, providing preliminary evidence in support of the feasibility of BSS-based neurofeedback. Beyond this evidence of neurofeedback learning trends, the study also demonstrated volitional control of the feedback signal upon completion of the neurofeedback training period in three of the neurofeedback participants. No differences in behavioral performance were observed for the neurofeedback group on the navigation task from which the training source was derived, when compared to an age and gender matched no-treatment control group. Thus, the findings of this study suggest learned regulation of oscillatory activity derived from a BSS component represents a plausible line of inquiry for future research.

within sessions learning **(C)**, but no linear increases across sessions.

#### **NON-LEARNERS AND STUDY LIMITATIONS**

In the present study, two participants failed to show evidence of learned control over peak theta activity beyond baseline levels. An important consideration in analyzing neurofeedback learning is that not all those exposed to training will gain significant control over the feedback signal. Previous explorations of neurofeedback learners and non-learners suggest that as much as half of those participating in neurofeedback training may not demonstrate significant learning (Weber et al., 2011). In line with this, 50% of the present sample belongs to this non-learner group, while 25%

**neurofeedback training**. For each participant, the mean source pθ/totθ ratio for each trial is shown. A significantly elevated ratio was observed in up trials, contrasted with down trials using a randomization t-test, for three of the four participants, where \* = p < 0.05.

showed no evidence of volitional control of the feedback signal upon completion of the training. This aspect of neurofeedback training is relatively unexplored, and the characteristics and predictors of learners and non-learners is an area requiring further empirical exploration.

One possible explanation for the difficulty in training source peak theta activity in these non-learners is that the source contained a spectral profile which included a low alpha component. The low alpha band shows basic attentional correlates and desynchronizes in response to task demands (for a review, see Klimesch, 1999). Individually determined peak theta for the two non-learners was slightly higher than in the other two participants. As the training frequency for these participants was adjacent to the low alpha band, spectral power associated with alpha activity may have leaked into peak theta estimates. Indeed, the functional source on which the feedback was based showed a clear alpha peak in the original group data around 10– 11 Hz during a resting baseline (see White et al., 2012). Thus, a reduction in low alpha activity during training could have confounded efforts to increase peak theta activity for these nonlearners during neurofeedback trials, as the increased attention associated with the training periods would be reasonably expected to reduce low alpha activity. Whilst the BSS-derived training source is argued to be functionally and spatially specific, this does not preclude spectral activity across multiple frequencies. Thus, the lack of learning in these participants may have been a result of contamination of the desired peak theta signal from the adjacent low alpha band. This may be particularly relevant for Participant A, who showed significant reductions in the feedback signal during feedback when contrasted with baseline. The findings of the present study further emphasize the need to account for fluctuations in a number of frequency bands beyond the training band in neurofeedback research.

As this study represents a preliminary investigation into BSSbased neurofeedback, interpretation of findings remain limited by the design implemented and small sample size. Neurofeedback research is increasingly utilizing experimental designs which incorporate control conditions to allow for stronger evidence of efficacy and specificity, but also the feasibility of learning. In examining efficacy and specificity of clinical and experimental neurofeedback protocols, the use of non-contingent feedback, variable feedback contingencies (e.g., Hoedlmoser et al., 2008), or alternate target bands for neurofeedback control groups have been used to minimize concerns to do with comparable therapist contact, placebo effects, and other non-specific effects of training. Having provided this preliminary evidence of the feasibility of training a BSS-based source, future work may further explore the validity and potential applications of neurofeedback training using controlled designs which can target sources identified from functional or resting state brain processes.

#### **APPLICATIONS OF BSS-BASED NEUROFEEDBACK**

Developing neurofeedback methods are increasingly refining the spatial and functional specificity with which the training neural substrate can be targeted. Using a BSS-derived source as the feedback source offers advantages in this respect, and the findings of the present study support the feasibility of such methods. Thus, future applications of BSS-based neurofeedback are only limited by the extent to which a stable estimate of the target source can be obtained. As BSS-derived sources can be identified from functional or resting activity, future research can extend the present work by applying neurofeedback protocols based on BSS components based on manipulating functional activity such as that described herein, or training problematic BSS components identified in clinical applications. The potential for clinical applications of BSS-based neurofeedback has recently been explored by Kop ˇrivová et al. (2013), who tested neurofeedback training of a medial frontal EEG source identified as showing abnormally elevated low-frequency activity in Obsessive-Compulsive Disorder patients compared to healthy controls. This BSS-based neurofeedback training was associated with greater clinical improvement than a sham feedback control group and non-significant trends towards a shift in the trained frequency band, however, clinical improvement was not associated with EEG changes. In demonstrating the capacity for regulation of a task-derived BSS source, this study supports the use of BSSderived sources in neurofeedback applications, as task-related decompositions such as that used to derive the feedback source herein can be considered stable enough to target outside the context of the specific task from which they were based. As such, experimental BSS-based neurofeedback may train functionally and spatially isolated brain sources, facilitating the study of these sources as the independent variable, in turn providing a powerful method for studying the functional significance of functionally and spatially isolated endogenous brain processes. Using BSS-based neurofeedback may enhance the success of neurofeedback protocols, reducing the influence of artifacts, and providing optimal conditions for training of the target activity.

#### **CONCLUSIONS**

The research described herein builds upon the increasing use of BSS methods in the study of brain function. Adopting BSS methods across a range of methods for recording brain activity, including fMRI and EEG, has offered novel insights into brain function (eg. Greicius et al., 2004; Onton et al., 2005). These findings provide preliminary evidence of the feasibility of sourcebased neurofeedback training derived from BSS, future work may explore further validation and potential applications of BSSbased neurofeedback training, targeting sources identified from functional or resting state brain processes.

#### **ACKNOWLEDGMENTS**

This work was partially supported by a research grant from the Barbara Dicker Brain Sciences Foundation.

#### **REFERENCES**


**Conflict of Interest Statement**: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 17 April 2014; accepted: 09 October 2014; published online: 22 October 2014*. *Citation: White DJ, Congedo M and Ciorciari J (2014) Source-based neurofeedback methods using EEG recordings: training altered brain activity in a functional brain source derived from blind source separation. Front. Behav. Neurosci. 8:373. doi: 10.3389/fnbeh.2014.00373*

*This article was submitted to the journal Frontiers in Behavioral Neuroscience*.

*Copyright © 2014 White, Congedo and Ciorciari. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution and reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms*.

# Self-regulation of frontal-midline theta facilitates memory updating and mental set shifting

#### **Stefanie Enriquez-Geppert 1,2\*, René J. Huster 1,3 , Christian Figge<sup>2</sup> and Christoph S. Herrmann1,3,4**

<sup>1</sup> Experimental Psychology Laboratory, Department of Psychology, European Medical School, Carl von Ossietzky University of Oldenburg, Oldenburg, Germany

#### **Edited by:**

Sergio Ruiz, Pontificia Universidad Catolica de Chile, Chile

#### **Reviewed by:**

Andrew A. Fingelkurts, BM-Science-Brain and Mind Technologies Research Centre, Finland Silvia Erika Kober, University of Graz, Austria

#### **\*Correspondence:**

Stefanie Enriquez-Geppert, Experimental Psychology Laboratory, Department of Psychology, European Medical School, Carl von Ossietzky University of Oldenburg, A7, Ammerländer Heerstr. 114–118, 26129 Oldenburg, Germany e-mail: s.geppert@uni-oldenburg.de Frontal-midline (fm) theta oscillations as measured via the electroencephalogram (EEG) have been suggested as neural "working language" of executive functioning. Their power has been shown to increase when cognitive processing or task performance is enhanced. Thus, the question arises whether learning to increase fm-theta amplitudes would functionally impact the behavioral performance in tasks probing executive functions (EFs). Here, the effects of neurofeedback (NF), a learning method to self-up-regulate fmtheta over fm electrodes, on the four most representative EFs, memory updating, set shifting, conflict monitoring, and motor inhibition are presented. Before beginning and after completing an individualized, eight-session gap-spaced NF intervention, the three-back, letter/number task-switching, Stroop, and stop-signal tasks were tested while measuring the EEG. Self-determined up-regulation of fm-theta and its putative role for executive functioning were compared to an active control group, the so-called pseudo-neurofeedback group. Task-related fm-theta activity after training differed significantly between groups. More importantly, though, after NF significantly enhanced behavioral performance was observed. The training group showed higher accuracy scores in the three-back task and reduced mixing and shifting costs in letter/number task-switching. However, this specific protocol type did not affect performance in tasks probing conflict monitoring and motor inhibition. Thus, our results suggest a modulation of proactive but not reactive mechanisms of cognitive control. Furthermore, task-related EEG changes show a distinct pattern for fmtheta after training between the NF and the pseudo-neurofeedback group, which indicates that NF training indeed tackles EFs-networks. In sum, the modulation of fm-theta via NF may serve as potent treatment approach for executive dysfunctions.

**Keywords: frontal-midline theta, neurofeedback, cognitive enhancement, executive functions, EEG, proactive and reactive control**

# **INTRODUCTION**

Time-frequency analyses of electroencephalographic (EEG) recordings reveal synchronous processes of neural networks known as neuronal oscillations. Nowadays, these neural oscillations are considered to provide a linkage of neural activity with behavior and thought. As such, they are supposed to coordinate neuronal spiking between and within brain circuits (e.g., Buzsáki, 2006; Buzsáki et al., 2013), whereby different neural oscillations may appear at the same time and interact with each other in a hierarchical way in order to implement perception and cognition (Herrmann et al., 2004; Canolty et al., 2006; Basar and Güntekin, 2008; Fingelkurts and Fingelkurts, 2014).

Regarding higher cognitive functions, frontal-midline (fm) theta oscillations are of particular interest. Fm-theta oscillations are recorded over fronto-medial brain regions at frequencies between 4–8 Hz and are suggested to be generated in the midcingulate cortex (MCC; Mitchell et al., 2008; Cavanagh and Frank, 2014), a highly interconnected brain structure (Beckmann et al., 2009; Vogt, 2009) that is part of the superordinate cognitive control network (Niendam et al., 2012). The MCC is known to be crucially involved in executive functioning (Cavanagh et al., 2012), which enables goal directed behavior (e.g., Lezak, 2012). Increases of fm-theta power have been associated with enhanced coupling between neuronal spikes and the phase of the population theta cycle, and thus are suggested to organize neural processes during decision points where executive functioning is needed and information is integrated to inform action selection (Cavanagh and Frank, 2014). Enhanced cognitive processing is accompanied with increases of fm-theta (Mitchell et al., 2008), specifically in tasks involving working memory (WM; Mitchell et al., 2008) and executive functions (EFs; Nigbur et al., 2011). In addition, fm-theta activity has been related to efficient WM maintenance (Tóth et al., 2014), and increases of fm-theta activity during task processing have been shown to predict successful behavioral performance

<sup>2</sup> Karl-Jaspers Clinic, European Medical School Oldenburg-Groningen, Oldenburg, Germany

<sup>3</sup> Research Center Neurosensory Science, Carl von Ossietzky University, Oldenburg, Germany

<sup>4</sup> Center for Excellence 'Hearing4all', Carl von Ossietzky University of Oldenburg, Oldenburg, Germany

(Sederberg et al., 2003) and RTs in a Simon task involving conflict monitoring (Cohen and Donner, 2013). Correspondingly, the absence of fm-theta up-regulation, when executive functioning is required, has been reported to be associated with reduced performance (e.g., Donkers et al., 2011). In short, fm-theta has been proposed as a universal mechanism for EFs with the MCC acting as hub for the integration of relevant information (Cavanagh et al., 2012).

The possibility to modulate neural oscillations marks an important step beyond simply focusing on correlations between oscillations and cognitive performance. The feasibility to selfregulate endogenous neural activity has been nicely demonstrated in extracellular recordings for activity of single neurons (e.g., Olds and Olds, 1961), local field potentials (LFPs) in the animal model (e.g., Sterman et al., 1970), as well as for EEG activity measured on the scalp of humans (Kamiya, 1968). Nowadays the technique by which the modulation of neuronal activity is achieved in a reward-based fashion by giving feedback on the real-time status of the participants' brain activity who thereby learn to voluntarily control it, is termed neurofeedback (NF; Sherlin et al., 2011; Huster et al., 2014). NF-systems belong to a subclass of brain computer interfaces (BCIs) that aim at the regulation of oscillations and slow cortical potentials with EEG (Birbaumer et al., 2009) and magnetoencephalography (MEG; e.g., Sudre et al., 2011), or at the modulation of brain metabolism by functional magnetic resonance imaging (fMRI; e.g., Birbaumer et al., 2013; Ruiz et al., 2013) and near infrared spectroscopy (NIRS; e.g., Mihara et al., 2012; Kober et al., 2014). The modulation of neural oscillations by EEG-NF has been shown for diverse frequencies in association to different cognitive (sub) processes (see review of Gruzelier, 2014), for instance regarding enhanced upper alpha band and improved mental rotation (e.g., Hanslmayr et al., 2005; Zoefel et al., 2011), increased gamma-band activity and enhanced episodic retrieval (Keizer et al., 2010), or enhanced sensorimotor rhythm (SMR) and improved declarative learning (Hoedlmoser et al., 2008).

With respect to fm-theta oscillations, its basic modulability by NF has recently been demonstrated (e.g., Enriquez-Geppert et al., 2014). van Schie et al. (2014) designed a study to investigate effects of controlled fm-theta down-and up-regulation on WM performance. They demonstrated that the up-regulation led to increased, the down-regulation to attenuated performance. Concerning fm-theta NF training, positive effects on cognition have been shown for WM and attention (Wang and Hsieh, 2013). Notwithstanding this, effects of fm-theta modulation on EFs by NF remain to be elucidated.

The modulation of fm-theta by NF can be categorized within a broader framework of theta protocols. Generally, theta protocols can be performed at diverse electrode positions thereby focusing different neural networks (see review Gruzelier, 2014). A subdivision of protocols pertains to NF for theta up- or theta-downregulation. In case of the clinical application, down regulation of theta is most often combined with the modulation of other frequencies (e.g., theta-beta training, e.g., Gani et al., 2008; Arns et al., 2013), as for example seen with attention deficit hyperactivity disorder (ADHD). Here, EEG deviations have been observed during resting state compared to healthy controls, namely increased theta activity in context of beta activity (e.g., Chabot and Serfontein, 1996; Clarke et al., 2001). However, this deviation of theta measured at rest reflects a tonic condition that has to be dissociated from phasic theta responses, which in turn have been linked to specific cognitive functions. An important fact concerning the dissociation of tonic and phasic amplitude is the notion that successful behavior seems to be related to both of these EEG phenomena as good performance is related to a decrease in tonic and increased phasic theta activity (see review, Klimesch, 1999).

Taken together, NF seems perfectly suited to induce positive effects on EFs. This issue becomes imperative since EFs are crucial for success in daily life as they mediate learning processes (St Clair-Thompson et al., 2010), the control of emotions (e.g., Fikke et al., 2011), and predict academic achievement and social functioning (e.g., Miller et al., 2012b). Furthermore, age-related declines in EFs have been reported to lead to reduced success in everyday activities (Vaughan and Giovanello, 2010). Moreover, disturbances of EFs are associated with neurocognitive, and psychiatric impairments (e.g., Elliott, 2003).

The aim of this study is therefore to investigate the behavioral and neuronal effects of fm-theta NF on EFs. We set up an eight session personalized NF training for up-regulation of fm-theta and compared the achieved training effects to those of a pseudo NF training constituting an active control condition. A pre/post-test training design was implemented measuring the four most independent and representative EFs (Miyake et al., 2000; Miyake and Friedman, 2012) namely memory updating, set shifting, conflict monitoring, and motor inhibition, experimentally operationalized by a three-back, a number-letter task-switching, a Stroop-, and a stop-signal task. Concurrently, the participants' EEG was recorded. We hypothesized that NF for self-up-regulation of fm-theta would lead to enhanced performance in EFs at the behavioral level, and would also translate to enhanced fm-theta during executive functioning in the respective tasks compared to the pseudo NF group.

# **MATERIAL AND METHODS**

#### **PARTICIPANTS**

40 students (19 men; mean age: 24.8, standard deviation: 3.3) participated in this study. Participants were pseudo-randomly assigned to the experimental (NF; *n* = 19, 8 men; mean age: 23.8, standard deviation: 2.7) or the active control group (pseudo-NF, *n* = 21, 11 men; mean age: 25.8, standard deviation: 3.8) in order to balance the groups according to age, education level and gender. All subjects were right handed, as indicated by the Edinburgh Handedness Inventory (Oldfield, 1971), and had normal or corrected to normal vision. None of them reported a history of psychiatric or neurological disorders. Before the experiment, all participants gave written consent to the protocol approved by the ethics committee of the University of Oldenburg. For study participation, participants were rewarded with 8 e per hour. The study was conducted in accordance with the Declaration of Helsinki.

#### **PRE-POST TRAINING DESIGN**

To measure the effects of up-regulation of fm-theta on EFs, an individualized and adaptive eight session NF training was performed on consecutive working days within 2 weeks, comparing the effects to an active control group, the so-called pseudo-NF. One day before and one day after finishing the NF training, an EFs test-battery, assessing the four most important and independent EFs. The study was carried out at the Carl von Ossietzky University of Oldenburg.

#### **Executive function test-battery, EEG recordings and preprocessing**

In the first task of the EFs test-battery (see **Figure 1**), the visual three-back task, participants were presented with two kinds of letter sequences, the three-back and the zero-back sequence. In the three-back sequence, participants were instructed to respond via button press (using the right index finger), whenever a letter had already been presented three trials before the current one (three-back target condition). In the zero-back condition subjects were instructed to simply respond whenever a letter matched a target letter presented at the beginning of the letter sequence. In all other cases, participants were asked not to react. There were ten three-back and nine zero-back sequences that were presented in an alternating fashion with 24 white letters presented on a black background for 1000 ms, and followed by a fixation cross with a duration of 1000 ms (total trial length: 2000 ms; total trial number per sequence: 24; target numbers per sequence: 8). It is assumed that the three-back condition requires memory updating processes whereas the zero-back condition does not.

The number-letter task-switching consisted of number-letter pairs that were presented on a colored background. Participants were instructed to either classify the numbers (in even or odd numbers) or the letters (in vowels or consonants) by a right or left index button press. The classification depended of the specific background color (red, orange, pink vs. green, blue, and turquoise). Two versions with different number-color and letter-color assignments were utilized. Letter-number pairs were presented for 2000 ms followed by a white fixation cross presented for 1000 ms (trial length: 3000 ms). The task consisted of two parts. In the first part that included unmix-blocks, only the letters (unmix-block 1; 60 trials) had to be classified, followed by a second block during which only numbers (unmixblock 2; 60 trials) had to be processed. In the second part that included mixing blocks, participants had to switch between both classification types (trial number: 234; switches: 70). In contrast to the unmix-blocks, the mixing blocks required the switching between the two categorization tasks, thus it involves set-shifting.

The Stroop task contained the presentation of the colorwords blue, green, yellow, and red, which were either presented congruently in colors matching the word meaning (e.g., the color-word "blue" presented in blue) or incongruently in an unmatching fashion (e.g., the color-word "blue" presented in red). Participants were instructed to indicate via a right or left index button press the presentation color, but not the word-color. A trial started with a fixation cross presented for a randomized duration of 1200–1400 ms, and followed by the color word for 500 ms. Afterwards a second fixation cross was presented for a randomized duration of 100–500 ms. Trials were separated by an inter-trial interval (ITI) of a randomized duration of 400–800 ms (total trial number: 128, number of incongruent trials: 64). Regarding the involved EFs, conflict-monitoring is supposed to detect the conflict associated with incongruent trials relative to the congruent condition.

In the visual stop-signal task, left- or rightward pointing arrows were presented that changed their color during presentation time from purple to green, blue, or orange. Participants were instructed to press the button of a two-button box according to the arrow direction either with the right or the left index in a fast and accurate way directly after beginning of the arrow presentation (go-trials). However, a specific color change (for instance from purple to orange) was indicating that participants had to abort their initiated response (stop-trials). Two versions with different color-response assignments were used. The timing of the color-change was adjusted dynamically via a certain stimulus onset asynchrony (SOA; Logan et al., 1997) such that participants could stop their response in 75% of the stop-trials. Every trial with a length of 2 s started with a fixation cross that was randomly set with a duration of 300–600 ms. Then, the arrow was presented with an initial duration of 250 ms (this SOA was adjusted by adding 50 ms after every second correct or subtracting 50 ms after every failed stop trial to reach the intended error rate) before the color changed. The color change remained on screen for another 200 ms. The trial was ended with the presentation of a fixation cross (total number of trials: 300; number of stop trials: 100). The stop-trials of this task require motor inhibition as EFs, in contrast to motor execution during go-trials.

Before each task, a short exercise period was implemented to familiarize participants with the task requirements and the presentation. The four tasks each had a duration of roughly eight to 18 min. All four EFs tasks were implemented using the Presentation software<sup>1</sup> .

The measurements of the EFs test-battery, as well as the NF training, were conducted in an electrically shielded and sound attenuated room. EEG recordings were done using the Brain Vision Recorder software and a Brain Amp EEG amplifier (Brain Products GmbH, Filching, Germany). Electrode impendences were kept below 5 k for the continuous EEG recording, data was sampled at 500 Hz, and a low-pass online filter of 250 Hz was used. During the measurement of all four EFs, EEG activity was recorded from 32 Ag/AgCl electrodes (Easycap, Falk Minov Services, Munich, Germany), placed in accordance with the extended version of the international 10–20 system, with a nose electrode as online reference, and an electrode attached beneath the right eye for recording the electrooculogram (EOG) and quantification of ocular artifacts.

The offline-preprocessing was performed using the EEGLAB software<sup>2</sup> and included the following steps. First, data were lowpass (80 Hz) and high-pass filtered (0.5 Hz), and then downsampled to 250 Hz. To correct for ocular artifact, the Infomax ICA algorithm (Bell and Sejnowski, 1995; Makeig et al., 2004) was subsequently used on the continuous data. Blink-related independent components (ICs) were identified by comparing the IC activity to the eye blink artifacts in the EOG. The corresponding topographical maps of putative ICs had to show a frontal distribution. Eye-blink related ICs were then excluded from back-projection to the EEG channels. Afterwards, stimuluslocked epochs were computed comprising an interval of −1250 to +1250 ms corresponding to the stimulus presentation. Trials were then baseline-corrected. To correct for residual artifacts, a semiautomatic correction procedure was used based on the epoched data. Thereby single trials that crossed a self-set threshold (set to 60 µV) were marked automatically for visual inspection and rejection. Thereby 5.3 epochs were rejected on average per condition and task. Incorrect responses were discarded from further analyses.

Afterwards, so-called event-related spectral perturbations (ERSPs) were calculated for each task that represent logtransformed changes of power in dB relative to the baseline (Delorme and Makeig, 2004). For this time-frequency decomposition, a sinusoidal wavelet transform, using an increasing number of cycles with increasing frequency was used (range: 1–50 Hz; starting with 1 cycle at 1 Hz and increasing by 0.5 Hz per frequency; using 300 frequency steps). To visualize power changes relative to the pre-stimulus activity, the average power across the trials was divided by the frequency specific baseline values separately for each frequency. Mean ERSP values were calculated for the fm ROI over electrodes Fz, FC1, FC2, Cz.

#### **Individualized and adaptive neurofeedback training**

For up-regulating fm-theta by NF, eight 30-minutes training sessions were conducted. Because it was shown that fm-theta exhibits large inter-individual variability, but high intra-individual stability (e.g., Näpflin et al., 2008), a procedure for the detection of the individual fm-theta peak was used. Thus, the dominant individual fm-theta frequency peak was estimated based on the average of four peaks (+/−1 Hz), detected by the ERSPs computed from the four EF tasks. Fm-theta was shown to fall

<sup>1</sup>Version 14.8, www.neurobs.com

<sup>2</sup> freely available from http://www.sccn.ucsd.edu/eeglab/

into the time range of the N200/P300 complex (Huster et al., 2013), as well as fm-negativities (Huster et al., 2013), and furthermore to have a maximum at Fz (Ishihara et al., 1981; Cavanagh et al., 2012). Thus, the individual fm-theta frequency was determined from electrode Fz, between 4–8 Hz in the corresponding ERP time range of conditions requiring EFs. Each of the eight NF sessions furthermore consisted of six five-minute blocks of NF with self-paced breaks in-between. Before and after these blocks, a five minute start/end-baseline was measured to assess resting state activity. Frequency spectra were computed using a fast Fourier transformation and a hamming window for data segments of 2 s, shifted along the data in steps of 200 ms. For NF, the software NF Suite 1.0 (Huster et al., 2014) was used.

In the following, all five steps of the basic set-up for NF (Huster et al., 2014), forming the real-time processing pipeline, are described. First, the data acquisition was based on five electrodes placed at positions Fz, FC1, FC2, FCz, and Cz, thus covering the fm-brain region, with the nose as reference, and Fp1 and Fp2 for monitoring ocular activity. Before the start-baseline measurement, an EOG calibration method (3 min) was implemented that calculates the subject-specific, artifact-associated frequency band. This was used for all following measurements for eye blink detection and rejection during further measurements (for details see Huster et al., 2014). The second step refers to the data online-preprocessing. A crucial feature of this step is the monitoring of artifacts, specifically that of eye activity. This technique is important to avoid falsely modulating eye- rather than actual brain activity, since the power of eye activity unfolds in several frequency bands. Thus, the subject-specific artifactassociated frequency band that was calculated in the EOG calibration measure was monitored. Whenever the mean amplitudes of a 2 s segment was higher than the subject-specific artifactassociated frequency band (minus one standard deviation), the segment was rejected and not used for feedback. Third, for feature generation and selection, the raw power value of each 2 s segment for the individualized fm-theta frequency was compared to the baseline power of the same frequency. Fourth, to feed back the extracted feature to the participant, a visual procedure was used in form of a visual display of a colored square (range: highly saturated red over gray to highly saturated blue, each with 40 color steps). The color saturation depended on the fm-theta activity changes, and was updated every 200 ms. The color red indicated an fm-theta power increase, blue a decrease, and gray either an eye blink or no power change relative to the start-baseline of the specific NF session. The feedback saturation scale enclosed 95% of the amplitude range, whereby values above 97.5% or below 2.5% were indicated by a maximum red or blue saturation. Fifth, participants were instructed to color the square as red and as often as possible. To do so, they received a list of strategies based on the NF literature and were encouraged to find own self-invented strategies.

Participants of the pseudo-NF were matched to participants of the actual NF group according to age and gender. During a NF block, they received a playback of the same session and block of their matched participant. Thereby, both groups received a similar visual stimulation. To enhance the credibility of the pseudo manipulation, participants of the pseudo-NF received real feedback of their own eyeblink activity, by haltering the feedback replay and introducing the gray color feedback as it is done in the actual NF group.

# **DATA ANALYSIS**

# **NF training effects**

To analyze the self-upregulation of the individual fm-theta (indfm-th) by NF, the relative change of the fm-theta amplitude was quantified for all training sessions during training blocks as change in percent relative to the values of the first training session. This procedure was chosen because it minimizes some of the inter-subject variability caused by unspecific effects such as amplitude variability due to the visual stimulation during feedback. Thus, this specific calculation has the advantage to compare fm-theta increases relative to a condition in which subjects are also up-regulating their brain activity, but with lower success, thereby making sure that the same system or process is engaged. However, as a drawback, it attenuates some of the relevant inter-group variance (by essentially removing training effects of the first session), therefore reducing the likelihood to find a group by session interaction. Correspondingly, a main effect is expected, as the dependent variable represents a difference measure, indicating that proper NF induces increased fm-theta not seen with the pseudo-NF training. Consequently, NF training effects were investigated by a repeated-measures ANOVA with the factors SESSION (2–8) and GROUP (NF vs. pseudo-NF group). These calculations and analyzes were also performed for the alpha (ind-fm-th + 2 Hz to ind-fm-th + 7 Hz) and beta (ind-fm-th + 7 Hz to ind-fm-th + 15 Hz) frequency bands. In case of violations of sphericity, Greenhouse-Geisser corrections were performed, and corrected *p*-values and -values are reported.

# **Data analysis: behavioral performance of executive function tasks**

To investigate transfer effects of NF training on cognitive performance, mean RT and mean accuracy of correct responses were calculated for all conditions of the four EFs tasks, namely for the three-back vs. zero-back conditions of the three-back task; the unmix vs. stay vs. switch conditions of the letter number task-switching task, the congruent vs. incongruent conditions of the Stroop-task, and the go vs. stop conditions of the stopsignal task. Extreme values exceeding the mean by more than 2.5 standard deviations were excluded for statistical analyses. To investigate performance gains after NF training, performance differences between pre- and post measurements were calculated for all the conditions of the four EF tasks. A stronger performance gain was expected in conditions requiring EFs. In other words, concerning the three-back task, performance gains are expected after NF in the three-back condition, because for this condition a higher load of memory-updating is involved compared to the zero-back condition (larger effects in NF compared to pseudo-NF training). Similarly, larger effects were expected in the stay and switch conditions compared to the unmix condition after NF-training relative to the pseudo-NF training. Likewise, stronger performance gains regarding conflict monitoring were expected after NF in the incongruent compared to the congruent condition of the Stroop task. Finally, behavioral performance gains were expected in the stop condition as reflected in reduced SSRT after proper NF. Thus, pre-post measurement differences were tested for by means of independent-samples *t*-tests comparing the training and the control group. To examine the range of significant results, Cohen's d (Cohen, 1988) was calculated for the pretest-posttest differences or the standardized mean difference in performance between pre- and post measurements (the difference was divided by the pooled standard deviation for the measurements occasions) for each group and corrected for small sample bias using the Hedges and Olkin (Hedges and Olkin, 1985) correction factor (d') only for significant results. As result, a pre-post measurement effect size (ES) of *d*' = 1 reflects a mean difference between pre-and post-measurements of one standard deviation.

## **Data analysis: effects of NF training on fm-theta in executive functions tasks**

For the evaluation of NF effects on fm-theta during the four EFs tasks, ERSP values were extracted for the time range used for the detection of the individual fm-theta peaks (three-back task: 100–300 ms, number/letter task-switching: 100–300 ms, Stroop: 220–500 ms, stop-signal task: 300–500 ms) and the theta frequency range of 4–8 Hz. These values were then averaged for each condition, task and subject. Then, difference scores were calculated by subtracting the values before from values after training: (pre-post difference of letter number task-switching, the congruent and incongruent pre-post-differences of the Strooptask, and the go and stop pre-post-differences of the stop-signal task. To investigate if these difference scores differed significantly between the NF and the pseudo-NF group, a MANOVA with the above-mentioned pre-post-difference scores as dependent variables and GROUP (NF vs. pseudo-NF group) as independent variable was run.

# **RESULTS**

#### **NF TRAINING EFFECTS**

NF training effects on the fm-theta amplitude are depicted in **Figure 2**. The repeated-measures ANOVA resulted in a main effect of SESSION (*F*(4.346,39) = 6.487, = 0.724, *p* < 0.001) showing that every training session let to a change in fm-theta selfupregulation. The further expected main effect was the GROUP (*F*(6,39) = 6.225, *p* < 0.05) effect, demonstrating the validity of the experimental manipulation resulting in stronger enhancements of activity in the NF group regarding fm-theta amplitude, starting already at the second training session when compared to the pseudo-NF group.

Regarding the alpha range, a trend for a main effect of SES-SION (*F*(6,37) = 8.173, *p* < 0.1) was observed, no further effects were detected. Similarly, within the beta frequency range also only a trend for the main effect of SESSION (*F*(6,37) = 3.157, = 0.308, *p* < 0.1) was observed.

#### **BEHAVIORAL PERFORMANCE IN EXECUTIVE FUNCTIONS TASKS**

The results of the pre- and post measurements (RTs and accuracy) are shown in **Figures 3**–**6** computed from correct responses of all conditions for the three-back, task-switch, Stroop-, and

**FIGURE 2 | shows frequency effects during NF. (A)** Depicts the means of fm-theta increases of the NF (blue lines) and the pseudo-NF group (green) within the eight training sessions (error bars represent standard error of mean). In **(B)** the frequency spectra are shown for the start baseline measure of the first session (in gray) and during the last training session (S8, black) for each group (left side: NF, right side: pseudo-NF group). Unspecific alpha effects are observed in both groups, whereas fm-theta is specifically increased in the experimental group.

**group (right) for each condition (three-back vs. zero-back) before and after the training intervention**. Significant differences (marked with a star) between accuracy enhancements (pre- post differences as depicted with curly brackets) were detected in the three-back condition between both groups showing stronger improvements after proper NF training.

stop-signal tasks for both training types (NF and pseudo-NF). Descriptively, these figures hint to behavioral changes, particularly in the three-back and task-switching tasks. Although these enhancements are more pronounced in the NF group, behavioral

**FIGURE 4 | Shows the behavioral results of the task-switching (mean RT and standard error of mean) for the NF group (left) and the pseudo-NF group (right) for all three conditions (unmix, stay, switch) before and after the training intervention**. Significant differences (pre-post differences as depicted with curly brackets) are marked with stars and are depicted between both groups as training results. Stronger enhanced RTs can be observed for the NF group in the stay and switch condition.

changes appear also in the pseudo-NF group. However, any study that is using a pre-post design has to expect repetitionrelated effects that may simply arise from repeatedly testing a task. Such unspecific repetition-related effects are particularly well known in behavioral training studies and are found in all subjects, for instance in task-switching (e.g., Karbach and Kray, 2009) or three-back tasks (e.g., Schneiders et al., 2011). Therefore, the utilization of a control group is urgently needed

to dissociate such repetition-related or non-specific effects from true enhancements of cognitive functions, which in turn are reflected in stronger improvements in the intervention group than in the control group (Campbell and Stanley, 1963; Oken et al., 2008).

Concerning the three-back task, the expected stronger increases in accuracy was statistically confirmed in the three-back condition (*t*(38) = 1.786; *p* < 0.05) after proper NF training when compared to the pseudo-NF training; such effects were absent in the zero-back condition (see **Figure 3**). These results were supported by the pretest-posttest ES. ES were larger after NF training (three-back: *d*' = 1.059) than after pseudo-NF training (threeback: *d*' = 0.346). For the zero-back condition, the following ES are reported (NF training: *d*' = −0.076; pseudo-NF training: *d*' = −0.494).

I. As expected, stronger RT decreases were observed in the switch condition (*t*(38) = 1.966; *p* < 0.05), and in the stay condition (*t*(38) = 1.695; *p* < 0.05) after proper NF training as compared to the pseudo intervention (see **Figure 4**). No differences were found for neither the unmix condition (*t*(38) = 1.036; *p* = n.s.) between both training types nor for accuracy scores (switch: *t*(38) = 1.633; *p* = n.s.; stay *t*(38) = −1.651; *p* = n.s. unmix: *t*(38) = −0.388; *p* = n.s.). These results were supported by the pretest-posttest ES. For both, switch and stay conditions, EF were larger after NF training (switch: *d*' = 1.67; stay: d' = 1.09) than after pseudo-NF training (switch: *d*' = 0.582; stay: *d*' = 0.649). However, no differences in performance gains were shown in accuracy (switch: *t*(38) = 1.633, *p* = n.s.; stay: *t*(38) = −1.651, *p* = n.s.). ES for the unmix condition were the following, NF training: *d*' = −0.408 and pseudo-NF training: *d*' = −0.063, (the ES for differences in accuracy scores are also given subsequently, NF training: switch *d*' = 0.607, stay *d*' = 0.718, unmix *d*' = 0.091; pseudo-NF training: switch *d*' = 0.879, stay *d*' = 0.929 unmix *d*' = 0.142 ).

II. Different from what was expected, no differences were observed when comparing NF to pseudo-NF training regarding the Stroop task, neither concerning RTs (incongruent: *t*(34) = 0.247, *p* = n.s.; congruent: *t*(38) = 0.142, *p* = n.s.) nor accuracy (incongruent: *t*(38) = 0.989, *p* = n.s.; incongruent: *t*(38) = 0.81, *p* = n.s.), see **Figure 5**. Similarly, low ES values were recorded for the NF and the pseudo-NF training (RTs differences in the incongruent condition: NF training: *d*' = 0.337 and pseudo-NF training: *d*' = 0.749; RTs differences in the congruent condition: NF training: *d*' = −0.469 and pseudo-NF training: *d*' = −0.143; accuracy differences in the incongruent condition: NF training *d*' = 0.846 and pseudo-NF training *d*' = 0.55; accuracy differences in the congruent condition: NF training: *d*' = 0.42 and pseudo-NF training: *d*' = 0.244).

III. Similarly, in the stop-signal (see **Figure 6**) task no differences were observed comparing the pre-post difference of the NF- and the pseudo-NF-training groups neither concerning SSRTs of the stop condition (*t*(38) = −1.558, *p* = n.s.) nor RTs of the go-condition (*t*(36) = 1.65, *p* = n.s.). Accordingly, low ES scores were registered: differences in the stop condition: NF training *d*' = 0.159, pseudo-NF training *d*' = 1.147; differences in the go condition: NF training *d*' = 0.159, pseudo-NF training *d*' = 0.147.

# **EFFECTS OF NF ON FM-THETA IN EXECUTIVE FUNCTION TASKS**

In the following, a descriptive overview will be given for each task and group before and after the training; thereafter the fmtheta effects will be reported, as computed within the multivariate analysis framework.

I. Irrespective of group, power enhancements can especially be observed in the delta and theta frequency range around 300 ms post stimulus onset, accompanied by power reductions in the beta and slight power enhancements in the gamma range in the threeback condition of the three-back task (see **Figure 7**), which are in agreement with those obtained by e.g., Pesonen et al. (2007). Slight frequency changes can be observed in both groups after training.

II. The ERSPs in the task-switching task before and after the training intervention are characterized by a distinct thetaalpha power enhancement at around 200 ms, accompanied by weaker effects with a similar time pattern in the beta and gamma frequency range with a duration of approximately 100 ms, in accordance with previous studies as Prada et al. (2014). In the subsequent time frame, the power enhancement of theta and delta remains stable and the beta power reduction (20 Hz) becomes apparent.

III. To turn to the ERSPs in the Stroop task, here theta and beta power enhancement effects are observed corresponding with prior finding in literature (e.g., Cavanagh et al., 2012) beginning at 180 ms, whereby the theta power enhancement becomes maximal at 300 ms and remains stable until the end of the analyses time window. Minor frequency changes can visually be detected after the training intervention for both groups.

IV. With respect to the stop-signal task, maximal theta power enhancements are detected starting at about 220 ms till 600 ms post stimulus, consistent with those reported in the review of Huster et al. (2013), these power enhancements are very similar before and after training.

V. To inspect training induced changes in fm-theta more closely, pre-post fm-theta difference scores, representing changes in fm-theta power after training compared to the initial power before training, are depicted in **Figure 7** (lower panels of A–D) for each EF condition of the four EFs task and groups (NF and pseudo-NF group). As can be seen in **Figure 7**, fm-theta is decreasing in the pseudo-NF group after the control intervention (an exception is the congruent condition in the Stroop task), whereas the NF group shows amplitude increases. An exception is the stop-signal task, in which the fm-theta activity was decreasing for both groups. Indeed, the training intervention led to significantly different fm-theta changes between the NF and the pseudo-NF group as indexed by the multivariate assessment that revealed a multivariate main effect of GROUP (Wilks' λ = 0.468, *F*(9,25) = 2.774, *p* < 0.05), meaning an increase of 0.0036 dB in the NF group and a decrease of −0.2122 dB in the pseudo-NF group over all conditions and tasks. Univariate analyses for the effects of the group type significantly predicted fm-theta changes in the three-back task, such as significant fm-theta reductions were observed in the zero-back condition (*p* < 0.05) in the pseudo-NF group. A trend (*p* < 0.1) was furthermore observed within the univariate analyses for the prediction of group type concerning fm-theta power changes in the incongruent condition.

# **DISCUSSION**

Since fm-theta has been proposed as neural "working language" of brain communication for EFs, the current study investigated the effects of an individualized and adaptive eight session NF training to up-regulate fm-theta compared to a pseudo-NF intervention by utilizing a pre/post-test training design, analyzing memory updating, set-shifting, conflict monitoring, and motor inhibition. As a result, it was shown that proper NF training led to facilitated memory updating and mental set shifting, two EFs relying on proactive control, but not to enhanced conflict monitoring and motor inhibition, two EFs that built on reactive control. Furthermore, analyses of neural effects after NF training demonstrated that learning to self-upregulate fm-theta during NF translated to fm-theta changes in tasks engaging EFs after proper training compared to the pseudo-NF intervention. In the following, aspects concerning the subtypes of EFs, the fm-theta NF protocols, and the neural transfer effects are discussed.

With regard to the dual nature of the findings, the differential NF training effects of this fm-theta NF protocol on EFs reflect the current qualitative distinction of cognitive control mechanisms into proactive and reactive control. Indeed, a dual mechanisms framework (DMS; Braver, 2012) has been suggested that works by means of these two distinct operating modes that probably differ concerning their temporal dynamics and relevant neural networks. As such, within the DMS, proactive control is conceptualized as an anticipatory mechanism, actively maintaining task goals that serve as a source of top-down bias, thus supporting facilitated processing of expected events with a high cognitive demand before they actually occur. For instance, in the threeback task, subjects can process upcoming targets by subvocally repeating the sequence they have to keep in WM, and comparing it with their mental representation of the target stimulus. In contrast, reactive control is conceptualized as a reactive bottomup mechanism that is recruited only when it is required, for instance when interference is detected by a conflict monitoring

system or when a stop-signal requires the inhibition of an initiated response as is the case with the Stroop and the stop-signal tasks.

Interestingly, although all EFs, regardless of whether they require proactive or reactive control mechanisms, recruit the superordinate cognitive control network (Niendam et al., 2012), the subtypes seem to rely on different sub-networks. Menon (2011) distinguishes the central executive network, a frontoparietal system, anchored in the dorsolateral prefrontal cortex (DLPFC) and lateral posterior parietal cortex (PPC), crucially involved in actively maintaining task goals, from the salience network, a cingulate-frontal-opercular system, reacting to detected task events. Similarly, Dosenbach et al. (2008) proposes a dualnetwork architecture for EFs based on graph analytic methods, differentiating the fronto-parietal from a cingulo-opercular network, implementing proactive and reactive cognitive control, respectively.

From the lack of behavioral effects in the Stroop- and stopsignal tasks, one might be tempted to conclude that fm-theta is not necessarily crucial for the implementation of reactive control. Yet, the current fm-theta protocol of this study might have primarily targeted fm-theta used in the proactive control network and might less effectively address the network implementing reactive control, as fm-theta might primarily affect the detection signals generated in the MCC and less the resolution activity or the compensatory mechanisms processed in other brain regions of the EFs network (Botvinick et al., 2004). Indeed, fm-theta is thought to enable the transmission of information over different cortical brain areas by entraining activity in disparate neural systems (Cavanagh and Frank, 2014). More precisely, the properties of theta oscillations showing high-amplitude-low-frequency modulations denote an ideal neural parameter for neural organization over distal brain regions (Buzsáki and Draguhn, 2004). Theta phase synchrony between the MCC and distal brain regions has been observed in different studies on EFs, such as the study of Cohen and Cavanagh (2011), who reported of single-trial phase synchrony between the MCC and lateral prefrontal brain areas that was modulated by RTs during conflict monitoring. Nigbur et al. (2011) presented enhanced synchrony in the theta range between frontomedial and lateral frontal electrode sites, interpreted as cooperated work to allocate control during conflict monitoring, as well as between fronto-medial electrode sites with those over the contra-lateral motor area during conflict monitoring, possibly reflecting the renewed need of response selection during conflicts. Just recently, Oehrn et al. demonstrated by means of intra-cranial recordings that fm-theta originated from MCC as conflict detection signal, causally leading to entrained theta in the DLPFC, and finally accomplishing a coupling between DLPFC-gamma power and MCC oscillations for conflict resolution (Oehrn et al., 2014). Ultimately, accounting for the dual nature of EFs provides potentially new avenues for more elaborative feature extraction procedures concerning fm-theta NF protocols.

Apart from the beneficial behavioral effects of proactive control, the current study revealed an overall alteration of fmtheta in EF-tasks after proper NF training as compared to the active control intervention. This finding generally demonstrates that the targeted EF network has been affected by fm-theta NF. However, one would have expected EF-specific fm-theta effects in the proactive tasks to be paralleled by the above described behavioral effects. When looking more specifically at the neural effects, this does not seem to be the case, since results demonstrate that fm-theta effects between the proper NF group and the active control group are detected in the non EF-condition of memory updating, whereas no differences in fm-theta changes were found between both groups with task-switching. Although both the behavioral and the neural findings are interesting in itself, one should keep in mind that only one of the possible neural parameters, namely the fm-theta amplitude, has been assessed. It is conceivable that the fm-theta NF training could have influenced other neural parameters, such as coherence, frequencycoupling, or even myelination, paralleling the behavioral effects in proactive control in a more consistent way. The behavioral effects observed in proactive control could also correspond to structural changes within axonal pathways of relevant networks (e.g., changes in the integrity of white matter or the velocity of conductivity). In fact, glance at the EEG-NF literature indicates that the neural mechanisms underlying NF and putative transfer effects to neurocognition are still not well understood. Little is known about which specific network elements are changed and how the elements are altered due to NF. Behavioral performance increases aren't necessarily associated with increases in neural activity; rather, reduced neural activity may also result from stimulus and task repetition (see review Grill-Spector et al., 2006). Furthermore, associations between behavioral task proficiency and neural activity after training do not necessarily have to follow a linear function. For example, domain-proficient relative to domain-naive participants may show lower neural activity with simple tasks, but stronger neural processing with higher task difficulty (e.g., Prat and Just, 2011; Dunst et al., 2014). Hence, complex interactions of repetition-related performance changes, training-induced changes in neural responding and behavior, and task difficulty may exist.

Those functional and structural studies, on neural mechanism mediating behavioral improvements, undertaken so far either focus: (a) on immediate and long-lasting functional EEG effects, (b) on immediate fMRI effects and network activity; and finally (c) on long-lasting structural effects found in white matter pathways, usually examined via diffusion tensor imaging (DTI). For instance, Egner and Gruzelier (2001, 2004) were among the first to study neural changes induced by NF by using a protocol to increase activity in the 12–18 Hz frequency range. In their study EEG training effects were accompanied by improved attention. More importantly, after NF an increase of the oddball P300 was detected and interpreted as indicator for improved neural integration of relevant environmental stimuli. In order to investigate immediate neural effects after NF, Ros et al. (2010) examined possible changes of motor evoked potentials (MEP) by means of transcranial magnetic stimulation (TMS) to assess variations in the strength of neurotransmission from the motor cortex to the muscle. By using two types of TMS protocols (single vs. paired pulses), they could dissociate increases in cortico-spinal excitability (CSE), short-interval intra-cortical inhibition (SICI), and intra-cortical facilitation (ICF). This way, Ros et al. (2010) demonstrated that down-regulation of alpha oscillations, considered as a marker of cortical activation, indeed led to increased CSE and decreases of SICI. Similar increases of SICI were recently reported after completion of a 20 session theta/beta training (Studer et al., 2014). Here as well, a decreased P300 observed in the Attention Network Test (ANT) was considered to indicate increased efficiency of stimulus processing. Later on, Ros et al. (2013) conducted an independent component analysis (ICA) of brain activity recorded via fMRI during an oddball task to identify functionally coupled brain regions and to assess putative connectivity changes directly after a single session NF intervention. Immediately after alpha down-regulation, increased activation of brain regions belonging to the salience network were detected, namely within the MCC, the bilateral insular, thalamic, basal ganglia, cerebellar and ponto-mesencephalic regions. Structural changes after NF were assessed by DTI by Ghaziri et al. (2013) after a 3 months beta up-regulation training that resulted in enhanced sustained attention as evidenced in behavioral measures. Interestingly, microstructural changes were observed in three white matter pathways that are part of the sustained attention network, namely the cingulum bundle (a tract connecting the MCC with the DLPFC and the PPC), the anterior corona radiata (connecting the frontal cortex with the brainstem), and the splenium of the corpus callosum (a tract supporting inter-hemispheric processing), which seem not only to speak for enhanced neural transmission, but for increased myelination and faster conduction velocity (CV) underlying improved performance triggered by NF.

Regarding the relation between NF and myelination, participants learn to change oscillatory activity by recruiting and synchronizing the activity of neurons in a specific brain region, which again may affect myelination that preferable takes place at electrically active axons (e.g., Demerens et al., 1996; Ishibashi et al., 2006; Fields, 2010; Wake et al., 2011). Myelin plasticity has been furthermore suggested as an experience and learning dependent mechanisms (Wang and Young, 2014), which is thought to be relevant throughout adulthood (Miller et al., 2012a). Overall, myelin plasticity seems to crucially reflect an important lifelong process of relevance for oscillatory activity and cross-regional coupling (e.g., Pajevic et al., 2014; O'Rourke et al., 2014), and thus may also represent important mechanisms underlying NF-induced neural modulations.

Altogether, self-regulation of endogenous fm-theta increases capacities of EFs and represent a feasible neuroscientific training approach. Theta protocols may also be adapted according to the specific needs of the participants, with a focus on enhancing general EFs or proactive and reactive subtypes, in neurological and psychiatric disorders. Given that age-related changes of fmtheta seem to be related to EFs declines in age (Kardos et al., 2014), fm-theta NF protocols may also serve as an ideal tool to decelerate aging effects.

### **REFERENCES**


Buzsáki, G. (2006). *Rhythms of the Brain.* Oxford, NY: Oxford University Press.

Buzsáki, G., and Draguhn, A. (2004). Neuronal oscillations in cortical networks. *Science* 304, 1926–1929. doi: 10.1126/science.1099745


motor imagery related cortical activation. *PloS One* 7:e32234. doi: 10. 1371/journal.pone.0032234


network connectivity in schizophrenia. *Hum. Brain Mapp.* 34, 200–212. doi: 10. 1002/hbm.21427


**Conflict of Interest Statement**: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 27 August 2014; accepted: 17 November 2014; published online: 05 December 2014*.

*Citation: Enriquez-Geppert S, Huster RJ, Figge C and Herrmann CS (2014) Self-regulation of frontal-midline theta facilitates memory updating and mental set shifting. Front. Behav. Neurosci. 8:420. doi: 10.3389/fnbeh.2014.00420*

*This article was submitted to the journal Frontiers in Behavioral Neuroscience*.

*Copyright © 2014 Enriquez-Geppert, Huster, Figge and Herrmann. This is an openaccess article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution and reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms*.

# Detaching from the negative by reappraisal: the role of right superior frontal gyrus (BA9/32)

#### *Rosalux Falquez <sup>1</sup> \*, Blas Couto2,3, Agustin Ibanez 2,3,4, Martin T. Freitag5, Moritz Berger 5, Elisabeth A. Arens 1, Simone Lang1 and Sven Barnow1*

*<sup>1</sup> Department of Clinical Psychology and Psychotherapy, Institute of Psychology, University of Heidelberg, Heidelberg, Germany*

*<sup>2</sup> Laboratory of Experimental Psychology and Neuroscience, Institute of Cognitive Neurology, Favaloro University, Buenos Aires, Argentina*

*<sup>3</sup> UDP-INECO Foundation Core on Neuroscience, Diego Portales University, Santiago, Chile*

*<sup>4</sup> Departamento de Psicología, Universidad Autónoma del Caribe, Barranquilla, Colombia*

*<sup>5</sup> Department of Radiology, German Cancer Research Center, Heidelberg, Germany*

#### *Edited by:*

*Sergio Ruiz, Pontificia Universidad Catolica de Chile, Chile*

#### *Reviewed by:*

*Christian E. Salas Riquelme, Head Forward Rehabilitation Centre, UK Bryan Thomas Denny, Mount Sinai School of Medicine, USA*

#### *\*Correspondence:*

*Rosalux Falquez, Department of Clinical Psychology and Psychotherapy, Institute of Psychology, University of Heidelberg, Hauptstrasse 47-51, 69117 Heidelberg, Germany e-mail: rosalux.falquez@ psychologie.uni-heidelberg.de*

The ability to reappraise the emotional impact of events is related to long-term mental health. Self-focused reappraisal (REAPPself), i.e., reducing the personal relevance of the negative events, has been previously associated with neural activity in regions near right medial prefrontal cortex, but rarely investigated among brain-damaged individuals. Thus, we aimed to examine the REAPPself ability of brain-damaged patients and healthy controls considering structural atrophies and gray matter intensities, respectively. Twenty patients with well-defined cortex lesions due to an acquired circumscribed tumor or cyst and 23 healthy controls performed a REAPPself task, in which they had to either observe negative stimuli or decrease emotional responding by REAPPself. Next, they rated the impact of negative arousal and valence. REAPPself ability scores were calculated by subtracting the negative picture ratings after applying REAPPself from the ratings of the observing condition. The scores of the patients were included in a voxel-based lesion-symptom mapping (VLSM) analysis to identify deficit related areas (ROI). Then, a ROI group-wise comparison was performed. Additionally, a whole-brain voxel-based-morphometry (VBM) analysis was run, in which healthy participant's REAPPself ability scores were correlated with gray matter intensities. Results showed that (1) regions in the right superior frontal gyrus (SFG), comprising the right dorsolateral prefrontal cortex (BA9) and the right dorsal anterior cingulate cortex (BA32), were associated with patient's impaired down-regulation of arousal, (2) a lesion in the depicted ROI occasioned significant REAPPself impairments, (3) REAPPself ability of controls was linked with increased gray matter intensities in the ROI regions. Our findings show for the first time that the neural integrity and the structural volume of right SFG regions (BA9/32) might be indispensable for REAPPself. Implications for neurofeedback research are discussed.

#### **Keywords: emotion regulation, self-focused reappraisal, VLSM, VBM, right SFG**

# **INTRODUCTION**

Cognitive emotion regulation (ER) is conceptualized as the ability to modulate the spontaneous flow of emotional states by means of cognitive control. That means, the ability to manage not only which emotion we feel, but when and how this emotion is experienced and expressed (Gross, 1999). Thus, people can effectively take control over their own emotional responses by adjusting them to everyday events and context demands. Conversely, disturbances in ER abilities might disrupt human adaptation and therefore compromise well-being as well as lead to unhealthy social functioning (Aldao and Dixon-Gordon, 2013). For instance, previous studies have emphasized that an impaired ER constitutes a core feature of affective (Goldin et al., 2009; Hermann et al., 2009; New et al., 2009; Abler et al., 2010) and personality disorders (Slee et al., 2008; Lang et al., 2012). According to Gross's *process model of ER*, some regulation strategies might be more protective than others, particularly when they act early in the emotion-generative process (Gross, 1999, 2002). The early change of the way an emotional stimulus is appraised in order to decrease its emotional impact (i.e., reappraisal) is associated with long-term psychological health outcomes (Gross and John, 2003; Goldin et al., 2008; Barnow, 2012). Reappraisal is held to be very effective for the down-regulation of negative emotions, as it has been shown to decrease peripheral psychophysiology (Ray et al., 2010; Kim and Hamann, 2012) and self-reported negative affect (Gross, 1998; Ochsner et al., 2002). Moreover, previous studies investigating neural activation pattern during reappraisal show decreased activation of emotional limbic regions such as the amygdala (Ochsner et al., 2002; Banks et al., 2007; Wager et al., 2008).

Importantly, the down-regulation of emotional limbic regions through reappraisal has been shown to occur by means of topdown influences of cognitive control regions in the prefrontal cortex (PFC; Buhle et al., 2013). Reappraisal is one of the most complex strategies; it involves a variety of cognitive control abilities which serve to generate alternative explanations about an emotionally arousing cue. For example, participants need to rely on working memory (WM) so as to keep or update alternative reinterpretations in mind (Hofmann et al., 2012; Schweizer et al., 2013). Furthermore, flexibility skills are crucial in order to choose between new reinterpretations (Joormann and Gotlib, 2008; Malooly et al., 2013). Also, cognitive inhibition skills are similarly important, especially for the decrease of automatic emotional appraisals (Joormann, 2010; Pe et al., 2013). Following, all of these processes need to be monitored in order to keep track of the regulation success according to internal and external demands (Paret et al., 2011). In line with these assumptions, neural structures commonly shown in functional magnetic resonance imaging (fMRI) investigations examining reappraisal of negative stimuli include mainly regions in the superior frontal gyrus (SFG), like the dorsolateral PFC (dlPFC; BA9/46) and the anterior cingulate cortex (ACC, BA32; Harenski and Hamann, 2006; Banks et al., 2007; Erk et al., 2010; Leiberg et al., 2012; Ochsner et al., 2012). These regions are found to be highly involved in WM and inhibition performance (Lutcke and Frahm, 2008; Shackman et al., 2009; Balconi, 2013). Similarly, an increased blood-oxygen-level-dependent (BOLD) contrast of the dorsal ACC (dACC;BA32) has been observed during reappraisal of negative social situations (Koenigsberg et al., 2010), as well as during situations where error monitoring (van Veen et al., 2001; Ichikawa et al., 2011) and cognitive flexibility are required (Zastrow et al., 2009; Vriend et al., 2013). Therefore, reappraisal function relies to a great extent on the same regions involved in several complex cognitive control tasks (Schweizer et al., 2013).

Besides its functional complexity, there are several types of reappraisal (McRae et al., 2012a). To date, two main variants of strategies have been investigated with fMRI. The most investigated is the situation-focused strategy (REAPPsit), which involves reinterpreting the meaning of the emotional actions or events presented, in order to reduce the emotional response (e.g., seeing a diseased person and thinking the person will get better). The other type is the self-focused reappraisal strategy (REAPPself), which is also known as distancing (e.g., Ochsner et al., 2004), i.e., adopting the role of a detached third-person observer during the presentation of the emotional stimuli (e.g., thinking that the presented stimuli are randomly seen in a newspaper). Ochsner et al. (2004) compared the neural response of both reappraisal strategies, linking regions of the medial prefrontal cortex (mPFC) to REAPPself and more lateral prefrontal cortex regions (lPFC) to REAPPsit (Ochsner et al., 2004). Furthermore, it has been discussed that the ability to assume a subjective distance to emotional cues implies a change in the perceived self-relevance of emotion-inducing objects, which is associated with a rightlateralized, mPFC activity (Kalisch et al., 2005; Ochsner et al., 2005; Ochsner and Gross, 2008; Leiberg et al., 2012). On the other hand, the act of reinterpret negative events might require a more left-lateralized, lPFC involvement because of a highly verbal, externally focused control processing during REAPPsit (see also Ochsner et al., 2012).

In order to examine whether specific regions in the PFC are crucial for effective reappraisal of negative stimuli, brain lesion studies might be an excellent extension to fMRI data, as they do not only reflect which areas are associated with a given ability, but also show which regions are critical for function integrity (Rorden and Karnath, 2004). Furthermore, the need for this type of studies on ER has been highlighted in the past (firstly addressed by Beer and Lombardo, 2007). However, to date there have been only two studies investigating reappraisal performance after acquired brain damage. One recent case-study showed that emotional reappraisal is impaired after a left PFC stroke lesion (Salas et al., 2013). Another recently published study of the same author investigated behavioral data of reappraisal difficulty and productivity in brain injured patients. Results showed that cognitive skills, such as inhibition and verbal fluency might be strongly associated with the generation of reappraisals. Still, the reappraisal ability was not addressed (Salas et al., 2014). Therefore, the goal of the current study was to explore the reappraisal ability in brain-injured patients, and to infer which area of the PFC impairs this ability the most. For this purpose, considering that the effects of a brain lesion might impair the manipulation of thoughts (as held by Salas et al., 2014), we chose to hold the reappraisal strategy constant and instruct the REAPPself strategy. First, this strategy has been shown to be more effective than REAPPsit at overall reduction of affective responding (Shiota and Levenson, 2012). Second, REAPPself might be less difficult than REAPPsit for the braindamaged patients, as REAPPsit might require more cognitive abilities for the spontaneous generation of alternative reinterpretations and involves more contextual encoding of stimuli than REAPPself (Ochsner et al., 2004, 2012; Ochsner and Gross, 2008). Furthermore, REAPPself might be a relevant strategy for braindamaged patients, given the widely recognized importance of self-distancing for adaptive coping with autobiographical negative events (Ayduk and Kross, 2008, 2010).

To explore which lesion location is most likely to impair REAPPself, we first ran an exploratory voxel-based lesionsymptom mapping analysis (VLSM; for further explanation of this method see Bates et al., 2003; Rorden et al., 2007) in 20 patients with single brain lesions. We hypothesized that SFG regions, particularly regions near the mPFC, would be indispensable for REAPPself ability (Ochsner et al., 2004), as this region has anatomically been defined as a mediator between the lateral PFC and amygdala regions (Johnstone et al., 2007; Ray and Zald, 2012). Second, we predicted that subjects with a lesion in the areas highlighted by the VLSM analysis, defined as region-of-interest (ROI), would have deficits on REAPPself, whereas patients with a lesion sparing these ROI would show a better REAPPself ability. Third, we ran a regional voxel-based-morphometry (VBM; Ashburner and Friston, 2000) analysis with structural data of healthy participants expecting to find significant associations between ROI gray matter intensities and REAPPself function. All participants completed cognitive and affective screening tasks in order to characterize potential deficits of the patients. Here, we also aimed to explore which cognitive functions are more impaired in the group with a lesion in the ROI compared to the other groups.

### **METHODS AND MATERIALS PARTICIPANTS**

A total of 27 patients with lesions in different parts of the cortex, but predominantly affecting the frontal lobes, and 23 healthy volunteers were assessed. The patients were recruited either from the department of Neuro-Oncology of the University Hospital of Heidelberg, Germany, or from self-help groups in the community. The inclusion criterion was the presence of a lesion involving well-defined parts of the cortex with definable and segmentable margins in the T2 weighted FLAIR (fluid-attenuated inversion recovery) sequence, assessed by an experienced radiologist. For all analyses, a potential edema zone was, if present, taken into account. Patients with multifocal brain lesions, previous history of head trauma or neurological disorders independent of brain injury, clinically detectable aphasia symptoms as well as presence of serious functional impairments (i.e., patients with a Karnofsky index below 80%; Clark and Fallowfield, 1986) were excluded. Healthy control participants (15 women; *M* age = 39.65 ± 11.23) had no history of neurological or psychiatric disorders. No control participant was taking psychoactive drugs. They were recruited through advertisements posted in newspapers, and were matched as closely as possible to the patients for sex and age.

Nineteen patients with an acquired brain tumor and one patient with a cyst met the inclusion criterion. The diagnosis of patients with brain tumors has been histopathologically confirmed either by operation (*n* = 17) or by biopsy (*n* = 2) in agreement with the WHO staging system (Kleihues and Sobin, 2000). Seven patients had to be excluded, due to non-corrected vision problems (*n* = 1), multifocal lesions (*n* = 5) and severe microangiopathy (*n* = 1). The remaining 20 patients took part in the analysis (11 women, *M* age = 45 ± 10.04 years; *M* lesion volume <sup>=</sup> 35.27 <sup>±</sup> 32.19 cm3). Of 19 brain tumor patients, 10 had single low-grade tumors (WHO◦II) and 9 had single high-grade tumors (WHO◦III-IV). Patients with brain tumors received radio- and/or chemotherapy. Brain tumor patients were tested at least 1 year after biopsy or maximum safe resection (*M* biopsy/resection = 3.95 ± 4.20 years, range = 1–16 years), and the whole group was tested at least 2 years since lesion onset (*M* onset = 5.00 ± 3.96 years, range = 2–15 years). Sixty-five percent of the 19 patients with brain tumors had no evidence for tumor recurrence and 35% had tumor recurrence in the same lesion site. Taking perifocal edemas into account, no lesion was bigger than 110 cm<sup>3</sup> (see **Table 1** for more detailed information about etiology, lesion laterality and tumor location).

All participants were informed about the risks of the study and signed a written informed consent prior to participation. This research was conducted with the approval of the ethical board of the University of Heidelberg according to the declaration of Helsinki. All participants were paid after finished assessment.

#### **NEUROPSYCHOLOGICAL AND AFFECTIVE ASSESSMENT**

All participants completed a brief screening of cognitive abilities, which included measures of cognitive flexibility (Trail Making Task A,B; Tombaugh, 2004); fluid intelligence using the *Culture-Fair-Intelligence-Test* (CFT-20; Cattell, 1960; Weiss, 1998); memory performance and processing speed (measured by the cognitive screening of the German Aphasia-Check-List ACL; Kalbe et al., 2002, 2005) and behavioral inhibition (Go/Nogo task from the German Attention Test Battery TAP; Zimmermann and Fimm, 2002). The Beck Depression Inventory (BDI-II; Kuhner et al., 2007) was included for the assessment of depressive symptoms (for more information see the Supplementary Material).

#### **STIMULUS MATERIAL**

A set of 20 negative and 20 neutral pictures was selected from the *International Affective Picture System* (IAPS; Lang et al., 2005). The selected negative pictures mainly contained unpleasant images of injured or mutilated persons, violent situations and diseases (*M* arousal = 6.91 ± 0.30; *M* valence = 2.01 ± 0.53), while the neutral pictures showed mainly ordinary home objects (*M* arousal = 2.69 ± 0.52; *M* valence = 4.98 ± 0.23). Neutral and negative pictures differed significantly in arousal [*t*(26) = −22*.*96; *p <* 0*.*001] and valence ratings [*t*(30*.*58) = 31*.*59; *p <* 0*.*001)].

#### **REAPPself TASK**

The task consisted of three conditions: the neutral condition, in which participants had to watch neutral pictures (LNeu); the negative condition, in which participants had to watch negative affective pictures (LNeg); and the REAPPself condition, in which participants had to decrease the triggered negative emotions by means of REAPPself during negative picture presentation (Dec). Sixty pseudo-randomized trials were presented using the *Presentation experiment driver* (www*.*neurobs*.*com). A typical trial started with a white fixation cross on a black background, which was presented for 2 s. Afterwards, the instruction was presented for 4 s: either "LOOK" (solely look at the picture without trying to manipulate the induced emotion) or "DISTANCING" (e. g. "It is a newspaper picture, and I am not involved"). Then, a brief fixation cross was presented again, followed by the negative emotional picture, presented for 6 s. Participants had to process the pictures according to the instructions. Self-assessment manikin (SAM Ratings; Bradley and Lang, 1994) were presented directly afterwards. Here, participants had to spontaneously rate the amount of emotional arousal as well as how displeasing the emotion induced by the previously presented picture was (valence); patients rated on a 1–9 scale. As the task was originally design for fMRI, a random jitter (6–9 s) relax trial appeared on the screen afterwards (Amaro and Barker, 2006). For a graphical description of the experimental order see **Figure 1**.

# **PROCEDURE**

The assessment was separated into two sessions, with less than a week between assessments. The first session comprised cognitive and affective assessment, whereas the second session involved the magnetic resonance imaging (MRI) scan and the REAPPself fMRI-task. In the current article, only behavioral results are reported.

First, the examiner provided the reappraisal task instructions in a written form. Then, participants were confronted with negative pictures of the IAPS while becoming clear instructions of REAPPself: to decrease the induced negative emotion by perceiving the content of the stimuli in a detached, third-person perspective (as viewing the picture in a newspaper; e.g., Ochsner et al., 2004; Lang et al., 2012). Thus, participants had to think, for example, that the event showed in the picture occurs in a


#### **Table 1 | Brain-damaged patient sample.**

*f, female; m, male.*

place far away or is randomly seen in the newspaper. Further, they informed the examiner by saying out loud how they detached. Subsequently, participants were trained in REAPPself by performing three practice trials. The session ended after ensuring that the participant understood the task procedure and applied REAPPself properly. In the second session, four more practice trials were conducted in the MRI scanner, in order to ensure that the participants felt comfortable. After the investigator was sure that the task was understood, participants began with the task assessment, which was separated into two runs of 30 trials each.

#### **STRUCTURAL IMAGING RECORDINGS AND LESION ANALYSIS**

Structural images were obtained on a 3 Tesla MRI scanner system (MAGNETOM Trio, Siemens Medical Systems, Erlangen, Germany) equipped with a 32-channel head coil. All brains were visualized with high resolution scans, which were acquired using a T1-weighted flash 3D sequence (*TR* = 1680 ms; *TE* = 2*.*6 ms; voxel size = 1.1 × 1.1 × 1.1 mm). Individual edema and tumor tissue were traced from T2-weighted FLAIR anatomical scans (*TR* = 9000 ms; *TE* = 95*.*0 ms; voxel size = 0.9 × 0.9 × 4.0 mm) by a radiologist blind to task performances using MRIcron software (Rorden et al., 2007). Using a procedure endorsed by Crinion et al. (2007), the T2 scans with the identified lesions were co-registered to the T1-weighted scans. Finally, the T1 scans were normalized to standardized MNI-space via *Unified segmentation and normalization* of the MATLAB toolbox *Statistical Parametrical Mapping* (SPM8; Crinion et al., 2007; Seghier et al., 2008). The Montreal Neurological Institute (MNI) brain standardized lesions were used to estimate lesion sites on aal templates of the MRIcron software (www*.*mricro*.*com/mricron) and to create lesion overlap images.

#### **VLSM ANALYSIS**

The first analysis was run in VLSM (Bates et al., 2003) in order to explore which regions are associated with impairments of REAPPself ability. The input variables were the operationalization of the REAPPself ability (i.e., the amount of down-regulation), which comprised the subtractions in arousal and valence ratings from "LOOK" and "DISTANCING" conditions (LNeg-Dec). Then, a series of Brunner-Munzel (BM) *t*-tests at every voxel were run in order to compare the input variables in patients with and without a lesion in the voxel. Based on the results (*p <* 0*.*05), a colorized map was generated, showing which lesioned region/s is/are associated with poorer performance. For instance, if patients with a lesion in specific voxels show significantly poorer REAPPself ability, then the region in which these specific voxels are located would be visualized in the statistical map. We also generated a map to determine the distribution of statistical power for our sample, based on an effect size of 0.8 (Kimberg et al., 2007) and an alpha level of 0.05, which shows voxels with enough power to detect significant differences. As shown in **Figure 2**, mostly right PFC and left superior PFC areas had adequate power. For instance, predictions for the VLSM analysis were restricted to these regions. In order to prevent spurious results, solely voxels in which a minimum of three patients was affected were included (analogously to Tsuchida et al., 2010; Tsuchida and Fellows, 2012). Significant results were overlaid to an MRIcron template (http://www*.*mccauslandcenter*.* sc*.*edu/mricro/mricron/) in order to identify involved Brodmann areas (BA).

#### **VLSM REGION-OF-INTEREST (ROI) GROUPWISE COMPARISON**

The second analysis depicts group comparisons between patients presenting a lesion in the overlap area calculated by VLSM (ROI), patients with a lesion sparing the ROI (IntactROI), and the healthy control (HC) group. Mean arousal and valence ratings were included as dependent variables in 3 *groups*(ROI, IntactROI, HC) × 2 *tasks* (LNeg, Dec) repeated measures ANOVA, followed by *post-hoc* Tukey HSD pair-wise comparisons. For relevant control variables, we used the Kruskal-Wallis-Test. Variables differing significantly between groups were included as covariate in the ANOVA.

# **VBM ANALYSIS**

Images for control subjects were preprocessed for VBM analysis using DARTEL Toolbox and followed procedures previously described (Ashburner and Friston, 2000). Following, the modulated images were smoothed with a Gaussian 12 mm full-width half-maximum kernel as suggested in other reports (Good et al., 2001) and normalized to the MNI standard space. Finally, these images were analyzed within different general linear models in SPM-8 2nd level analyses (http://www*.*fil*.*ion*.*ucl*.*ac*.*uk/spm/ software/spm8). These consisted of multiple linear regressions accounting for total intracranial volume, age and gender as noninterest or nuisance covariates. First, two whole brain analyses were performed: (1) for arousal differences (LNeg-Dec) and (2) for valence difference scores (LNeg-Dec) of REAPPself ability. The analyses were performed and examined at *p <* 0*.*001, two-tailed uncorrected threshold. Second, in order to assess the specific regional pattern of gray matter involved in each domain, two linear regressions were performed within the VLSM depicted ROI of the patients group as a small volume correction.

# **RESULTS**

### **SOCIODEMOGRAPHIC, NEUROPSYCHOLOGICAL, AND AFFECTIVE ASSESSMENT**

The independent *t*-test revealed no significant differences between healthy controls (*n* = 23) and patients (*n* = 20) in age

**FIGURE 2 | Three-dimensional and multislice views of voxels (yellow), where there is sufficient statistical power to detect an effect of lesion on behavior.**

[*t*(41) = −1*.*64, *p* = 0*.*11], but significant differences in educational level [*t*(26*.*81) = 2*.*37, *p* = 0*.*03], depression [*t*(20*.*13) = − 5*.*05, *p <* 0*.*001] and fluid intelligence scores [*t*(37) = 2*.*77, *p* = 0*.*01] (see **Table 2**). Regarding the neuropsychological screening, patients differed significantly from the healthy control performance in processing speed assessed by phonemic verbal fluency [*t*(41) = 3*.*49, *p* = 0*.*001], short-term memory [*t*(29*.*97) = 2*.*92, *p* = 0*.*007], behavioral inhibition assessed by the number of Go/NoGo errors [*t*(19*.*87) = −2*.*70, *p* = 0*.*01] and cognitive flexibility [*t*(31*.*51) = −2*.*49, *p* = 0*.*02]. The cognitive profile of the patient group is consistent with several descriptions of performance on individuals with frontal lobe lesions (see **Table 3**; Dimitrov et al., 2003; Niki et al., 2009; Rodriguez-Bailon et al., 2012).

#### **GROUP COMPARISONS IN EMOTIONAL REACTIVITY AND REGULATION**

In order to examine the differences on emotional reactivity, we subtracted the arousal and valence ratings of LNeu conditions from LNeg. The independent *t*-test with both groups showed that on average, patient and control groups did not significantly differ in emotional reactivity, neither for arousal [*t*(29*.*98) = 0*.*72, *p* = 0*.*48], nor for valence [*t*(41) = −0*.*52, *p* = 0*.*61].

Following, to investigate whether patients and HC differed significantly in REAPPself ability, we ran two repeated-measures ANOVAs with arousal and valence ratings of LNeg and Dec *tasks* as within-subject factors, and *group* (patients vs. HC) as between factors. Given that depressive symptoms (arousal: *r* = 0*.*43; *p* = 0*.*006, valence: *r* = 0*.*47; *p* = 0*.*002), fluid intelligence (arousal: *r* = −0*.*34; *p* = 0*.*04), short-term/immediate memory (arousal: *r* = −0*.*35; *p* = 0*.*02, valence: *r* = −0*.*36; *p* = 0*.*02) and inhibition deficits (arousal: *r* = 0*.*31; *p* = 0*.*05) significantly correlated with arousal and valence ratings in the Dec condition, we included each of these variables as covariates in an ANOVA design of arousal and valence ratings, and tested their significance regarding interaction effects with the dependent variables. It is important to mention that the variables did not correlate with the ratings in the LNeg condition. No significant interaction effects with the dependent variables were found. Only the depression (BDI) scores reached significant main effects as a covariate for the arousal [*F*(1*,* 38) = 5*.*57; *p* = 0*.*02] and valence [*F*(1*,* 38) = 7*.*27; *p* = 0*.*01] ratings. So, controlling for depression, there was no main effect for group [arousal: *F*(1*,* 38) = 0*.*54, *p* = 0*.*47; valence: *F*(1*,* 38) = 0*.*70, *p* = 0*.*41] or interaction effects *group* x *task* [arousal: *F*(1*,* 38) = 0*.*23, *p* = 0*.*63; valence: *F*(1*,* 38) = 1*.*41, *p* = 0*.*24], whereas effects of task were highly significant [arousal: *F*(1*,* 38) = 63*.*18, *p <* 0*.*001; valence: *F*(1*,* 38) = 58*.*23, *p <* 0*.*001]. Thus, all participants were able to down-regulate the emotional valence of the pictures (see **Figure 3**). Interestingly, by excluding the depression covariate from the analysis, the valence ratings showed significant interaction effects *group* x *task* [*F*(1*,* 41) = 4*.*38,

#### **Table 2 | Demographic variables of brain-damaged patients and healthy controls.**


*BDI, Becks Depression Inventory; CFT, Culture Fair Intelligence Test; f, female.*

#### **Table 3 | Summarized performance results on selected neuropsychological screening tests for brain-damaged patients and healthy controls.**


*aIndependent t-test results, bCognitive assessment section of the Aphasia Check List test battery; SD, standard deviation; ACL, Aphasia Check-List; TMT, Trail Making Task.*

*p* = 0*.*04], reflecting that the valence-related down-regulation was influenced by depressive symptoms.

#### **BRAIN-DAMAGED PATIENTS' GRAY MATTER RESULTS**

The VLSM analysis of the arousal-related REAPPself ability scores showed the involvement of a region in SFG overlapping the right dlPFC (BA9) and the right dACC (BA32). The valence-related REAPPself ability did not show any cortex involvement. The statistical map generated by the BM-test on each voxel is shown in **Figure 4A**. The color scale indicates BM-test *Z*-scores. It is important to mention that no voxel survived correction for multiple comparisons using conventional false discovery rate (FDR) thresholds (e.g., Tsuchida et al., 2010). However, the power map in **Figure 2** shows that this region in the right SFG had adequate power to detect effects at the uncorrected threshold depicted in **Figure 4A**. Nevertheless, the statistical map of the BM-test should be interpreted with caution, because of the risk of false-positive findings. We conducted a ROI-based analysis to further analyze the functional effects of lesions in the depicted voxels (Kimberg et al., 2007). For this purpose, we focused on the brighter regions showing results thresholded with a cut-off value of *Z >* 2*.*5.

#### **ROI GROUPWISE COMPARISON**

The patient sample was subsequently divided into two groups. The ROI group consisted of five patients presenting a lesion overlapping the region of the computed VLSM analysis (*N* = 5; see **Figure 4B**). The IntactROI group consisted of patients presenting a lesion sparing this region (*N* = 15; see **Figure 4C**). Compared with the HC in a Kruksall-Wallis-test, groups showed no significant differences in age (*p* = 0*.*22), education level (*p* = 0*.*21), or lesion volume (*p* = 0*.*13). However, groups differed significantly in depressive symptoms (*p <* 0*.*001; ROI*>*IntactROI*>*HC), fluid intelligence (*p* = 0*.*01; ROI*<*IntactROI*<*HC), phonemic verbal fluency (*p* = 0*.*01; ROI*<*IntactROI*<*HC), short-term memory (*p* = 0*.*02; ROI*<*IntactROI*<*HC), and inhibition deficits, as assessed by the number of Go/Nogo errors (*p* = 0*.*002; ROI*>*IntactROI*>*HC). Interestingly, the covariance main effect of depressive symptoms was significant for valence rating scores [*F*(1*,* 37) = 4*.*27; *p* = 0*.*05], but not for arousal scores [*F*(1*,* 37) = 2*.*71; *p* = 0*.*11]. However, no significant interaction effects BDI x *task* [arousal: *F*(1*,* 38) = 0*.*001, *p* = 0*.*98; valence: *F*(1*,* 37) = 0*.*15, *p* = 0*.*71] and no significant correlations with REAPPself ability scores were found (arousal: *r* = −0*.*20, *p* = 0*.*21; valence: *r* = − 0*.*30, *p* = 0*.*06), so that they were not included in the analysis. In addition, the Kruksal-Wallis test showed that the groups did not significantly differ in emotional reactivity scores (LNeg-LNeu; arousal: *p* = 0*.*54; valence: *p* = 0*.*87).

Rating values demonstrated significant overall main effects of *group* [arousal: *F*(2*,*40) = 6*.*07, *p* = 0*.*005; valence: *F*(2*,*40) = 4*.*93, *p* = 0*.*01] and *task* [arousal: *F*(1*,* 40) = 94*.*35, *p <* 0*.*001; valence: *F*(1*,* 40) = 80*.*65, *p <* 0*.*001]. A significant interaction effect *group* × *task* of arousal ratings [*F*(2*,* 40) = 3*.*28, *p* = 0*.*05] demonstrated that patients with a lesion in the ROI could not down-regulate arousal induced by negative emotions in the same manner as the two other control groups, as shown in **Figure 5** (see also Tukey comparisons in **Table 4**). In other words, negative arousal in the Dec condition was significantly higher for the ROI group compared to the other control groups. In addition, valence *group* × *task* interactions were marginally significant [*F*(2*,* 40) = 3*.*07, *p* = 0*.*06]. Results of Tukey's test for multiple comparisons revealed that arousal and valence rating scores were significantly higher for the ROI group than the two other groups, which did not differ significantly in arousal and valence scores (*p* = 0*.*99; see **Table 5**). These results indicate a more successful down-regulation of negative emotions for the two control groups (REAPPself ability), but less successful for the ROI group. Regarding the valence ratings, this difference might be significantly influenced by the amount of depressive symptoms of the ROI group.

#### **HEALTHY CONTROLS' GRAY MATTER RESULTS** *Whole brain analyses*

In the whole brain analyses, it was found that REAPPself ability scores (LNeg-Dec) had a positive correlation with gray matter

**FIGURE 5 | Subjective ratings of arousal and valence after look and decrease conditions for each group.** There was a significant group difference of decrease scores showing that the ROI-group was the less successful in down-regulating negative emotions. ∗*p <* 0*.*05.


#### **Table 4 | Multiple comparisons (Tukey HD** *post-hoc* **tests) of arousal and valence ratings of ROI injured, ROI intact patients, and healthy controls.**

#### **Table 5 | Whole brain Patterns of GM intensity correlated with task performance in controls.**


of a series of cortical and subcortical structures in the right and left hemispheres: at the frontal lobe, more specifically the right SFG (BA 9-32; see **Figure 6A**), left insula, basal ganglia and mid temporal gyrus, and bilateral cerebellum (See **Table 5A** for MNI coordinates). For the valence (LNeg-Dec) domain, this relation appeared at the left SFG (BA 9-32; see **Figure 6B**), right SFG, left mid and inferior frontal gyri, temporal cortex, parietal cortex, basal ganglia and cerebellum (See **Table 6B** for MNI coordinates). No significant negative correlations were found.

#### *Regional brain analyses*

Similar results were found in the regressions done at the patient's lesion ROI. HC showed greater gray matter amount in the right SFG (BA 9-32) for higher arousal-related REAPPself ability scores (LNeg-Dec; see **Figure 7A**), as well as for the valence domain (see **Figure 7B**). The MNI coordinates are presented in **Table 6**.

In summary, following results were obtained: First, we found an association between deficits in REAPPself ability performance and a dlSFG area overlapping the right SFG (BA9/32) using exploratory VLSM analysis. Second, repeated-measures ANOVA confirmed that the ROI group was less successful in regulating negative emotional responding (i.e., poorer REAPPself ability) compared to IntactROI and HC groups. Third, regional VBM analyses of the mentioned areas in the HC revealed that REAPPself ability was positively related to BA9/32 gray matter intensities.

# **DISCUSSION**

The cognitive regulation of negative emotions is crucial for mental health, yet there is a lack of lesion studies investigating reappraisal. Therefore, the goal of this study was to identify critical PFC regions for reappraisal ability by analyzing the performance

of brain-damaged patients and healthy participants. Taking in consideration the cognitive difficulties of the patients, we thoroughly trained REAPPself assuring that the participants apply the proper strategy. VLSM analysis among the patients and VBM analysis of the healthy control's gray matter showed that specific regions of the right SFG including the dlPFC (BA9) and the dACC (BA32) might be indispensable for REAPPself. Furthermore, ROI-based group comparisons supported the results, demonstrating that a lesion located in the mentioned areas significantly impaired down-regulation of negative arousal. To the best of our knowledge, the current study is the first lesion study using neuroimaging methods for the identification of circumscribed brain regions indispensable for the REAPPself ability.

#### **THE SFG AND REAPPRAISAL OF NEGATIVE STIMULI**

In line with previous investigations, the current study linked down-regulation of negative emotions by REAPPself to regions near the mPFC (Ochsner et al., 2004). Our results were also in


**Table 6 | Patterns of regional GM intensity correlated with task performance in controls.**

*VBM, voxel-based morphometry; BA, Brodmann area; R, right; L, left; GM, gray matter; FDR, false-discovery rate correction.*

accordance with latest fMRI data, as six from the 7 fMRI studies investigating down-regulation of negative emotions by REAPPself (as classified by Ochsner et al., 2012) have consistently demonstrated the involvement of areas in the depicted SFG, including the dlPFC (Erk et al., 2010), dorsomedial PFC (Schardt et al., 2010), and dACC (Koenigsberg et al., 2010; Lang et al., 2012). Given that the first descriptive comparison between HC and the whole patient group showed no significant differences, the current ROI groupwise comparisons demonstrate that a lesion in the target region was indeed crucial for REAPPself, especially for the down-regulation of arousal. Interestingly, a previous study has discussed that REAPPself might be particulary effective in the down-regulation of physiological responding (Shiota and Levenson, 2012).

Current results can be interpreted in the frame of a cognitive control of emotions based on anatomical connections between (1) ACC, insula and basal ganglia (Ongur and Price, 2000; Ibanez et al., 2010; Ibanez and Manes, 2012), (2) amygdala-OFC-ACC (Carmichael and Price, 1995; Cavada et al., 2000) as well as (3) dlPFC and basal ganglia (Heekeren et al., 2008). The SFG (BA6/BA8/BA9/BA32) can be divided in an anteromedial (amSFG), a dorsolateral (dlPFC) and a posterior region (pSFG), and is also supposed to be involved in several cognitive control tasks (du Boisgueheneuc et al., 2006; Moreno-Lopez et al., 2012; Li et al., 2013). Moreover, the SFG is highly interconnected, with pathways extending to the ACC, middle frontal gyrus, inferior frontal gyrus (IFG) as well as thalamus and caudate nucleus in the basal ganglia. This important PFC region lies in a unique position between emotional limbic regions and highly cognitive and executive process networks in the dorsal and medial areas of the PFC (Li et al., 2013). The dACC is also one of the few PFC areas that presents strong projections to amygdala nuclei, and might be the cue connection between prefrontal executive and limbic emotional areas during ER (Ray and Zald, 2012). Furthermore, our findings support previous studies in showing that the activation of dACC regions predicts cognitive reappraisal success (Ochsner et al., 2002). This assumption has been supported by *neurofeedback* techniques, in which the down-regulation of emotion-related insula activity was accompanied by the right SFG including ACC (BA32) involvement during reappraisal of threat-related stimuli (Veit et al., 2012). Moreover, conscious self-regulation of brain activity (e.g., right SFG top down control) may depend on an interaction with unconscious subcortical processes, involving not only emotional (amygdala) but also motor skill learning (basal ganglia) as shown in recent models of neurofeedback (Birbaumer et al., 2013).

#### **RIGHT SFG LESION AND REAPPself IMPAIRMENTS**

Several studies have demonstrated that the ROI depicted by our VLSM results (including the right dACC) is relevant not only for REAPPself, but also for inhibitory control, an executive function that excludes irrelevant information from WM in order to prevent undesirable behavioral responses (Garavan et al., 1999; Vanderhasselt et al., 2012). Thus, REAPPself of negative stimuli may imply the inhibition of dominant negative thoughts, permitting a detached third-person perspective. For instance, a recent study showed that a habitual reappraisal use is positively associated with the ability to inhibit dominant thoughts to negative cues (Vanderhasselt et al., 2013). Similarly, Salas et al. (2014) present a reappraisal model, in which behavioral inhibition is presented as an essential skill for reappraisal generation (Salas et al., 2014). Accordingly, our results showed that inhibition failures during the Go/NoGo task, assessed by the number of errors, were positively correlated with the raw scores of arousal in the Dec condition. Moreover, the ROI group not only showed deficits in REAPPself, but was also the group with most inhibition failures (number of errors) during Go/NoGo tasks. This is of particular interest, as one case study reported that inhibition impairments after a left frontoparietal lesion generated difficulties to spontaneously generate reappraisals (Salas et al., 2013). As previously shown in lesion studies, patients with right PFC lesions typically show inhibition difficulties, reflected by increased error rates in the interference condition of the Stroop task (Vendrell et al., 1995). Furthermore, studies investigating lesions of the right ACC and dlPFC regions report inhibition deficits (Turken and Swick, 1999; Swick and Turken, 2002), rule breaking and difficulties in strategy planning (Burgess et al., 2000). Considering this findings, it would be expectable to spot more right lateralized regions comprising overlapping areas for cognitive inhibition as being crucial for REAPPself (i.e., detaching) of negative events, which in turn, implies the inhibition of negative meanings. However, as inhibition failures (number of Go/NoGo errors) did not significantly correlate with the REAPPself ability scores (LNeg-Dec), our findings support the reappraisal model of Salas et al. (2014). That is, inhibition failures might have influenced reappraisal generation (Salas et al., 2014). As Go/NoGo errors were positively associated with the amount of negative arousal in the Dec condition (i.e., the more errors, the more arousal), it might be interpreted that automatic negative meaning of the stimuli could not be inhibited appropriately. For instance, our results support the findings of McRae et al. (2012a,b), who did not find significant associations of general reappraisal ability with response inhibition, but with WM capacity (McRae et al., 2012b). Therefore, our results revealed that REAPPself might not be dependent only on dACC and inhibition, but also on WM process and strategy planning functions that rely on dlPFC areas (Heyder et al., 2004; Kaller et al., 2011, 2013).

A previous fMRI study examining the contribution of PFC areas in ER shows that the right dlPFC is strongly involved in reappraisal function, regardless of the kind of stimuli that are reappraised (Golkar et al., 2012). This is not surprising, as reappraisal relies on executive functions that update emotional to new non-emotional "reappraised" thoughts and maintain these reinterpretations in mind (Malooly et al., 2013). The displayed areas in the dlSFG region enclose the dlPFC areas in BA9, which have not only been linked to WM and executive processes, but also with metacognitive evaluations of oneself and others, particularly the right dlPFC (Schmitz et al., 2004). That is, the right PFC might be recruited when self-evaluations are produced. For REAPPself, a self-focused strategy, evaluations about the self might be essential. Moreover, since a part of the identified ROI is placed in the white matter of the PFC (between BA9 and BA32), REAPPself ability might also be dependent on the interaction and connectivity of the mentioned areas. Further analysis of SFG connectivity and the influence on REAPPself might clarify these issues.

Interestingly, the ROI group showed not only more inhibition deficits, but also more immediate memory recall and processing speed deficits, as well as lower fluid intelligence scores as the IntactROI and HC group. Therefore, a lesion in the depicted region might lead to other cognitive impairments, besides of those of REAPPself and inhibition. These variables showed no significant correlations with REAPPself ability scores, but with the raw arousal and valence scores of the Dec condition. In other words, these cognitive variables had a significant influence on the amount of negative emotion during reappraisal. Here, results showed that cognitive abilities such as fluid intelligence and immediate memory, correlated negatively with arousal and valence scores during the Dec condition. This result might support the assumption of less negative affect through heightened cognitive control abilities (Williams et al., 2009).

#### **ROI LESION AND DEPRESSION SYMPTOMS**

The current findings show that the ROI group suffered more from depression than the two other groups. These results are in agreement with previous studies investigating lesions in the right hemisphere, which also show associations with impaired affective processing, reflected by the presence of anxiety and depression (Berg et al., 2000; Zorzon et al., 2001, 2002). In addition, our results support the findings of Königs and colleagues, which demonstrated increased vulnerability for depression after a bilateral dlPFC lesion (Koenigs et al., 2008). Apart from this, depression was also positively associated with the amount of negative emotionality during the Dec condition. Depression and other affective disorders have been related to inhibition failures of negative stimuli (Joormann and Gotlib, 2008; Joormann, 2010) and to impaired reappraisal (Johnstone et al., 2007; Moore et al., 2008; Ehring et al., 2010). For these reasons, depression might reflect a confounder; particularly for the analysis of valence rating scores (were depression was a significant influence). This finding might lead to the assumption that the depicted right SFG region might be important for a valence-related REAPPself ability, but probably not indispensable, as it might be with an arousal-related REAPPself ability. In addition to the lesion, depression might have influenced the subjective down-regulation of negative valence.

#### **VBM RESULTS OF GRAY MATTER IN HEALTHY CONTROLS**

Our whole-brain VBM findings showed that partially different neural structures were correlated with arousal- and valencerelated REAPPself ability. For arousal REAPPself, we found positive associations with more subcortical regions as the insula, whereas valence was associated with highly cognitive areas as the middle and inferior frontal gyrus, as well as with the inferior parietal lobule. Although both of the constructs are assumed to be difficult to separate in the subjective experience (Kuppens et al., 2013), the obtained results lead to the assumption that arousal down-regulation comprise the involvement of limbic regions mainly related to emotional awareness and physiological responding, whereas valence down-regulation is a more elaborated process, in which highly cognitive regions are involved (Citron et al., 2014). Accordingly, insula activity has been consistently observed during changes in autonomic arousal (Critchley et al., 2002, 2003). However, in studies examining the evaluation of valence, more cortical, attentional structures are observed (Kensinger, 2004; Kensinger and Corkin, 2004). In addition, the SFG was significantly associated with the down-regulation of both, arousal, and valence self-reports. This was confirmed by the regional VBM analysis, were right SFG regions showed significant positive correlations with REAPPself ability. These findings support previous evidence showing that the anatomical volume of ACC (BA32) is positively associated with a cognitive reappraisal ability (Giuliani et al., 2011).

# **IMPLICATIONS FOR FURTHER RESEARCH**

Summarizing, the involvement of SFG regions during reappraisal of negative stimuli has been strongly underlined in the majority of fMRI studies, and included in several reappraisal models (Ochsner et al., 2002, 2004, 2012; Wager et al., 2008; Koenigsberg et al., 2010; Buhle et al., 2013). Inhibition performance, which is supposed to be a right lateralized function (Garavan et al., 1999), might influence the ability of decreasing automatic negative appraisals, thus constituting a corner pillar for the architecture of reappraisal and especially, REAPPself (Salas et al., 2013, 2014). However, we cannot rule out the influences of depression symptoms, particularly on the down-regulation of valence. Although inhibition, depression and impaired REAPPself ability are strongly associated (Joormann and Gotlib, 2008; Joormann, 2010; Aldao and Nolen-Hoeksema, 2012; Barnow et al., 2013), additional research is needed to explain the direction of these associations with a bigger sample. Analyses about these associations are, unfortunately, outside the scope of this work. Nevertheless, to gain further insight into the effects of a trained modulation of the right SFG (specifically dACC and dlPFC areas) on REAPPself performance and their related limbic responses might be useful in the treatment of various psychiatric disorders involving emotional dysregulation. Furthermore, it would be of great interest to examine whether teaching patients to gain control over the neural activity (right SFG and related subcortical networks) via neurofeedback would yield positive therapeutic effects.

# **LIMITATIONS**

No study is free of limitations or possible improvements. In the present study, the sample size was not large enough to reach significance with conventional multiple-measure corrections (see for example Medina et al., 2010). Therefore, our VLSM results have to be interpreted with caution due to the risk of false-positive findings. Additional research should replicate these findings with a larger sample of patients. However, the ROI based group comparison supported the VLSM results. Furthermore, although we were very conservative in lesion reconstruction, we cannot rule out the possibility of etiology and treatment confounders. We therefore controlled for the lesion volume in the statistical analysis, and it was not significantly different between groups. However, we have to take the influence of cortical reorganization of functions by slow-growing tumors into consideration (Desmurget et al., 2007). It is also important to mention that the infiltration pattern of brain tumors is diffuse *per se* and generally difficult to assess. The growth of such a tumor results in T2-weighted hyperintense signal alteration; the current methods in MRI make it impossible to differentiate between tumor and perifocal edema, as both features may lead to the MRI pattern (Essig et al., 1998). However, the employed T2-FLAIR sequence is regarded to be one of the most sensitive MRI sequences to detect the extensiveness of damaged brain tissue. Thus, by considering the whole T2 hyperintensity, the analysis was performed conservatively, as this type of segmentation includes the maximum area of damaged brain tissue, detectable with current methods. The highly educated control group might also represent a source of bias, although this variable did not significantly correlate with any of the outcome variables and no significant differences in demographic variables were found in the ROI group-wise comparison. Finally, although previous investigations studying reappraisal function in brain-damaged patients argue that reappraisal ability might be a problematic variable to measure due to the potential cognitive impairments of brain-damaged patients (Salas et al., 2014), our results show that these reappraisal difficulties might be dependent of the localization of the lesion (right SFG), as the descriptive patients-HC group comparison did not show any significant results in reappraisal ability (except for the significant influence of depression).

#### **CONCLUSIONS**

Considering our limitations, it is safe to conclude that the integrity of the right dACC and dlPFC might be of crucial importance not only for REAPPself ability, but also for affective and cognitive health. To the best of our knowledge, the current work is the first lesion study on cognitive reappraisal that targets circumscribed brain regions using imaging methods. It brings useful insights in the importance of specific right SFG areas for REAPPself. These findings might have important implications for studies using real-time fMRI techniques (Decharms et al., 2004). It would be of great interest to investigate whether the conscious modulation of right SFG BOLD activity could influence limbic responses using neurofeedback methods. The development of evidence-based neurofeedback trainings would be of prime importance in patients suffering from emotional dysregulation, depression, and other types of psychopathology.

# **ACKNOWLEDGMENTS**

This project was supported by the Neuro-oncology section of the University Hospital and the German Cancer Research Center in Heidelberg, Germany. Agustin Ibanez is supported by grants CONICYT/FONDECYT Regular 1130920 and 1140114, FONCyT-PICT 2012-0412, FONCyT-PICT 2012- 1309, CONICET and the INECO Foundation. The authors are very thankful for the professional collaboration of Dr. Benedikt Wiestler. We would also like to thank Adelheid Fuxa and Moritz Riese for their friendly assistance.

# **SUPPLEMENTARY MATERIAL**

The Supplementary Material for this article can be found online at: http://www.frontiersin.org/journal/10.3389/fnbeh. 2014.00165/abstract

# **REFERENCES**


lesion-symptom mapping studies. *Neuropsychologia* 48, 341–343. doi: 10.1016/ j.neuropsychologia.2009.09.016


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 20 January 2014; accepted: 18 April 2014; published online: 09 May 2014. Citation: Falquez R, Couto B, Ibanez A, Freitag MT, Berger M, Arens EA, Lang S and Barnow S (2014) Detaching from the negative by reappraisal: the role of right superior frontal gyrus (BA9/32). Front. Behav. Neurosci. 8:165. doi: 10.3389/fnbeh.2014.00165 This article was submitted to the journal Frontiers in Behavioral Neuroscience.*

*Copyright © 2014 Falquez, Couto, Ibanez, Freitag, Berger, Arens, Lang and Barnow. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Corrigendum: Detaching from the negative by reappraisal: the role of right superior frontal gyrus (BA9/32)

#### *Rosalux Falquez <sup>1</sup> \*, Blas Couto2,3, Agustin Ibanez 2,3,4, Martin T. Freitag 5, Moritz Berger 5, Elisabeth A. Arens 1, Simone Lang1 and Sven Barnow1*

*<sup>1</sup> Department of Clinical Psychology and Psychotherapy, Institute of Psychology University of Heidelberg, Heidelberg, Germany*

*<sup>2</sup> Laboratory of Experimental Psychology and Neuroscience (LPEN), Institute of Cognitive Neurology (INECO), Favaloro University, Buenos Aires, Argentina*

*<sup>3</sup> Diego Portales University, UDP-INECO Foundation Core on Neuroscience (UIFCoN), Santiago, Chile*

*<sup>4</sup> Departamento de Psicología, Universidad Autónoma del Caribe, Barranquilla, Colombia*

*<sup>5</sup> Department of Radiology, German Cancer Research Center (DKFZ), Heidelberg, Germany*

*\*Correspondence: rosalux.falquez@psychologie.uni-heidelberg.de*

#### *Edited and reviewed by:*

*Sergio Ruiz, Pontificia Universidad Catolica de Chile, Chile*

**Keywords: right SFG, reappraisal, VBM, VLSM, lesion**

## **A corrigendum on**

# **Detaching from the negative by reappraisal: the role of right superior frontal gyrus (BA9/32)**

*by Falquez, R., Couto, B., Ibanez, A., Freitag, M. T., Berger, M., Arens, E. A., et al. (2014). Front. Behav. Neurosci. 8:165. doi: 10.3389/fnbeh.2014.00165*

We noticed an error in one of our presented analyses. One mismatched brain image was accidentally included in the voxel-based-morphometry (VBM) analysis. Thus, arousal and valence values were consecutively not properly assigned to the morphological brain data of the other included participants. Of three analyses implemented in this study (VSLM, ROI-based), only the VBM analysis was affected but the others remain untouched. Therefore, we re-conducted the whole VBM analysis with the correct allocation of data. The corrected results showed changes in the whole-brain and regional correlations compared to the originally presented results. However, the correlations in the expected areas of the original manuscript remain significant for arousal and valence difference scores.

Fortunately, these results do not impact the main implications and neither invalidate the conclusions derived from the study nor introduce differing directions of inference. The right superior frontal gyrus (SFG/BA9) and anterior cingulate cortex (ACC/BA32) remain significant at wholebrain *p <* 0*.*001 uncorrected level, and the ROI analysis still showed significant correlations with gray matter intensities in the right SFG (BA9).

The corrected results affect **Figures 6**, **7**, **Tables 5**, **6**, and small parts in results and discussion which are attached below.

The authors deeply regret this error and apologize for any confusion it might have caused.

# **RESULTS**

The sentence in the results section for the arousal scores

". . . more specifically the right SFG (BA 9–32; see **Figure 6A**), left insula, basal ganglia and mid temporal gyrus, and bilateral cerebellum (See **Table 5A** for MNI coordinates)."

should be replaced by:

". . . more specifically the right SFG (BA 9– 32, see **Figure 6A**), orbital gyrus, leftmiddle occipital gyrus, right angular gyrus, superior temporal gyrus, left rolandic operculum, right inferior parietal cortex and bilateral mid-frontal gyrus (See **Table 5A** for MNI coordinates)."

The sentence in the results section for the valence scores

"For the valence domain, this relation appeared at the left SFG (BA 9–32; see **Figure 6B**), right SFG, left mid and inferior frontal gyri, temporal cortex, parietal cortex, basal ganglia and cerebellum (See **Table 6B**, for MNI coordinates)."

should be replaced by:

"For the valence domain, this relation appeared at the left SFG (BA10; see **Figure 6B**), right SFG (BA9) including ACC (BA32), left mid and inferior frontal gyri, temporal cortex, parietal cortex, parahippocampal gyri and opercula (See **Table 6B**, for MNI coordinates)."

# **DISCUSSION**

The sentences in the discussion section: "For arousal REAPPself, we found positive associations with more subcortical regions as the insula, whereas valence was associated with highly cognitive areas as the middle and inferior frontal gyrus, as well as with the inferior parietal lobule. Although both of the constructs are assumed to be difficult to separate in the subjective experience (Kuppens et al., 2013), the obtained results lead to the assumption that arousal down-regulation comprise the involvement of limbic regions mainly related to emotional awareness and physiological responding, whereas valence down-regulation is a more elaborated process, in which highly cognitive regions are involved (Citron et al., 2014)."

should be replaced by:

"For arousal REAPPself, we found positive associations with the right orbital gyrus and left middle occipital gyrus whereas the down-regulation of both (valence and arousal) were associated with highly cognitive areas as the middle and inferior frontal gyrus, as well as with the inferior parietal lobule. Although both of the constructs are assumed to be difficult to separate in the subjective experience (Kuppens et al., 2013), the obtained results lead to the assumption that arousal down-regulation comprise the involvement of limbic regions mainly related to emotional awareness and multisensory

**FIGURE 7 | Graphic display of regional gray matter patterns of volume using the ROI depicted by the VLSM analysis (BA9/32) correlated with task performance in controls for (A) Arousal and (B) Valence rating differences (presented at a level of** *p <* **0***.***05 unc).**

#### **Table 5 | Whole brain Patterns of GM volume correlated with task performance in controls.**


*VBM, voxel-based morphometry; BA, Brodmann area; R, right; L, left; GM, gray matter.*

#### **Table 6 | Regional brain Patterns of GM volume correlated with task performance in controls.**


*VBM, voxel-based morphometry; BA, Brodmann area; R, right; L, left; GM, gray matter; FWE, family-wise error correction.*

integration (Ongur and Price, 2000). In contrast, the down-regulation of both arousal and valence is an elaborated process, in which highly cognitive regions are involved. These findings are in accordance with an earlier report, in which arousal and valence are described as separable dimensions (Citron et al., 2014)."

# **REFERENCES**


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 03 July 2014; accepted: 17 July 2014; published online: 22 August 2014.*

*Citation: Falquez R, Couto B, Ibanez A, Freitag MT, Berger M, Arens EA, Lang S and Barnow S (2014) Corrigendum: Detaching from the negative by reappraisal: the role of right superior frontal gyrus (BA9/32). Front. Behav. Neurosci. 8:264. doi: 10.3389/fnbeh. 2014.00264*

*This article was submitted to the journal Frontiers in Behavioral Neuroscience.*

*Copyright © 2014 Falquez, Couto, Ibanez, Freitag, Berger, Arens, Lang and Barnow. This is an openaccess article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Insula and inferior frontal triangularis activations distinguish between conditioned brain responses using emotional sounds for basic BCI communication

*Linda van der Heiden1,2 ‡, Giulia Liberati 2,3,4\*‡, Ranganatha Sitaram2,5,6, Sunjung Kim2, Piotr Jaskowski ´ 1†, Antonino Raffone3,7, Marta Olivetti Belardinelli 3,7, Niels Birbaumer 2,8 and Ralf Veit <sup>2</sup>*

*<sup>6</sup> Biomedical Engineering, Sri Chitra Tirunal Institute of Medical Sciences and Technology, Trivandrum, India*

*<sup>8</sup> Ospedale San Camillo—IRCCS, Istituto di Ricovero e Cura a Carattere Scientifico, Venezia Lido, Italy*

#### *Edited by:*

*Carmen Sandi, Ecole Polytechnique Federale de Lausanne, Switzerland*

#### *Reviewed by:*

*Giuseppina Rota, University of Pisa, Italy, Italy*

*James Sulzer, University of Texas at Austin, USA*

#### *\*Correspondence:*

*Giulia Liberati, Institute of Neuroscience, Université catholique de Louvain, Avenue Mounier 53 bte B1.53.02 à 1200 Woluwe-Saint-Lambert, Claude Bernard, Etage 02, Local 0826, Site Bruxelles Woluwe, Brussels, Louvain-la-Neuve, Belgium e-mail: giulia.liberati@uclouvainb.be*

*†Deceased 6 January 2011.*

*‡These authors have contributed equally to this work.*

In order to enable communication through a brain-computer interface (BCI), it is necessary to discriminate between distinct brain responses. As a first step, we probed the possibility to discriminate between affirmative ("yes") and negative ("no") responses using a semantic classical conditioning paradigm, within an fMRI setting. Subjects were presented with congruent and incongruent word-pairs as conditioned stimuli (CS), respectively eliciting affirmative and negative responses. Incongruent word-pairs were associated to an unpleasant unconditioned stimulus (scream, US1) and congruent word-pairs were associated to a pleasant unconditioned stimulus (baby-laughter, US2), in order to elicit emotional conditioned responses (CR). The aim was to discriminate between affirmative and negative responses, enabled by their association with the positive and negative affective stimuli. In the late acquisition phase, when the US were not present anymore, there was a strong significant differential activation for incongruent and congruent word-pairs in a cluster comprising the left insula and the inferior frontal triangularis. This association was not found in the habituation phase. These results suggest that the difference in affirmative and negative brain responses was established as an effect of conditioning, allowing to further investigate the possibility of using this paradigm for a binary choice BCI.

**Keywords: classical conditioning, emotions, fMRI, Insula, inferior frontal triangularis, BCI**

## **INTRODUCTION**

The possibility to produce two differentiable conditioned responses, corresponding to "affirmative" and "negative" thinking, could be exploited for basic yes/no communication through a brain-computer interface (BCI). This would allow individuals who are not able to use standard communication pathways (e.g., because of severe motor disability or expressive deficits) to convey information on their basic needs. BCIs are traditionally based on operant conditioning (Wolpaw et al., 2002), which may however be problematic for some users, such as completely locked-in state individuals (Birbaumer, 2006) or persons with dementia (Liberati et al., 2012a).

In the present study, we investigated the possibility to use classical conditioning to discriminate between different brain responses. The advantages of such paradigm shift is that, since classical conditioning does not require users to perform cognitively challenging tasks, it could be used for communicating with cognitively impaired patients (e.g., with dementia Liberati et al., 2012a,b).

Differential classical conditioning has been well studied and described over the last century. The basic principle of this learning mechanism is that a conditioned stimulus (CS+) is paired with an unconditioned stimulus (US), while another CS remains unpaired (CS−). The pairing of CS with US results in a conditioned response (CR; Pavlov, 1927) reflecting a new learned stimulus-response association. Many variations of this procedure have been applied, using different CS modalities and pleasant/appetitive or unpleasant/aversive US. Semantic classical conditioning refers to the conditioning of responses to meaningful words or sentences, irrespective of the specific letters or sounds that constitute the words (Razran, 1939, 1961). The repeated association of words or sentences with a significant US, such as a painful stimulus, produces a CR, i.e., measured at the level of cortical evoked responses (Montoya et al., 1996). Recently, semantic

*<sup>1</sup> Department of Cognitive Psychology, University of Finance and Management, Pawia, Warsaw, Poland*

*<sup>2</sup> Institute of Medical Psychology and Behavioral Neurobiology, Eberhard Karls-University, Tübingen, Germany*

*<sup>3</sup> Interuniversity Centre for Research on Cognitive Processing in Natural and Artificial Systems (ECONA), Rome, Italy*

*<sup>4</sup> Institute of Neuroscience, Université Catholique de Louvain, Brussels, Louvain-la-Neuve, Belgium*

*<sup>5</sup> Department of Biomedical Engineering, University of Florida, Gainesville, FL, USA*

*<sup>7</sup> Department of Psychology, University "Sapienza" of Rome, Rome, Italy*

conditioning experiments have been performed while electroencephalographic (EEG) potentials were measured (De Massari et al., 2012; Furdea et al., 2012; Ruf et al., 2013) using two different aversive auditory US or a single aversive electrical shock US. Up to now, semantic conditioning has not been investigated using functional magnetic resonance imaging (fMRI).

In the present fMRI study, we introduced a semantic double conditioning paradigm to condition two different responses, an affirmative and a negative one, in view of developing a BCI to allow basic yes/no communication. For this purpose, an innovative approach was tested in which congruent and incongruent word-pairs, eliciting respectively affirmative and negative thinking, were associated with two different emotional sounds, a pleasant and an unpleasant one.

Our aim was to assess whether it is possible to discriminate between the congruent and incongruent word-pairs, using a semantic conditioning protocol to condition two distinct brain responses at the same time, by associating two emotional stimuli to word-pairs presented aurally. We hypothesized that, after classical conditioning, the semantic stimuli would elicit differentially conditioned responses in emotion-related brain areas. More specifically, we expected that word-pairs associated with unpleasant emotional stimuli would elicit a negative emotional response (e.g., associated with ACC, insula and amygdala activation) (Büchel et al., 1998, 1999) and word-pairs associated with pleasant emotional stimuli a positive emotional response (e.g., associated with amygdala, hippocampus and prefrontal cortex activation) (Ito et al., 2005; Costa et al., 2010). In other words, we addressed two challenges, namely double semantic auditory conditioning using emotional sounds and—through this approach—the elicitation of distinctive brain responses, to enable basic yes/no BCI communication.

#### **MATERIALS AND METHODS**

#### **PARTICIPANTS**

Ten right-handed, native German speaking, healthy individuals (5 males, 5 females), ranging in age from 21 to 28 (mean age = 25.3, *SD* = 1*.*77 years), participated in this study. All participants gave written informed consent prior to participation in the fMRI experiment. The study was approved by the Ethics Committee of the Medical Faculty of the University of Tübingen and was performed in compliance with the Code of Ethics of the World Medical Association (Declaration of Helsinki).

#### **STIMULI**

The stimuli consisted of 300 German word-pairs, read aloud by a native German speaker, recorded using a SpeedLink USB microphone and QuickTime Player 7 program for Macintosh.

Each word-pair included a superordinate category (e.g., "animals") and a subordinate object (e.g., "dog"). Half of the wordpairs were congruent (e.g., "Obst-Apfel," "Fruit-Apple"), half incongruent (e.g., "Obst-Hund," "Fruit-Dog"). Congruence was given by the belonging of the object to the category. The categories that were used were Animals, Countries, Fruit, Furniture, Sports, Clothing, Instruments, Drinks, Crockery, and Jobs. The word-pairs were short (1.5 s) and simple, so that they did not represent a cognitive challenge. Word-pairs were chosen, instead of questions or sentences, as they could be more easily standardized in length, and could be also understandable by individuals with some degree of cognitive deficit. In fact, different studies have shown that even in presence of cognitive impairment, basic semantic information that may not be explicitly accessible could be relatively intact at the implicit level (Nebes, 1994; Laisney et al., 2011).

The brain responses associated to the congruent and incongruent word-pairs (associated respectively to "yes" and "no" thinking) constituted the conditioned stimuli (CS). The unconditioned stimuli (US) were two standardized emotional sounds drawn from the International Affective Digitized Sounds (IADS, Bradley and Lang, 1999; Stevenson and James, 2008): a pleasant emotional stimulus (a baby-laughter) and an unpleasant emotional stimulus (a scream). The duration of each US was also 1.5 s. To ascertain that all stimuli had the same precise length, their duration was adjusted using the software program Audacity 1.3.14 Beta for Mac OS X. As our final aim was communication, congruent word-pairs (eliciting affirmative thinking) where always associated with the pleasant emotional stimulus, and incongruent word-pairs (eliciting negative thinking) were always associated with the unpleasant emotional stimulus, in order to maximize the strength of the rewarding/aversive effects.

Stimuli presentation in the fMRI scanner was performed with a software interface developed in Matlab v. 6.5 (Mathworks, Inc., Sherbon, MA). Participants heard all auditory stimuli through MRI-compatible headphones with efficient gradient noise suppression (up to 45 dB) and a filter system with more than 90 dB RF-suppression (MR confon System, Leibniz-Institute for Neurobiology at Magdeburg, Germany).

#### **EXPERIMENTAL PARADIGM**

The paradigm consisted of a single session divided into six blocks (**Table 1**). In the first block, defined as habituation phase, 25 incongruent word-pairs (CS1), 25 congruent word-pairs (CS2), 25 unpleasant emotional stimuli (scream, US1) and 25 pleasant


*The classical conditioning procedure comprised six blocks: one habituation block (Wolpaw et al., 2002), two early acquisition blocks (Birbaumer, 2006; Liberati et al., 2012a), two late acquisition blocks (Pavlov, 1927; Liberati et al., 2012b), and one extinction block (Razran, 1939). In the habituation, incongruent wordpairs (CS1), congruent word-pairs (CS2), the scream (US1) and the laughter (US2) were all presented individually (25 times each). In the early acquisition, CS1 was always paired with US1 (25 times) and CS2 was always paired with US2 (25 times). In the late acquisition, the number of CS paired with US progressively decreased. In the extinction, only CS1 (25 times) and CS2 (25 times) were presented, without US.*

emotional stimuli (laughter, US2) were presented to the subject in a pseudo-random order, so that the same stimulus was never presented consecutively more than three times. The inter-stimulus interval (ISI) was also randomized to optimize design efficiency, and could last for 6, 7.5, or 9 s. The habituation phase served to evaluate the activations relative to each type of stimulus individually, before their association in the conditioning process. In the second and third blocks, which were structured identically and constituted the early acquisition phase, 25 incongruent wordpairs (CS1) and 25 congruent word-pairs (CS2) were presented in a pseudo-random order, such that the same condition (congruent or incongruent) was never presented consecutively more than three times. Each word-pair was immediately followed by an US: scream (US1) after incongruent word-pairs and laughter (US2) after congruent word-pairs. Also in the fourth and fifth block (late acquisition phase), 25 CS1 and 25 CS2 were presented in a pseudo-random order. In the fourth block, only 10 CS1 and 10 CS2 were paired (40% of the word-pairs were followed by the appropriate US) with US1 and US2, respectively. In the fifth block, only 5 CS1 and 5 CS2 were paired (20%). In the sixth and final block (extinction block), none of the 50 word-pairs, also presented in a pseudo-random order, were followed by an US. The gradual diminution of the percentage of word-pairs followed by emotional stimulation served to verify whether the conditioning had taken place, meaning that congruent and incongruent word-pairs could be discriminated thanks to their association to the emotional sounds, even when the emotional sounds were not present anymore.

#### **BEHAVIORAL MEASURES**

Participants were instructed to use the Self-Assessment Manikin (SAM, Bradley and Lang, 1994) to rate the valence (pleasantness/unpleasantness) and the arousal related to the two emotional US (scream and laughter), both at the end of the first block and at the end of the fifth block, to allow for the comparison of the two stimuli and to assess whether the subjects habituated to them throughout the measurement. The SAM comprises two 9–point scales, ranging from "pleasant" to "unpleasant" and from "not arousing at all" to "very arousing," respectively.

#### **fMRI DATA ACQUISITION**

The experiment was performed using a Siemens AG (Erlangen, Germany) 3T Trio MRI scanner. Functional T2∗-weighted images were acquired with a standard 12-channels head coil, in transversal orientation (*TR* = 1*.*5 s, *TE* = 30 ms, flip angle = 70◦, matrix <sup>64</sup> <sup>×</sup> 64, voxel size <sup>=</sup> 3.3 <sup>×</sup> 3.3 <sup>×</sup> 5.0 mm3, 16 slices, 1 mm gap, bandwidth = 1.954 kHz/pixel) covering the whole brain. Moreover, a high-resolution T1-weighted anatomical scan of the whole brain was acquired from each participant (MPRAGE, matrix size = 256 × 256, 160 slices, 1 mmisotropic voxels, *TR* = 2300 ms., *TE* = 3*.*93 ms., *TI* = 1100 ms, flip angle α = 8◦). The first 10 volumes of every block were discarded to permit T1 equilibrium.

#### **fMRI DATA ANALYSIS**

Data were analyzed using Statistical Parametric Mapping (SPM8, Wellcome Department of Imaging Neuroscience, London, UK) run on Matlab R2008b (Mathworks, Inc., Sherborn, MA, USA). Images of each subject were realigned and unwarped to correct for head movement, and were normalized to a standard Echo-Planar Imaging (EPI) template in Montreal Neurological Institute (MNI) space. Spatial smoothing was applied using a Gaussian kernel with full width at half-maximum of 9 mm. Prior to statistical analyses data were high-pass filtered (cutoff 128 s) and low-pass filtered (AR, Wolpaw et al., 2002).

Statistical analysis was carried out using the general linear model (GLM) with the canonical hemodynamic response function (HRF) as a basis set. In a first level analysis, regressors were defined to discriminate between paired and unpaired word-pairs separately for each block, condition and conditioning phase. The regressors were CS1 paired, CS2 paired, CS1 unpaired, CS2 unpaired, US1 and US2. The six movement regressors for each block were included as confounds in the design matrix to capture residual movement-related variance. The following contrasts were defined on the first level: US1 vs. US2, and unpaired CS1 compared to CS2 in the habituation phase CS1 paired, CS2 paired in the early acquisition phase and CS1 unpaired vs. CS2 unpaired, in the late acquisition phase as well as during extinction. Moreover contrasts were defined combining paired and unpaired word-pairs in the early and late acquisition. In a second level analysis, corresponding contrast images of all subjects were used to assess the main effects of conditioning. A paired *t*-test, performed by including the individual contrast images for US1 and US2 during habituation, was computed in order to detect significantly activated brain regions related to these emotional unconditioned stimuli during habituation. A paired *t*-tests was also performed to detect differences in brain regions involved in the laughter vs. scream and congruent vs. incongruent word-pairs in the extinction phases. To investigate the difference in brain regions for congruent and incongruent word-pairs, we performed a 2 x 2 full-factorial model with the factors Phase (early acquisition and late acquisition) and Condition (congruent and incongruent). Moreover, two different models were computed to observe activations in the early acquisition and late acquisition individually, to evaluate the progression of the conditioning process. For all group statistics a cluster-level threshold of *p <* 0*.*05 corrected for the Family Wise Error rate (FWER) of the whole brain was used.

# **RESULTS**

# **BEHAVIORAL DATA**

#### *SAM ratings*

A two-sample *t*-test indicated that participants rated the scream as significantly more unpleasant [block 1: *t*(20) = 10*.*62, block 5: *t*(20) = 3*.*09, *p <* 0*.*01] and more arousing [block 1: *t*(20) = 5*.*87, block 5: *t*(20) = 2*.*66, *p <* 0*.*02] than the laughter, both at the beginning and at the end of the measurement. The arousal associated to the scream was significantly less at the end of block 5 compared to block 1 [*t*(20) = 2*.*96, *p <* 0*.*01], although no significant difference was found for the laughter (p = 0.3). The valence associated to the scream and laughter did not change significantly during the experiment (*p* = 0*.*2 in both cases).

#### **NEUROIMAGING DATA**

In the habituation phase (**Table 2**), significant differential activations were only found for the scream vs. laughter (US1*>*US2) contrast, in the left middle cingulate gyrus (MCG), in the right inferior frontal triangularis (IFT), and in the superior frontal gyrus (SFG). No difference was found for the reverse contrast (US2*>*US1), nor for the incongruent vs. congruent (CS1*>*CS2) and congruent vs. incongruent (CS2*>*CS1) contrasts.

In the acquisition phase (**Table 3**), significant differential activation for the incongruent vs. congruent contrast (CS1*>*CS2) was found in the left IFT, adjacent to the insula (**Figure 1**). Differential activation in the left insula was found for the incongruent vs. congruent (CS1*>*CS2) contrast in the early acquisition, as well as in the late acquisition, for the word-pairs that were no more followed by emotional stimuli (**Figure 2**).

In the extinction phase (**Table 4**), the only differential activation was found for the congruent vs. incongruent contrast (CS2*>*CS1) in the anterior cingulate cortex (ACC).

Finally, paired *t-tests* were computed to directly compare the incongruent vs. congruent contrast pairs in the habituation and in the extinction, as well as in the habituation and in the acquisition, not showing significant differences for corrected *p*-values.

# **DISCUSSION**

The present study assessed the possibility to apply classical conditioning to two different responses (affirmative and negative) using positive and negative emotional US, in order to discriminate them. This was the first attempt to use such a protocol with fMRI. Our main hypothesis was that the effect of classical conditioning would have emerged as a differential activation for the

#### **Table 2 | Habituation.**


*P < 0.05 FWE corrected on cluster level; R, Right; L, Left.*

incongruent and congruent word-pairs in emotion related areas. In fact, our results showed an effect of the aversive semantic classical conditioning, as demonstrated by the differential activation of the insula, known to be related to emotion processing (Phillips et al., 1998; Phan et al., 2002; Radua et al., 2010; Sitaram et al., 2011), and of a region adjacent to the insula, the IFT, for the incongruent vs. congruent word-pairs during acquisition. This is supported by the observation that there was no significant difference in the activation of the insula between the word-pairs in the habituation phase. The differential activation of the insula and of the IFT could be interpreted as a conditioned brain response for the incongruent word-pairs associated to the unpleasant sound. The IFT, corresponding to the BA 45 area, belongs to the inferior frontal articulatory network and is involved in phonological and semantic processing of language (McDermott et al., 2003; Amunts et al., 2004; Gold et al., 2005). More specifically, several studies have associated this area to verbal fluency (Abrahams et al., 2003) lexical search (Fiebach et al., 2002; Heim et al., 2005), and semantic memory retrieval (Rugg et al., 1999; Düzel et al., 2001). Most interestingly for our aim of developing a communication system, this region was also associated with the process of internal word generation (Friedman et al., 1998; Tremblay and Gracco, 2006). We may therefore speculate that IFT activation for the incongruent vs. congruent contrast could be related to the difference in the patterns of activation within this region pertaining "yes" and "no" responses. Hence, we propose that a multivariate pattern classifier that can discriminate between fMRI spatio-temporal patterns can be trained to decode "yes" and "no" answers for the purpose of communication.

To be able to use the present paradigm for communication, it is necessary to be able to discriminate incongruent and congruent responses also in the extinction phase. The fact that the conditioned response was evident in the late acquisition phase (when the word-pairs were not followed by emotional stimuli anymore),

#### **Table 3 | Acquisition. Contrast Region** *t***-value MNI coordinate** *xyz* **ACQUISITION** Congruent *>* Incongruent No differential activations Incongruent *>* Congruent Inferior frontal triangularis L 4.57 −45 26 13 **EARLY ACQUISITION** Congruent *>* Incongruent No differential activations Incongruent *>* Congruent Insula L 4.80 −36 17 −14 **LATE ACQUISITION** Congruent *>* Incongruent No differential activations Incongruent *>* Congruent Insula L 7.77 −30 23 −5

*P < 0.05 FWE corrected on cluster level; R, Right; L, Left.*

**FIGURE 1 | Acquisition phase trials (early and late acquisition combined) for the incongruent** *>* **congruent contrast.** Showing left insula (left) and left inferior frontal gyrus pars trianglaris (right) on axial **(top)** and coronal **(bottom)** slices, Color map represents *t*-values.

**FIGURE 2 | Late acquisition phase unpaired trials for the incongruent** *>* **congruent contrast.** Showing left inferior frontal gyrus pars trianglaris (left) and left insula (right). Color map represents *t*-values.

but not in the extinction phase, may indicate that the extinction was rapid, suggesting the need of increasing the number of trials during the acquisition phase and modifying the reinforcement schedule to establish sustained CR's. The number of acquisition trials was limited in this study, because the conditioning process took place in the fMRI scanner and we wanted to avoid



*P < 0.05 FWE cluster < 0.05; R, Right; L, Left.*

tiring participants. In fact, once ascertained that the conditioning effect is visible in the acquisition phase, the first blocks of the protocol could be performed outside the scanner, and only the unpaired word-pairs would be subject to fMRI. After successful learning, the CR could be renewed faster and subsequently used for communication.

The comparisons between extinction and habituation, and between acquisition and habituation, concerning the incongruent vs. congruent contrast, are certainly important to evaluate the effect of conditioning. Although the present paradigm did not show significant differentiation for these phases, for binary communication aims, the decisive factor is the differentiation between two responses (i.e., "yes" and "no" answers). In fact, we could show that in the late acquisition phase, for the trials in which the word-pairs were not followed my emotional stimuli, there was a significant difference between incongruent and congruent word-pairs. Such differential activation could not be found in the habituation phase.

One possibility for the lack of strong differential activation for the congruent vs. incongruent word-pairs is that the appetitive US (baby-laugh) may have not been pleasant enough, although this explanation could be ruled out by considering the subjects' SAM ratings. It is possible that the presence of two distinct emotional US is confounding, so that the activations that would usually emerge with single conditioning are not elicited in double conditioning, in agreement with Lachnit (1991) and with Watt and Honey (1997). Another possible explanation for our results could be that appetitive conditioning is weaker compared to aversive conditioning, leading to weaker activations.

The lack of an independent parameter of the conditioning process, such as skin conductance response (SCR), startle response or contingency ratings, makes it difficult to judge whether the association between CS and US was identified, especially investigating a double conditioning. Nevertheless, the activations we found during acquisition, especially in the latest stage, suggests a conditioning effect for the incongruent word-pairs.

Improvements for this study comprise the reversed CS-US combination and the usage of different emotional US. To judge whether the conditioning of both CS was successful, the reversed CS-US combination should be tested, associating the congruent word-pairs with the scream and the incongruent word-pairs van der Heiden et al. Insula and IFT for BCI

with the laughter, to investigate whether this results in similar CRs. Concerning a different US, Metereau et al. (Metereau and Dreher, 2012) found that both appetitive and aversive reinforcers activate the ACC, anterior insula and striatum, suggesting that the usage of appetitive and aversive US could activate the same areas, which may not be ideal for discriminating between the two conditioned responses. In our study the MCG, IFT, and SFG were activated more for the scream compared to the laughter, indicating a differentiable brain pattern but future studies should take possible overlap into consideration. It is well known that US intensity is a critical factor in effective classical conditioning. A more intense US improves learning and results in increased conditioned responses. Our suggestion for a different US would be to replace the laughter with an auditory stimulus eliciting disgust. Sitaram et al. (2011) showed that "disgust" and "sad" imagery are differentiable, indicating the possibility to classify the CRs and enable further investigation of double conditioning. Another possibility for the positive emotional US would be to use other appetitive US, such as smell or taste, depending on the individual salience of the stimuli.

The possibility to classify differential activations would be a first step in the direction of developing clinical applications, so that emotional stimuli may become tools for the conditioning of desirable behavior or brain responses. The demonstration that the response to congruent and incongruent word-pairs are differentiable (in the insula and in the IFT), at least until the late acquisition phase, introduces the possibility of applying such a paradigm for practical purposes, such as a binary BCI, which could allow basic "yes/no" communication with patients suffering from cognitive impairment.

Another aspect that should be considered when applying this classical conditioning protocol for basic communication is subject variability. Some individuals may be more easily conditioned, while others may require a higher number of trials in which wordpairs are associated to emotional stimuli. This introduces the need of developing an "adapted acquisition" protocol, based on the subjective ease of conditioning. For instance, if specialized pulse sequences become more sensitive to amygdala activation while enabling real-time fMRI for pattern classification, or if other biomarkers of acquisition are identified, it would be possible to obtain extra information about the conditioning process, both to improve the paradigm in the future, and to adapt it to different individuals.

#### **CONCLUSION**

This study represents a first step into a new direction of double conditioning of brain responses. The insula activations for unpaired word-pairs during the late acquisition suggest that conditioning took place for the incongruent word-pairs. The importance of the insula and neighboring inferior frontal gyrus pars triangularis throughout the association process is shown by the continuous differential activation during the different phases of conditioning. The possibility of using classical conditioning with emotional US opens the door to investigating more variations on double conditioning with different US or testing the usability for clinical applications.

The application we aimed at is a BCI for communication. If the negative and affirmative response can be classified from the BOLD response, this possibility could become reality (i.e., allowing to discriminate between affirmative and negative thinking). This application would be most beneficial for completely locked-in and cognitively impaired patients.

# **ACKNOWLEDGMENTS**

This work was supported by the European Commission 7th Framework Programme (FP7) Marie Curie Initial Training Networks: ITN-LAN [PITN-GA-2008-214570], and by the Badenwürttemberg-Singapore Life Sciences Grant. We are grateful to Adelheid Kumpf and Dr. Daniele De Massari for helping in the preparation of the stimuli, Dr. Andrea Caria, Mohit Rana and Dr. Sangkyung Lee for their support in data acquisition, and Dr. Tamara Matuz, Dr. Carolin Ruf and Dr. Adrian Furdea for fruitful discussions.

# **REFERENCES**


phonological processing: evidence from FMRI adaptation. *Cereb. Cortex* 15, 1438–1450. doi: 10.1093/cercor/bhi024


schizophrenic adults. *Neuroimage* 49, 939–946. doi: 10.1016/j.neuroimage.2009. 08.030


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 12 March 2014; accepted: 30 June 2014; published online: 21 July 2014. Citation: van der Heiden L, Liberati G, Sitaram R, Kim S, Ja´skowski P, Raffone A, Olivetti Belardinelli M, Birbaumer N and Veit R (2014) Insula and inferior frontal triangularis activations distinguish between conditioned brain responses using emotional sounds for basic BCI communication. Front. Behav. Neurosci. 8:247. doi: 10.3389/ fnbeh.2014.00247*

*This article was submitted to the journal Frontiers in Behavioral Neuroscience. Copyright © 2014 van der Heiden, Liberati, Sitaram, Kim, Ja´skowski, Raffone, Olivetti Belardinelli, Birbaumer and Veit. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Down-regulation of amygdala activation with real-time fMRI neurofeedback in a healthy female sample

# *Christian Paret 1,2\*, Rosemarie Kluetsch2, Matthias Ruf 1, Traute Demirakca1, Steffen Hoesterey1, Gabriele Ende1 † and Christian Schmahl 2 †*

*<sup>1</sup> Department Neuroimaging, Central Institute of Mental Health Mannheim, Medical Faculty Mannheim/Heidelberg University, Mannheim, Germany <sup>2</sup> Department of Psychosomatic Medicine and Psychotherapy, Central Institute of Mental Health Mannheim, Medical Faculty Mannheim/Heidelberg University,*

*Mannheim, Germany*

#### *Edited by:*

*Sergio Ruiz, Pontificia Universidad Catolica de Chile, Chile*

#### *Reviewed by:*

*René Hurlemann, University of Bonn, Germany Kathrin Cohen Kadosh, University of Oxford, UK*

#### *\*Correspondence:*

*Christian Paret, Department Neuroimaging, Central Institute of Mental Health, J5, Mannheim, D-68159, Germany e-mail: christian.paret@ zi-mannheim.de*

*†These authors have contributed equally to this work.*

Psychiatric conditions of emotion dysregulation are often characterized by difficulties in regulating the activity of limbic regions such as the amygdala. Real-time functional magnetic resonance imaging (rt-fMRI) allows to feedback brain activation and opens the possibility to establish a neurofeedback (NF) training of amygdala activation, e.g., for subjects suffering from emotion dysregulation. As a first step, we investigated whether feedback of the amygdala response to aversive scenes can improve down-regulation of amygdala activation. One group of healthy female participants received amygdala feedback (*N* = 16) and a control group was presented with feedback from a control region located in the basal ganglia [*N*(sum) = 32]. Subjects completed a one-session rt-fMRI-NF training where they viewed aversive pictures and received continuous visual feedback on brain activation (REGULATE condition). In a control condition, subjects were advised to respond naturally to aversive pictures (VIEW), and a neutral condition served as the non-affective control (NEUTRAL). In an adjacent run, subjects were presented with aversive pictures without feedback to test for transfer effects of learning. In a region of interest (ROI) analysis, the VIEW and the REGULATE conditions were contrasted to estimate brain regulation success. The ROI analysis was complemented by an exploratory analysis of activations at the whole-brain level. Both groups showed down-regulation of the amygdala response during training. Feedback from the amygdala but not from the control region was associated with down-regulation of the right amygdala in the transfer test. The whole-brain analysis did not detect significant group interactions. Results of the group whole-brain analyses are discussed. We present a proof-of-concept study using rt-fMRI-NF for amygdala down-regulation in the presence of aversive scenes. Results are in line with a potential benefit of NF training for amygdala regulation.

**Keywords: affective disorders, amygdala, emotion regulation, mPFC, emotions, real-time fMRI neurofeedback, affective symptoms, instrumental learning**

# **INTRODUCTION**

The amygdala constitutes a core structure of emotion processing (Phan et al., 2002; Kober et al., 2008). It plays a major role in the generation and modulation of emotional responses in animals (LaBar and LeDoux, 1996) and humans (LaBar et al., 1995; Hermans et al., 2012; Haaker et al., 2013). Accumulating data indicate a close relationship between psychiatric symptoms and an exaggerated amygdala response to emotional material. An excessive response of this structure to emotional information has been shown for borderline personality disorder (BPD) (Niedtfeld et al., 2010; Schulze et al., 2011), depression (Sheline et al., 2001; Victor et al., 2010), phobias (Phan et al., 2006; Goossens et al., 2007), and post-traumatic stress disorder (PTSD) (Fonzo et al., 2010; Simmons et al., 2011)—all of which involve emotional dysregulation. Consequently, improving emotion regulation is one of the central goals of psychotherapy. On the neural level, there is evidence for normalization of the amygdala response with psychotherapy (Sheline et al., 2001; Goossens et al., 2007; Godlewska et al., 2012; Lipka et al., 2013). This is in line with a recent meta-analysis by Buhle et al. (2013), who identified the amygdala as a robust target of cognitive emotion regulation. The decrease of amygdala activation by cognitive reappraisal has been shown to correlate with perceived emotion regulation success (Wager et al., 2008). The neural top-down control of the amygdala is achieved by prefrontal-limbic coupling: specifically, a decrease in amygdala activation is associated with activation in lateral and medial regions of the prefrontal cortex (PFC) (Urry et al., 2006; Wager et al., 2008; Erk et al., 2010; Diekhof et al., 2011). In addition to amygdala hyperactivity, individuals with the aforementioned disorders have previously shown dysfunctional amygdala-prefrontal coupling (Johnstone et al., 2007; Fonzo et al., 2010; Schulze et al., 2011; Simmons et al., 2011; Lang et al., 2012).

Taken together, improving amygdala self-regulation might constitute a pathway to mental health. Hence, there is a need to develop effective therapeutic interventions which could help patients to better control amygdala activation. Several studies have begun to test real-time functional magnetic resonance imaging (rt-fMRI) as a potential therapeutic tool in psychiatric conditions (Linden et al., 2012; Ruiz et al., 2013). With rt-fMRI, a volume of brain data can be processed as soon as it has been scanned. This enables the estimation of the activation in a brain region-of-interest (ROI) for each newly acquired volume. In rtfMRI neurofeedback (NF), the patient is supplied e.g., with a visual display indicating the current activation level in a ROI. There is increasing evidence that the control of neural circuits of emotion can be achieved by rt-fMRI-NF in healthy individuals as well as psychiatric populations. For example, Young et al. (2014) recently showed that depressed subjects who received feedback from the left amygdala during recall of positive autobiographical memories were able to up-regulate their amygdala response. This corroborates earlier findings in healthy male volunteers (Zotev et al., 2011), who showed a significant Blood Oxygenation Level Dependent (BOLD) signal increase in the left amygdala, which was not found in a control group receiving sham feedback. Another study by Brühl et al. (2014) investigated amygdala downregulation by rt-fMRI-NF. Specifically, the authors instructed a group of healthy subjects to down-regulate a region in their right amygdala while viewing faces with different emotional expressions. Their results demonstrated an increasing down-regulation effect over the course of four scanning sessions. However, due to the lack of a control group it is not possible to rule out unspecific effects that may have influenced learning, like task repetition or rehearsal of a regulation strategy (Sulzer et al., 2013).

In this proof-of-concept study, we aimed to assess whether subjects can profit from a rt-fMRI-NF training of the amygdala with the aim of down-regulating their amygdala response to aversive scenes, and whether such a training effect can be attributed specifically to having received contingent amygdala feedback. Previous results of our group have shown that the amygdala responds with sustained BOLD activation in aversive picture viewing and identified this region as a potential target region for down-regulation by a rt-fMRI-NF training (Paret et al., 2014). While the reduction of the amygdala response emerges from the literature as one consistent effect of successful emotion regulation (see Diekhof et al., 2011; Buhle et al., 2013 for metaanalyses), the precise prefrontal structures involved in regulation may differ with the cognitive strategy applied (Ochsner et al., 2004; Kalisch et al., 2006; McRae et al., 2010; Kanske et al., 2011; Opialla et al., 2014). We assume, that choosing the signal from the amygdala opposed to a prefrontal region allows for flexibility in strategy-selection. Studies on down-regulation of the amygdala response to aversive material either endorse preferential involvement of the left (Diekhof et al., 2011), right (Brühl et al., 2014), or bilateral (Buhle et al., 2013) amygdala. We combined an anatomical delineation of the bilateral amygdala with a functional voxel selection based on the activation profile during the experiment. This procedure was flexible for including voxels from the right as well as the left amygdala, depending on their activation to the experimental stimuli.

Only female participants were included to control for variance due to potential effects of gender. We chose to elicit emotional activation by presenting aversive pictures which are widely used in the neuroimaging literature of emotion regulation (Kalisch, 2009). A one-session learning protocol was used, closing with a transfer test without NF, where subjects were again presented with aversive pictures and instructed to regulate, but did not receive any feedback. We hypothesized a decrease of the BOLD signal amplitude of the amygdala in blocks where subjects of the experimental group were instructed to regulate vs. view aversive pictures. This finding was expected to be more pronounced in the experimental group compared to a control group receiving feedback from a region located in another part of the brain. Further, we expected the experimental group to show stronger down-regulation of amygdala activation during the transfer run compared to the control group.

Given previous reports of run-to-run improvements during NF training (Zotev et al., 2011; Lawrence et al., 2013), we also evaluated our data for an improvement in amygdala downregulation over the course of the training.

Veit et al. (2012) have recently used the anterior insula response to aversive pictures as a NF signal for a training of brain self-regulation. The authors found a modulation of the anterior insula response consistent with the instruction to up-regulate and down-regulate a thermometer, which displayed the activation of the region. Together with the amygdala, the anterior insula plays an important role in the processing of emotional information (Damasio et al., 2000; Phan et al., 2002; Kober et al., 2008). To explore the specifity of amygdala NF training on the regulation of brain structures of the affective brain system, we investigated the effect of the training on the activation of the anterior insula.

Finally, we conducted a whole-brain analysis to explore the involvement of other regions.

# **METHODS SAMPLE**

*N* = 32 right-handed female participants (*N* = 16 per group) were tested. There were no significant differences in age between the experimental (24.19 ± 4.17, range: 19–34) and the control group (24.94 ± 3.87, range: 20 to 34) [*T*(30) = 0*.*53]. 14 participants of the experimental group and 15 participants of the control group had a university entrance diploma (German Abitur) (Fisher's Exact Test: *p* = 1*.*000). Participants reported no current and past DSM-IV Axis I and II disorder or family history of neurological or psychiatric disorders, as confirmed by the Structured Clinical Interview for DSM-IV (SCID; First et al., 1997) as well as the International Personality Disorder Examination (IPDE; Loranger, 1999), which had been conducted before inviting subjects for the experiment. The study was approved by the Ethics Committee of the Medical Faculty Mannheim of the University of Heidelberg, and all subjects provided written informed consent before participation. Compensation for expenses was 24 Euro.

#### **GROUP ASSIGNMENT**

Subject assignment to groups was randomized and blinded to both the participant and the investigator welcoming and instructing the participant before training. Subjects were informed before the investigation that part of the experiment was to find out which of two regions would be best suited for the training.

#### **INSTRUCTION AND VISUAL FEEDBACK PRESENTATION**

Subjects were instructed to regulate "the feeling center" of their brains and were told that this brain region is involved in the "perception and processing of emotions." No further instructions on the use of a specific regulation strategy were provided. During training, feedback on brain activation was given by the level of a thermometer displayed to both sides of the picture stimulus. The bilateral presentation was chosen to ensure high visibility of the thermometer regardless of gaze orientation. Participants were presented with the following instruction in German language: "If you see the statement "Regulate," an unpleasant image will be shown. Your task is to regulate the number of bars in the thermometer." Additionally, subjects were briefed that an orange line in the lower half of the thermometer indicated "the activation of the feeling center under rest or non-emotional conditions" and the aim is "that the bars remain at or below the baseline." In terms of activation magnitude, one bar of the thermometer display corresponded to 0.2% signal change. The orange line divided the thermometer in an upper part displaying activation (maximum of 2.8% signal change, derived from reported BOLD signal changes in the literature, Zotev et al., 2011 and confirmed by piloting experiments in our group) and a lower part indicating deactivation from baseline (maximum of 1.2% signal change, since we expected more positive signal change compared to baseline in the REGULATE and VIEW conditions, we chose to leave more space for thermometer bars above compared to below). To prevent regulation by just looking at the thermometer or nonemotional details of the picture, participants were advised not to avert their gaze or keep their eyes closed, nor to focus exclusively on the thermometer, but rather to look at the picture for its entire presentation. The participants' eyes were tracked by a camera system (MRC Systems, Heidelberg, Germany) to encourage subjects to adhere to the instructions and to control for drowsiness during training. Data were not statistically analyzed. Subjects were also instructed to consider the temporal latency of the BOLD signal amplitude when evaluating the success of regulating their brain activation.

When a NF run was finished, subjects rated perceived regulation success. They were asked whether they had been able to regulate the thermometer on a 9-point scale (0 = not at all, 9 = very much).

# **EXPERIMENTAL CONDITIONS**

In addition to the "regulate"-condition (REGULATE), the protocol included two control conditions: a "view-negative" (VIEW) and a "view-neutral" (NEUTRAL) condition. Subjects were instructed to refrain from controlling the thermometer during the control conditions. While aversive pictures were presented in VIEW trials, scrambled pictures with no meaningful content were presented during NEUTRAL trials. Trials were separated by an inter-trial interval (ITI) with a fixation cross displayed on the screen.

The structure of an experimental trial is shown in **Figure 1A**. An experimental run lasted approximately 9 min and consisted of 15 trials, with 5 of each condition. Each subject participated in 3 consecutive NF training runs, followed by 1 transfer run.

### **STIMULUS MATERIAL AND PRESENTATION**

Stimuli were taken from standardized picture series (Lang et al., 2008; Wessa et al., 2010) and were chosen to elicit moderate to high negative valence and arousal. Pictures included in the training runs had a valence level of 2.17 ± 0.49 (mean ± standard deviation (SD), values were taken from the original reports by Lang and colleagues and Wessa and colleagues) and an arousal level of 6.35 ± 0.70. Stimuli included in the transfer run were matched to training stimuli with a valence of 2.12 ± 0.54 and arousal values of 6.26 ± 0.71. Stimuli were assigned to the REGULATE and VIEW conditions matching valence and arousal between runs and conditions (significance value of *p <* 0*.*05 in a comparison of means) and the stimulus-to-condition assignment was counterbalanced (picture numbers can be obtained from the corresponding author). The condition order was semirandomized with the restriction of ≤2 consecutive stimuli of the same condition. Each subject of the control group received the same version of the experiment as another subject from the experimental group. An overview on the experimental procedure can be obtained from **Figure 1B**. Stimuli were presented with Presentation software (Neurobehavioral Systems, Berkeley, CA). After completion of the experiment, subjects rated the pictures outside the MRI suite.

### **DELINEATION OF BRAIN AREA AND REAL-TIME DATA PROCESSING**

The anatomical scans were imported into BrainVoyager software (version QX2.4, Brain Innovations, Maastricht, Netherlands), skull-stripped and transformed into Talairach space. Normalization parameters were loaded into TurboBrainVoyager (TBV) (version 3.0, Brain Innovations, Maastricht, Netherlands). Depending on group assignment, an anatomical mask of the bilateral amygdala or a mask of a region located in the rostral part of the basal ganglia (control region) was loaded (**Figure 2**). Due to a technical problem, the Talairach transformation failed in three subjects of the control group. Instead of the control region, these subjects received feedback from a manually drawn square region of variable size (thickness: 5 axial slices), covering parts of the corpus callosum, gray matter, and ventricular areas.

The initial 2 volumes of the functional scans were discarded before real-time processing started. A motion correction feature implemented in TBV was enabled to correct for head movements and spatial smoothing with a 4 mm kernel (full width at half maximum, FWHM) was applied. For the calculation of the BOLD signal amplitude, the "best voxel selection" tool implemented in TBV was used to identify the 33% voxels with beta-values discriminating best between VIEW and NEUTRAL conditions. The voxels were dynamically determined by a score defined by Goebel (2014) based "(a) on the maximum condition beta value and (b) on the amount of deviation from the mean of all condition betas. The first criterion selects those voxels, which have the largest beta value. The second criterion calculates first the mean of all betas and then adds the absolute differences of each beta value from the mean. This deviation index biases the selection to those voxels with a "irregular" profile, i.e., which will show high values for contrasts between betas." The overall voxel-score is calculated by: voxel-score = (b\_max + b\_dev)/b\_constant. Thereby, the selection of voxels within the spatial region was dynamically

refined along the course of training and counterbalanced moderate shifts of the anatomical delineation due to alignment errors across successive runs as well as movement-related slice shifts. Furthermore, the procedure guaranteed that there was no difference in the number of voxels used for signal extraction between subjects and groups. To permit an initial selection of voxels based on their response patterns during VIEW and NEUTRAL, the first two trials of each NF run consisted of those conditions.

The BOLD signal amplitude was passed to Presentation as soon as a new volume had been processed. For each trial, the mean of the last 4 data points before picture onset was taken as a baseline. The signal was smoothed by calculating the mean of the current and the preceding 3 data points. The subtraction of the baseline resulted in the feedback signal amplitude [(X + [X − 1]+[X − 2]+[X − 3])*/*4-baseline; X = current data point]. The feedback display was updated as soon as information of a new volume had been available. Thus, the latency of the feedback was composed of the TR (2 s) plus the time needed for real-time calculation and display actualization by the presentation software (about half a second).

# **DATA ACQUISTION AND** *POST-HOC* **ANALYSIS OF IMAGING DATA** *Image acquisition*

For brain imaging, a 3 Tesla MRI Scanner (Trio, Siemens Medical Solutions, Erlangen, Germany) with a 32 channel head coil was used. Functional images of the BOLD contrast were acquired with a gradient echo T2∗ weighted echo-planar-imaging sequence (*TE* = 30 ms, *TR* = 2 s, *FOV* = 192 × 192 mm, flip angle = 80◦). One volume comprised 36 slices in AC-PC orientation with a thickness of 3 mm and slice gap of 1 mm. Participants' heads were lightly restrained using soft pads. The four experimental runs comprised 284 volumes each. The T1-weighted

**neurofeedback signal during training.** The amygdala mask was prepared using the Talairach Daemon (Lancaster et al., 2000) and included voxels which were exclusively assigned to the amygdala by the online tool. The mask delineating the control region was the same size and shape as the amygdala mask, but was located in another part of the brain, comprising parts of rostral basal ganglia, and white matter.

anatomical image recording parameters were as follows: *TE* = 3*.*03 ms, *TR* = 2*.*3 s, 192 slices and *FOV* = 256 × 256 mm.

#### *Preprocessing*

FMRI data were analyzed with SPM version 8 (Wellcome Department of Cognitive Neurology, London, UK). Before preprocessing of functional data, nine initial volumes were discarded to avoid T1-equilibrium effects. A slice timing correction of the functional scans was performed with reference to the 18th slice to correct for differences in acquisition time between slices. The functional volumes were spatially aligned to the mean image using a rigid body transformation and images were resliced. Functional images were coregistered to the anatomical image, normalized to the SPM standard template and brought into Montreal Neurological Institute (MNI) coordinate space. Finally, images were smoothed with a kernel of 6 mm (FWHM).

#### *Statistical analysis*

*First-level analysis.* We formulated separate models in SPM for the NF training and the transfer run. The three NF runs were modeled as separate sessions. Three experimental conditions were modeled (REGULATE, VIEW, NEUTRAL) and the movement vectors taken from the spatial realignment procedure were included in the model as nuisance variables. All events were modeled as blocks of brain activation and were convolved with the hemodynamic response function. Data were high-pass filtered (128 s) and a correction for serial correlations was implemented by autoregressive modeling.

*ROI analysis.* To test our hypotheses, voxel-wise *T*-tests of parameter estimates for the contrasts VIEW*>*REGULATE, REGULATE*>*NEUTRAL, and VIEW*>*NEUTRAL were conducted on the subject level. The mean contrast value was then extracted from all voxels of the amygdala and from the control region. Anatomical templates were used as provided by the Wake Forest University (WFU) PickAtlas toolbox (Maldjian et al., 2003) to delineate the amygdala (left: 71 voxels, right: 76 voxels). To assess results of the control region, we extracted mean contrast values using the control region mask (left and right: 43 voxels each).

Values were screened for outliers before the analysis. As an exclusion criterion, we set a 3 SD threshold below and above the group mean, based on the mean contrast value taken from the bilateral region masks. When outliers were identified, we report test-values with and without outlier exclusion.

Extracted contrast values were passed to SPSS version 20 for statistical analyses.

To test our main hypothesis regarding down-regulation of brain activation (REGULATE) as contrasted with natural viewing of the aversive stimulus (VIEW), a Region (2) × Hemisphere (2) × Group (2) analysis of variance (ANOVA) was calculated for the REGULATE*>*VIEW contrast. Since the dynamic voxel selection principally allowed the inclusion of more voxels of the one or the other laterality into the ROI, we included the factor of Hemisphere in our analysis and assessed down-regulation for each hemisphere separately.

Where we had directional a-priori hypotheses, one-tailed *t*-tests were calculated to estimate significance (*p <* 0*.*05). We report a trend when the *p*-value was below *p* = 0*.*10.

To further characterize group differences and conditioneffects, we report results of an additional ANOVA for each region (amygdala, control region; separately for the left and right hemisphere), taking into account the factor Condition (REGULATE*>* NEUTRAL, VIEW*>*NEUTRAL).

To explore the effect of NF training on anterior insula regulation, a ROI analysis of parameter estimates was calculated. Mean contrast values were extracted from spherical masks with a radius of 8 mm and centers taken from the literature (left peak: [−33, 20, 0], 82 voxels; right: [36, 26, 6], 81 voxels) (Caria et al., 2007).

*Exploratory whole-brain analysis.* To elucidate task-related effects, we conducted exploratory whole-brain random effects analyses for both groups independently on the t-contrasts VIEW*>*REGULATE, REGULATE*>*NEUTRAL and VIEW*>* NEUTRAL. Additionally we explored group differences, assuming independent measurements and equal variances of errors. To protect against false positives, we used Monte-Carlo simulations to estimate the cluster-extend at a voxel-threshold of *p <* 0*.*001 and a cluster-threshold of *p <* 0*.*05. Simulations were performed with 3dClustSim, implemented in AFNI (Cox, 1996). Estimation of cluster-extend was determined in 10.000 iterations, based on the number of voxels included in the masks and the smoothness (FWHM) of the residuals of the second-level SPM8-analyses. The resulting number of voxels (k) is indicated in the results tables.

# **ANALYSIS OF RATING DATA**

Picture-rating data of 2 participants was lost. We looked for group differences in subjective arousal and valence elicited by picture viewing and conducted Group × Condition (VIEW, REGULATE) ANOVAs for pictures presented during training and transfer. To further determine whether groups differed in their perceived regulation success, we conducted a repeated measures ANOVA on regulation success, with "Run" as the within-subjects factor and "Group" as the between-subjects factor.

#### **ANALYSIS OF THERMOMETER VARIABILITY**

To compare the dynamics of the thermometer display between the groups, the number of bars presented to a subject at any given time point in the experiment was derived from the data. The variance of the number of bars as a measure of thermometer variability was calculated for each subject and each condition, and was taken to a Condition (3) × Run (3) × Group (2) ANOVA.

# **RESULTS**

#### **ROI ANALYSIS**

Since three participants of the control group did not receive feedback from the standardized control region due to a technical error, we additionally report results of an analysis leaving out these subjects.

#### *Neurofeedback training*

Neither the Region × Hemisphere × Group ANOVA nor the Hemisphere × Group ANOVA of the amygdala revealed significant interactions. The main effect of Region was significant [*F*(1*,* 31) = 33*.*163, *p <* 0*.*001]. Subjects from the experimental group showed down-regulation of the left amygdala in the REGULATE*>*VIEW contrast [−0*.*25 ± 0*.*38 (mean ± *SD*), *T*(15) = 2*.*675, *p* = 0*.*009, one-tailed] in the hypothesized direction (**Figure 3**). The effect was less pronounced in the right amygdala [−0*.*15 ± 0*.*34, *T*(15) = 1*.*684, *p* = 0*.*057]. The control group showed a similar effect in the left [−0*.*31 ± 0*.*53, *T*(15) = 2*.*324, *p* = 0*.*018] and right amygdala [−0*.*36 ± 0*.*62, *T*(15) = 2*.*338, *p* = 0*.*017]. All *t*-tests were not significant when correcting for multiple comparisons.

An exploratory ANOVA of the run-to-run change in the REGULATE*>*VIEW contrast revealed a trend for a Hemisphere by Group interaction [*F*(1*,* 30) = 3*.*164, *p* = 0*.*085] and a significant main effect of Region [*F*(1*,* 30) = 33*.*394, *p <* 0*.*001]. Since a visual inspection did not suggest a linear trend of the outcome measure in the experimental group, we did not further explore within-group run-to-run changes in parameter estimates.

Excluding subjects who did not receive feedback from the standardized control region did not change significance of the test values.

To further elucidate condition effects, we analyzed the REGULATE*>*NEUTRAL and VIEW*>*NEUTRAL contrasts (**Table 1**). Consistent with the previous results, we detect a significant main effect of Condition in the analysis of the left and right amygdala, while Group-interactions were not significant.

### *Transfer run*

The screening for outlier values had identified one outlier in the control group (i.e., contrast-value of 3 SD below the group mean). The Region by Hemisphere by Group interaction was significant [*F*(1*,* 29) = 4*.*550, *p <* 0*.*05], with outlier inclusion there was still a trend [*F*(1*,* 30) = 3*.*452, *p* = 0*.*073] (**Figure 4**). There was a main effect of Region [*F*(1*,* 29) = 5*.*172, *p <* 0*.*05; including the outlier: *F*(1*,* 30) = 5*.*701, *p <* 0*.*05] and Hemisphere [*F*(1*,* 29) = 19*.*012, *p <* 0*.*001, including the outlier: *F*(1*,* 30) = 21*.*023, *p <* 0*.*001]. When excluding subjects who had received feedback from a nonstandardized control region, the results did not change in terms of significance regarding the three-way interaction [*F*(1*,* 26) = 4*.*618,

BOLD signal amplitude in the regulate condition (REGULATE) compared to natural responding toward aversive pictures (VIEW). White box plots: participants receiving feedback from the amygdala during training (experimental group). Gray box plots: participants receiving feedback from region during training due to a technical error (*N* = 3). *P*-values indicate probability of the findings for each group and mask under the null-hypothesis. *T* -tests were not significant when correcting for multiple comparisons. A.u., artificial units.

*p <* 0*.*05; including the outlier: *F*(1*,* 27) = 3*.*339, *p* = 0*.*079]. The main effects of Region [*F*(1*,* 26) = 5*.*675, *p <* 0*.*05; including the outlier: *F*(1*,* 27) = 6*.*326, *p <* 0*.*05] and Hemisphere [*F*(1*,* 26) = 18*.*314, *p <* 0*.*001; including the outlier: *F*(1*,* 27) = 20*.*942, *p <* 0*.*001] were still significant.

The two-way ANOVA of the amygdala proved the Hemisphere by Group interaction to be at trend [*F*(1*,* 29) = 3*.*921, *p* = 0*.*057; including the outlier: *F*(1*,* 30) = 2*.*526, *p* = 0*.*122] with a significant main effect of Hemisphere [*F*(1*,* 29) = 10*.*246, *p <* 0*.*01; including the outlier: *F*(1*,* 30) = 11*.*922, *p <* 0*.*01]. When excluding subjects receiving feedback from a non-standardized control region, the trend regarding the interaction was robust [*F*(1*,* 26) = 3*.*435, *p* = 0*.*075; including the outlier: *F*(1*,* 27) = 2*.*016, *p* = 0*.*167] as was the significance of the main effect of Hemisphere [*F*(1*,* 26) = 8*.*934, *p <* 0*.*01; including the outlier: *F*(1*,* 27) = 10*.*944, *p <* 0*.*01].

An inspection of subject means suggested a right-lateralized effect. We conducted a Region × Group ANOVA of the values from the right hemisphere masks. The Region by Group interaction was found at trend level [*F*(1*,* 29) = 3*.*491, *p* = 0*.*072; including the outlier: *F*(1*,* 30) = 3*.*046, *p* = 0*.*091] and the main effect of Region was significant [*F*(1*,* 29) = 7*.*765, *p <* 0*.*01; including the outlier: *F*(1*,* 30) = 9*.*028, *p <* 0*.*01], also when excluding subjects who had received feedback from a non-standardized control region [Region by Group interaction: *F*(1*,* 26) = 2*.*956, *p* = 0*.*097; including the outlier: *F*(1*,* 27) = 2*.*500, *p* = 0*.*125; main effect of Hemisphere: *F*(1*,* 26) = 7*.*468, *p <* 0*.*05; including the outlier: *F*(1*,* 27) = 8*.*964, *p <* 0*.*01]. The Region × Group ANOVA did not indicate a significant interaction effect in the left hemisphere.

When contrasting groups, we observed a trend for lower values in the experimental group compared to the control group in the right amygdala [*T*(29) = 1*.*522, *p* = 0*.*070, one-tailed; including the outlier subject: *T*(30) = 0*.*648, *p* = 0*.*261, one-tailed]. Contrast estimates in the left amygdala did not differ between groups. When excluding subjects who had received feedback from a non-standardized control region, the group difference in the right amygdala was not anymore at trend level [*T*(26) = 0*.*921, *p* = 0*.*183; including the outlier subject: *T*(27) = 0*.*062, *p* = 0*.*476]. In the control region, we did not find any significant group differences [left: *T*(29) = 0*.*391, *p* = 0*.*698, right: *T*(29) = 0*.*020, *p* = 0*.*985; including outlier: left: *T*(30) = 0*.*073, *p* = 0*.*942, right: *T*(30) = 0*.*470, *p* = 0*.*642].

The experimental group showed down-regulation of the right amygdala [REGULATE*>*VIEW contrast: −0*.*14 ± 0*.*20, *T*(15) = 2*.*797, *p* = 0*.*007, one-tailed] (**Figure 5A**) but not the left amygdala [−0*.*01 ± 0*.*20, *T*(15) = 0*.*178, *p* = 0. 861, two-tailed]. The control group did not show down-regulation in the left [0.01 ± 0.22, *T*(15) = 0*.*099, *p* = 0*.*923, two-tailed; including outlier subject: −0*.*03 ± 0*.*26, *T*(15) = 0*.*506, *p* = 0*.*310, one-tailed] and right amygdala [−0*.*03 ± 0*.*22, *T*(14) = 0*.*451, *p* = 0*.*330, onetailed; including the outlier subject: −0*.*08 ± 0*.*31, *T*(15) = 1*.*056, *p* = 0*.*154, one-tailed]. However, all *t*-tests were not significant after correcting for multiple comparisons.

Results didn't change in terms of significance when excluding subjects from the analysis who had received feedback from a nonstandardized ROI [left amygdala: −0*.*07 ± 0*.*20, *T*(11) = 0*.*708, *p* = 0*.*247; including the outlier subject: −0*.*08 ± 0*.*24, *T*(12) = 1*.*235, *p* = 0*.*120, one-tailed; right amygdala: −0*.*14 ± 0*.*31,

#### **Table 1 | Group statistics and ANOVA results of neurofeedback training.**


*Table shows parameter-estimate-contrast values from the regions of interest. (A) Amygdala, (B) Control region. SD, standard deviation; REG, regulate; NEU, neutral.*

*T*(12) = 1*.*602, *p* = 0*.*068, including outlier subject: −0*.*07 ± 0*.*20, *T*(11) = 1*.*200, *p* = 0*.*127].

We examined training effects on activation of the control region and did not find significant regulation in the REGULATE*>*VIEW contrast in both groups, as well as no groupdifferences of the contrast values.

The results of the Condition × Group ANOVA can be obtained from **Table 2**. Descriptively, subjects of the experimental group showed lower activations in both REGULATE and VIEW trials compared to the control group (**Figure 5B**). In the right amygdala, the main effect of Group showed a trend and the main effect of Condition was significant, however, the Condition × Group interaction was not significant. *Post-hoc t*-tests of group differences in the single conditions brought a trend for the experimental group showing lower values than the control group when instructed to regulate. In line with the results presented above, the experimental group and not the control group did show a significant reduction of the right amygdala response in the REGULATE vs. the VIEW condition in the transfer run. Regarding the control region, we detected a significant effect of Group, with participants of the control group showing higher values than the experimental group during both conditions.

#### *Anterior insula: neurofeedback training and transfer run*

Results from the Run × Condition × Group ANOVA can be obtained from **Table 3A**. There was a significant main effect of Condition with higher anterior insula activation during REGULATE vs. VIEW (**Figure 6A**). None of the interactions were significant. The Run × Group interaction was found at trend level. A linear decrease of anterior insula activation, however, was not visible from the data of both groups.

**Table 3B** lists the results of the transfer run. The Condition × Group effect was found at trend-level. The main effect of Condition was significant with higher anterior insula activation in the REGULATE vs. the VIEW condition. *Posthoc t*-tests paralleled the findings from the amygdala ROI analysis: the experimental group showed less right anterior insula activation compared to the control group in the REGULATE condition. This difference was not present in the VIEW condition. Increased activation in the REGULATE vs. VIEW contrast was revealed in the bilateral anterior

experimental group (*N* = 16), gray bars: control group (*N* = 16). Error bars indicate standard error of mean. A.u., artificial units.

#### **Table 2 | Group statistics and ANOVA results of the transfer run.**

insula in the control group. The experimental group, however, did not significantly increase right anterior insula activation (**Figure 6B**).

# **EXPLORATORY WHOLE-BRAIN ANALYSIS** *Neurofeedback training*

No significant group comparisons were detected with a whole-brain analysis. When inspecting brain activation associated with natural responding to aversive picture viewing (VIEW*>*NEUTRAL), the experimental (Supplementary Table 1A) and the control group (Supplementary Table 1B) showed a similar pattern of activated brain regions, including the occipital lobe, the inferior temporal lobe and medial temporal areas. Activations were also found in the ventrolateral cortex, extending to the anterior insula. An analysis of brain areas implicated in regulation (REGULATE*>*NEUTRAL) brought only one significant cluster for the experimental group in the medial parietal lobe. In contrast, the analysis of the control group showed activated clusters covering occipital, parietal, inferior temporal, and prefontal areas. Activations were also detected in the anterior insula and in the medial temporal lobe. The analysis of brain regions with reduced responding during regulation with amygdala NF (VIEW*>*REGULATE) brought an activation cluster in the ventromedial PFC and another one in the medial occipital/temporal lobes. In contrast, no significant clusters were detected for the control group. The inspection of areas activated by regulation compared to natural responding (REGULATE*>*VIEW) in the experimental group brought one significant cluster in the right dorsolateral PFC. The control group, in contrast, activated an area in the dorsomedial frontal cortex and the thalamus.

#### *Transfer run*

Again, no group differences were found for neither of the contrasts. Supplementary Table 2 summarizes the findings of the


*Table shows parameter-estimate-contrast values from the regions of interest. SD, standard deviation; REG, regulate; NEU, neutral.*

#### **Table 3 | Group statistics and ANOVA results of the anterior insula analysis.**


*Table shows parameter-estimate-contrast values from the anterior insula spheres. (A) Neurofeedback training, (B) Transfer run. SD, standard deviation; REG, regulate; NEU, neutral.*

single-group analyses. Taken together, activations were largely congruent in the VIEW*>*NEUTRAL contrast. Brain regions involved were found in the occipital lobe, inferior temporal lobe (including the fusiform gyrus) and medial temporal lobe (including the bilateral amygdala). At the whole-brain, both groups also responded similar when instructed to regulate (REGULATE*>*NEUTRAL). Besides activation clusters spanning the occipital lobe and inferior temporal lobe, both groups activated the dorsomedial frontal cortex and the dorsolateral PFC. Consistent with the ROI analysis, the experimental group showed activation in the left but not right amygdala in this contrast. No activations were found in the VIEW*>*REGULATE contrast. While control participants activated clusters in the dorsomedial frontal cortex, anterior insula and left ventrolateral PFC contrasting REGULATE vs. VIEW, no activations exceeded the cluster-threshold in the experimental group.

# **RATING DATA**

A repeated measures ANOVA for arousal and valence ratings of the stimuli used in the training neither showed significant group differences nor within-subjects effects (VIEW vs. REGULATE). There was also no significant Group interaction or main effect of Group found. An analysis of the arousal and valence ratings of the stimuli used in the transfer run also did not show significant group or within-subjects effects.

In terms of perceived regulation success, we did not find any significant differences between the experimental group [3.04 ± 1.89 (mean ± *SD*)] and the control group [3.85 ± 1.82; *T*(30) =

**FIGURE 6 | Results of the region-of-interest analysis of the anterior insula.** Mean parameter estimate of all voxels within the anterior insula masks of the REGULATE*>*NEUTRAL and VIEW*>*NEUTRAL contrast. Error bars indicate standard error of mean. A.u., artificial units. **(A)** Neurofeedback training (mean over all runs). **(B)** Transfer run.

1*.*530, *p* = 0*.*226]. Results did not change significantly when participants' ratings were compared on a run-by-run basis.

#### **THERMOMETER VARIABILITY**

The three-way ANOVA did not indicate an effect of Group on the variance of the number of bars displayed during the experimental runs (interactions: Condition × Run × Group: *p* = 0*.*949, Run × Group: *p* = 0*.*149, Condition × Group: *p* = 0*.*582; main effect of Group: *p* = 0*.*685).

# **DISCUSSION**

Within one session of rt-fMRI-NF training, subjects receiving contingent amygdala feedback were successful in down-regulating amygdala activation in response to aversive pictures. In a subsequent run, subjects were able to decrease right amygdala activation without the feedback signal, which is in line with the hypothesis of a transfer of learned amygdala down-regulation. A control group receiving feedback from a standardized control region located in the rostal caudate did also reduce amygdala activation during training. A significant Region by Hemisphere by Group interaction indicated a specific effect of rt-fMRI-NF training on brain self-regulation in the transfer test. A trend in the two-way interactions of the amygdala (Hemisphere by Group) and right hemisphere (Region by Group) may suggest that this effect was lateralized to the right side. This proof of concept study is in line with previous reports on rt-fMRI-NF training of amygdala regulation (Zotev et al., 2011; Brühl et al., 2014) and extends the existing literature by providing evidence for the feasibility to use affective pictures as stimulus material to train amygdala down-regulation.

To provide evidence for a specificity of the treatment, the effect of the intervention needs to exceed the effect of a sham treatment which mimics external aspects of the real treatment. In other words, a placebo control is needed. In our study, we used the signal of a spatially well-defined control region located at the rostral caudate. We chose this region for its similarity with the experimental region in tissue composition and we matched the number of voxels taken to extract the NF signal for both groups. Furthermore, the literature does not indicate a function of the caudate comparable to the amygdala in emotion processing. This suggests that feedback from the control region may not specifically help in down-regulating amygdala activation in response to aversive stimuli. *Post-hoc* tests of the training phase illustrate, that the control region showed a positive change in BOLD signal amplitude in response to aversive pictures during REGULATE and VIEW (Supplementary Figure 1) and an analysis on the dynamics of the thermometer display does not indicate a significant difference between groups. The belief of receiving contingent feedback of affective brain activation may have prevented control subjects to become suspicious of a sham treatment or to become frustrated because of failure during the course of training. Due to the instructions, subjects knew that they would somehow have to engage in emotion regulation. Since we tested healthy participants, it is conceivable that all subjects were equipped with effective strategies to control their emotions in everyday life, and they may have employed these strategies to try to regulate the thermometer during training. Subjects of both groups may have applied strategies they had evaluated as successful based on their interpretation of the feedback signal and this may have impeded the detection of significant group differences during training. In this context it is not surprising that ratings of regulation success do not indicate a group difference. However, regulation-success was only estimated to be in the medium to low range. The lack of a difference in subjective success is in line with a previous report by Lawrence et al. (2013), who had trained one group with anterior insula NF and a control group with feedback from a control region located in the parietal lobe. It may be that a retrospective evaluation of regulation success is not sensible to capture between-group differences, especially when using placebo-feedback as a control. A better way to assess group differences may be via behavioral measures which are associated with the specific psychological function the intervention is thought to modulate. Regarding amygdala NF, future studies could let participants rate the subjective arousal and valence after regulation vs. natural viewing, which is known to be associated with amygdala activation to affective pictures (Zald, 2003; Anders et al., 2004).

The results of the transfer run provide first evidence that a rt-fMRI-NF training of amygdala down-regulation can have a differential effect on brain self-regulation in a placebo-controlled study. Further, results indicate a hemispheric asymmetry of brain self-regulation. There is an ongoing debate on a functional differentiation between left and right amygdala (Baas et al., 2004; Costafreda et al., 2008; Sergerie et al., 2008). However, methodological issues have been raised on interpreting laterality differences (LaBar et al., 2001; Mathiak et al., 2012) and no consistent laterality difference of amygdala activation has been reported in the emotion regulation literature (Buhle et al., 2013). A possible but rather speculative explanation for a lateralized transfer effect may state that subjects had tried different emotion regulation strategies during training which may have been successful in down-regulating the right as well as left amygdala. In the transfer run, however, participants may have used a strategy they had evaluated during training as most successful. Strategies approved as most successful may have preferentially involved neural processes of top-down control of the right amygdala. This interpretation endorses the choice of Brühl et al. (2014)to use the right amygdala for NF training.

The results of the anterior insula analysis largely complemented the results of the amygdala analysis. In the transfer run, the experimental group did not show a significant difference of right anterior insula responding when comparing the REGULATE to the VIEW condition, while the control group significantly up-regulated the anterior insula when instructed to regulate. In the group comparison, the experimental group showed less activation during REGULATE than the control group, which is in support of improved regulation of affective brain structures after amygdala NF. The group interaction, however, did only show a trend. As opposed to a training effect on the regulation of a rather circumscribed component of the brain's affective system, this result suggests that amygdala NF training may extend to the self-regulation of other brain regions in the domain of emotional processing. Future studies are needed to corroborate this finding.

To evaluate general effects of training and transfer, we conducted a whole-brain analysis between and within groups. The between-group analysis did not show significant group interactions. When instructed to naturally respond to aversive picture presentation, both groups congruently activated regions of the medial temporal lobe and the anterior insula, which are consistently found to be involved in the processing of emotion stimuli (Phan et al., 2002; Kober et al., 2008). This finding was consistent across the training and the transfer phase. When contrasting the REGULATE-condition against the non-affective control condition, both groups showed highly overlapping brain regions in the analysis of the transfer run. Results indicate that subjects from both groups had engaged in the top-down control of limbic and para-limbic regions when instructed to regulate their brain responses to aversive stimuli after training. The involvement of lateral PFC (Kalisch, 2009; Buhle et al., 2013), medial PFC/anterior cingulate cortex (Etkin et al., 2011; Paret et al., 2011), and anterior insula (Dosenbach et al., 2006; Hollmann et al., 2012; Veit et al., 2012) in emotion regulation and cognitive control is consistent with the existing fMRI literature. The singlegroup analyses of the REGULATE condition differed, however, regarding the rt-fMRI-NF training. Contrasting regulation with natural viewing of aversive pictures in the experimental group brought an acitivation cluster in the right lateral PFC, while the control group displayed activation of the dorsomedial frontal cortex. In the transfer run, the dorsomedial frontal cortex was again found activated only by the control group, indicating the involvement of similar brain structures during training and transfer in this group. The results may give a hint for the neural patterns implicated in down-regulation of the amygdala with and without contingent amygdala NF training. However, betweengroup differences may result from voxel-thresholding and may not reflect real differences in the involvement of brain regions between groups. Since we did not find significant group interactions, between-group differences remain descriptive and should be interpreted with caution.

We did not find evidence for a run-to-run improvement of amygdala down-regulation in the experimental group. This could have had several reasons, like exhaustion due to the duration of the scanning session or task difficulty. Future studies should try to improve the task design in order to make the NF training more efficient.

There are several limitations for this study. Regarding our hypotheses, we cannot conclude from the results that rt-fMRI-NF had a specific effect on the down-regulation of the BOLD signal amplitude of the amygdala response during the training phase. Additionally, we did not include a pre-training test to evaluate pre-post comparisons in amygdala regulation success. However, without any experience of rt-fMRI-NF, the instruction to regulate a brain region (the "feeling center") may be difficult to follow. Therefore, and to prevent exhaustion due to long scanning duration, we decided against a run similar to the transfer run before the training. Data from pre-post comparisons, however, could advance our understanding of whether and to what extent amygdala NF training can change the activation of neural circuitries of emotion regulation and may be easier to implement with another experimental design. Another potential limitation may be the display of contingent feedback during the control conditions (VIEW and NEUTRAL). First, subjects of both groups reported that the thermometer did sometimes rise during the NEUTRAL condition. Signal increases without an external aversive stimulus may have had several reasons, such as affective responses to internal stimuli or signal noise. This may have confused participants and could have made them suspicious of receiving a sham feedback. Second, contingent amygdala feedback during VIEW may have triggered evaluation and control processes similar as in the REGULATE condition. An at-trend main effect of the factor Group in the Condition × Group ANOVA of the right amygdala is in line with this interpretation and may result from the generalization of learned regulation to the natural viewing condition. Future studies are needed for replication and should consider the potential confound of a feedback display during the control condition in the study design. In this study only healthy female subjects were tested. There are fMRI studies reporting gender differences in the involvement of prefrontal and limbic regions during emotion regulation (McRae et al., 2008; Mak et al., 2009; Domes et al., 2010). Thus, the results reported here may not generalize to samples of male participants.

control region on brain self-regulation. This study is a starting point for further research toward the application of rt-fMRI-NF of the amygdala as a potential intervention in psychiatric populations. In particular, down-regulation of the amygdala as demonstrated in the current study and elsewhere (Brühl et al., 2014) may be helpful for disorders characterized by problems in emotion regulation and elevated amygdala activity such as borderline personality disorder. In these patients, training skills for emotion regulation is a decisive aspect of successful psychotherapies (Stoffers et al., 2012). NF may be used to test and train individual emotion regulation skills and therefore provide an excellent tool to increase efficacy and time-to-success of psychotherapy in such conditions.

# **AUTHOR CONTRIBUTIONS**

Christian Paret, Rosemarie Kluetsch, Matthias Ruf, Traute Demirakca, Gabriele Ende, and Christian Schmahl were involved in the design and conception of the work. Rosemarie Kluetsch, Steffen Hoesterey, and Christian Paret conducted the data acquisition. Christian Paret analyzed the fMRI data and wrote the manuscript. All authors (Christian Paret, Rosemarie Kluetsch, Matthias Ruf, Traute Demirakca, Steffen Hoesterey, Gabriele Ende, Christian Schmahl) were involved in the interpretation of data and revised the draft critically for important intellectual content. All authors approved the final version to be published and agreed to be accountable for all aspects of the work.

# **ACKNOWLEDGMENTS**

We would like to thank Lydia Robnik and Martin Jungkunz for their support in study organization, Rebekka Knies, Kathrin Haeussler, and Marie-Luise Zeitler for recruitment and conducting the diagnostic interviews, Anja Voigt for her help with the preparation of figures, and Bettina Kirr for assisting in data acquisition. The study was part of the clinical research unit 256, funded by the German Research Foundation (DFG, SCHM 1526/14-1, EN 361/13-1).

# **SUPPLEMENTARY MATERIAL**

The Supplementary Material for this article can be found online at: http://www*.*frontiersin*.*org/journal/10*.*3389/fnbeh*.*2014*.* 00299/abstract

### **REFERENCES**


regulation. *J. Cogn. Neurosci.* 18, 1266–1276. doi: 10.1162/jocn.2006.18. 8.1266


and cognitive reappraisal strategies. *Eur. Arch. Psychiatry Clin. Neurosci*. doi: 10.1007/s00406-014-0510-z. [Epub ahead of print].


patients with major depressive disorder. *PLoS ONE* 9:e88785. doi: 10.1371/journal.pone.0088785


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 28 May 2014; accepted: 14 August 2014; published online: 18 September 2014.*

*Citation: Paret C, Kluetsch R, Ruf M, Demirakca T, Hoesterey S, Ende G and Schmahl C (2014) Down-regulation of amygdala activation with real-time fMRI neurofeedback in a healthy female sample. Front. Behav. Neurosci. 8:299. doi: 10.3389/fnbeh. 2014.00299*

*This article was submitted to the journal Frontiers in Behavioral Neuroscience.*

*Copyright © 2014 Paret, Kluetsch, Ruf, Demirakca, Hoesterey, Ende and Schmahl. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Self-regulation of circumscribed brain activity modulates spatially selective and frequency specific connectivity of distributed resting state networks

Mathias Vukeli´c1,2\* and Alireza Gharabaghi 1,2 \*

<sup>1</sup> Division of Functional and Restorative Neurosurgery and Division of Translational Neurosurgery, Department of Neurosurgery, Eberhard Karls University Tuebingen, Tuebingen, Germany, <sup>2</sup> Neuroprosthetics Research Group, Werner Reichardt Centre for Integrative Neuroscience, Eberhard Karls University Tuebingen, Tuebingen, Germany

#### Edited by:

Francisco Javier Zamorano, Universidad del Desarrollo, Chile

#### Reviewed by:

Francisco Gomez, University of Liege, Belgium Jose Miguel Miguel Sanchez Bornot, University of Ulster, UK

#### \*Correspondence:

Mathias Vukeli ´c and Alireza Gharabaghi, Division of Functional and Restorative Neurosurgery and Division of Translational Neurosurgery, Department of Neurosurgery, Eberhard Karls University Tuebingen, Otfried-Müller-Straße 45, 72076 Tuebingen, Germany mathias.vukelic@cin.unituebingen.de; alireza.gharabaghi@uni-tuebingen.de

> Received: 30 June 2014 Accepted: 29 June 2015 Published: 14 July 2015

#### Citation:

Vukeli ´c M and Gharabaghi A (2015) Self-regulation of circumscribed brain activity modulates spatially selective and frequency specific connectivity of distributed resting state networks. Front. Behav. Neurosci. 9:181. doi: 10.3389/fnbeh.2015.00181 The mechanisms of learning involved in brain self-regulation have still to be unveiled to exploit the full potential of this methodology for therapeutic interventions. This skill of volitionally changing brain activity presumably resembles motor skill learning which in turn is accompanied by plastic changes modulating resting state networks. Along these lines, we hypothesized that brain regulation and neurofeedback would similarly modify intrinsic networks at rest while presenting a distinct spatio-temporal pattern. High-resolution electroencephalography preceded and followed a single neurofeedback training intervention of modulating circumscribed sensorimotor low βactivity by kinesthetic motor imagery in eleven healthy participants. The participants were kept in the deliberative phase of skill acquisition with high demands for learning self-regulation through stepwise increases of task difficulty. By applying the corrected imaginary part of the coherency function, we observed increased functional connectivity of both the primary motor and the primary somatosensory cortex with their respective contralateral homologous cortices in the low β-frequency band which was self-regulated during feedback. At the same time, the primary motor cortex—but none of the surrounding cortical areas—showed connectivity to contralateral supplementary motor and dorsal premotor areas in the high β-band. Simultaneously, the neurofeedback target displayed a specific increase of functional connectivity with an ipsilateral fronto-parietal network in the α-band while presenting a de-coupling with contralateral primary and secondary sensorimotor areas in the very same frequency band. Brain self-regulation modifies resting state connections spatially selective to the neurofeedback target of the dominant hemisphere. These are anatomically distinct with regard to the corticocortical connectivity pattern and are functionally specific with regard to the time domain of coherent activity consistent with a Hebbian-like sharpening concept.

Keywords: self-regulation of brain activity, neurofeedback, brain-computer interface, resting state networks, functional connectivity, corrected imaginary part of coherency, neuronal reorganization, Hebbian-like plasticity

# Introduction

Brain-computer interfaces are currently being applied in neurofeedback training for a variety of brain-related pathological conditions to alleviate related symptoms (Wyckoff and Birbaumer, 2014). In such an environment, contingent feedback of the neuronal state is provided to enhance self-regulation of brain activity via operant conditioning. This neurofeedback training is expected to selectively induce use-dependent neuroplasticity for re-normalizing pathological brain activity and achieving behavioral gains (Daly and Wolpaw, 2008). Although variances of brain selfregulation could be attributed to different neuronal processes (Blankertz et al., 2010; Grosse-Wentrup et al., 2011; Halder et al., 2011; Vukeli´c et al., 2014; Vukeli´c and Gharabaghi, 2015), the underlying mechanisms of learning this skill still have to be uncovered to exploit the full potential of this technique for clinical application (Bauer and Gharabaghi, 2015a,b).

Due to its procedural nature and the involvement of the cortical-basal ganglia loop (Birbaumer et al., 2013), the skill of volitionally changing brain activity has been proposed to be comparable to implicit motor skill learning. Several neuroimaging studies revealed that a distributed network consisting of prefrontal, premotor, supplementary motor, primary sensorimotor, and parietal regions is recruited when acquired motor skills are executed (Hallett and Grafman, 1997; Halsband and Lange, 2006; Hardwick et al., 2013). What is more, resting state measurements, being unbiased by activity during any task, revealed that-particularly in frontoparietal areas these networks were specifically modulated by previous motor skill learning but not by the motor performance (Albert et al., 2009). In addition, motor learning resulted in functionally distinct changes in subsequent intrinsic networks, revealing a distributed pattern of sensory and motor plasticity (Vahdat et al., 2011). These studies suggested that intrinsic resting state activity may reflect the processing of memory during consolidation, thereby resembling functional neuronal networks involved in skill learning (Albert et al., 2009).

In this context, we hypothesized that volitional modulation of brain activity modifies subsequent intrinsic networks similar to motor learning. Moreover, we expected these resting state networks to show a topographic distribution of synchronized cortical regions similar to that observed during neurofeedback training (Grosse-Wentrup et al., 2011; Halder et al., 2011; Vukeli´c et al., 2014; Vukeli´c and Gharabaghi, 2015) due to the cognitive demanding nature of brain self-regulation (Wander et al., 2013). On the basis of our previous findings during volitional brain control (Bauer et al., 2015; Vukeli´c et al., 2014; Vukeli´c and Gharabaghi, 2015) we went on to hypothesize that frequency-specific and spatially selective changes of functional connectivity occur and therefore applied a highdensity electroencephalography study to capture connectivity patterns via the concept of the imaginary part of the coherency function (Nolte et al., 2004; Ewald et al., 2012). This approach has been applied in recent studies as a robust method to interfere functional connectivity (Martino et al., 2011; Dubovik et al., 2012; Westlake et al., 2012; Mottaz et al., 2014; Notturno et al., 2014).

# Materials and Methods

# Subjects

We recruited eleven healthy subjects (mean age = 25.83 ± 3.1 years, four female), all of them right-handed as assessed by the Edinburgh Handedness Inventory (Oldfield, 1971). Subjects gave their written informed consent before participation and received monetary compensation. The study protocol was approved by the local ethics committee of the Medical Faculty of the University of Tuebingen, Germany. The current data were collected as part of a larger research project investigating the neurophysiology of neurofeedback; whereas previous work analyzed the cortical physiology during neurofeedback training (Vukeli´c and Gharabaghi, 2015), this evaluation focused on the resting state networks after the interventions.

# Data Acquisition and Experimental Paradigm

All subjects were comfortably seated upright in a chair. High resolution scalp EEG potentials were recorded (BrainAmp, Brainproducts GmbH, Germany) from 128 positions according to the extended international 10–05 system, with active electrodes based on Ag/AgCl (actiCAP, Brainproducts GmbH, Germany). The left mastoid was used as common reference and grounded to AFz. All impedances were kept below 20 k at the onset of each session. EEG data was digitized at 1 kHz, high-pass filtered with a time constant of 10 s and stored for off-line analysis (Brainvision, Brain Products GmbH, Germany).

Each subject was exposed to one neurofeedback training experiment, lasting 48 min, to acquire volitional control of regional low β-oscillations (16–22 Hz) induced by kinesthetic motor imagery of hand movements (right and left hand) which resulted in strong sensorimotor power fluctuations contralateral to movement imagination. The successful control of contralateral sensorimotor β-oscillations was translated into contingent neurofeedback. In order to reduce the impact of the feedback modality, participants received 24 min of haptic feedback (control of a hand orthosis which was attached to the right or left hand of the subjects) and 24 min of visual feedback (control of a cursor ball towards a selected target on a computer screen) in a randomized order. In order to balance for the impact of cerebral specialization, the subjects had to self-regulate either left (FC3, C3, and CP3) or right (FC4, C4, and CP4) cortex in half of all trials, respectively. This resulted in a total of four feedback sessions each of which lasted 12 min, i.e., regulating left hemisphere with haptic feedback, regulating left hemisphere with visual feedback, regulating right hemisphere with haptic feedback, regulating right hemisphere with visual feedback. For the classification of successful brain self-regulation an adaptive linear classifier procedure was used as described recently (Gharabaghi et al., 2014a; Vukeli´c et al., 2014; Vukeli´c and Gharabaghi, 2015). Each feedback session was subdivided into three runs with each run separated into 16 trials. To ensure that the participants remained in the deliberative phase of skill acquisition with high demands for learning self-regulation, we increased the task difficulty after each run, i.e., we increased the threshold value of the online classifier to ensure that feedback was provided only when the subjects reached either 50% (low difficulty), 30% (moderate difficulty), or 10% (high difficulty) of the strongest β-eventrelated desynchronization (ERD) modulation in the first, second and third run, respectively.

Before (PRE) and after (POST) the neurofeedback training, we recorded 6 min of resting state activity with the subjects alternating between the conditions ''relax with eyes open (EO)'' and ''relax with eyes closed (EC)'' every 15 s (Blankertz et al., 2010). During the EO condition, the subjects fixated a central cross on a computer screen. An auditory beep tone caused the subjects to switch between EC and EO conditions.

#### Data Pre-Processing

The present analysis considered EEG data during the EO condition (Blankertz et al., 2010). Each 15 s period was concatenated, resulting in a data stream of 3 min per subject both for each PRE and POST neurofeedback recording. Artifacted EEG channels (PO9), that had been detected by visual inspection were not taken into account. The EEG data were detrended, zero-padded and band-pass filtered between 1–42 Hz, using a first order zero-phase lag Finite Impulsive Response (FIR) filter. We divided the whole data set into 3 s epochs, automatically rejecting any epochs that contained artifacts with an amplitude >100 µV (Sanei, 2007). Finally, the artifact-free EEG data was re-referenced to mathematically linked mastoids (Nunez, 2006).

#### Estimation of β-Modulation Range

To estimate the ability of each subjects to self-regulate his/her oscillatory activity during the training experiment we calculated the β-modulation range as described in detail elsewhere (Vukeli´c et al., 2014; Vukeli´c and Gharabaghi, 2015). In short, the β-modulation range was calculated off-line from the event-related spectral perturbation, as implemented in EEGLab toolbox (Delorme and Makeig, 2004), in the same frequency band (16–22 Hz) and from the same electrodes (FC3/4, C3/4, and CP3/4) as used for online control. This value described the maximal potential of each subject to synchronize and de-synchronize local oscillatory β-activity. The overall β-modulation range, i.e., for the haptic and visual feedback sessions, was calculated trial-wise for regulating the left (**Figure 1A**) and right hemisphere (**Figure 1B**) respectively. In cases where no trials had to be removed due to artifacts, we excluded the first trials in each run to adjust the number of trials in each subject. This resulted in the trials 1–15, 16–30, 31–45 for the first, second and third run, respectively (see **Figure 1**).

#### Estimation of Functional Connectivity Networks

To calculate functional connectivity we utilized the imaginary part of coherence (iCOH; Nolte et al., 2004). iCOH is a robust connectivity measure ignoring relations at zero phase lag and is therefore insensitive to volume conduction properties. Since the original proposed iCOH might exhibit a spatial bias towards long-range synchronizations, we used the corrected version of the iCOH function (ciCOH) as suggested by Ewald et al. (2012). This version shares the same properties as the original iCOH function but includes additional features to compensate for the preference of remote interactions. ciCOH was calculated for each artifact free epoch, where the ciCOH function is based on an estimation of the complex coherency function. Hence, epochs were further divided into 1 s segments with 50% overlap resulting in a frequency resolution of δf = 1 Hz (Nolte et al., 2004). The segments were subsequently multiplied with a Hanning window, and the cross-spectrum between two time series, was defined by calculating the Fourier transformation and averaging over 1 s segments (Nolte et al., 2004):

$$S\_{\vec{ij}}(f) = \frac{1}{N} \sum\_{k=1}^{N} z\_i(f,k) z\_j^\*(f,k) \tag{1}$$

where zi(·) and zj(·) represent the Fourier transform of the time series for channels i and j, k the segments of length 1 s, and N the total number of segments.

For each channel pair i and j the complex coherency function was defined as the normalized cross-spectrum:

$$COH\_{\vec{ij}}(f) = \frac{S\_{\vec{ij}}(f)}{\sqrt{S\_{\vec{ii}}(f)S\_{\vec{jj}}(f)}} \tag{2}$$

Where Sij(·) was the cross-spectrum between channels i and j, and Sii(·), Sjj(·) represented the auto-spectra for channels i and j, respectively.

Since, our neurofeedback training procedure focused on electrodes over selected sensorimotor regions, i.e., premotor (PM, FC3/4), primary motor (M1, C3/4) and primary somatosensory (S1, CP3/4) regions, we defined each of them separately as seed electrodes and evaluated systematically the functional connectivity between these circumscribed regions of interest (ROIs) and the whole brain (all other EEG channels). Hence, the ciCOH function was calculated from the complex coherency function (Ewald et al., 2012):

$$\text{ciCOOH}\_{\text{Seedj}}(f) = \frac{\text{Im}\left(\text{COH}\_{\text{Seedj}}(\text{f})\right)}{\sqrt{\left(1 - \text{Re}(\text{COH}\_{\text{Seedj}})^2\right)}}\tag{3}$$

where Seed denotes the seed electrode f indicate frequency bins and Im(·) and Re(·) denote the imaginary and real parts, respectively. The ciCOH was fisher z-transformed to fit a Gaussian distribution (Rosenberg et al., 1989; Nolte et al., 2004). We evaluated the functional connectivity within predefined frequency bands of interest (FOI): α (8–14 Hz), low β (15–25 Hz), and high β (26–40 Hz). In a next step, the functional connectivity measure was obtained by averaging the absolute value of ciCOH across frequencies within each predefined FOI.

Furthermore, control analyses (i.e., control for spatial selectivity) of functional connectivity were conducted by defining seed electrodes immediately surrounding the neurofeedback ROIs, i.e., the electrodes adjacent to the FC3, C3, and CP3

electrodes, respectively. All data analysis was performed offline with custom written scripts in MATLAB®.

## Statistics

To analyze networks changes induced by brain self-regulation we compared the functional connectivity (ciCOH) between the PRE and POST condition. Here, we conducted a cluster-based permutation analysis which offers the opportunity to incorporate neurophysiologically motivated constraints to the test statistic (i.e., spatially clustering neighboring electrodes). This increases the sensitivity of the statistical test and controls for the familywise error rate, thereby correcting for the multiple comparison problem (Nichols and Holmes, 2002; Maris and Oostenveld, 2007; Maris et al., 2007; Maris, 2012). This entailed the use of a cluster-based non-parametric randomization approach as implemented in FieldTrip (Oostenveld et al., 2011). Here, a multiple dependent sample t-statistic was conducted to establish the topography of resting state motor networks (i.e., seed electrodes) showing significant functional connectivity (ciCOH) differences between the POST and PRE training conditions for each predefined FOI. Thus, t-values exceeding a threshold of p < 0.01 (uncorrected) where spatially clustered based on neighboring electrodes. The cluster level statistics were defined as the sum of t-values within every cluster. The correction of multiple comparisons was carried out by considering the 95th percentile (two tailed) of the maximum values of summed t-values estimated from an empirical reference distribution. t-values exceeding this threshold were thus considered as significant at p < 0.05 (corrected).

The reference distribution of maximum values was obtained by means of a permutation test (randomly permuting the ciCOH across the POST and PRE training resting state EEG data for 1000 times). This non-parametric approach was used to evaluate the functional connectivity topographies of POST- vs. PRE-training differences of resting state brain activity.

# Results

The overall β-modulation, i.e., of the haptic and visual feedback sessions, were analyzed across trials for the left (**Figure 1A**) and right (**Figure 1B**) hemisphere, respectively, and revealed in a twoway ANOVA no main effects for ''runs'' F(2,84) = 0.32, p = 0.72 or ''hemisphere'' F(1,84) = 2.02, p = 0.16 nor for the interaction between these factors F(2,84) = 0.27, p = 0.76. Thus, participants showed a stable performance of brain-self regulation for both hemispheres throughout the experiment, i.e., they adapted to the different levels of difficulty in each run.

The non-parametric randomization test revealed significant changes of functional connectivity for the neurofeedback targets of the dominant left hemisphere (FC3, C3, CP3, see **Figures 2**, **3**), but not of the non-dominant right hemisphere (FC4, C4, CP4, see **Figure 4**). These findings were spatially selective, i.e., they were not observed in the surrounding electrodes (see **Figure 5**). More specifically, we observed increased functional connectivity of both the seed electrodes overlying the primary motor and the primary somatosensory cortex of the left hemisphere with their respective contralateral homologous cortices in the low β-frequency band which were self-regulated during feedback (see **Figures 2**, **3**, middle). Simultaneously, the seed electrode over the primary motor cortex presented a decrease of functional connectivity with electrodes in midline parietal area in the same frequency band (see **Figure 2**, middle).

At the same time, the seed electrode over the left primary motor cortex, and none of the surrounding electrodes in other cortical regions, exhibited an increased connectivity to contralateral electrodes over the supplementary motor and dorsal premotor areas in the high β-band (see **Figure 2**, right). For the α-band, all neurofeedback target ROIs of the left hemisphere showed an ipsilateral increase of functional connectivity with electrodes over frontal areas, while the electrode over the

show t-value topographies of ciCOH for POST vs. PRE differences of intrinsic oscillatory activity in the α-band, low β-band, and high β-band. Electrode clusters, displaying significant differences in the non-parametric statistical

indicates the seed electrode position in the primary motor cortex (M1). Red color indicates increase and blue color decrease in functional connectivity (ciCOH) in the POST training as compared to PRE training condition.

premotor cortex exhibited additional functional coupling with electrodes over parietal regions (see **Figures 2**, **3**, left). Simultaneously, the seed electrode over the left primary motor cortex showed a decrease of functional coupling with electrodes in contralateral primary and secondary sensorimotor areas in the very same frequency band (see **Figures 2**, **3**, left).

Neither the neurofeedback targets of the right hemisphere (FC4, C4, CP4, see **Figure 4**) nor any of the surrounding seed electrodes showed comparable changes of connectivity patterns (see **Figure 5**).

# Discussion

This study aimed to shed light on possible neurophysiological mechanisms of learning to volitionally modulate circumscribed brain activity by applying high-resolution electroencephalography to study the immediate after-effects of a single neurofeedback intervention on the resting state network architecture of oscillatory brain activity. While most previous studies exploring the influence of learning and neuroplastic changes on the subsequent intrinsic brain

connectivity used functional magnectic resonance (Albert et al., 2009; Vahdat et al., 2011; Harmelech et al., 2013), we decided to instead use neuroelectrical recordings to enable us to examine frequency-specific measures of connectivity. The reason for this was that patterns of coherent oscillations have been shown to match with a broad variety of attentional, cognitive and sensorimotor behavior (Destexhe et al., 1999; Steriade, 2006; Engel and Fries, 2010; Siegel et al., 2012; Engel et al., 2013). For the purpose of restoring lost motor functions for example, neurofeedback of sensorimotor β-band (15–30 Hz) activity seems to be particularly suited (Gharabaghi et al., 2014a,b,c) as this frequency band is linked to the natural communication between cortex and peripheral muscular activity. However, even these approaches have been shown to activate a distributed cortical network in a lower, i.e., α-frequency band (Vukeli´c et al., 2014), thereby bridging the abilities and cortical networks of motor imagery and motor execution (Bauer et al., 2015).

Previous studies on visuomotor skill learning suggested that the same networks which connected prefrontal cortices (PFC), premotor (PM) regions, supplementary motor areas (SMA), primary sensorimotor, and parietal cortices, and which were recruited in the course of training, shaped the pattern of the following intrinsic brain activity (Albert et al., 2009; Vahdat et al., 2011). These resting state patterns would therefore reflect the history of neuronal activation during the skill learning period encompassed as lasting increases and/or decreases of connectivity among these cortical regions. Such neuronal changes have been shown to involve both short-term (immediate) and long term (long-lasting) Hebbian-like effects of previous cortical activation (Harmelech et al., 2013). This very study was the first to describe such effects on intrinsic networks following brain self-regulation via functional Magnetic Resonance maging (MRI)-based neurofeedback. However, due to the nature of the technique applied, the co-activations of distant cortical areas could not be characterized on different frequency scales. Here, we successfully extended this line of research by using EEG as a tool to capture frequency-specific measures of functional connectivity.

By regularly switching the feedback modality and the trained cortical hemisphere, and by continuously increasing the difficulty of the feedback task, we succeeded in keeping the subjects in the deliberative phase of skill acquisition throughout the whole experiment to trace learning and not

performance related connectivity changes in the subsequent intrinsic networks.

Switching between feedback modalities might have caused the overall effects of the intervention to be determined by the characteristic features of only one modality, e.g., the sensory stimulation of the haptic feedback. A recent study addressed this question by contrasting the very same two feedback modalities as in the present study while capturing the entrained cortical networks during the task (Vukeli´c and Gharabaghi, 2015). This comparison between haptic/proprioceptive and visual feedback revealed, with respect to the same frequency spectrum analyzed in the present study, significant differences only for the low β-band. In this low β-frequency band, the haptic condition revealed a significantly stronger decoupling of the trained, i.e., left, motor cortex from bilateral premotor and frontal areas as compared to the visual feedback condition, i.e., a pattern relevantly different from those connectivity changes observed in the present study. It is therefore plausible to assume that the findings of the current study were not determined by one feedback modality only.

In this context, it is remarkable that the learning-related connectivity changes were lateralized to the dominant left hemisphere of the participants despite the fact that both hemispheres underwent the same amount of feedback training. This observation might reflect the functional specialization, i.e., that planning of manual actions of either hand involves the left posterior parietal and the left motor area (Rushworth et al., 2003; Johnson-Frey et al., 2005; Bauer et al., 2015). Similarly, the ability to execute movements of the left hand is also characterized by connectivity within bilateral motor regions, especially by signals from the left to the right motor areas. This might reflect the relay of planned movements from the left to the right hemisphere, in accordance with hemispheric specialization in right-handers (van den Berg et al., 2011; Bauer et al., 2015). Interestingly enough, we were able to demonstrate how these interhemispheric communications were mediated in different frequency bands in a complex way, i.e., increased functional connectivity of the seed electrode overlying the primary motor cortex of the left hemisphere with its contralateral homologous cortex in the low β-frequency and with contralateral electrodes over the supplementary motor and dorsal premotor areas in the high β-band. At the same time, there was a decrease of functional connectivity of the very same seed electrode over the primary motor cortex with contralateral electrodes over primary and secondary sensorimotor areas in the α-band and with electrodes over midline parietal area in the low β-band.

Bilateral somatomotor regions are known to have a high inclination to oscillate synchronously in the β-band during intrinsic natural brain activity (Marzetti et al., 2013). The stronger engagement of interhemispheric sensorimotor cortices might represent an initial recruitment of homologues regions, where the interplay of these interactions undergoes dynamic plastic changes. This is crucial for motor control and motor skill learning (Beaulé et al., 2012; Takeuchi et al., 2012). With regard to brain lesions, such as occur following a stroke, maladaptive neuronal reorganization of interhemispheric primary sensorimotor cortices are related to impaired motor and cognitive behavior (Rehme et al., 2011a,b; Dubovik et al., 2012). Furthermore, abnormal alterations of intrinsic functional communication between the sensorimotor network and higher order supplementary motor cortex is also related to impaired motor behavior after stroke (Inman et al., 2012). It is worth mentioning that we also detected frequency (high β-band) specific effects of increased communication between the electrode over the left primary motor network connected with higher order motor regions such as electrodes over the SMA and dorsal PM regions in the right hemisphere (**Figure 2**). This could be due to the fact that already motor skill learning involves two parallel cortico-subcorticalcerebellar circuits (frontoparietal striatum-cerebellar loop and sensorimotor striatum-cerebellar loop) which coordinate both spatial and motor features of learning (Hikosaka et al., 2002). In this context, the communication between primary motor, SMA, and PM regions is liable to coordinate the transformation between these two systems related to different aspects of skill acquisition (Hikosaka et al., 2002; Vahdat et al., 2011). Recent results showing a higher recruitment of SMA in relation to positive brain-computer interface (BCI) control (Halder et al., 2011) are also in agreement with our observation and further highlights the special relevance of SMA for neurofeedback training.

Synchronization of oscillations in the α-band has been proposed to underlie attentional states, memory processes and motor planning during sensorimotor behavior (Sauseng and Klimesch, 2008; Palva and Palva, 2011; Siegel et al., 2012). Our results highlight an immediate after-effect of neurofeedback training on distributed fronto-centro-parietal networks synchronously oscillating in the α-band (**Figures 2**, **3**). We found a consistent increase of the functional connections between electrodes over the left PFC with electrodes over the left PM and primary sensorimotor regions. Left PM regions and PFC are primarily involved in the skill acquisition of new motor sequences and in the short-term storage and encoding of these new learned sequences (Schubotz and von Cramon, 2003; Hardwick et al., 2013). Increased functional connectivity between PFC and PM are probably related to high attentional demands (Hikosaka et al., 2002; Sun et al., 2007) during skill learning. The increased connections between PFC and primary sensorimotor regions could therefore be related to short-term storage of information and encoding of unfamiliar new skills and could also reflect higher cognitive and attentional demands (Grafton et al., 2008; Kantak et al., 2012). Moreover, we found a dissociation of interhemispheric communication between electrodes in bilateral primary sensorimotor regions in the α- and β-band (**Figure 2**). The α-band showed a down-regulation, while the β-band showed an up-regulation of functional connectivity. These results possibly reflect a cross-frequency interaction between these two components, which has frequently been observed during cognitive tasks (Palva et al., 2005) and which has also been found to be present during intrinsic brain oscillations (Nikulin and Brismar, 2006; Chella et al., 2014). Along these lines, we recently demonstrated that synchronized coupling of global αoscillations regulated the volitional modulation of regional βband sensorimotor activity related to the successful control of these oscillations (Vukeli´c et al., 2014). In such a crossfrequency framework, a hierarchy seems to exist in which the lower frequencies modulate the oscillations of higher frequencies (Jensen and Colgin, 2007; Canolty and Knight, 2010).

From the methodological point of view, brain connectivity analysis has to disentangle true neuronal interferences from the phenomenon of volume conduction or field spread, which occurs at zero time (or phase) lag (Nolte et al., 2004; Stam et al., 2007; Schoffelen and Gross, 2009; Ewald et al., 2012). However, true neuronal activity measured with EEG might show zero (or close to zero) time lag at local or distant cortical regions as well (Stam et al., 2007; Schoffelen and Gross, 2009; Ewald et al., 2012). Both empirical data (Roelfsema et al., 1997) and modeling findings (Vicente et al., 2008), have revealed symmetrical interaction, i.e., in phase or in phase opposition, among distant neuronal populations. This common source problem might affect connectivity measures such as the classical iCOH (Stam et al., 2007; Vinck et al., 2011; Ewald et al., 2012). The corrected form of the iCOH function (ciCOH), which is used in the present study, intends to address this challenge by maximizing the imaginary part of the complex cross-spectrum (Ewald et al., 2012).

Furthermore, the present study has certain limitations with regard to the localization of coherent effects among distant cortical regions. The anatomical relationship between EEG potentials from surface electrodes and specific cortical structures is unsatisfactory due to the field spread effect of neuronal signals recorded at scalp EEG electrodes. However, the signals obtained are highly weighted by the proximity and radial orientation of the cortical area under the electrode (Nunez, 2006). Furthermore, it has been demonstrated that high resolution electrode systems with up to 128 channels, such as was used in this study, facilitates the spatial resolution more significantly than standard low resolution systems (32 and 64 electrodes; Luu et al., 2001). The use of the ciCOH function improves the spatial specificity further when connectivity is studied among EEG sensors. This diminishes the tendency to favor long-range interactions, thus also highlighting short-range interactions that would remain hidden (Ewald et al., 2012). One possible way of improving spatial specificity among scalp related EEG potentials could be the use of surface laplacian. However, it is important to note that such a transformation could unintentionally distort phase synchronization effects due to distortions of physiologically generated phase differences, thereby precluding meaningful physiological results (Nunez, 2006).

In conclusion, our results demonstrated that a single neurofeedback intervention suffices to induce immediate reorganization of neuronal communications involving functional connectivity of frequency specific networks indicative for short term Hebbian-like processes.

# Acknowledgments

MV was supported by the Graduate Training Centre of Neuroscience, International Max Planck Research School, Tuebingen, Germany. AG was supported by grants from the German Research Council [DFG GH 94/2–1, DFG

# References


EC 307], and from the Federal Ministry for Education and Research [BFNT 01GQ0761, BMBF 16SV3783, BMBF 03160064B, BMBF V4UKF014]. We thank Dr. Robert Bauer for fruitful discussions. We acknowledge support by the Deutsche Forschungsgemeinschaft and the Open Access Publishing Fund of University of Tuebingen.


**Conflict of Interest Statement**: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2015 Vukeli ´c and Gharabaghi. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution and reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Comparison of anterior cingulate vs. insular cortex as targets for real-time fMRI regulation during pain stimulation

# *Kirsten Emmert 1,2\* †, Markus Breimhorst 3 †, Thomas Bauermann4, Frank Birklein3, Dimitri Van De Ville1,2 and Sven Haller <sup>1</sup>*

*<sup>1</sup> Department of Radiology and Medical Informatics, University of Geneva, Geneva, Switzerland*

*<sup>2</sup> Medical Image Processing Laboratory, Institute of Bioengineering, Ecole Polytechnique Fédérale de Lausanne, Lausanne, Switzerland*

*<sup>3</sup> Department of Neurology, University Medical Center of the Johannes Gutenberg-University Mainz, Mainz, Germany*

*<sup>4</sup> Institute of Neuroradiology, University Medical Center of the Johannes Gutenberg-University Mainz, Mainz, Germany*

#### *Edited by:*

*Ranganatha Sitaram, University of Florida, USA*

#### *Reviewed by:*

*Annette Beatrix Bruehl, University of Cambridge, UK Ralf Veit, Eberhard Karls-University, Germany*

#### *\*Correspondence:*

*Kirsten Emmert, Department of Radiology and Medical Informatics, Hôpitaux Universitaires de Genève, University of Geneva, Rue Gabrielle Perret-Gentil 4, 1211 Genève 14, Switzerland*

*e-mail: emmert.kirsten@gmail.com*

*†These authors have contributed equally to this work.*

Real-time functional magnetic resonance imaging (rt-fMRI) neurofeedback allows learning voluntary control over specific brain areas by means of operant conditioning and has been shown to decrease pain perception. To further increase the effect of rt-fMRI neurofeedback on pain, we directly compared two different target regions of the pain network, notably the anterior insular cortex (AIC) and the anterior cingulate cortex (ACC). Participants for this prospective study were randomly assigned to two age-matched groups of 14 participants each (7 females per group) for AIC and ACC feedback. First, a functional localizer using block-design heat pain stimulation was performed to define the pain-sensitive target region within the AIC or ACC. Second, subjects were asked to down-regulate the BOLD activation in four neurofeedback runs during identical pain stimulation. Data analysis included task-related and functional connectivity analysis. At the behavioral level, pain ratings significantly decreased during feedback vs. localizer runs, but there was no difference between AIC and ACC groups. Concerning neuroimaging, ACC and AIC showed consistent involvement of the caudate nucleus for subjects that learned down-regulation (17/28) in both task-related and functional connectivity analysis. The functional connectivity toward the caudate nucleus is stronger for the ACC while the AIC is more heavily connected to the ventrolateral prefrontal cortex. Consequently, the ACC and AIC are suitable targets for real-time fMRI neurofeedback during pain perception as they both affect the caudate nucleus, although functional connectivity indicates that the direct connection seems to be stronger with the ACC. Additionally, the caudate, an important area involved in pain perception and suppression, could be a good rt-fMRI target itself. Future studies are needed to identify parameters characterizing successful regulators and to assess the effect of repeated rt-fMRI neurofeedback on pain perception.

**Keywords: real-time fMRI neurofeedback, realtime fMRI, pain, anterior cingulate cortex (ACC), anterior insular cortex, insular cortex**

# **INTRODUCTION**

Pain perception has a great impact on individual emotional health as pain is associated with anxiety (Asmundson and Katz, 2009), anger (Trost et al., 2012), fear (Leeuw et al., 2007a,b; Vlaeyen and Linton, 2012), and worry (Eccleston and Crombez, 2007; Linton, 2013). Thus, not surprisingly, chronic pain increases the risk of depression and suicide (Turk et al., 1995; Geisser et al.,

**Abbreviations:** ACC, anterior cingulate cortex; AIC, anterior insular cortex; aMCC, anterior mid-cingulate cortex; ANOVA, analysis of variance; BOLD, blood oxygenation level dependent; fMRI, functional magnetic resonance imaging; GLM, general linear model; ICA, independent component analysis; MELODIC, multivariate exploratory linear optimized decomposition into independent components; MPRAGE, magnetization prepared rapid gradient echo; NRS, numeric rating scale; PIC, posterior insular cortex; PFC, prefrontal cortex; ROI, region of interest; rt-fMRI, real-time fMRI; SEM, standard error of the mean.

2000; Bair et al., 2003; Ilgen et al., 2008; Denkinger et al., 2014). Pharmacological intervention remains the mainstay of chronic pain treatment. As most chronic pain patients are treated with a combination of pain medications and over long periods of time (Muller-Schwefe et al., 2011), cumulative drug-related side effects pose a considerable risk of adverse effects for these patients (Jouini et al., 2014), highlighting the importance of alternative and supplementary pain therapies.

One novel technique that shows potential in the treatment of chronic pain is real-time functional magnetic resonance imaging (rt-fMRI), which allows volitionally influencing activation of a targeted brain area by means of operant conditioning when being supplied with a corresponding feedback signal. This technique could be employed to reduce brain activation in pain network target areas with the aim to decrease the subjective pain perception. A pilot study showed that it is possible to regulate the anterior cingulate cortex (ACC) as a target brain region using rt-fMRI for chronic pain patients as well as healthy participants during pain perception (Decharms et al., 2005). However, according to subsequent reports of the same group, these findings could not be replicated (Decharms, 2012). In line with this observation, rtfMRI is generally still in its early days, facing some limitations and confounds. High inter-individual differences in regulation success and small effect sizes make it difficult to assess the therapeutic use of this method. In an attempt to optimize the choice of the target region, which is a key factor of the rt-fMRI experiment, this study compares two possible target brain regions for feedback involved in pain processing in healthy subjects. The effect of the feedback on these target regions and other brain regions within the pain-responsive network will be assessed.

Acute pain perception starts with an external stimulus that activates peripheral receptors such as the vanilloid receptor (TRPV1), which is sensitive to temperatures above 43◦C (Cesare and McNaughton, 1996) eliciting a depolarization of peripheral sensory neurons synapsing onto second-order dorsal horn neurons (Basbaum and Jessell, 2000) in the spinal cord. These fibers ascend to the thalamus relaying information to the somatosensory cortex, the ACC and the insular cortex (IC). Additional projection neurons from the dorsal horn to the parabrachial nucleus in the brainstem engage the ACC and the IC via the amygdala. Apart from this ascending connection, cortical pain areas such as the primary and secondary somatosensory cortex as well as the posterior insula (PIC), which are implicated in basic pain perception, are heavily interconnected (Apkarian et al., 2005). The same is true for higher-level areas involved in pain processing, including the ACC, the anterior insula (AIC) and prefrontal cortical areas exerting top-down regulation on the thalamus and the amygdala in turn. In addition, the basal ganglia are activated through multiple pathways including the thalamus, the amygdala and cortical areas (Borsook et al., 2010). While areas that are involved in basic sensory pain processing, such as the PIC, are predominantly activated contralateral to the pain stimulus, higher-level processing areas implicated in pain interpretation including the AIC are activated in a bilateral fashion (Brooks et al., 2002).

Ongoing nociceptive input from injuries leads to a hyperexcitability of the nervous system, in a process that resembles long-term potentiation called central sensitization (Drdla and Sandkuhler, 2008; Woolf, 2011), in addition to a decrease of tonic inhibition (Moore et al., 2002; Keller et al., 2007). This hyperalgesia has the purpose of facilitating the healing processes of the injured tissue. However, central sensitization can persist after tissue healing leading to chronic hyperalgesia and even pain perception in the absence of painful stimuli (Voscopoulos and Lema, 2010; Woolf, 2011). Furthermore, pathological changes in the descending modulatory pathways might also contribute to the emergence of chronic pain (Porreca et al., 2002; Ossipov et al., 2010).

Functional brain imaging showed abnormal activation in the rostral ACC and the frontal cortex in certain chronic pain populations (Baliki et al., 2006; Berman et al., 2008; Jensen et al., 2009; Burgmer et al., 2010). Additionally, chronic pain patients show altered functional connectivity of the prefrontal cortex (PFC) and the insula with the default mode network (Napadow et al., 2010; Baliki et al., 2011). Similarly, structural imaging revealed gray matter reductions within the PFC, the ACC and the IC (Bushnell et al., 2013). On a molecular level, chronic pain patients seem to show altered endogenous release for the glutamatergic and GABAergic system as well as a decrease in receptor binding of the opioidergic system in these areas (Bushnell et al., 2013). These anatomical and molecular changes might not only alter pain regulation, but also affect decision making (Grace et al., 1999; Leavitt and Katz, 2006; Munguia-Izquierdo and Legaz-Arrese, 2007).

Some studies also suggest that these changes can be partly reversed, for example, in cases where there is an underlying painful condition that can be removed after years (Gwilym et al., 2010; Seminowicz et al., 2011). Moreover, the pain modulation system consisting of the PFC, ACC, and AIC was shown to be modulated by cognitive measures such as meditation or cognitive behavior therapy (Grant et al., 2011; Gard et al., 2012; Jensen et al., 2012). Thus, it seems useful and feasible to regulate these areas using rt-fMRI neurofeedback. Before looking into possible neurofeedback effects for chronic pain patients, we aim to optimize target ROI selection for pain neurofeedback in healthy subjects during pain stimulation as a first step. Future studies are needed to make sure that these target ROIs can be regulated in chronic pain patients as well.

The ACC and the AIC seem to be particularly important in perceiving pain intensity (Favilla et al., 2014). Therefore, these two regions of the medial pain system (Treede et al., 1999) were considered the most promising rt-fMRI target regions for cortical pain processing. The ACC was also the subject of a recent rt-fMRI neurofeedback study testing feasibility of pain regulation for the rostral ACC and PIC (Rance et al., 2014). They postulated that sensory pain aspects might be more related to PIC activation while affective aspects are more related to ACC activation. In this context, it is interesting to investigate how the AIC—implicated in another aspect of pain, namely cognitive control processes—can be regulated.

The ACC has been associated to several functions relevant to pain processing including saliency (Seeley et al., 2007; Iannetti and Mouraux, 2010), attention (Bush et al., 2000; Weissman et al., 2005), and emotion (Bush et al., 2000; Shackman et al., 2011). It is furthermore linked to affective processing of painful stimuli (Vogt et al., 1996; Rainville et al., 1997). Studies already showed that it is possible to target the ACC in smokers (Canterberry et al., 2013; Hartwell et al., 2013; Li et al., 2013) and chronic pain patients as well as healthy participants during pain perception (Decharms et al., 2005). In the latter study, regulation of the ACC activation using rt-fMRI neurofeedback even resulted in a decrease of pain intensity ratings. Other behavioral interventions that have been shown to modulate ACC activation include hypnosis (Rainville et al., 1997; Faymonville et al., 2000), modulation of pain expectation (Sawamoto et al., 2000; Bingel et al., 2011), and distraction (Bantick et al., 2002; Valet et al., 2004).

The IC can be divided into the anterior and the PIC that serve distinct functions in pain processing. The PIC seems to be involved in basic pain and touch sensation (Greenspan and Winfield, 1992), receiving direct spinothalamic input (Garcia-Larrea, 2012). Lesions in this area lead to pain and temperature deficits (Greenspan et al., 1999; Birklein et al., 2005). In contrast, AIC lesions usually do not seem to have a direct impact on pain perception *per se* (Greenspan et al., 1999). The AIC is implicated in a wide variety of functions, including visceral sensation and an integrative role in perception-action coupling possibly by mediating heightened alertness to prepare for action (Sterzer and Kleinschmidt, 2010). It seems to be engaged in affective-motivational processes of pain perception as a disconnection of the AIC from the PIC leads to a decrease of emotional pain reaction while nociceptive recognition remains intact (Berthier et al., 1988). Up-regulation of the AIC was shown to be possible (Caria et al., 2007; Veit et al., 2012) using recall of personal and affectively relevant events or focused attention on arising bodily sensations (Lawrence et al., 2013). It was shown that it is even possible to target subjects with clinical disorders such as schizophrenia (Ruiz et al., 2013) or depression (Linden et al., 2012). While these studies suggest that AIC regulation can be used to increase certain affective states and control, there is no specific data looking at the influence of the AIC down-regulation on pain perception.

In this work, we directly compared two possible target regions for rt-fMRI neurofeedback in pain, notably the AIC and the ACC, in order to determine the most efficient target region for future neurofeedback studies in pain processing.

# **MATERIALS AND METHODS**

#### **PARTICIPANTS**

The local ethics committee in Mainz approved the study that adhered to the Declaration of Helsinki. Twenty-eight healthy subjects (mean age: 27.5 ± 2.3 years, 14 male, 14 female) gave written informed consent prior to participation. Participants were randomly split into two groups of *N* = 14 each, including seven male and seven female participants per group (group 1: 27.6 years ± 2.1, group 2: 27.4 ± 2.6 years). The first group received feedback from the left anterior insula (lAIC) as a target region, while the second group did so from the ACC. Exclusion criteria were defined by acute or chronic pain, pregnancy, severe neurological or internal disorders, intake of painkillers and contraindications for MR-measurements. Participants were paid for participation in the study.

# **REAL-TIME EXPERIMENT**

The experiment consisted of two stages. First, a functional localizer run with an ON-OFF block design of eight blocks alternating between continuous painful stimulation for 30 s and rest for 30 s each was performed to identify the individual target regions. The target region was chosen based on significant activation within the lAIC/ACC during the functional localizer. Thereafter, four identical neurofeedback runs were performed consisting of a block design of four rest and regulation blocks (30 s each) preceded by 15 s of initial rest before the first block. Online data analysis was performed using TurboBrainVoyager version 2.8 (Brain Innovation, Maastricht, The Netherlands).

The target region was chosen based on significant activation within the lAIC/ACC during the functional localizer (summarized in Supplementary Table 1). Regulation blocks included the same pain stimulation as during the localizer. During this period of the neurofeedback runs, subjects were asked to decrease the blood oxygen level dependent (BOLD) activation level in the target region, which was visualized to them by a yellow line. The background color of the yellow line indicated to either keep the yellow line constant (black = rest blocks, no heat pain) or to decrease the amplitude of the yellow line (blue = downregulation, heat pain). Subjects could freely choose their mental strategy to reach this objective.

## **PAIN STIMULATION AND RATING**

An MR compatible thermode (TSA 2001, Medoc Ltd, Ramat Yishai, Israel), placed at middle of the lower right volar forearm, was used for pain stimulation. This 30 × 30 mm Peltier device has a default temperature of 32◦C. Before the start of the experiment the thermode temperature was adjusted for each participant to elicit a subjective pain intensity of 7 out of 10 on the numeric rating scale (NRS). The thermode temperature for pain stimulation remained constant throughout the experiment [Ramp rate: 4◦C/s, mean ramp and fall time for AIC-group: 3.83 s (*SD* 0.26) and for ACC-group: 3.64 s (*SD* 0.32), mean plateau for AIC-Group: 22.35 s (*SD* 0.53) and for ACC-Group: 22.71 s (*SD* 0.64), mean temperature for AIC-Group: 47.08◦C (*SD* 1.1) and for ACC-group: 46.42◦C (*SD* 1.4)]. After each run pain ratings were obtained using a 11-point NRS ranging from 0 (not painful) to 10 (most painful).

# **fMRI DATA ACQUISITION**

Imaging was performed on a 3T MRI Scanner (Siemens Tim Trio, Erlangen, Germany) using a 32-channel head-coil. For functional data acquisition an echo-planar imaging sequence (EPI, *TR* = 1500 ms, *TE* = 30 ms, matrix size 64 × 64, 24 slices, slice thickness 3 mm without gap) was utilized. Additionally, a highresolution T1-weighted anatomical scan [magnetization prepared rapid gradient echo (MPRAGE), 1 mm isotropic] was acquired for later co-registration with the lower resolution EPI images.

#### **STATISTICAL ANALYSIS BETWEEN RUNS AND GROUPS**

Statistical testing for differences between runs and groups [pain ratings, region of interest (ROI) activation, s-modes] was performed in MATLAB 2012b (The MathWorks, Inc., Natick, USA). First, parameters were tested for normality using D'Agostino K-squared test. As normality was rejected for all our parameters of interest (pain ratings, ROI beta values, s-mode values), we used the non-parametric Friedman test (comparison between all runs) and *post-hoc* Wilcoxon tests (comparison between groups, and comparison of two runs when the Friedman test showed significant results). Bonferroni correction was applied to correct for multiple comparisons in the s-mode analysis (i.e., the number of independent components).

# *POST-HOC* **GLM ACTIVATION ANALYSIS OF THE FUNCTIONAL LOCALIZER**

Off-line analysis was performed with SPM 8 (UCL, London, UK) and FSL 5.0 (FMRIB Analysis Group, University of Oxford, UK). Functional data was spatially realigned, co-registered to the anatomical data, normalized and smoothed (8 mm kernel) before group analysis on the basis of a general linear model (GLM) using the block design described under Section Real-Time Experiment. For the fMRI analysis, family-wise error (FWE) corrected values of *p <* 0*.*05 are considered significant.

#### *POST-HOC* **ROI ACTIVATION ANALYSIS OF THE NEUROFEEDBACK RUNS**

GLM analysis for all four neurofeedback runs was performed analogous to the localizer run. As self-regulation was expected to increase with practice, we compared the first neurofeedback run with the subsequent runs in a ROI analysis for regions that were activated during the localizer run and known to be involved in pain processing, namely ACC, AIC, PIC. Based on our functional connectivity and ICA results in combination with its know implication in pain processing (Borsook et al., 2010), we included the caudate nucleus as an additional (a posteriori) ROI. Then, ROIs were defined as spheres with 1-cm diameter centered at the activation peaks within the relevant clusters from the group analysis of the functional localizer. This approach seemed more suitable than defining the ROIs on an individual level, as done for target ROI analysis, as not all subjects showed significant activation in all of the ROIs in the localizer run. Since regulation using rtfMRI neurofeedback fails in some subjects, we restricted extensive *post-hoc* ROI analysis to those subjects who showed a decrease in activation in the target ROI; i.e., 9/14 for the AIC group and 8/14 subjects for the ACC group.

### *POST-HOC* **fMRI CONNECTIVITY ANALYSIS OF THE NEUROFEEDBACK RUNS**

Using FSL 5.0, functional connectivity was assessed with a seedbased approach testing for correlation with the seed's time course orthogonalized to the global signal and the GLM regressor of main effect. Seed regions were both rt-fMRI targets, ACC and lAIC, respectively. The resulting connectivity maps of each subject were fed into a 2nd level GLM analysis to obtain group results.

In addition, an independent component analysis (ICA) was carried out in FSL using multi-session multivariate exploratory linear optimized decomposition into independent components (MELODIC) tensor ICA. So-called s-modes (i.e., measures of activation strength for every component in each subject) were compared between groups.

### **RESULTS**

#### **EFFECT OF NEUROFEEDBACK ON PAIN RATINGS**

Pain ratings were lower in the neurofeedback runs compared to the localizer run [non-parametric, p(AIC group) *<* 0.001; p(ACC) *<* 0.01] in both groups, but did not show any significant differences between neurofeedback runs (see **Figure 1**, **Table 1**). Pain ratings did not differ between regulators and non-regulators (*p >* 0*.*1).

Neither pain ratings of the regulators nor the non-regulators changed significantly between neurofeedback runs.

#### **FUNCTIONAL LOCALIZER**

As expected, the functional localizer revealed significant activation within the insula, PFC and the ACC, all regions involved in pain processing (see **Figure 2**). Activation of the target region in each subject enabled the individual region of interest placement (see Supplementary Figure 1).

#### **NEUROFEEDBACK RUNS**

#### *Seed-based connectivity of the left AIC and the ACC*

Seed-based analysis at the group level showed the functional connectivity of the ACC and the AIC to other regions of the pain network (see **Figure 3A**). The analysis confirmed that ACC and

box indicates 25%/75% confidence intervals and the whiskers indicate the most extreme points within 1.5 times of the box length.

the AIC are strongly interconnected as well as showing connections to prefrontal areas. Interestingly, the ACC has high functional connectivity with the caudate nucleus that did not show up in the AIC connectivity map while the AIC group has an increased connectivity with the ventrolateral PFC (see **Figure 3B**).



# *Effect of training (runs over time)*

To assess a possible improvement in self-regulation over time, we looked for a decrease in activation in the later runs compared to the first run. To that aim, we first analyzed the activation within each individual target ROI (see **Table 2** and **Figure 4**). Data from subjects that showed a successful down-regulation (i.e., decrease of target region's activation level from the first to the average of the following runs) were used for a more extensive ROI analysis including the main brain areas involved in pain regulation (see **Table 3**).

The analysis of these regions showed that the decrease of the left AIC in the AIC group is accompanied by a similar significant decrease in the contralateral anterior insula (*p <* 0*.*05). In addition, both groups show a significant decrease of the caudate nucleus with the effect being more pronounced in the AIC group (*p <* 0*.*01, ACC: *p <* 0*.*05, see **Figure 5**).

#### *Independent component analysis*

Using ICA, we identified 33 components of which one is significantly different between groups according to its s-mode (*p <* 0*.*05, corrected for multiple comparison). This component includes AIC, ACC, and small portions of the occipital and parietal lobes (see **Figure 6**). In addition, we looked for components that exhibit a linear trend over runs and identified one component with slope significantly different from zero for the ACC group (*p <* 0*.*05, corrected for multiple comparison). This component includes thalamus and parts of the basal ganglia (see **Figure 7**).

# **DISCUSSION**

In the current investigation, we compared the effectiveness in pain regulation using real-time fMRI neurofeedback from two different target regions, notably ACC and AIC. At the behavioral level, both for ACC and AIC feedback, the neurofeedback runs showed a decrease in pain perception with respect to the identical pain stimulation in the localizer runs. However, there was no significant behavioral difference in the direct comparison between ACC and AIC and between runs. Despite the absence of behavioral differences between runs, we found effects in neuroimaging for the two target regions. This observation is in line with the known higher sensitivity of neuroimaging, as compared to behavioral measures, in functional MRI studies investigating subtle effects

(Weiskopf et al., 2003; Haller et al., 2005, 2013; Johnston et al.,

**connectivity blue, A) and left AIC (yellow, connectivity orange, A). (B)** Areas that had a significantly greater connection to the ACC than

2011). At the neuroimaging level, AIC and ACC regulation led to a significant down-regulation of parts of the pain network with practice, notably the caudate nucleus for successful regulators.

Functional-connectivity analyses further demonstrated that both target regions are functionally well connected to other parts of the pain network. Therefore, based on this neuroimaging evidence, we found that both AIC and ACC influence the pain network in a similar fashion through the caudate nucleus.

#### **ACC REGULATION DURING PAIN**

Contrary to two previous studies about rt-fMRI ACC regulation of pain processing (Decharms et al., 2005; Rance et al., 2014), we did not find a significant down-regulation effect of ACC regulation over runs within the ACC for all subjects. This might be due to the different experimental paradigm that compared down-regulation vs. no regulation in our setting, while deCharms et al. compared up- vs. down-regulation. Considering that downregulation might be harder to obtain than up-regulation, as it is easier to explicitly focus on acute pain than to find a strategy to decrease pain, the effect of down-regulation might be smaller. In addition, this particular finding could not be replicated by deCharms et al. in a later follow-up study; as publicly stated at the rt-fMRI conference in Zurich, 2012 (Decharms, 2012). One factor that possibly complicates ACC regulation is that the adjacent anterior mid-cingulate cortex (aMCC) is also thought to be involved in rt-fMRI neurofeedback regulation (Lee et al., 2012), inducing activation during the regulation and thus making it to the ACC (orange) in a direct comparison. Arrows indicate the target seed location.

harder to detect the deactivation in nearby ACC, and possibly also confounding the participants' feedback signal itself to some extent. This possible confound is less strong in the recent study of Rance et al. as they used a more rostral part of the ACC leading to a significant down-regulation of this ROI. Therefore, future studies should preferably use a more rostral part of the ACC.

Nevertheless, ACC rt-fMRI neurofeedback did induce a downregulation of the ACC in a large group of subjects (8/14) as well as a significant change within the caudate nucleus, a brain region involved in planning of goal directed actions (Grahn et al., 2008) and affective processing of pain (Borsook et al., 2010). This part of the basal ganglia is anatomically closely connected to the ACC, with functional relevance, for example, in pain avoidance behavior in monkeys (Koyama et al., 2000). Similarly, previous studies found caudate nucleus involvement when participants suppressed pain (Freund et al., 2009; Wunderlich et al., 2011). Thus, the caudate nucleus, regulated via the ACC, seems to be important in deliberate pain control. This result is supported by the seed-based functional connectivity analysis showing a strong ACC—caudate nucleus interaction and the ICA analysis that revealed a specific component including the caudate nucleus and thalamus that showed significantly decreasing s-modes as a function of runs. These results also indicate that the caudate, the thalamus or a combination of these regions could be considered as suitable targets for future pain real-time neurofeedback studies.

#### **AIC REGULATION DURING PAIN**

Similar to the ACC group, AIC down-regulation was not significant when looking at all subjects. This difficulty in AIC regulation **Table 2 | Beta values of the target ROI for all subjects, classification criteria (beta value decrease from run 1 to the average of run 2–4), and classification label (+, regulator; −, non-regulator).**


might be explained by competing processes within the AIC. On one hand, the AIC was selected as the target for down-regulation as it is a core component of the network involved in pain processing (Apkarian et al., 2005). On the other hand it is likely to be activated in neurofeedback regulation processes (Haller et al., 2010). In addition, the AIC is involved in many other cognitive processes such as saliency detection (Cauda et al., 2012) and emotion regulation and representation (Singer et al., 2004; Eippert et al., 2007). Due to the regulation procedure, saliency of the visual display (focus on the line and the lower part of the "scale") as well as saliency of the pain stimulus (less focus on pain) could be modulated. In addition, the feedback could induce emotions such as frustration or contentment, thus possibly increasing insula activation, thereby counteracting insula down-regulation. This might also explain why all previous studies only reported reliable up-regulation while voluntary down-regulation of the AIC by rt-fMRI neurofeedback was less successful (Veit et al., 2012). The possible interaction of cognitive and emotional processes within the AIC was also underlined by an fMRI study showing increased reaction times and error rates for cognitively demanding tasks during presentation of painful compared to non-painful pictures (Gu et al., 2013).

However, 9 out of 14 subjects showed a trend to downregulation of the AIC. In these subjects the ROI analysis also showed a down-regulation of the contralateral AIC. This corresponding contralateral change could be expected, given the bilateral processing of higher-level pain functions and the high connectivity between the left and right AIC as confirmed in the functional connectivity analysis. Additionally, the left and right caudate nucleus showed a down-regulation when comparing the first and later feedback runs. The fact that in both groups successful target region regulation is accompanied by a decrease in caudate nucleus activation underlines its importance in pain regulation.

## **DIFFERENCES IN THE FUNCTIONAL CONNECTIVITY AND ICA BETWEEN GROUPS**

Functional connectivity analysis revealed that the ACC shows a stronger functional connectivity to the caudate nucleus while the AIC is more heavily connected to the ventrolateral PFC. These differences might reflect different pathways of pain regulation. While the ACC might directly influence caudate nucleus activity, the AIC has a stronger connection to higher-level processing via the PFC that in turn might regulate caudate activity.

ICA revealed one functional connectivity ICA component involving the ACC and the AIC that showed significantly lower s-mode values (a measure of effect size) in the ACC group in comparison to the AIC group. This implies that AIC and ACC activity overall was higher in the AIC group. One possible explanation might be that AIC regulation is harder to obtain in the beginning due to competing processes within this brain region. This might lead to an increase in pain processing within the AIC and ACC that is compensated at a later phase when subjects learned down-regulation.

#### **EFFECT OF rt-fMRI ON PAIN RATINGS**

In addition to our main goal of comparing two targets for rt-fMRI neurofeedback, we also looked at the pain rating as a function of runs. The finding that pain ratings decreased in neurofeedback runs compared to the localizer run suggests that ACC and AIC down-regulation by means of rt-fMRI neurofeedback decreases pain perception. Two contradictory factors potentially confound the interpretation of decrease in pain perception. Habituation might reduce, while sensitization might increase subjective pain perception despite identical physical stimulation. The observed result of decreased pain ratings in feedback as compared to localizer runs would not be expected from a regular pain study as short-term repeated pain stimulation in general causes sensitization rather than habituation (Drdla and Sandkuhler, 2008; Breimhorst et al., 2012). The same trend was seen in another recent pain real-time neurofeedback study (Rance et al., 2014) where slightly higher pain intensity was applied and pain unpleasantness ratings were compared for the last against the first run, indicating a pain sensitization over run. However, we cannot exclude the possibility that the placebo effect, caused by the neurofeedback intervention, might have confounded pain ratings

**Table 3 | Overview of ROIs with their location and** *p***-value of Friedman test for change in Beta-value across neurofeedback runs (AIC:** *n* **= 9, ACC:** *n* **= 8).**


*Bold numbers indicate significant results (p <* 0*.*05*), values for the corresponding target area are highlighted red.*

during neurofeedback runs. Pain perception is known to vary depending on the context (Rhudy and Meagher, 2000; Iannetti et al., 2008; Wang and Mitchell, 2011), therefore, making it hard to distinguish the factors that contribute to the pain reduction between localizer run and feedback runs. The fact that subjects were directing attention toward a cognitively demanding task itself could decrease pain perception as shown in a study working with different distraction tasks (Verhoeven et al., 2011). Both effects might be particularly high in the first neurofeedback runs when the task is new and subjects exert more effort than later on, thus possibly counteracting the desired effect of increasing regulation. The difference between localizer and neurofeedback pain rating in the AIC group can also be explained by competing processes within the ROI and the effect of cognitively highly demanding task engagement. These confounding effects might be similar in size to the effects of rt-fMRI, which are expected to be rather small, considering that pain perception has been experienced for years while cognitive modulation of pain has been practiced for minutes only. Some other neuroimaging studies already showed a similar phenomenon: significant neuroimaging effects were not accompanied by corresponding behavioral changes (Weiskopf et al., 2003; Haller et al., 2005, 2009, 2013, 2014; Johnston et al., 2011). This might indicate that objective fMRI data are more sensitive to small-scale changes within a rather small group than subjective behavioral measures. Therefore, it is not surprising that the decreased caudate activity over runs in the AIC and ACC group did not directly lead to a significant decrease in pain rating between feedback runs.

#### **STRENGTH AND LIMITATIONS**

The current investigation is a comparison of two possible target regions for rt-fMRI neurofeedback in pain. It clearly indicates that the AIC and the ACC could serve as a pain neurofeedback target in future studies. The following limitations should however be taken into account when interpreting the current results. First, this study did not aim at assessing the absolute behavioral effect of neurofeedback on pain ratings. Thus, further studies including sham feedback as well as modified pain stimulation are needed to separate specific effects of rt-fMRI neurofeedback from habituation/sensitization over time. Additionally, these studies should aim to compare neurofeedback to a sham method with a similar cognitive load, as a high cognitive load could influence pain ratings as well (Verhoeven et al., 2011). Second, as in previous real-time fMRI studies (Decharms et al., 2005;

Bray et al., 2007; Scharnowski et al., 2012; Robineau et al., 2014) not all subjects learned to down regulate the target area. Future studies should aim at identifying the parameters that lead to successful rtfMRI neurofeedback regulation in order to maximize the number of subjects that succeed. Another limitation lies in the use of a GLM on the basis of a box-type function convolved with the hemodynamic response function. Due to this hypothesis about the shape of the response, differently shaped responses such as a decrease in BOLD response after a certain period of pain stimulation, as it has been reported for the thalamus (Tran et al., 2010), would lead to underestimated statistical values.

The ACC and the AIC were judged as the most suitable neurofeedback targets based on literature (see Introduction). Based on our results the caudate nucleus and the thalamus or measures of the connectivity between the ACC and the caudate nucleus (e.g., intrinsic connectivity contrast degree) might be an additional target for future rt-fMRI neurofeedback studies in the domain of pain. As a next step, the potential long-term effects of neurofeedback training on pain perception should be assessed using the AIC, the ACC, thalamus or caudate nucleus as ROI in healthy subjects and as a next step also in chronic pain patients. Due to the possible involvement of the aMCC in neurofeedback regulation processes, the target area should be sufficiently separated

from the aMCC. These future studies could be another important step toward a possible supplemental pain therapy to reduce the impact of pain on patients' life.

# **ACKNOWLEDGMENTS**

This work was supported in part by the Swiss National Science Foundation (projects 320030\_147126/1, 320030\_127079/1 and PP00P2-146318), the foundation "Stiftung Rheinland-Pfalz" (Project 936) and the Center for Biomedical Imaging (CIBM).

# **SUPPLEMENTARY MATERIAL**

The Supplementary Material for this article can be found online at: http://www*.*frontiersin*.*org/journal/10*.*3389/fnbeh*.*2014*.* 00350/abstract

# **REFERENCES**


experimental pain: a controlled fMRI Study. *J. Neural Transm.* 117, 123–131. doi: 10.1007/s00702-009-0339-1


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 25 April 2014; accepted: 18 September 2014; published online: 09 October 2014.*

*Citation: Emmert K, Breimhorst M, Bauermann T, Birklein F, Van De Ville D and Haller S (2014) Comparison of anterior cingulate vs. insular cortex as targets for realtime fMRI regulation during pain stimulation. Front. Behav. Neurosci. 8:350. doi: 10.3389/fnbeh.2014.00350*

*This article was submitted to the journal Frontiers in Behavioral Neuroscience.*

*Copyright © 2014 Emmert, Breimhorst, Bauermann, Birklein, Van De Ville and Haller. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Neurofeedback of the difference in activation of the anterior cingulate cortex and posterior insular cortex: two functionally connected areas in the processing of pain

#### *Mariela Rance1, Michaela Ruttorf 2, Frauke Nees 1, Lothar R. Schad2 and Herta Flor <sup>1</sup> \**

*<sup>1</sup> Department of Cognitive and Clinical Neuroscience, Medical Faculty Mannheim, Central Institute of Mental Health, Heidelberg University, Mannheim, Germany <sup>2</sup> Computer Assisted Clinical Medicine, Medical Faculty Mannheim, Heidelberg University, Mannheim, Germany*

#### *Edited by:*

*Sergio Ruiz, Pontificia Universidad Católica de Chile, Chile*

#### *Reviewed by:*

*Li Yao, Beijing Normal University, China Nikolaus Weiskopf, University College London, UK*

#### *\*Correspondence:*

*Herta Flor, Department of Cognitive and Clinical Neuroscience, Central Institute of Mental Health, Square J5, D-68159 Mannheim, Germany e-mail: herta.flor@zi-mannheim.de*

The aim of this study was the analysis of the effect of a learned increase in the dissociation between the rostral anterior cingulate cortex (rACC) and the left posterior insula (pInsL) on pain intensity and unpleasantness and the contribution of each region to the effect, exploring the possibility to influence the perception of pain with neurofeedback methods. We trained ten healthy subjects to increase the difference in the blood oxygenation level-dependent response between the rACC and pInsL to painful electric stimuli. Subjects learned to increase the dissociation with either the rACC (state 1) or the pInsL (state 2) being higher. For feedback we subtracted the signal of one region from the other and provided feedback in four conditions with six trials each yielding two different states: [rACC—pInsL increase (state 1), rACC—pInsL decrease (state 2), pInsL—rACC increase (state 2), pInsL—rACC decrease (state 1)]. Significant changes in the dissociation from trial one to six were seen in all conditions. There were significant changes from trial one to six in the pInsL in three of the four conditions, the rACC showed no significant change. Pain intensity or unpleasantness ratings were unrelated to the dissociation between the regions and the activation in each region. Learning success in the conditions did not significantly correlate and there was no significant correlation between the two respective conditions of one state, i.e., learning to achieve a specific state is not a stable ability. The pInsL seems to be the driving force behind changes in the learned dissociation between the regions. Despite successful differential modulation of activation in areas responsive to the painful stimulus, no corresponding changes in the perception of pain intensity or unpleasantness emerged. Learning to induce different states of dissociation between the areas is not a stable ability since success did not correlate overall or between two conditions of the the same state.

**Keywords: real-time fMRI, pain, anterior cingulate cortex, posterior insula, neuromodulation, connectivity**

# **INTRODUCTION**

The number of studies utilizing neuromodulation to alter behavior, cognition, and emotional processing has been growing in the past ten years. Applications include the modulation of the blood oxygenation level-dependent (BOLD) response in single brain regions such as altering motor function via modulation of the ventral or dorsal premotor cortex (Sitaram et al., 2012; Zhao et al., 2013), processing of emotional visual cues by modulating the anterior insula (Caria et al., 2010), craving in smokers by modulating the anterior cingulate cortex (Li et al., 2013), or improving working memory performance by modulating the dorsal lateral prefrontal cortex (Zhang et al., 2013). While in some studies altered brain activation was associated with changes in behavior, other studies showed that self-modulation of brain activation is possible in the absence of behavioral effects (Weiskopf et al., 2003; Johnston et al., 2011; McCaig et al., 2011; Birbaumer et al., 2013). More recently, real-time functional magnetic resonance imaging (rt-fMRI) has not only been used in the modulation of specific regions of the brain but of active networks defined by connectivity analyses or real-time pattern classification (Esposito et al., 2003; Laconte et al., 2007; Sitaram et al., 2011; Berman et al., 2013; Koush et al., 2013; Ruiz et al., 2013; Zotev et al., 2013; Scharnowski et al., 2014; Zilverstand et al., 2014). In the light of these results an important but so far underinvestigated issue is the local specificity of neuromodulation of single regions, differential effects of up- or downregulation of BOLD activation and their influence on the active network and behavior.

Pain is a complex sensory and emotional experience and its alteration by rt-fMRI might be a challenge. It has been shown that pain perception involves a distributed network (Apkarian et al., 2005; Brodersen et al., 2012; Wager et al., 2013) that is, moreover, also involved in other functions (Iannetti and Mouraux, 2010; Legrain et al., 2011; Cauda et al., 2012b). Moreover, pain perception is subject of many modulating factors such as cognitive, emotional, and learning processes (Bingel et al., 2006; Diesch and Flor, 2007) that involve other brain circuits (Villemure and Bushnell, 2002, 2009; Wiech et al., 2006). Decharms et al. (2005) trained healthy controls while they received painful thermal stimulation to decrease and increase the BOLD signal in the rACC. The magnitude of the change in the activation was associated with the magnitude of the change in the ratings. The authors also transferred their design to chronic pain patients who subsequently reported pain relief. However, a replication of these results is still missing (Chapin et al., 2012).

We (Rance et al., 2014) previously examined and compared the effect of separately up- and downregulating two regions that are part of the pain processing network: the rACC, involved in the tonic aversive state elicited by pain (Apkarian et al., 2005; Qu et al., 2011) and the left posterior insula (pInsL), a region that processes the sensory aspect of pain perception (Rainville et al., 1997; Frot et al., 2007). In this study successful up- and downregulation of the rACC and pInsL was trained. The subjects acquired successful pInsL up- and downregulation and successful rACC down- but not upregulation. Successful modulation was not accompanied by a change in perceived pain intensity or unpleasantness. Modulation of one region also affected the second region, implying that the network seen active in the presence of the painful electrical stimulus was affected. Better learning was associated with higher dissociation between the two regions. Moreover, higher dissociation during upregulation of the pInsL correlated positively with an increase in pain unpleasantness ratings. These results suggest that the state of the pain-related network plays a role in the learning of self-modulation of the activation of single nodes.

In the light of these findings the current study aimed at examining and comparing the effects of a combined difference feedback of the two regions, rACC and pInsL, thus permitting to analyze the effect of the disruption of a part of the pain processing network. Similar to our previous study ten healthy subjects trained on four consecutive days. Two distinct states were trained: activation of the rACC higher than pInsL activation (state 1) or activation of pInsL higher than rACC activation (state 2). The goal of the training was to increase the difference. Each state was practiced in two distinct conditions; state 1A: rACC—pInsL increase (arrow up), state 1B: pInsL—rACC decrease (arrow down), state 2A: pInsL—rACC increase (arrow up), state 2B rACC—pInsL decrease (arrow down). For a detailed list of the balancing protocol see Supplementary Table SI. We assessed pain intensity and unpleasantness ratings after each training trial and recorded the strategy used. In the previous study we saw no differential contribution of the modulation of the two regions on the perception of pain intensity and unpleasantness. Based on our previous findings, we expected to find an effect of dissociating two functionally connected regions, which are part of a wider network active in the presence of painful stimulation, on pain intensity and unpleasantness ratings. We examined differences in the contributions of the two regions to the successful modulation of both. We thus compared within each condition (state) changes in the single regions of those subjects who were successful with those who did not learn the modulation of the combined regions.

# **MATERIALS AND METHODS**

### **PARTICIPANTS**

Ten healthy adults with a mean age of 27.8 years (*SD* = 4*.*71, range: 22–35), six females (*M* = 29.0, *SD* = 5*.*25) and four males (*M* = 26*.*0, *SD* = 3*.*65), were examined. All subjects were righthanded as assessed by the Edinburgh Handedness Inventory (Oldfield, 1971) and recruited by announcements at the local university. We defined cardiovascular or neurological disorders, brain injury, current or chronic pain, current analgesic medication, pregnancy, lifetime and current substance abuse or dependence, any mental disorder, and metallic implants as exclusion criteria. The study adhered to the Declaration of Helsinki and was approved by the Ethics Committee of the Medical Faculty Mannheim, Heidelberg University, Germany. All subjects gave written informed consent after a detailed description of the complete study and received a reimbursement of 60 C.

# **STIMULATION PROTOCOL**

Painful monopolar transcutaneous electric stimuli (Digitimer, DS7A, Welwyn Garden City, UK) were applied to the base of the right fourth digit using a concentric stainless steel bipolar needle electrode (Nihon Kohden, Tokyo, Japan), which permitted the stimulation of Aδ nociceptive fibers located in the epidermis (Inui et al., 2002; Yoshino et al., 2010). Pulses lasted 1 ms and were applied with a frequency of 2 Hz. The method of limits was used to determine individual detection and pain thresholds, averaging over the last 2 of 3 ascending and descending stimulation sequences. Stimulation strength was set at 70% between pain threshold and pain tolerance and adjusted to be rated between 6 and 7 on an 11 point verbal rating scale (ranging from 0 = no pain to 10 = strongest imaginable pain). The individually adjusted mean stimulation current was 1.97 mA (*SD* = 1*.*51), the pre-baseline intensity of this stimulus was rated as 6.30 (*SD* = 1*.*06) and the unpleasantness as 5.80 (*SD* = 1*.*23). The postbaseline stimulus intensity was rated 6.10 (*SD* = 2*.*44) and the pain unpleasantness 6.50 (*SD* = 2*.*69) and served as reference for the ratings of the following training trials.

# **RT-fMRI FEEDBACK PROTOCOL**

The neurofeedback protocol was identical to the one described in detail in our previous study (Rance et al., 2014): a baseline run accompanied by an anatomical scan on one day followed by 24 training trials spread over the course of four consecutive days. Each training day consisted of six successive training trials; each trial was composed of six regulation phases (45.0 s) and six nonregulation phases (22.5 s) evenly distributed across each session. During the regulation phases, painful electric stimuli were applied along with the real-time feedback (**Figure 1**). The feedback screen consisted of a moving blue or yellow ball on a black background accompanied by a stationary white arrow indicating the vertical direction in which the ball should be moved. Ball movements were calculated from changes in the computed BOLD signal of the regions (rACC and pInsL). The positions of the regions of interest (ROIs) were determined online and monitored throughout the trials. The criteria for the positions of the ROIs were a) lying in the most significant cluster active during the regulation phase and b) being in the respective areas in the rACC and pInsL. If

two foci of online activation in the rACC and pInsL were present, the location of the ROIs of the baseline run was used to guide the positioning. The feedback was updated every 1.5 s according to the repetition time of the fMRI protocol and calculated as the difference of the percent signal change in the two ROIs. The ball color was similar, blue or yellow, for state 1A/state 2B and state 1B/state 2A, ensuring that the two conditions of a state never had the same ball color or the same direction instruction. The trial sequence of the conditions and the color assignment were counterbalanced over the four days (see Supplementary Table SI) with the first training trial of each of the four states taking place on the first day and the last training trial on the last day. During the baseline session, a non-changing feedback screen with a stationary white ball was shown.

# **INSTRUCTIONS**

Identical to our previous study (Rance et al., 2014), subjects were told that they could learn to control their own brain activity in previously defined brain regions, the vertical change of the blue or yellow ball being an indicator of the change in brain activation. They were informed that they would be able to observe these changes with a delay of a few seconds and that the two colors represented feedback of different brain regions. It was explained that the goal of the training was to assess if and to what extent it was possible to learn to alter brain activation in these brain areas. The function of the selected brain regions (pain processing) was not mentioned neither was the possibility of a relation between a change in activation to a change in the perception of pain intensity or unpleasantness. Subjects were also unaware that there were only two states and that two conditions per state were identical with regards to feedback computation but were presented in a different ball color. The subjects were advised to not use strategies involving body moment (e.g., muscle tension and relaxation), but were otherwise free to choose any other strategy in the regulation phases. During non-regulation phases, subjects were performing simple mental arithmetic. After each trial the subjects had to rate their perceived control over the ball movement on an 11 point verbal rating scale (ranging from 0 = no control to 10 = absolute control) and the average perceived pain intensity and unpleasantness of the nociceptive stimuli given during the regulation phases. The subjects were asked to report their individual regulation strategies verbally after each training trial.

# **RT-fMRI DATA ACQUISITION AND ONLINE ANALYSIS**

The fMRI data were acquired on a 3 T MAGNETOM Trio whole body MR scanner using a standard 12-channel head coil (Siemens Medical Solutions, Erlangen, Germany). Imaging protocols and sequence parameters, as well as the scan procedure and preparation were identical to those used in our previous study (Rance et al., 2014). Before each training day for every subject a standard double gradient-echo field map to measure the static magnetic field was recorded. FMRI data were acquired with a gradientecho echo planar imaging (EPI) sequence (*TR* = 1500 ms, echo time *TE* = 22 ms, matrix size = 96 × 96, voxel size = 2*.*2 × 2*.*2 × 3*.*5 mm3, gap = 0.5 mm, flip angle α = 90◦, bandwidth *BW* = 1270 Hz/px, parallel acquisition technique GRAPPA acceleration factor 2) to record 24 axial slices aligned along the line connecting the anterior-posterior commissure. An anatomical scan was recorded using a three-dimensional (3D) fast low angle shot high-resolution T1-weighted anatomical scan (*TR* = 23 ms, *TE* = 5*.*02 ms, matrix size = 448 × 448, α = 25◦, *BW* = 190 Hz/px, voxel size <sup>=</sup> <sup>0</sup>*.*<sup>5</sup> <sup>×</sup> <sup>0</sup>*.*<sup>5</sup> <sup>×</sup> <sup>1</sup>*.*0 mm3) from each subject. To ensure maximum comparability between all scans, all steps of the training, scanning, and feedback procedures were always performed by the same person.

Online data analysis was based on Turbo BrainVoyager Version 1.1 (Brain Innovation, Maastricht, The Netherlands) consisting of online preprocessing including linear detrending, 3D motion detection and correction, as well as drift removal, and computation of statistical maps for every scan based on the General Linear Model (GLM). Feedback computation and visualization was performed with in-house written scripts based on Presentation® Version 13.0 Build 01.23.09 (Neurobehavioral Systems Inc., Albany, CA, USA).

#### **OFFLINE ANALYSES OF THE fMRI DATA**

Offline data preprocessing of the fMRI scans was performed using BrainVoyager QX 2.3 (Brain Innovation, Maastricht, The Netherlands). Gradient field map based distortion correction was performed on the EPI images as well as 3D motion correction, spatial smoothing with a Gaussian kernel with a full width half maximum of 8 mm3, linear detrending before high frequency artifacts were removed applying a highpass filter (0.006 Hz). Estimation of a response function was done by convolution of a condition box-car time course with a two-gamma hemodynamic response function.

Anatomical datasets were transformed into standard Talairach space and coregistered with their respective functional datasets applying the same transformation parameters using BrainVoyager. A group analysis based on the increase in BOLD percent signal change during the regulation phase with respect to the non-regulation phase was performed using a GLM. All results were Bonferroni corrected for multiple testing.

Offline time course data analysis of the feedback was automated with MATLAB R2011b (The MathWorks Inc., Natick, MA, USA), statistical analyses were performed with IBM SPSS Statistics Desktop (IBM, Armonk, NY, USA) for windows, version 21.0 using 2 (state) × 2 (direction) analysis of variance (ANOVA), paired samples *t*-tests, Pearson correlations, and the Friedman Test for ordinal data. A threshold of *p <* 0*.*05 was employed to determine statistical significance.

The ROI positions for each trial were equally sized (standard dimension of 12 × 12 × 1 voxels selected on the transverse slice). Time courses for the analyses of the change in the activation difference of the regions (feedback signal) were extracted from the ROI positions that were individually saved on every training trial of each subject for all conditions. For group analyses, the saved ROI positions were averaged per condition and region. For analyses of the activation of single regions requiring correction by an unrelated region to be comparable to the difference feedback, a ROI of equal size was determined after group analyses per condition at a location in Brodman area 39 that did not show activation or deactivation, analogous to our previous study (Rance et al., 2014). Time courses per training trial and subject for each condition were then extracted from this ROI and averaged according to regulation and non-regulation phases for each trial of each condition, leaving six single means per subject and condition for regulation and non-regulation phases. The averaged differences of these six single means were averaged into a resulting mean per subject and condition and trial. Later analyses employed onetailed significance levels when a specific a priori hypothesis was present and two-tailed significance levels if we had no assumptions about the direction of an association. Group ROI activations for the baseline run and the first and last training trial are shown in Supplementary Tables SII, SIII.

Similarly to Rance et al. (2014) we identified brain regions active upon the painful stimulation alone and those active on the first and last training trial. For all conditions the success and the effect of the regulation training was assessed. We compared the magnitude of the learned regulation and the differential contribution of the single regions and assessed differences in the use of the chosen strategies. The effect of self-regulation (modulation effect) is seen in the BOLD percent signal change of the rACC, the pInsL, as well as the dissociation (difference between rACC and pInsL, constituting the feedback) in a training trial. In the baseline, where no regulation attempts took place, the BOLD percent signal change in the rACC and pInsL as well as their difference reflects the effect of the stimulation. The effect of the training can be seen in the change of the feedback from the first to the last training trial in a condition. Comparing the rACC and pInsL of the first and last training trial can determine whether both regions are similarly affected by the training. Paired samples *t*-tests were used to compare the feedback and single unrelated region corrected signal changes of the rACC and the pInsL of the first and last training trial of every condition to assess the training effect. The baseline stimulation effect was compared to the modulation effect of the first and the last training trial to assess the stability of the activation due to the stimulation as well as changes due to training (see **Figure 3** in the Results Section). To achieve state 1A and 1B rACC activation had to be higher than pInsL activation, state 2A and 2B required pInsL activation to be higher than rACC activation. In order to determine, whether successful regulation was possible for all subjects, the modulation effect was calculated for every training trial. To compare states regarding both successful and non-successful regulation attempts and to investigate possible effects of modulation as opposed to effects of non-successful modulation, within each condition subjects were additionally categorized as learners and non-learners. Analogously to our previous study, if the average difference of the activation of the two regions was in the right direction (i.e., the feedback was positive), and this was the case for at least four out of six training trials, and if the modulation effect for the specific condition improved from trial 1 to trial 6, a subject was considered a learner for the condition. The other subjects were categorized as non-learners.

#### **RESULTS**

#### **ACTIVE REGIONS**

A whole brain random effect analysis of the baseline session where no regulation attempts took place confirmed active clusters suitable for feedback in the rACC and the pInsL (**Figure 2**). Other active frontal regions were the left and right superior frontal gyrus, the left middle frontal gyrus, left and right inferior frontal gyrus. Parietal regions included the right posterior insula, the

secondary somatosensory cortex, the left supramarginal gyrus, and the right precuneus. The left thalamus and caudate nucleus region was also active (see Supplementary Table SII). With the exception of the right posterior insula in the state 1 pInsL—rACC decrease condition (trial 1), all regions found to show either activation or deactivation in the baseline run, were also active on the first training trial and the last training trial. An additional region in the left inferior gyrus was found to be significantly deactivated in all conditions on the first and last training trial but not significantly active in the baseline run. A region in the precuneus was found to be deactivated in the baseline run and activated on the first and last training trial. The right thalamus and caudate nucleus region was found to be active on the first and last training trial in four conditions. For a detailed overview of activated and deactivated regions in the baseline run and the first and last trial of all conditions see Supplementary Tables SII, SIII.

#### **TRAINING EFFECTS**

Subjects trained to achieve two states, each state had two conditions. For state 1 subjects trained to increase the difference between the rACC and pInsL activation with the rACC activation being higher. This state was trained in two conditions, each having a different ball color and an arrow pointing up in one condition and down in the other. State 2 was also trained in two separate conditions, with the goal of increasing the difference between the activation of the regions so that pInsL activation would be higher than rACC activation. The effect of the stimulation is seen in the activation of rACC and pInsL in the baseline. The modulation effect is seen in the activation of the single target regions and the difference of the rACC and pInsL activation of the training trials. This difference was translated into the vertical ball movement, i.e., the feedback signal. The training effect is seen in the magnitude of the difference of the two regions from the first to the last training trial. **Figure 3** summarizes the results separately for the four conditions, showing changes in the unrelated region corrected activation of both regions and the difference (feedback signal) for all ten subjects and the subgroups of *learners* and *non-learners*.

# *State 1A (rACC—pInsL increase)*

There was a positive training effect with a significantly higher rACC-pInsL difference on the last training trial than on the first [*t*(9) = −1*.*974; *p <* 0*.*05] indicating that subjects had learned to significantly increase dissociation between the regions in the indicated direction from the first to the last training trial. The differential brain activation on the first training trial was significantly lower than the difference of the stimulation effects of the regions in the baseline [*t*(9) = 2*.*544; *p <* 0*.*05]. The stimulation effect of the rACC and pInsL in the baseline did not differ significantly. The modulation effect of the regions did not differ significantly on the first and the last training trial. There was no significant change in the modulation effect of either the rACC or the pInsL from the first to the last training trial. In the *subgroup of the learners* there was a positive training effect with a significantly higher rACC-pInsL difference on the last training trial than on the first training trial [*t*(5) = −7*.*995; *p <* 0*.*001]. The rACC-pInsL difference on the first training trial was significantly lower than the difference of the stimulation effects of the regions in the baseline [*t*(5) = 2*.*576; *p <* 0*.*05]. The rACC-pInsL difference on the last training trial [*t*(5) = −8*.*029; *p <* 0*.*001] was significantly higher. There was no significant difference in the stimulation effect of the rACC and pInsL and no significant difference between the modulation effects of the regions on the first training trial. We observed a significant difference between the modulation effects of the regions on the last training trial [*t*(5) = 5*.*950; *p <* 0*.*01] with the activation in the pInsL being lower. The modulation effect in the pInsL was significantly lower on the last training trial than on the first [*t*(5) = 2*.*678; *p <* 0*.*05] and significantly lower than in the baseline [*t*(5) = 3*.*367; *p <* 0*.*05]. The modulation effects of the rACC did not change significantly from the first to the last training trial. In the *subgroup of nonlearners* there was no significant change modulation effects from the first to the last training trial. There was no significant difference between the stimulation effects of the regions in the baseline and no difference of the modulation effects between the regions on the first and on the last training trial. The modulation effects of the regions did not change from the first to the last training trial.

#### *State 1B (pInsL—rACC decrease)*

The pInsL-rACC difference on the last training trial was significantly lower than on the first training trial [*t*(9) = 1*.*860; *p <* 0*.*05], indicating a positive training effect. The stimulation effect in the baseline and the modulation effect on the first and last training trial of the regions did not differ significantly. There was a significant change in the modulation effect of the pInsL from the first to the last training trial [*t*(9) = 2*.*340; *p <* 0*.*05] with the activation being lower on the last training trial. The modulation effect of the pInsL on the last training trial was significantly lower than the stimulation effect in the baseline [*t*(9) = 2*.*502; *p <* 0*.*05]. The modulation effect of the rACC did not change significantly from the first to the last training trial. For the *learners* there was a positive training effect with the pInsL-rACC difference on the last training trial being significantly lower than on the first [*t*(4) = 3*.*198; *p <* 0*.*05]. The difference of the stimulation effect of the rACC and pInsL in the baseline was significantly lower than the

**directions of the two states.** Black lines indicate the change of the feedback signal, red lines the change in the pInsL only, and the blue line the change in the rACC only. Significant changes in the feedback, rACC, and pInsL activation are indicated in brackets of the same color. A significant difference between the activation in the two regions is indicated by a green bracket. The first column depicts the development from the baseline run to the first and the

last training trial for all ten subjects, the second for the learners (L) and the third for the non-learners (NL) only. A positive training effect is seen in the significant change of the differences between the regions (feedback) from the first to the last training trial in the right direction.

pInsL-rACC difference on the first training trial [*t*(4) = −2*.*881; *p <* 0*.*05]. In the baseline there was no significant difference between the stimulation effects of the regions. There was a significant difference between the modulation effects between the regions on the first training trial [*t*(4) = −2*.*786; *p <* 0*.*05] with pInsL being lower, but no significant difference between the modulation effects on the last training trial. The modulation effect of the pInsL was significantly lower on the last training trial than on the first training trial [*t*(4) = 3*.*514; *p <* 0*.*05]. The modulation effect of the rACC did not change significantly between the trials. In the *subgroup of the non-learners* there was no significant change of the pInsL-rACC difference from the first to the last training trial, the stimulation effect in the baseline did not differ significantly between the regions, nor did the modulation effect differ between the regions on the first and the last training trial.

#### *State 2A (pInsL—rACC increase)*

There was a positive training effect with the pInsL-rACC difference on the last training trial being significantly higher than on the first training trial [*t*(9) = −2*.*268; *p <* 0*.*05]. The modulation effect on the last training trial was significantly higher than the difference of the stimulation effect of the rACC and pInsL in the baseline [*t*(9) = 3*.*598; *p <* 0*.*01]. The stimulation effect in the baseline did not differ significantly between the regions, nor did the modulation effect on the first training trial. On the last training trial the modulation effect was significantly higher in the pInsL than in the rACC [*t*(9) = −2*.*499; *p <* 0*.*05]. There was no significant change in the modulation effect of the rACC and the pInsL from the first to the last training trial. In the *subgroup of the learners*, the pInsL-rACC difference on the last training trial was significantly higher than on the first [*t*(5) = −7*.*453; *p <* 0*.*01] resulting in a positive training effect. The modulation effect on the last training trial was significantly higher than the difference of the stimulation effect of the region in the baseline [*t*(5) = −2*.*683; *p <* 0*.*05]. There was no significant difference between the regions' stimulation effect in the baseline. There was a significant difference of the modulation effect between the regions on the last training trial [*t*(5) = 2*.*69; *p <* 0*.*05], but no significant change in the modulation effects of either region from the first to the last training trial. For the *non-learners* there was a significant negative training effect with the pInsL-rACC difference in the last training trial being lower than on the first training trial [*t*(3) = 2*.*649; *p <* 0*.*05]. The difference in the stimulation effect of the regions in the baseline was lower than the pInsL-rACC difference in the first training trial [*t*(3) = −4*.*788; *p <* 0*.*05] and the pInsL-rACC difference in the last training trial [*t*(3) = −2*.*687; *p <* 0*.*05]. The stimulation effect in the baseline and the modulation effect on the first and last training trials did not differ between the rACC and pInsL. There was no significant change in the modulation effect of the regions from the first to the last training trial.

#### *State 2B (rACC – pInsL decrease)*

The rACC-pInsL difference on the last training trial was significantly lower than on the first training trial [*t*(9) = 2*.*024; *p <* 0*.*05] resulting in a positive training effect, and significantly lower than the difference of the stimulation effect of the regions in the baseline [*t*(9) = 2*.*214; *p <* 0*.*05]. There is no significant difference in the stimulation effect of the regions in the baseline and no significant difference between the modulation effects of the regions on the first and last training trial. The modulation effect of the pInsL is significantly higher on the last training trial than on the first [*t*(9) = −3*.*093; *p <* 0*.*05]. The modulation effect of the rACC does not change significantly from the first to the last training trial. For the *subgroup of the learners* there was a significant positive training effect with the rACC-pInsL difference being lower on the last than on the first training trial [*t*(6) = 7*.*861; *p <* 0*.*001] and significantly lower than the difference of the stimulation effect of the regions in the baseline [*t*(6) = 4*.*313; *p <* 0*.*01]. The stimulation effect of the regions did not differ significantly in the baseline. The modulation effect of the regions differed significantly on the first [*t*(6) = 2*.*792; *p <* 0*.*05] and on the last training trial [*t*(6) = −11*.*695; *p <* 0*.*001]. The modulation effect of the pInsL was significantly higher on the last than on the first training trial [*t*(6) = −6*.*411; *p <* 0*.*01], but the modulation effect was not significantly different for the rACC. In the group of the *non-learners*, there was no significant training effect. The stimulation effect in the baseline was not significantly different between the regions, nor was the modulation effect of the regions on the first and last training trial. For an overview of the results see **Figure 3**.

#### **COMPARISON OF THE CONTROLLABILITY OF REGIONS**

A correlation analysis between the training effects of the conditions showed that there was no correlation between the two conditions of state 1 and state 2. There was a significant negative correlation between the state 1 pInsL—rACC decrease and the state 2 rACC—pInsL decrease conditions [*r*(8) = −0*.*80; *p <* 0*.*01].

To directly compare the controllability of the states and conditions, we compared the training effects of the two states and the two directions (increase, decrease). An analysis of variance revealed a significant effect of the regulation direction [*F*(33*.*26*,*0*.*87) = 38*.*22; *p <* 0*.*001]. There was no significant effect of the state and no significant interaction effect. Follow-up paired samples *t*-tests between the increase and decrease conditions of the different states, state 1 rACC—pInsL increase and state 2 rACC—pInsL decrease; state 1 pInsL—rACC increase and state 2 pInsL—rACC decrease revealed significant differences in the expected directions in both cases [*t*(9) = 3*.*14; *p <* 0*.*05; *t*(9) = 2*.*48; *p <* 0*.*05].

To examine whether one state was easier to achieve, the direction-independent magnitude of the training effect was compared (see **Figure 4**). There were no significant differences between the four conditions (both states and both directions).

We compared learner and non-learner for each condition. State 1 had six learners in the increase condition and five in the decrease condition, state 2 had six learners in the increase and seven in the decrease condition. Being a learner or a non-learner did not significantly correlate between the four conditions and was not correlated with other person-specific variables such as age, gender, stimulation current, or pain intensity and unpleasantness ratings.

#### **EFFECTS ON PAIN INTENSITY AND UNPLEASANTNESS RATINGS**

To identify a possible effect of the learned dissociation of the regions on the perception of pain intensity and unpleasantness, paired samples *t*-tests were used to compare the ratings of the first and the last training trial of every condition (see **Figure 5**). There was no significant difference between the ratings on the first and on the last training trial in any of the four conditions. The same was true for the subgroups of the learners and non-learners. Pain ratings from the baseline run and on the first and on the last training trial did not correlate with rACC and pInsL percent signal change or the difference between the activations.

To investigate the association of the dissociation of rACC and pInsL (training effect) and the evaluation of pain intensity and pain unpleasantness, a correlation analysis was performed. There was no significant correlation between the change in pain intensity and unpleasantness ratings [rating(training trial 6−1)] and the respective activation difference between the regions [computed feedback signal(training trial 6−1)] in the conditions, neither for the whole group nor for the respective subgroups. A similar

**FIGURE 4 | Box plot showing the median, interquartile range, sample minimum and maximum of the direction independent magnitude of the training effect in the four conditions calculated from the unrelated region corrected (urc) difference of the rostral anterior cingulate gyrus (rACC) and the left posterior insula (pInsL) blood oxygenation level-dependent (BOLD) percent signal change.**

correlation analysis between the activation change in the single regions [rACC/pInsL(training trial 6−1)] from the first to the last training trial and change in the pain intensity and unpleasantness ratings also revealed no significant correlations.

#### **STRATEGIES**

The reported strategies were subsumed under four categories. Category 1: ball focus: picturing the ball in motion or silent verbal instructions to move the ball; Category 2: distraction: focusing on another body part, continuing with the mental arithmetic, imagining a scene that was not described as being particularly emotional or relaxing; Category 3: positive or negative emotional memories; Category 4: stimulus focus: imagined changing of the stimulus quality or intensity, the location or frequency. If a subject reported to have been using two strategies during one training trial, both were recorded and categorized.

We compared the frequency of the strategy use per category over all conditions using a Friedman test. There was no significant difference between the strategy use over all conditions [χ<sup>2</sup> (3*,* 10) = 9*.*255; *p* = 0*.*51]. For a distribution of the strategies per trial over all conditions see Supplementary Figure S1.

# **DISCUSSION**

#### **ACTIVE REGIONS DURING STIMULATION AND MODULATION**

A whole brain offline analysis of the baseline run when subjects received no feedback and did not attempt to modulate brain activation, showed that active regions generally consisted of areas involved in the processing of pain (Apkarian et al., 2005; Iannetti et al., 2005). They included the rACC and pInsL, which were the source of the feedback signal in the following training trials, and other regions that are frequently found to be involved in the processing of pain, including the right posterior insula, bilaterally the secondary somatosensory cortex, and the posterior cingulate cortex in accordance with our previous study (Rance et al., 2014). Additional regions found in the baseline run were the right thalamus and caudate nucleus region, several frontal regions in the mid, inferior, and superior frontal gyrus that showed either activation or deactivation. In the parietal lobe we found deactivation in the left primary somatosensory cortex, the left precuneus, and the left gyrus supramarginalis. Almost all regions found to be active in the baseline were also active on both the first and the last trials of all conditions. One region in the inferior frontal gyrus and the left thalamus and caudate nucleus region was additionally active on the first and last training trials. Similar to our previous study (Rance et al., 2014) the right posterior insula was significantly active in the baseline run, shifting to deactivation on the last training trial in all but one condition. Whereas in the previous study, on the first training trial the pInsR was mainly activated, here the results are mixed, with the pInsR being predominantly deactivated in two conditions of different states, not significantly activated or deactivated in one condition, and activated in one condition. The prevalent change from activation during the painful electric stimulation to deactivation on the last training trial of both regulation of single regions and the difference feedback suggests that this region is not only involved in the processing of pain but also in the regulation effort. Deactivation of this region is observed within the framework of task-induced deactivations seen in the default mode network (Harrison et al., 2011). It was also placed in the pattern of the deactivation of regions involved in the processing of pain during a reduction of pain perception through shifting attention away from the pain or placebo analgesia (Tracey and Mantyh, 2007; Amanzio et al., 2013). An increase in pInsR activation was seen during painful stimulation when subjects were in a state of mindfulness, an increased non-judgmental awareness of present experiences, thus effectively focusing attention on sensory aspects of the stimulus, such as the skin surface of the stimulated area (Gard et al., 2012). Taken together, the results of the present study suggest that during the course of the training the perception of painful stimulation was of increasingly lesser importance, consistent with the increased task demand of the training trials, i.e., finding and memorizing an adequate modulation strategy while observing the feedback and judging the success of the modulation efforts.

A region that was found to be deactivated in the baseline, the right precuneus, was consistently activated in all conditions on the first and the last training trial. The precuneus has been described to be involved in the process of integrating salient stimuli into self-relevant experiences. It is thus not necessarily involved in the processing of pain, but in the evaluation of a salient stimulus in the context of present experience (Cavanna and Trimble, 2006; Goffaux et al., 2014). Increased attention to the stimulation induced feedback might thus lead to an activation of the precuneus independent from the processing of the nociceptive quality of the stimulus.

#### **MODULATION**

In our previous study (Rance et al., 2014) the rACC and the pInsL were modulated separately, and subjects learned modulation of all but one condition. Using the same training paradigm in this study with the feedback consisting of a difference feedback of both regions, subjects successfully learned to achieve both states in the two respective conditions. In both studies, there was no significant correlation of the learning success between the conditions and states. When modulation efforts are focused on one region and direction, the absence of a significant correlation of the regulation success between the conditions suggests that subjects are not able to gain control over the activation of one of the regions better than the other and that there is no "easier" direction of modulation. In the present study, subjects trained to increase the difference in the activation of the two regions, with two seemingly different conditions per state. Here, we again did not find evidence that one state is achieved more easily. Moreover, even if one state is successfully learned in one condition, the same state is not necessarily achieved in another condition. We did find a difference in the utilization of the modulated regions. Regarding the regions separately, a significant change from the first to the last training trial was only evident in the pInsL in the state 1 pInsL rACC decrease and the state 2 rACC—pInsL decrease in the pInsL activation. This was especially true in the subgroup of the learners, where this significant change was additionally observed in the state 1 rACC—pInsL increase condition. These results indicate that the target ROI in the insula was the driving force behind the change in the feedback consisting of the activation difference of the regions. There was also some evidence that the pInsL can be regulated in a more stable manner in our previous study (Rance et al., 2014), where the rACC could be trained overall, however, with a much larger range of training success, especially in the upregulation condition, where the group as a whole did not show a significant training effect. Both the rACC and the pInsL are implicated in a variety of functions. The posterior part of the insula is involved in interoception, emotional processing, and pain perception (Cauda et al., 2012a,b; Dowman, 2014; Uddin et al., 2014). In the presence of ongoing salient painful stimulation, activation in the posterior insula might be more readily accessible. The rACC on the other hand is involved in both emotional and cognitive-evaluative functions apart from playing a crucial role in the network active during pain perception (Bush et al., 2000; Li et al., 2013). It might be involved in both the modulation process itself and the processing of the painful stimulus, yielding it more difficult to regulate especially when the feedback does not depend on rACC activation alone.

Similar to our previous study (Rance et al., 2014), strategy use was equally distributed across the training trials. Together with the result that there also was no significant correlation between learning success between the regions or states, the regulation of rACC and pInsL in the presence of a painful stimulus does not require or favor a specific strategy. This might be connected to the multimodal nature of the regions. In other real-time neurofeedback studies, a concrete strategy has sometimes been linked to the modulation and the function of the modulated area such as emotion induction in the regulation of the right anterior insula (Caria et al., 2007). Because of the distributed nature of pain processing, there might not be generally useful strategies.

#### **PAIN INTENSITY AND UNPLEASANTNESS RATINGS**

In our previous study (Rance et al., 2014) no significant effect of the successful modulation of either the rACC or the pInsL on pain intensity or unpleasantness ratings was found. However, a connection between the modulation in the pInsL and unpleasantness ratings was observed when the difference between the target region (pInsL) and the rACC was high. In the present study, subjects successfully learned to increase the activation difference achieving two different states, however, there was no statistical evidence that the ability to dissociate the two regions was significantly correlated with a change in the pain intensity and unpleasantness ratings. This was also true in the subgroup of learners. Moreover, there was no significant correlation between either the differences of the activation of the target ROIs, or the unrelated region corrected activation of either region, and the pain intensity and unpleasantness ratings. The ratings were not significantly correlated with activation in either region or the combined feedback. Due to the exploratory nature of this study, we examined several specific aspects of feedback modulation and therefor refrained from an overall correction of the *p*-values. Our results are not in line with previous findings by deCharms et al. who found a correlation of the increase and decrease of rACC activation and a change in pain intensity and unpleasantness ratings, thus linking regulation of brain activation with changes in pain perception. One possible reason might be that activation in the rACC and pInsL are the result of both the stimulation or task and modulation efforts (Papageorgiou et al., 2009). In the current literature on neuromodulation there are mixed results on behavioral effects of modulation. Pain perception is a variable experience, involving many modulating cognitive, emotional, and physiological factors (Rhudy and Meagher, 2000; Bantick et al., 2002; Apkarian et al., 2005; Forys and Dahlquist, 2007; Tracey and Mantyh, 2007; Gard et al., 2012) and a wide network of contributing regions (Hofbauer et al., 2001; Tracey and Mantyh, 2007; Iannetti and Mouraux, 2010; Cauda et al., 2012b). The modulation of the perception of pain intensity and unpleasantness of a painful electrical stimulus does not seem possible by regulation of the rACC and pInsL either singularly or combined. Attempting modulation should thus involve the active network to a greater part. This has been suggested especially in the context of rt-fMRI since many connections between brain activation and behavior depend not only on the activity of single regions but on the connectivity of involved networks (Zilverstand et al., 2014). Connectivity feedback seems to be a more feasible method especially when attempting behavioral changes, since changes in connectivity go along with learning to regulate activation in brain areas (Berman et al., 2013; Zotev et al., 2013; Scharnowski et al., 2014). This approach might, moreover, help in identifying regions involved in the process of learning modulation. When considering employing rt-fMRI in the treatment of chronic pain, it is further important to distinguish experimental pain from ongoing chronic pain and the respective activated networks, which may be very different (Baliki et al., 2010). Taken together, rt-fMRI might be valuable in not only identifying regions involved in the processing of pain in pain disorders, but also help understanding the contribution to pain perception of single regions within the active network, which could be the targets of modulation in the treatment of chronic pain.

#### **CONCLUSION**

In our previous study we found that the state of the network played a crucial role in regulating pain-related activation and might be key in altering the perception of pain. In the present study we therefore explored the extent to which an increase in the difference in the activation of two functionally connected areas in response to a painful stimulus is possible and the effects on the perception of pain. We thus not only examined the results of the group but also looked at differences between subjects who learned regulation and those who did not, similar to the previous study. We found that subjects were able to increase the difference in all four conditions after six trainings trials, thus successfully achieving the two states of either the rACC or the pInsL BOLD percent signal change being higher. When looking at the contribution of the single regions to the combined difference feedback, the pInsL was found to be driving force in three out of the four conditions with significant changes in the activation from the first to the last training trial. Activation in the rACC did not change significantly. In our previous study we saw that control over rACC and pInsL in both directions can be acquired by the majority of subjects (Rance et al., 2014). Mirroring these results, learning success did not correlate between the conditions or states. This indicates that among our subjects, there is no overall ability to learn regulation to achieve the two states in general. This means that subjects who were successful in one or two conditions did not necessarily learn all, and that if a subject learned to establish one state in one condition the same subject did not always learn to establish the state in the other condition. Since the group as a whole did learn to establish both states in the conditions, our results suggest that, given enough training trials, both states can be successfully established. Furthermore, learning was unrelated to chosen strategies. In line with this, the magnitude of the feedback change was similar between all conditions. Despite successful modulation, there was no change in the perception of pain intensity or unpleasantness. Our results suggest that in the modulation of pain intensity and unpleasantness, both the rACC and pInsL either alone or combined, are not sufficient to alter perception of the painful experimental electric stimulus. However, it is possible that increasing automatization of the response would free the respective region from dual tasking and could thus have an effect. This could be tested by using extended training sessions. It is also necessary to identify networks not only involved in the processing of the stimulus but also in learning regulation of the network. It might then be possible to specifically modulate communication of parts of the network to alter perception and even correct altered states of the pain processing network in patients with chronic pain.

#### **ACKNOWLEDGMENTS**

The authors would like to thank Dr. C. Christmann from the Central Institute of Mental Health, Mannheim, Germany, for help with the set-up of the experimental hardware and the experimental procedure. We would also like to thank Professor K. Inui from National Institute for Physiological Sciences, Okazaki, Japan for providing the electrodes. This study was supported by a grant from the Deutsche Forschungsgemeinschaft (Fl 156/33) and the Prize for Basic Research of the State of Baden-Württemberg to Herta Flor.

#### **SUPPLEMENTARY MATERIAL**

The Supplementary Material for this article can be found online at: http://www*.*frontiersin*.*org/journal/10*.*3389/fnbeh*.*2014*.* 00357/abstract

#### **REFERENCES**

Amanzio, M., Benedetti, F., Porro, C. A., Palermo, S., and Cauda, F. (2013). Activation likelihood estimation meta-analysis of brain correlates of placebo analgesia in human experimental pain. *Hum. Brain Mapp.* 34, 738–752. doi: 10.1002/hbm.21471


exploratory real-time FMRI and TMS study. *Neurorehabil. Neural Repair* 26, 256–265. doi: 10.1177/1545968311418345


**Conflict of Interest Statement:** This study was supported by the Mück-Weymann-Prize of the Deutsche Gesellschaft für Biofeedback e. V. granted to Mariela Rance. The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 23 April 2014; accepted: 26 September 2014; published online: 15 October 2014.*

*Citation: Rance M, Ruttorf M, Nees F, Schad LR and Flor H (2014) Neurofeedback of the difference in activation of the anterior cingulate cortex and posterior insular cortex: two functionally connected areas in the processing of pain. Front. Behav. Neurosci. 8:357. doi: 10.3389/fnbeh.2014.00357*

*This article was submitted to the journal Frontiers in Behavioral Neuroscience.*

*Copyright © 2014 Rance, Ruttorf, Nees, Schad and Flor. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Cognitive and neural strategies during control of the anterior cingulate cortex by fMRI neurofeedback in patients with schizophrenia

#### Julia S. Cordes 1,2 , Krystyna A. Mathiak 1,2,3 , Miriam Dyck 1,2 , Eliza M. Alawi 1,2 , Tilman J. Gaber 1,3 , Florian D. Zepf 2,3,4,5 , Martin Klasen1,2 , Mikhail Zvyagintsev 1,2 , Ruben C. Gur <sup>6</sup> and Klaus Mathiak 1,2,7 \*

<sup>1</sup> Department of Psychiatry, Psychotherapy, and Psychosomatics, Medical School, RWTH Aachen University, Aachen, Germany, <sup>2</sup> JARA-Translational Brain Medicine, RWTH Aachen University, Aachen, Germany, <sup>3</sup> Clinic for Child and Adolescent Psychiatry, Psychosomatics and Psychotherapy, RWTH Aachen University, Aachen, Germany, <sup>4</sup> Department of Child and Adolescent Psychiatry, School of Psychiatry and Clinical Neurosciences and School of Paediatrics and Child Health, Faculty of Medicine, Dentistry and Health Sciences, The University of Western Australia (M561), Perth, WA, Australia, <sup>5</sup> Specialised Child and Adolescent Mental Health Services (CAMHS), Department of Health in Western Australia, Perth, WA, Australia, <sup>6</sup> Department of Psychiatry, University of Pennsylvania, Philadelphia, PA, USA, <sup>7</sup> Institute of Psychiatry, Psychology and Neuroscience, King's College London, London, UK

## Edited by:

Francisco Javier Zamorano, Universidad del Desarrollo, Chile

#### Reviewed by:

Megan Teresa DeBettencourt, Princeton University, USA Christian Paret, Central Institute for Mental Health, Germany

#### \*Correspondence:

Klaus Mathiak, Department of Psychiatry, Psychotherapy, and Psychosomatics, Medical School, RWTH Aachen University, Pauwelsstraße 30, 52074 Aachen, Germany kmathiak@ukaachen.de

> Received: 15 November 2014 Accepted: 15 June 2015 Published: 25 June 2015

#### Citation:

Cordes JS, Mathiak KA, Dyck M, Alawi EM, Gaber TJ, Zepf FD, Klasen M, Zvyagintsev M, Gur RC and Mathiak K (2015) Cognitive and neural strategies during control of the anterior cingulate cortex by fMRI neurofeedback in patients with schizophrenia. Front. Behav. Neurosci. 9:169. doi: 10.3389/fnbeh.2015.00169 Cognitive functioning is impaired in patients with schizophrenia, leading to significant disabilities in everyday functioning. Its improvement is an important treatment target. Neurofeedback (NF) seems a promising method to address the neural dysfunctions underlying those cognitive impairments. The anterior cingulate cortex (ACC), a central hub for cognitive processing, is one of the brain regions known to be dysfunctional in schizophrenia. Here we conducted NF training based on real-time functional magnetic resonance imaging (fMRI) in patients with schizophrenia to enable them to control their ACC activity. Training was performed over 3 days in a group of 11 patients with schizophrenia and 11 healthy controls. Social feedback was provided in accordance with the evoked activity in the selected region of interest (ROI). Neural and cognitive strategies were examined off-line. Both groups learned to control the activity of their ACC but used different neural strategies: patients activated the dorsal and healthy controls the rostral subdivision. Patients mainly used imagination of music to elicit activity and the control group imagination of sports. In a stepwise regression analysis, the difference in neural control did not result from the differences in cognitive strategies but from diagnosis alone. Based on social reinforcers, patients with schizophrenia can learn to regulate localized brain activity. However, cognitive strategies and neural network location differ from healthy controls. These data emphasize that for therapeutic interventions in patients with schizophrenia compensatory strategies may emerge. Specific cognitive skills or specific dysfunctional networks should be addressed to train impaired skills. Social NF based on fMRI may be one method to accomplish precise learning targets.

Keywords: social reinforcement, cognitive therapy, psychotherapy, remediation therapy, cognitive strategies, rostral and dorsal ACC, brain computer interface, self-regulation

# Introduction

Aspects of cognitive functioning, such as memory, attentional performance, or face recognition, are impaired in patients with schizophrenia, and can lead to significant disabilities in occupational, social, and economic functioning (see Keefe and Harvey, 2012, for a review). These particular impairments often persist throughout all stages of illness (Censits et al., 1997). Antipsychotic agents often improve positive symptoms, but treatments that improve social and cognitive functioning are still warranted (Keefe et al., 1999). Cognitive skills have become an important therapeutic target in patients with schizophrenia, and are associated with long-term outcomes and prognosis (Purdon et al., 2000). Psychological strategies, such as cognitive remediation therapy, have been introduced to improve cognitive deficits (Turkington et al., 2004) and should target different symptom domains, e.g., executive function, attentional performance, and aspects of memory (Kurtz et al., 2001). Such improvements of specific cognitive functions (Medalia and Lim, 2004) can be expected to normalize dysregulated neural activity (Kurtz, 2012; Penadés et al., 2013). Although one may expect that addressing a supposed neural dysregulation in a direct manner should lead to improvements in cognitive functions, so far, none of the existing therapeutic approaches directly addresses the underlying neural deficits (Turkington et al., 2004).

The anterior cingulate cortex (ACC) plays an important role in the pathogenesis of schizophrenia. Structural magnetic resonance imaging (MRI) and neuropathological findings demonstrate gray matter reductions of the ACC in patients with psychosis, occurring already prior to its onset and, eventually, progressing with illness duration (Fornito et al., 2009). In functional MRI (fMRI) studies, patients with schizophrenia showed reduced conflict- (Snitz et al., 2005) as well as errorrelated activity in the ACC (see Melcher et al., 2008, for a review; Carter et al., 2001; Alain et al., 2002; Kerns et al., 2005), which may normalize upon administration of antipsychotic medications (Snitz et al., 2005; Adams and David, 2007). Studies reported hypo-activation in patients only for the rostral division of the ACC during Stroop (Carter et al., 1997), Go/NoGo (Laurens et al., 2003), oddball (Liddle et al., 2006), or emotion recognition tasks (Habel et al., 2010). Reduced ACC activity in patients with schizophrenia plays an important role in the development of deficits in different cognitive domains, such as attention, working memory, verbal production, response monitoring, and inhibition (Sanders et al., 2002).

fMRI-based neurofeedback (NF) trains subjects to control localized brain activity (Bray et al., 2007). Using real-time fMRI, a Brain-Computer Interface (BCI) provides feedback of the momentary activity in a selected brain area (Weiskopf et al., 2004). This has been suggested as a method to modulate specific functions of neural networks (Yoo et al., 2006). Several studies have demonstrated that healthy participants can learn the control of circumscribed brain regions using fMRI-based NF (Yoo et al., 2006; Caria et al., 2007; Rota et al., 2009; Hamilton et al., 2011; Scharnowski et al., 2012; Lawrence et al., 2013). Moreover, first attempts of using fMRI-NF as a therapeutic intervention have been made (e.g., Subramanian et al., 2011). Patients with treatment-resistant depression showed clinical improvements after localized regulation (Linden et al., 2012). In patients with chronic pain, improved symptomatic control was associated with ACC regulation (deCharms et al., 2005). Moreover, successful ACC regulation was achieved in other study groups, i.e., such as patients with nicotine addiction (Li et al., 2013). In patients with schizophrenia, NF of the bilateral anterior insula activity led to an emotion recognition bias towards disgust, demonstrating that patients with schizophrenia were able to gain voluntary control over their regional brain activity (Ruiz et al., 2013). So far, however, no NF study aimed to improve cognitive functioning in patients with schizophrenia. The ACC seems a promising target region for such an intervention, because the ACC is considered to be a central hub for cognitive processing. In particular, in patients with schizophrenia changes in ACC functioning were associated with impaired responses in cognitive tasks, especially in a Stroop cognitive interference task (Minzenberg et al., 2009). Voluntary control of ACC function may help to overcome specific cognitive deficits, and the applied cognitive strategies may provide a significant pathway to improve cognitive skills in this particular disorder (Weiskopf et al., 2004), even over a longer period of time (Harmelech et al., 2013).

In patients with schizophrenia, the voluntary regulation of ACC function may influence circumscribed cognitive dysfunctions, and thereby may contribute to the understanding of cognitive remediation therapies. The present study applied fMRI-based NF over 3 days to train volitional control of the ACC in participants with schizophrenia and in healthy controls. Since this particular brain area underlies cognitive dysfunctions in the patient group, we expected differences between the groups with respect to the neural pattern of activation as well as the applied cognitive strategies. Therefore, reported strategies were content-coded and related to the activation pattern during ACC regulation. We expected different frequencies of emerging content categories between the groups. Further, we explored correlations of regulation amplitudes with symptom and mood scales.

# Materials and Methods

# Participants

We investigated 11 patients with a confirmed diagnosis of schizophrenia (five females) with a mean age of 38.9 ± 9.3 years, and an age- and gender-matched control group of 11 healthy subjects (see **Table 1**). Patients were recruited through the Department of Psychiatry, Psychotherapy and Psychosomatics of the University Hospital Aachen. All of the patients were diagnosed with schizophrenia according to the Diagnostic and Statistical Manual of Mental Disorders (DSM-IV) criteria, and all of them were medicated with a stable dose of antipsychotics (seven with a single and four with a combination of two). The diagnoses were ascertained by experienced clinicians using the

**Abbreviations:** ACC, anterior cingulate cortex; BCI, brain-computer interface; EEG, electroencephalography; EPI, echo planar image; NF, neurofeedback; ROI, region of interest; rt-fMRI, real-time functional magnetic resonance imaging; SPM, Statistical Parametric Mapping.

German version of Structured Clinical Interview for the DSM-IV (SKID; Wittchen et al., 1996). Patients with acute psychiatric or any neurological co-morbidity were excluded. The control group comprised 11 healthy subjects, matched for age (38.9 ± 9.3 years) and gender (five females), and was recruited via public advertisement. All but one participant were right-handed as rated with the Edinburgh Handedness Inventory (Oldfield, 1971; see **Table 1**) and gave written informed consent to the experimental protocol, which was in accordance with the Helsinki Declaration and approved by the Ethics Committee of the RWTH Aachen University, Germany.

# Procedure

Every participant underwent three NF training sessions on three different days during 1 week, with at least 1 day without training in between. Each session included three NF runs. The runs comprised eight regulation and nine baseline blocks, lasting 30 s each, starting and ending with a baseline block, resulting in 8.5 min per run. During the regulation blocks, the momentary BOLD activation was fed back to the participants via a BCI providing social rewards (**Figure 1**; for details see Mathiak et al., 2010): In short, the avatar of a dark-haired male human (created using Poser Pro, Smith Micro, Inc., USA) smiled at the participants with rising intensity when the activity of the ACC increased. In contrast, it gradually returned to a neutral expression when the activity decreased. A fair-haired, slightly smiling avatar indicated the baseline condition, instructing to count backwards from 100 in steps of three. The association of dark and fair hair

TABLE 1 | Demographic and clinical characteristics of schizophrenia (n = 11) and control group (n = 11).

with regulation and baseline condition were randomized and counterbalanced across subjects.

The Positive and Negative Affect Scale (PANAS; Watson et al., 1988) assessed mood before and after each of the three NF sessions, i.e., a pre and a post value for each day. After the experiment, imagination abilities were assessed with the short version of Betts' Questionnaire upon Mental Imagery (QMI; Sheehan, 1967). Patients' clinical status was assessed using the Positive and Negative Syndrome Scale (PANSS; Kay et al., 1987).

# Instructions and Strategies

During NF blocks, subjects were advised to up-regulate the smile intensity (feedback signal) using a personalized individual mental strategy. They were provided with a standardized protocol containing information on the hemodynamic delay and the instruction to only switch in between different strategies after trying one for at least 10 s. Some template strategies from different cognitive domains were named, i.e., positive autobiographic memories, picturing oneself doing sports or playing an instrument, and concentrating on certain perceptions like feeling the temperature of one's own left foot. However, it was clarified that the subjects needed to find individual ways and strategies to achieve successful regulation of the feedback signal, and that they would be asked to report what kind of strategies they applied after each feedback run as well as in an interview at the end of every training day. Further, they rated their perceived control success over the feedback signal on a scale from 1 to 10. During baseline blocks, subjects were instructed to count


PANAS: Positive and Negative Syndrom Scale; QMI: Questionnaire upon Mental Imagery; PANSS: Positive and Negative Syndrom Scale (X): currently abstinent from smoking F63.0: pathological gambling, F20.4: post-schizophrenic depression, F19.1: multiple drug abuse.

FIGURE 1 | NF paradigm using social reward. In blocks of 30 s, the dark-haired avatar gave feedback of localized brain activity by smiling with rising intensity, whereas the fair-haired face instructed to count backwards, serving as

baseline condition. Baseline and regulation blocks summed up to 8.5 min for one run. Three runs were conducted on each of the three training days.

backwards in steps of three, starting with 100. They were advised to concentrate on counting and not to think about anything else.

# Analysis of Cognitive Strategies

The reported strategies were content-coded by three independent raters with the categories Music (German: ''Musik''), Sports (''Sport''), Mention of other people (''Gedanken, in deren Formulierung explizit andere Menschen erwähnt werden''), and Others (''Sonstiges''). The system yielded an interrater reliability of α = 0.85 (Krippendorff ´s Alpha Reliability Estimate; Krippendorff, 1970). Two-sample t-tests compared the frequencies of the four categories between the groups. Furthermore, we conducted a stepwise regression analysis for the dorsal and the rostral ACC peak activity to analyze contributions of cognitive strategies to the neural activity.

# Data Acquisition

Imaging was performed on a 3T MRI scanner (Magnetom TRIO, Siemens medical systems, Erlangen, Germany). Sixteen transverse slices were acquired in parallel to the anterior commissure—posterior commissure (AC-PC) plane with echo planar imaging (EPI) at a repetition time TR = 1 s (echo time TE = 28 ms; matrix size = 64 × 64 with 3 × 3 mm<sup>2</sup> in-plane resolution; slice thickness = 3 mm; gap = 0.75 mm; flip angle = 67◦ ). Each NF run comprised 520 volumes. For optimal and reproducible ACC coverage, the slices were positioned so that the AC-PC plane was between the 7th and the 8th slice from the bottom. Additionally, we collected a T1-weighted structural scan for superposition of functional maps and brain anatomy (MPRAGE, field of view FOV = 256 × 256 mm, 176 sagittal slices, 1 mm<sup>3</sup> isotropic voxels, TR = 1900 ms, TE = 2.52 ms, flip angle = 8◦ ).

#### On-Line Data Analysis

An observer-independent anatomical template of the ACC determined the target region. In particular, it represented the cingulate cortex from the Automated Anatomical Labeling (AAL) atlas restricted to the quadrant with y and z coordinates greater than zero in the Montreal Neurological Institute (MNI) space. The individual images were realigned to a reference male or female EPI dataset and image raw data was extracted at and averaged over the predefined mask. First, exponential moving average algorithm removed low-frequency drift. Second, outliers, and high-frequency fluctuations were reduced with a modified Kalman filter (see Koush et al., 2012, for details). The data were scaled as to 1% signal change represented the full range of the display from the mild smile to the full smile of the avatars face. The baseline periods achieved a return to the initial signal level due to the drift removal. Data processing and feedback visualization were performed using Matlab 7.7 (The Mathworks, Natick, MA, USA).

# Off-Line Data Analysis

Statistical parametric mapping (SPM8)<sup>1</sup> was conducted following standard procedures with realignment for motion correction, normalization into the MNI template space (Collins et al., 1994), and spatial smoothing with 8 mm full-width at half-maximum Gaussian kernel. Blocks for regulation conditions entered into a general linear model convolved with hemodynamic response function as independent variables. Second-level statistics conducted a group comparison using a two-sample t-test. After model estimation, a mask was applied including only voxels within the ACC. We used the WFU Pickatlas<sup>2</sup> to define the ACC mask, taking the provided anatomical template of the cingulate cortex with the y coordinate ≥ 0. Levels of significance were set according to p < 0.05 after family-wise error correction (FWE).

A probit model tested whether there was predictive power of categories on the group variable. Post hoc t-tests identified the contributions of each category. Further, individual contrast estimates from dorsal and rostral subdivisions of the ACC entered linear regression analysis with group indicator, test scores, and category of strategy as predictors. Stepwise regression analysis investigated whether the subsequent

<sup>1</sup>http://www.fil.ion.ucl.ac.uk

<sup>2</sup>http://fmri.wfubmc.edu/software/PickAtlas

addition predictors added to the explanatory power of the hierarchical models more than chance effects. The level of statistical significance was set according to p < 0.05 without correction for multiple testing owing to the exploratory character.

# Results

#### ACC Regulation

During regulation as compared to the counting baseline blocks the patients yielded significant increase in activation in the a priori defined ACC mask (Tpeak = 14.75, p < 0.05, FWEcorrected). The bilateral activation cluster was located in the dorsal subdivision of the ACC (Brodmann area [BA] 24, and partly in BA 33; **Figure 2A**, see **Table 2**). The control group upregulated the ACC as well, but activated the rostral subdivision (Tpeak = 5.19, p < 0.05, FWE-corrected; BA 32 and 33; **Figure 2B**; **Table 2**). The difference in localization was confirmed within a direct comparison: the patient group exhibited a greater activation of the dorsal subdivision (Tpeak = 7.32, p < 0.05, FWE-corrected), whereas the control group yielded stronger rostral subdivision activity (Tpeak = 5.00, p < 0.05, FWEcorrected; **Figure 2C**). These findings were located within the a priori ACC mask, but also yielded significance without small volume correction. The explorative correlations between brain activity at the cluster peaks and scales for mood, imagery, or symptom severity (average pre- and post-PANAS, QMI, PANSS) yielded no significant brain-behavior relationships at this level (all p > 0.1).

#### Whole-Brain Analysis

Exploratory mapping analysis of the entire volume indicated significantly higher activation during regulation blocks for patients with schizophrenia vs. controls in bilateral superior temporal gyri (Tpeak = 8.60, p < 0.05, FWE-corrected), rolandic opercula (Tpeak = 6.79, p < 0.05, FWE-corrected), pre- (Tpeak = 6.50, p < 0.05, FWE-corrected) and postcentral gyri (Tpeak = 6.63, p < 0.05, FWE-corrected), as well as in the left middle temporal gyrus (Tpeak = 7.22, p < 0.05, FWE-corrected) and the left inferior parietal gyrus (Tpeak = 6.49, p < 0.05, FWE-corrected). The reversed contrast revealed lower activity in the right supramarginal gyrus (Tpeak = 8.30, p < 0.05, FWE-corrected) and the right middle temporal gyrus for the patient group (Tpeak = 6.43, p < 0.05, FWE-corrected).


Cluster size refers to the whole brain analysis without ACC mask. In the "Pat > Cntl" contrast, the large volume reflects the confluence with a distributed pattern. Pat: patients; Cntl: controls.

FIGURE 3 | Activity at rostral and dorsal ACC clusters. The vertical axis shows individual activation amplitudes in the dorsal cluster and the horizontal axis in the rostral cluster (arbitrary units but same scaling for both axes). The background picture merely illustrates this relationship; it depicts the dorsal and rostral ACC clusters from Figure 2C. In the left panel for the control group, the data points are closer to the rostral cluster at the right side; on the right panel for the group of patients with schizophrenia, the data points tend towards the dorsal cluster positioned at the top. Further, color and form of the data points reveal the prominent strategy of the individual for the given regulation success (see insert). Cognitive strategies involving music (blue circles) were mostly used by patients and led in some of them but not in controls to high dorsal activation. Strategies mentioning sports (green crosses) were more frequently found in the controls.

# Cognitive Strategies

All subjects reported a feeling of control over their ACC activity with a mean rating of 5.65 ± 1.35 (first day), 6.52 ± 1.33 (second day), and 6.76 ± 1.14 (third day) on a scale from 1 to 10. The strategies reported by patients and healthy controls ranged widely. For instance, subjects reported they applied thoughts on ''my favorite music, songs from ABBA, Smokie, and The Sweet, on ''being at the beach with my family, and on ''playing soccer.'' Based on the list of all reported strategies, an inductive coding scheme was derived, similar to a previous method for fMRI analysis (Mathiak and Weber, 2006). The induction and the coding were conducted blind to the diagnosis, i.e., from the list of verbatim reports without information on the respective participant. In the team of the authors, a list of potential categories was created reflecting the previous experience on reported control strategies. These initially 13 categories were iteratively fused to yield a sufficient frequency of items and an inter-rater reliability above 0.8 from two independent coders. In the final content coding scheme with four categories, the reported strategies differed between the diagnostic groups (probit model, χ 2 (4) = 10.8, p < 0.05). The frequencies of Music and Sports differed significantly between the groups: while Music was used by eight patients and only four controls, Sports was applied by only three patients, but seven controls (Music: T = 2.05, p = 0.045; Sports: T = −2.22, p = 0.03; see **Figure 3**). Likewise, the contrast Music—Sports yielded a significant group difference (T = 2.91, p = 0.005). In contrast, no differences emerged for Mention of others (seven patients, six controls) and Others (10 patients, nine controls; all p > 0.2).

In a stepwise regression analysis, we investigated which variable contributed to the ACC signals in an independent manner. Dorsal and rostral ACC signals were extracted from the two clusters in the group comparison, rendering the group effect trivial. Indeed, in the dorsal subdivision, the group factor ''diagnosis of schizophrenia'' contributed to the model (T = 4.62, p < 0.0001). The group factor was also the most important predictor for rostral ACC activity as well (T = −3.48, p < 0.001); however, the strategies which were coded as Mention of others added slightly but significantly to the explained variance (T = 2.11, p = 0.039). Importantly, none of the other strategies was included by the stepwise regression procedure (all p > 0.2); in particular Sports and Music had no effect on ACC after correction for diagnosis (all p > 0.2). Finally, also linear learning over the days of assessment seemed not to contribute with regards to additionally explained variance in the regression model (all p > 0.2). Only in an exploratory repeated-measures ANOVA, regulation amplitude in the rACC changed significantly over days (F(2,40) = 3.64, p = 0.035); in the dACC, only a trend emerged (F(2,40) = 2.73, p = 0.078).

# Discussion

fMRI NF training of ACC activity in patients with schizophrenia led to activation of the dorsal ACC subsection, whereas controls activated the rostral subsection. In addition, different cognitive strategies were reported, i.e., related to music in patients with schizophrenia and to sports in healthy controls. The difference in strategies, however, did not contribute to the difference in neural activation. One can assume that with regards to the human body one choses pragmatically the most readily available strategy to increase reward intensity or probability, i.e., the smile on the avatar face. This may circumvent addressing the desired neural target, for instance, the rostral ACC in the group of patients with schizophrenia since it is easier for them to achieve control via signal changes in the dorsal ACC. Therefore, targets in the neural and cognitive domain need to be specific for normalizing dysfunctions in patients with schizophrenia.

The ACC can be divided into subdivisions in different ways, such as anatomically as well as functionally (Bush et al., 2000). In particular, differentiation between the rostral and the dorsal part of the ACC is of significant importance. If one of these parts is highly recruited, the other one tends to be suppressed with regards to related brain activity (Drevets and Raichle, 1998). The rostral ACC is mainly activated within emotional processes, whereas the dorsal part largely engages in cognitive tasks (see Bush et al., 2000, for a review). In our study, patients and controls performed identical tasks but activated different sub-regions of the ACC. Indeed, various studies have suggested that ACC dysfunction in patients with schizophrenia is primarily caused by impairments of the rostral subdivision of the ACC (Carter et al., 1997; Laurens et al., 2003; Liddle et al., 2006; Habel et al., 2010). Therefore, in order to achieve control over the global ACC signal, patients may have activated the dorsal part in particular. As compared to controls, in the schizophrenia group activity of the rostral part was even down regulated during feedback blocks. This finding is in line with other models suggesting an antagonistic action of both ACC subdivisions (Drevets and Raichle, 1998) but may counteract the intention of a global and thus also rostral activation increase in a therapeutic setting. This dysbalance may additionally contribute to the psychopathology of patients with schizophrenia. Thus, in particular in patients with schizophrenia, NF training with the aim to increase activity in the rostral ACC should consider to apply specific masks that do not cover the dorsal ACC.

Similar to the found neural strategies for achieving regulatory ACC control, the groups reported different cognitive strategies. The higher rate of music in the patients and sports in the controls, however, did not explain the neural pattern implying higher cognitive processing in patients with schizophrenia as compared to emotional processing in the healthy controls. Indeed, these particular categories predicted neither cognitive nor emotional ACC activity. Conceivably, deficits in motor activity and thus motor imagery may have been compensated by the patients using everyday experiences such as music (compare Zvyagintsev et al., 2013). The patients in our group of patients with schizophrenia were able to achieve a functional recovery in terms of effective regulation of the ACC signal by compensation from domains which seem to be less impaired (cognitive processing, music imagery). They did not apply the neural and cognitive strategies as expected from the healthy control group.

# Social Feedback

In contrast to most previous fMRI-based NF studies, we employed a feedback with direct social reward (Mathiak et al., 2015). Previously, the satisfaction of completing the task served as a reward in the operant conditioning procedure (Sitaram et al., 2007). A social context was found to enhance the reward value in NF (Goebel et al., 2004). The direct reward from the smiling face may reduce the influence of context, making the reward value more comparable in the group of patients with schizophrenia. Admittedly, emotion recognition deficits have been reported in patients with schizophrenia but hardly extend to smiling facial expressions (Kohler et al., 2003). Unimpaired recognition of happy emotional expression has been even documented in avatar faces; the unambiguous smile is expected as rewarding for the patients as for the controls (Dyck et al., 2010). In summary, direct reward using facial expressions may enhance the conditioning procedure and, thus, the long-term transfer improving therapeutic benefits in patients (compare Rathod and Turkington, 2005).

# Therapeutic Implications

Cognitive deficits are directly associated with impaired social functioning in patients with schizophrenia (Addington and Addington, 1999; Dickerson et al., 1999), and their improvement is of particular relevance for the course of the disorder. Even though pharmacotherapy is the major cornerstone with regard to the management of positive symptoms in patients with acute schizophrenia, it does not improve residual cognitive impairments (Keefe et al., 1999). Cognitive behavioral therapy was found to be effective in patients with schizophrenia (Pilling et al., 2002b; Rector and Beck, 2012), even as performed within brief interventions (Turkington et al., 2002) as well as in long-term follow-ups (Sensky et al., 2000; Gould et al., 2001). These approaches, however, aimed at positive symptoms, depression, and overall symptoms only (Turkington et al., 2004). Cognitive remediation therapy targets circumscribed cognitive functions and thus should improve the daily functioning of persons suffering from schizophrenia (Medalia and Lim, 2004), but its effectiveness is still unclear [see reviews in Pilling et al., 2002a; and the (National Institute for Clinical Excellence, 2002)]. Indeed, cognitive training may lead to the application of compensatory strategies only but may not enhance the specifically targeted processes. fMRI-based NF of localized brain activity creates the possibility to directly influence specific neural networks and simultaneously control for the achieved effects.

# Limitations

All of the patients were on a stable dose of antipsychotics (seven with a single pharmacological agent, and four with a combination of two mostly atypical substances, see **Table 1**). These drugs may block dopaminergic pathways as well as inhibit the ACC (Holcomb et al., 1996; Miller et al., 1997), which is the area with the highest dopaminergic innervations in the cerebral cortex of primates (Paus, 2001). Nevertheless, normalization of ACC activity was observed over middle- (4 weeks; Snitz et al., 2005) and long-term (more than 6 months) treatments with atypical antipsychotics (Braus et al., 2002). Furthermore, responses to social reward were unaltered after a single dose of an atypical antipsychotic (Klasen et al., 2013). Finally, cognitive therapy usually addressed patients on antipsychotic medications, and therefore this naturalistic sample may be more relevant to study than a group of unmedicated patients. In addition, the present sample is probably closer to the reality faced by clinicians in charge of treatment of patients with schizophrenia.

The content coding scheme used here was not only developed based on previous experiences, but also on the explicitly reported strategies from the current group. However, during this process the subject information was removed from the reports. Thus, the group differences may not be explained by an observer bias for the content code. Indeed, the choice of strategies may be influenced by the sample strategies from the standardized instruction during the introduction. However, the high contributions of the social (''Mention of others'') and the rest categories as well as reports of passive music imagery indicate that the development of independent strategies was possible to the participants. Future studies may attempt to make NF-naïve subjects learning ACC control without examples for explicit cognitive strategies. However, our informal experience was that in such a setting it may take a long time for some subjects to learn regulatory control, and a relevant proportion may actually never achieve to control their respective ACC activity. Therefore, we decided to facilitate the operant condition by providing a priori cognitive models.

As concerns the cognitive processes and neural activation pattern during regulation, the current study did not address the allover pattern of brain activations. Thereby, it remains elusive whether the observed activation is causal for the ACC regulation or merely epiphenomena. The limited volume coverage enabled fast acquisition with a TR of 1 s However, motor areas are not fully covered rendering connectivity analyses on this dataset. Dysconnectivity can be considered core pathology in schizophrenia (Friston, 2005) and therefore the effect on connectivity self-regulation may be an important research question for future studies.

The number of patients in our NF study group is low, so that the present sample possibly underlies a selection bias, and no long-term effects have been investigated. fMRI-based NF is a demanding technique. We pointed out to the participants that the regulation was voluntary. However, we cannot exclude that particular patients suffering from schizophrenia may be suspicious with respect to techniques intending to modulate brain activity, in particular with regard to some possible paranoid symptoms and features. Furthermore, to remain attentive to the paradigm over three consecutive runs on three different days and to adhere to the experimental protocol requires a certain degree of cognitive resources, which can be compromised in acutely ill patients when compared to healthy controls. Within the restrictions of such a pilot study, the clear dichotomy between neural and cognitive strategies suggests the unmasking of fundamentally impaired processes in these patients with schizophrenia.

In contrast to some previous studies (e.g., Yoo et al., 2006), the current study did not address the question whether subject learn voluntary regulation of the ACC. Indeed, a control group without effective NF or a good enough control condition would be required to address this question. However, this would render the study significantly more ambitious with the need for twice as many patients. Further, we do not directly suggest a therapeutic application of this technique. However, NF can be expected to address specific dysfunctions and symptoms rather than broad disease categories, e.g., auditory verbal hallucinations (cf. McCarthy-Jones, 2012). Therefore, such clusters should be selected and individual neural networks addressed that can be expected to underlie or to influence the symptomatology. Nevertheless, schizophrenia is characterized by complex and individual profiles of neural dysfunctions in addition to neuroadaptive and pharmacological effects (Gaebler et al., 2015). We suggest performing detailed studies on NF learning and potential target networks in well-characterized individuals may be an important step to gather sufficient information before conducting targeted randomized controlled trials (RCTs) with sufficient likelihood to show relevant clinical effects.

# Conclusion

In the present study, patients with schizophrenia learned to regulate the activity of their ACC in response to social NF.

# References


NF may target neural dysregulation underlying some cognitive impairments in patients with schizophrenia. The observation of divergent neurophysiology and cognition during social NF in this particular disorder emphasizes the need for orientating therapeutic interventions to specific impaired functions in patients with schizophrenia.

# Author Contributions


All authors read and approved the final version of the manuscript. All authors agree to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

# Acknowledgments

This study has been supported by the German Research Foundation (Deutsche Forschungsgemeinschaft, DFG; IRTG 1328, MA 2631/6-1), the Federal Ministry of Education and Research (APIC: 01EE1405B) and Faculty of Medicine, RWTH Aachen (START program and a habilitation scholarship to KAM). We gratefully acknowledge the participation of all our volunteers.

in chronic schizophrenia. Eur. Neuropsychopharmacol. 12, 145–152. doi: 10. 1016/s0924-977x(02)00003-2


in patients with schizophrenia: an event-related fMRI study. Am. J. Psychiatry 158, 1423–1428. doi: 10.1176/appi.ajp.158.9.1423


**Conflict of Interest Statement**: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2015 Cordes, Mathiak, Dyck, Alawi, Gaber, Zepf, Klasen, Zvyagintsev, Gur and Mathiak. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution and reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# A controlled study on the cognitive effect of alpha neurofeedback training in patients with major depressive disorder

# *Carlos Escolano1,2\*, Mayte Navarro-Gil 3, Javier Garcia-Campayo4,5, Marco Congedo6, Dirk De Ridder <sup>7</sup> and Javier Minguez 1,3*

*<sup>1</sup> Department of Robotics, Perception and Real Time Group, Aragon Institute of Engineering Research (I3A), University of Zaragoza, Zaragoza, Spain*

*<sup>2</sup> Department of Computer Science and Systems Engineering (DIIS), University of Zaragoza, Zaragoza, Spain*

*<sup>3</sup> BitBrain Technologies SL, Zaragoza, Spain*

*<sup>4</sup> Department of Mental Health in Primary Care, Aragon Health Sciences Institute (IACS), University of Zaragoza, Zaragoza, Spain*

*<sup>5</sup> Psychiatric Service in the Miguel Servet University Hospital, University of Zaragoza, Zaragoza, Spain*

*<sup>6</sup> GIPSA-Lab, Département Images et Signal, CNRS, University of Grenoble, Grenoble, France*

*<sup>7</sup> Department of Surgical Sciences, Dunedin School of Medicine, University of Otago, North Dunedin, New Zealand*

#### *Edited by:*

*Sergio Ruiz, Pontificia Universidad Catolica de Chile, Chile*

#### *Reviewed by:*

*Romain Grandchamp, École Normale Supérieure, France Ute Strehl, University of Tuebingen, Germany*

#### *\*Correspondence:*

*Carlos Escolano, Department of Robotics, Perception and Real Time Group, Aragon Institute of Engineering Research (I3A), University of Zaragoza, I*+*D*+*i Building, Mariano Esquillor, 50018 Zaragoza, Spain e-mail: cescolan@unizar.es*

Cognitive deficits are core symptoms of depression. This study aims to investigate whether neurofeedback (NF) training can improve working memory (WM) performance in patients with major depressive disorder (MDD). The NF group (*n* = 40) underwent eight NF sessions and was compared to a non-interventional control group (*n* = 20). The NF protocol aimed to increase the individual upper alpha power in the parieto-occipital area of the scalp. Main cognitive variable was WM, which was measured pre- and post- training along with other variables such as attention and executive functions. EEG was recorded in both eyes closed resting state and eyes open task-related activity, pre- and post- NF training, and pre- and post- the NF trials within each session. A power EEG analysis and an alpha asymmetry analysis were conducted at the sensor level. Frequency domain standardized low resolution tomography (sLORETA) was used to assess the effect at brain source level. Correlation analysis between the clinical/cognitive and EEG measurements was conducted at both the sensor and brain source level. The NF group showed increased performance as well as improved processing speed in a WM test after the training. The NF group showed pre-post enhancement in the upper alpha power after the training, better visible in task-related activity as compared to resting state. A current density increase appeared in the alpha band (8–12 Hz) for the NF group, localized in the subgenual anterior cingulate cortex (sgACC, BA 25). A positive correlation was found for the NF group between the improvement in processing speed and the increase of beta power at both the sensor and brain source level. These results show the effectiveness of this NF protocol in improving WM performance in patients with MDD.

**Keywords: major depressive disorder (MDD), neurofeedback (NF), EEG, cognitive enhancement, working memory (WM), upper alpha (UA)**

# **1. INTRODUCTION**

Major depressive disorder (MDD) is a severe, chronic, mood disorder characterized by episodes of sadness, loss of interest and motivation, pessimism, and suicidal thoughts (DeRubeis et al., 2008). Depression affects each year 13–14 million citizens in the United States, with a lifetime prevalence of 16.2% (Kessler et al., 2003). MDD is estimated to become the second cause of burden of disease in 2030 (Mathers and Loncar, 2006). The current standard treatment for depression is antidepressant medication. Unfortunately with this treatment about 33% of patients fail to achieve remission (Anderson et al., 2008). Moreover, the patient compliance to the treatment can decrease due to the side effects such as sexual dysfunction, gastrointestinal problems, and weight gain (Brunoni et al., 2009). For this reason, the development of new treatments is constantly explored. In particular, recently there has been a renewed interest on neuromodulation techniques such as repetitive transcranial magnetic stimulation, transcranial direct current stimulation, and neurofeedback (DeRubeis et al., 2008; Brunoni et al., 2009).

The aim of neurofeedback (NF) is to allow the subjects to selfregulate brain activity. NF consists in measuring the brain activity and providing the subjects with real-time feedback covarying with the brain patterns of interest. Thus, the subjects may acquire a certain degree of awareness of the underlying brain processes and learn to regulate them (Congedo et al., 2004). Most of the NF protocols applied to date to depressive patients are based on EEG findings of frontal asymmetry in the alpha frequency band, with depressive patients showing left hypoactivation (Henriques and Davidson, 1991; Davidson, 1998, 2004; Coan and Allen, 2004). These findings are interpreted as a dysfunction of the prefrontal cortex (PFC) and a predisposition toward negative emotions and behavioral withdrawal (Davidson, 2004; DeRubeis et al., 2008). While some studies support this theory (Gotlib, 1998; Lubar et al., 2003), controversial results have also been reported (Reid et al., 1998). Furthermore, some studies found asymmetry changes over recording sessions not related to the clinical symptoms of the patients (Allen et al., 2004b), thus its consideration as a marker for depression remains unclear to date (Allen and Cohen, 2010). Some NF studies have tried to reduce the alpha asymmetry in an attempt to alleviate the depressive symptoms, reporting promising results (Baehr et al., 1997, 2001; Hammond, 2000, 2005). However, to the best of our knowledge, none of the early studies were appropriately controlled. The first controlled study of this kind reported a reduction in depressive symptoms and an improvement in executive functions (Choi et al., 2011).

The present work is based on a different rationale. Our objective is to alleviate the cognitive symptoms of depression. Cognitive deficits are core symptoms of depressive disorders with a clear-cut impact on social and occupational functioning, with patients showing decreased performance in working memory (WM) and attention, among others (Austin et al., 2001; Castaneda et al., 2008; Gotlib and Joormann, 2010). Furthermore, depressive patients show biases in the processing of emotional contents in WM (Gotlib and Joormann, 2010; Levens and Gotlib, 2010). For example, Levens and Gotlib (2010) performed an emotion n-back WM task in which depressive and control individuals had to match the valence of stimuli (happy, neutral, or sad). Comparing the depressive vs. control individuals they found in the 2-back task that depressive individuals were slower to match emotional stimuli in WM (regardless of the valence), and they integrated faster and removed slower the sad stimuli from WM. Recent evidences thus suggest that cognitive deficits may not only be correlates of depression, but they may also increase the risk for depression (Gotlib and Joormann, 2010; Levens and Gotlib, 2010).

We hereby explore WM entrainment by means of a NF protocol aimed to up-regulate the alpha power in posterior locations of the scalp. A large body of research has highlighted the relation between alpha oscillations and WM performance through inhibitory mechanisms (Klimesch et al., 2007; Freunberger et al., 2011). Recent evidences suggest that the inhibition of irrelevant information (filtering efficiency to WM contents) is a key factor in WM performance (Vogel et al., 2005; McNab and Klingberg, 2007), whereas the neuronal substrates of inhibitory mechanisms is hypothetized to be related to alpha oscillations (Klimesch et al., 2007; Sauseng et al., 2009; Freunberger et al., 2011). For example, Sauseng et al. (2009) performed a visuospatial WM task in which a memory array displayed either on the right or left visual hemifield had to be retained. During the retention interval of the task, repetitive transcranial magnetic stimulation (rTMS) was delivered (at 10 Hz to increase alpha amplitude) in either contralateral or ipsilateral parietal sites to the items to be retained. Increased performance was found when rTMS was applied ipsilaterally to the items to be retained, suggesting that enhanced alpha oscillations effectively suppressed irrelevant information. In this line, some NF studies have reported higher cognitive performance in healthy users after regulating the alpha oscillations, specifically measured in WM (Escolano et al., 2011; Nan et al., 2012), visuospatial rotation (Hanslmayr et al., 2005; Zoefel et al., 2011), and procedural learning (Ros et al., 2014). The reader is directed to Gruzelier (2013) for a review on NF studies on cognitive enhancement.

To the best of our knowledge, this is the first NF study exploring the cognitive effect of WM entrainment in patients with MDD. Some preliminary results of the current study have been published in Escolano et al. (2013). The trained parameter was the power in the individual upper alpha band, since the upper part of alpha has been suggested to respond to cognitive demands (Klimesch, 1999) and is the direction followed by several alphabased NF studies (Hanslmayr et al., 2005; Escolano et al., 2011; Zoefel et al., 2011; Nan et al., 2012).

# **2. MATERIALS AND METHODS**

#### **2.1. PARTICIPANTS**

Seventy-four participants diagnosed with major depressive disorder (MDD) were allocated to the NF group (*n* = 50) or to the control group (*n* = 24). Participants were not randomly allocated to groups (see Section 4). Patients were recruited from different health centers in the city of Zaragoza (Spain). The inclusion criteria were age range (18–65 years), Spanish as the native language, diagnosis of MDD according to DSM-IV, and stable pharmacological/psychological treatment. The exclusion criteria were diagnosis of comorbid disorders (e.g., schizophrenia, drug addiction, dementia). The experimental design was approved by the Ethical Review Board of the regional health authority and followed the Declaration of Helsinki. All participants signed an informed consent. Five participants dropped out the study due to the inability to perform the cognitive assessments or recording sessions, four of whom belonging to the NF group. Among the remaining participants who completed the study, six subjects in the NF group and three in the control group were excluded due to excessive artifacts in the EEG. Thus, the final sample consisted of 40 participants in the NF group and 20 in the control group. Demographic and clinical variables for both groups are reported in **Table 1**. There were no significant differences between groups in any variable.

# **2.2. EXPERIMENTAL DESIGN**

The design of the study is shown in **Figure 1**. The severity of depressive symptoms was evaluated in a semi-structured interview using the Beck Depression Inventory (BDI-II, Beck et al., 1996) and the Patient Health Questionnaire (PHQ-9, Kroenke et al., 2001). After that, both groups performed a cognitive assessment and an EEG screening at the beginning and at the end of the study (5-week time interval). Cognitive assessments lasted approximately 1 h. The NF group performed eight NF sessions over 4 weeks (two sessions per week). Each session was composed of five trials of 4 min each for a total of 20 min of training, and a pre- and post- EEG screening. For each EEG screening we recorded 3-min of eyes closed resting state activity and 3-min of eyes open task-related activity. In the latter, participants faced a computer screen showing a square that changed saturation color randomly from gray to red or blue (gradually). Participants were instructed to count the number of saturation changes from gray to red as a cognitive challenge (Zoefel et al., 2011). A power EEG analysis and an alpha asymmetry analysis were conducted at the sensor level. Frequency domain standardized low resolution tomography (sLORETA) was used to assess the effect of training at the brain source level. Finally, a correlation analysis between the clinical/cognitive and EEG measurements was conducted at both the sensor and brain source level.

### **2.3. COGNITIVE PERFORMANCE**

Pre- and post- cognitive assessments were carried out at the beginning and at the end of the study. The main outcome variable was WM, which was measured using the Paced Auditory Serial Addition Task (PASAT, Gronwall, 1977) along with the processing speed. This test is sensitive to minimal changes in neurocognitive performance and presents high levels of internal consistency and test-retest reliability (Tombaugh, 2006). The test scores were the number of errors and elapsed time. Episodic memory, attention, and executive functions were also assessed using the following tests: (i) Rey Auditory Verbal Learning Test (RAVLT, Rey, 1964)



*Between-group differences were assessed by Fisher's exact test and independent samples t-tests for categorical and continuous variables, respectively.*

evaluated episodic memory and the test score was the number of recognized words. (ii) Stroop Color-Word Test (STROOP, Stroop, 1992) evaluated attention and concentration. The test score was the interference, which was standardized across age groups. (iii) Trail Making Test (TMT, Reitan, 1958) evaluated executive functions and was composed of parts A and B. The scores were the elapsed time to complete each part of the test. (iv) Fluency Verbal Test (FAS, Benton and Hamsher, 1976) evaluated verbal phonetic fluency. The test score was the number of evoked words. To determine statistical significance, Two-Way repeated-measures ANOVA (rm-ANOVA) was separately conducted for each score with the between-subject factor Group (NF, Control) and the within-subject factor Time (Pre, Post). Paired samples *t*-tests were performed for within-group (pre vs. post) comparisons.

# **2.4. EEG RECORDING AND NEUROFEEDBACK PROCEDURE**

EEG data was recorded from 16 electrodes placed at FP1, FP2, F3, Fz, F4, C3, Cz, C4, P7, P3, Pz, P4, P8, O1, Oz, and O2 (subset of the 10/10 system), with the ground and reference electrodes on FPz and on the left earlobe, respectively. EEG was amplified and digitized using a g.tec amplifier (Guger Technologies, Graz, Austria) at a sampling rate of 256 Hz, power-line notch-filtered at 50 Hz and (0.5–60) Hz band-pass filtered. EEG recording and the NF procedure were developed using software of *Bit & Brain Technologies, SL*.

The NF training focused on the increase of the individual upper alpha (UA) power averaged over parieto-occipital locations (P3, Pz, P4, O1, and O2, referred to as feedback electrodes). EEG power was calculated through a short-term fast Fourier transform (FFT) with 1 s hamming window, 30 ms of overlapping, and zero-padded to 1024 points (0.25 Hz resolution). For each session, the pre-NF EEG screening was recorded and then used to calibrate the training for each participant and session. In this calibration step, we automatically filtered out the blinking component from the task-related activity by Independent Component Analysis (ICA), using the FastICA algorithm (Hyvarinen, 1999). Furthermore, we removed the epochs with amplitude larger than 200μV at any electrode. The Individual Alpha Frequency (IAF) was computed for each electrode on the power spectra of the reconstructed EEG data as the frequency bin with the maximum power value in the extended (7–13 Hz) alpha range (Klimesch, 1999). Note that when no clear alpha peak was found, the IAF was computed on resting state instead. The UA band was thus

groups executed a pre- and post- cognitive assessment and EEG screening within a 5-week time interval. The NF group performed a total of eight NF

screening (6 min each) and five training trials (4 min each). The EEG screenings included eyes closed resting state and eyes open task-related activity.

defined as the (IAF, IAF+2) Hz interval (Klimesch, 1999). Finally, the baseline was computed in task-related activity as the mean UA power averaged across the feedback electrodes, and (5*th*–95*th*) percentiles established the lower and upper limits, respectively. After the calibration, the participants performed the training trials. During online training, EEG data was online filtered from blinking artifacts (through the aforementioned ICA filter) and a visual feedback was then displayed every 30 ms on a computer screen in the form of a square with changing saturation colors. A linear mapping was used to convert between the UA power and the square color. Power values above the baseline were displayed in a red color scale with increasing saturation. Similarly, power values below the baseline were displayed in a blue color scale. The color scales ranged from 0% saturation (baseline in gray color) to 100% saturation in both blue and red color scales set by the lower and upper limits, respectively. No additional information was displayed during training.

Participants in the NF group were instructed to turn the square into red color with the maximum possible saturation, and to maintain it as long as possible. They were not given any mental strategy nor they were aware of the EEG trained parameter. Instead, they were encouraged to try different mental strategies guided by the feedback. To the best of our knowledge, the effect of different mental strategies in the ability to self-regulate brain activity is unclear to date, which is the focus of recent NF studies (Kober et al., 2013).

### **2.5. OFFLINE EEG PRE-PROCESSING**

EEG data from the initial and final EEG screenings was carefully inspected for the presence of artifacts such as eye blinks, eye movements, body movements, and electrocardiogram artifacts. Initially, the extended infomax ICA (Lee et al., 1999) was applied to the task-related activity to remove the eye blinking component. Then, both resting state and task-related activity were imported into EureKa! software (Congedo, 2002) to reject the contaminated data by visual inspection. Participants with at least 30 s of artifact-free data were included in the analysis. EEG spectrum was computed following the same procedure as in the NF procedure. The EEG data of each NF session was cleaned from artifacts using a three-step automatic procedure: filtering of the blinking component by the extended infomax ICA (Lee et al., 1999), epoch rejection by a time-domain threshold (>200μV) at any electrode, and epoch rejection by a frequency-domain threshold. In the latter step, we computed the power values for each epoch in the bands (1–3 Hz) and (20–30 Hz), commonly affected by ocular and muscular artifacts (Delorme et al., 2007). Then, we converted the log-transformed power values to *z*-scores and removed the outliers (>2.5) at any electrode. Note that the automatic procedure was only used to compute the alpha power in parieto-occipital locations, and no significant differences were found between the two procedures in either resting state [paired samples *t*-test: *t*(59) = −0.77, *p* = 0.44] or task-related activity [*t*(59) = 0.26, *p* = 0.79].

#### **2.6. POWER EEG ANALYSIS IN THE TRAINED PARAMETER**

Power EEG analysis was conducted in the trained parameter: power in the individual UA band, averaged across the feedback electrodes (P3, Pz, P4, O1, O2). The pre-post enhancement was measured as the power change between the initial and final EEG screenings in resting state and task-related activity for both groups. The across- and within-session enhancement were also measured (for the NF group). Across-session enhancement was assessed by a linear trend analysis of the power values in the pre-NF screenings of all sessions and the final screening. The withinsession enhancement comprised two measurements, computed in the power values averaged across the NF sessions: a power change comparison between the pre- and post- EEG screenings, and a linear trend analysis of the power values in the pre-NF task-related EEG screening and the five training trials. To determine statistical significance of pre- and post- comparisons, log-transformed power values were entered into a Two-Way rm-ANOVA with the factors Group (NF, Control) and Time (Pre, Post). Paired samples *t*-tests were performed for within-group (pre vs. post) comparisons. Trend analysis consisted in the computation of the slope of a fitted regression line for each participant, and a *t*-test to test the hypothesis of a null slope. The type I error was set at α = 0.05.

#### **2.7. POWER EEG ANALYSIS IN THE SENSOR X FREQUENCY DOMAIN**

Power analysis was conducted for all sensors and frequencies in the (1–30 Hz) interval, separately applied for the resting state and task-related activity. The NF training effects were measured as the log-transformed power spectra comparison between the initial and final EEG screening: a between-group comparison (NF vs. control group) on the change power values, and a within-group comparison (pre vs. post) for each group. A cluster-based non-parametric randomization method (Nichols and Holmes, 2002; Maris and Oostenveld, 2007) was used, as implemented in the Fieldtrip toolbox (FC Donders Centre for Cognitive Neuroimaging, Nijmegen, The Netherlands; see http:// www.ru.nl/fcdonders/fieldtrip). This method first computes the difference between two conditions by performing *t*-tests in the (sensor, frequency)-pairs. Those pairs exceeding a threshold *q* are clustered on the basis of spatial and spectral adjacency, and then cluster-level statistics are calculated as the sum of the *t*-values within every cluster. The threshold was set to (*q* = 0.05) for resting state and to (*q* = 0.01) for task-related activity. Finally, the significance probability at the cluster-level was estimated by a permutation method (Pesarin, 2001). The distribution of the cluster values was constructed under the null hypothesis by 5000 random permutations and then the observed values were tested against the (1 − α)th percentile of the null distribution. This method controls for the type I error rate and corrects for multiple comparisons both across sensors and frequencies. The type I error at cluster-level was set to α = 0.05.

#### **2.8. ALPHA ASYMMETRY ANALYSIS**

Initial and final scores of alpha (8–12 Hz) asymmetry were computed in resting state and task-related activity. EEG data was re-referenced to Cz and asymmetry scores were computed as the normalized power difference between homologous right- and left-side locations, (R−L)/(R+L). See Allen et al. (2004a) for a review on methodological considerations. This score indicates the relative activation of the left over right locations. Thus positive scores indicate left-lateralized activation, i.e., more power over right-side locations due to the inverse relation between alpha power and brain activation (Coan and Allen, 2004). Alpha asymmetry scores were computed in five areas of the scalp: prefrontal (FP: FP2-FP1), frontal (F: F4-F3), central (C: C4-C3), parietal (P: (P7+P3)/2-(P8+P4)/2), and occipital (O: O2-O1). Independent samples *t*-tests were conducted to test for between-group differences in initial scores. We applied *t*-tests on the initial scores (for each group and area of the scalp) to test for null asymmetry scores. Two-Way rm-ANOVA with the factors Group (NF, Control) and Time (Pre, Post) was separately conducted for each area of the scalp to test for pre-post study changes. Bonferroni correction was applied to correct for multiple areas so as to keep the FWER at α = 0.05.

#### **2.9. EEG ANALYSIS AT THE BRAIN SOURCE LEVEL**

Frequency domain standardized low resolution tomography, sLORETA (Pascual-Marqui, 2002, 2007) was used to estimate the current density of brain sources in resting state and taskrelated activity. The current density changes were compared between groups (NF vs. control group) as well as within groups (pre vs. post) for each group. EEG data was re-referenced to a common average reference and Fourier cross-spectral matrices were computed for the following frequency bands: delta (1– 4 Hz), theta (4.5–7 Hz), alpha (8–12 Hz), beta1 (12–15 Hz), beta2 (15–20 Hz), and beta3 (20–30 Hz). These bands were defined according to the results obtained in the clustering analysis at the sensor level (Section 2.7). After that, sLORETA estimated the current density values in 6239 voxels (5 mm3 spatial resolution). sLORETA applies the boundary element method on the MNI-152 (Montreal Neurological Institute, Canada) template of Mazziotta et al. (2001), and the anatomical labeling is based on probabilities returned by the Daemon Atlas (Lancaster et al., 2000). Finally, current density values were log-transformed and statistical significance of each voxel was determined by a nonparametric randomization procedure using the *t*-max statistic to control the familywise type I error rate (FWER, Holmes et al., 1996). Following this procedure, the null distribution was estimated by 5000 random permutations under the null hypothesis of the maximum absolute *t*-value across all voxels. Then the absolute observed *t*-value for each voxel was tested against the (1 − α)th percentile of the null distribution. Bonferroni correction was applied to correct for multiple bands so as to keep the FWER at α = 0.05.

#### **2.10. CORRELATION ANALYSIS: EEG vs. BEHAVIORAL VARIABLES**

Spearman correlation was employed to test for a correlation in the initial scores between the clinical/cognitive and EEG variables, as well as in the change scores between the cognitive and EEG variables (clinical variables were not measured after NF training). A total of two clinical variables (Section 2.2) and seven cognitive variables (Section 2.3) were tested. The EEG variables were assessed in resting state and task-related activity and can be divided into two groups: power variables at the sensor level, and current density variables at the brain source level. In both cases, the analysis was conducted in the aforementioned frequency bands (delta, theta, alpha, beta1, beta2, beta3). In the case of the sensor level, power values in each band were averaged across five areas: prefrontal (FP: FP1, FP2), frontal (F: F3, Fz, F4), central (C: C3, Cz, C4), parietal (P: P7, P3, Pz, P4, P8), and occipital (O: O1, Oz, O2). In the case of the brain source level, a randomization procedure (similar to the one used in previous Section 2.9) was used to control the FWER (Holmes et al., 1996). Five-thousand random permutations were performed to construct a distribution of the maximum of the absolute *r*-value across all voxels under the null hypothesis. In both cases the Bonferroni correction was applied to keep the FWER at α = 0.05.

# **3. RESULTS**

## **3.1. COGNITIVE PERFORMANCE**

**Table 2** summarizes the scores in the cognitive assessments. A significant *Group* × *Time* ANOVA interaction appeared in the PASAT test [# errors: *F*(1, 46) = 5.42, *p* = 0.024; time: *F*(1, 46) = 4.97, *p* = 0.031], showing an improvement in WM performance and processing speed for the NF group only [# errors: *t*(30) = −5.21, *p* < 0.001; time: *t*(30) = −4.91, *p* < 0.001]. Cohen's *d* effect size (Cohen, 1988) revealed a medium-large effect for both the number of errors (*d* = 0.703) and elapsed time (*d* = 0.673). Note that 9 participants in the NF group did not complete the PASAT test in the initial assessment due to excessive cognitive effort, four of whom completed the final assessment. Three participants in the control group did not complete the PASAT test in both initial and final assessments. No significant ANOVA interaction appeared in the other variables. Regarding the within-group (pre vs. post) changes, the number of recognized words increased in the RAVLT test for the NF group [*t*(39) = 3.28, *p* < 0.005]. Interference score in the STROOP test was not significantly modified for any group. Both parts of the TMT test improved for the NF group [part A: *t*(37) = −1.95, *p* = 0.059; part B: *t*(37) = −2.85, *p* < 0.01]. The NF group also improved the number of evoked words in the FAS test [*t*(38) = 2.61, *p* < 0.05]. No significant changes were found for the control group.

### **3.2. POWER EEG ANALYSIS IN THE TRAINED PARAMETER**

The analysis of the pre-post enhancement in the trained parameter (power in the individual UA band averaged across the feedback electrodes: P3, Pz, P4, O1, O2) revealed a no significant *Group* × *Time* ANOVA interaction for the resting state activity. However, a statistical trend between the pre and post measurements was found in the NF group [*t*(39) = −1.72, *p* = 0.093], with an average increase of 22%. Regarding the task-related activity, a significant *Group* × *Time* interaction appeared [*F*(1, 58) = 14.88; *p* < 0.001]. *Post-hoc t*-tests showed a significant pre vs. post difference for the NF group only [*t*(39) = −5.44, *p* < 0.001], with an average increase of 56%. No significant change was found for the control group in either resting state or task-related activity. Note that groups did not differ statistically in initial IAF. Mean ± SD IAF measured in resting state activity was 9.81 ± 0.17 Hz for the NF group and 9.79 ± 0.25 Hz for the control group [*t*-test for independent samples, *t*(58) = 0.075, *p* = 0.94]; in task-related activity it was 9.71 ± 0.19 Hz for the NF group and 9.64 ± 0.24 Hz for the control group [*t*(58) = 0.21, *p* = 0.83]. Furthermore, IAF did not change significantly pre-post the study for either group. The across- and within- session enhancement was measured for the NF group (see **Figure 2**). Trend analysis



*Mean* ± *s.e.m. scores in pre and post cognitive assessments are shown, as well as the change scores (post-pre). The p-values of the paired samples t-tests are shown for each group (pre vs. post changes), as well as the p-values of the Group* × *Time interaction in the rm-ANOVAs. Significant effects are marked bold (p* < *0.05), statistical trends are underlined (p* < *0.1).*

revealed a significant UA power increase across the NF sessions in both resting state [*t*(39) = 2.56, *p* = 0.014] and task-related activity [*t*(39) = 4.04, *p* < 0.001]. Regarding the within-session enhancement, a significant power increase between the pre- and post- NF screenings appeared in resting state [*t*(39) = −3.10, *p* < 0.005], with an average increase of 15.1%; as well as in task-related activity [*t*(39) = −5.72, *p* < 0.001], with an average increase of 16.1%. Trend analysis revealed a significant power increase across the NF trials [*t*(39) = 7.81, *p* < 0.001].

#### **3.3. POWER EEG ANALYSIS IN THE SENSOR X FREQUENCY DOMAIN**

We have adjusted the frequency bands according to the results obtained in this analysis as follows: delta (1–4 Hz), theta (4.5–7 Hz), alpha (8–12 Hz), beta1 (12–15 Hz), beta2 (15–20 Hz), and beta3 (20–30 Hz). Regarding the power changes in the between-group (NF vs. control) comparison, a cluster appeared in the (5–9 Hz) frequency range for the resting state activity (*p* = 0.008), indicating a power increase for the NF group in theta, apparent in frontal and central locations, and lower part of alpha (8–10 Hz), in frontal, central and parietal locations. A cluster appeared in task-related activity covering the (5.5–12 Hz) range (*p* < 0.0001), indicating a power increase for the NF group in theta, apparent in frontal locations, and alpha, in frontal, central and parietal locations. **Figure 3** displays sensor x frequency maps of the power changes (pre vs. post NF training) for the NF group. Note that no significant clusters were found pre vs. post for the control group. Resting state activity showed a cluster in the (7–9.5 Hz) range at a trend level (*p* = 0.073), indicating a power increase in lower alpha in all the scalp areas. A cluster in the (4.5–20 Hz) range appeared for the task-related activity (*p* < 0.0001), indicating a power increase in theta, apparent in frontal locations, alpha and beta1 in all the scalp areas, and beta2, apparent in frontal, central, and parietal locations.

# **3.4. ALPHA ASYMMETRY ANALYSIS**

No significant differences were found between groups in the initial asymmetry scores. We applied *t*-tests on the initial scores (for each area of the scalp) to test for null asymmetry scores within each group. No significant results appeared after strict control of the type I error (see Section 4). Regarding the pre-post changes in asymmetry scores, no significant *Group* × *Time* ANOVA interaction was found in any area of the scalp, as well as no significant pre vs. post differences within each group.

# **3.5. EEG ANALYSIS AT THE BRAIN SOURCE LEVEL**

A significant effect was found for the NF group only (pre vs. post NF training) in task-related activity, measured in the alpha band (8–12 Hz), see **Figure 4**. Forty-two voxels showed a current density increase (threshold *t* = 3.37, α = 0.05). The current density

**FIGURE 3 | Sensor x frequency maps displaying the significant clusters (in the power spectra) for the NF group in the within-group (pre vs. post) comparison.** Left figure displays the resting state activity, and right figure the task-related activity. Significant clusters are shown at a given *q* threshold. X axis shows the frequency bins in the (1–30 Hz) frequency

range, whereas Y axis shows the sensor locations. Topoplots are displayed in the frequency bands with power changes above the *q* threshold (in significant clusters), and the involved sensors are marked with a cross. Color scale represent *t*-values, indicating a power increase after the NF training.

increase was localized in the subgenual anterior cingulate cortex, sgACC (BA 25; XYZ1= 0, 5, −5; *t* = 4.54), subcallosal gyrus (BA 34; XYZ = −10, 5, −15; *t* = 4.35), parahippocampal gyrus (BA 28; XYZ = −15, −5, −15; *t* = 4.13), anterior cingulate cortex, ACC (BA 32; XYZ = −5, 20, −10; *t* = 4.11), and rectal gyrus (BA 11; XYZ = −5, 15, −20; *t* = 4.02).

#### **3.6. CORRELATION ANALYSIS: EEG vs. BEHAVIORAL VARIABLES**

No significant correlation was found between the clinical and EEG variables at study entry. Significant correlations were found between the elapsed time variable of the PASAT test and EEG variables at both the sensor and brain source level for the NF group only, measured in task-related activity. Regarding the sensor level, a positive correlation appeared in the initial scores between the beta2 (15–20 Hz) power in parietal area (P: P7, P3, Pz, P4, P8) and the elapsed time [*r*(29) = 0.64, *p* = 0.029], i.e., higher beta2 power in parietal locations correlated with slower processing speed. The analysis of the pre vs. post change scores revealed negative correlations between the power increase in each beta sub-band in prefrontal area (FP: FP1, FP2) and the increment in elapsed time: beta1 [12–15 Hz, *r*(29) = −0.68, *p* = 0.007], beta2 [15–20 Hz, *r*(29) = −0.65, *p* = 0.019)], and beta3 [20–30 Hz, *r*(29) = −0.79, *p* < 0.0001]. Thus, the beta enhancement in prefrontal locations correlated with the improvement in processing speed. Regarding the brain source level, the analysis of the pre vs. post change scores revealed a negative correlation between the current density increase measured in the beta3 band (20–30 Hz) and the increment in elapsed time, see **Figure 5**. One hundred and thirty-three voxels were significant (α = 0.05), which were localized in the ACC (BA 32; XYZ = −5, 40, −10; *r* = −0.761), medial frontal gyrus (BA 11; XYZ = −5, 35, −15; *r* = −0.757), medial frontal gyrus (BA 10; XYZ = −10, 40, −10; *r* = −0.755), pregenual ACC, pgACC (BA 24; XYZ = −5, 30, −5; *r* = −0.747), and subgenual ACC, sgACC (BA 25; XYZ = −5, 25, −20; *r* = −0.741). Thus, the current density increase in the aforementioned regions correlated with the improvement in processing speed. No significant correlations were found for the control group.

# **4. DISCUSSION**

The objective of the current work was to explore whether the cognitive symptoms of patients with major depressive disorder (MDD) can be alleviated by EEG-based NF training. Depression is associated with cognitive deficits such as decreased working memory (WM) and attention, among others, which have a clear-cut impact on social and occupational functioning (Austin et al., 2001; Castaneda et al., 2008; Gotlib and Joormann, 2010). We hereby explored the application of a NF protocol based on upper alpha up-regulation to improve WM performance in patients with MDD. The rationale beyond this protocol consists in evidences showing the relation between alpha oscillations and WM performance through inhibitory mechanisms (Klimesch et al., 2007; Freunberger et al., 2011). This NF protocol has obtained promising results in healthy users (Escolano et al., 2011; Nan et al., 2012). Recent evidences suggest that the WM deficits in depression (biases toward negative emotions) may not only be correlates of depression but also increase the vulnerability and recurrence to depression (Gotlib and Joormann, 2010; Levens and Gotlib, 2010). Thus this NF protocol has potential to improve depressive symptoms.

#### **4.1. COGNITIVE PERFORMANCE**

WM performance and processing speed (PASAT test) were improved for the NF group in comparison with the control group, showing medium-large effect sizes. While positive findings had been already reported in healthy users (Escolano et al., 2011; Nan et al., 2012) this paper supports the effectiveness of such a protocol in improving WM performance in patients with MDD. Episodic memory, executive functions, and verbal fluency were also improved for the NF group only as revealed by the RAVLT, TMT, and FAS tests. However the improvement in these cognitive functions was not significantly superior to the improvement observed in the control group. These results suggest that the stronger effect of the NF training is specifically found in WM performance and processing speed, whereas the improvement in the

**FIGURE 5 | Correlation analysis for the NF group at the brain source level between the pre vs. post increase in the elapsed time variable of the PASAT test and the increase in current density, measured in beta3 band (20–30 Hz) in task-related activity.** Axial (left), sagittal (middle), and coronal (right) sections of sLORETA are displayed through

the voxel with maximal absolute *r*-value. Color scale represent *r*-values, with negative values indicating a positive correlation between the current density increase and the improvement in processing speed (note the inverse relation between the elapsed time in the PASAT test and processing speed).

<sup>1</sup>MNI (Montreal Neurological Institute, Canada) coordinates are used throughout this paper.

latter cognitive functions is marginal and may be explained by an enhancement of cognitive processing as a whole.

#### **4.2. EEG ANALYSIS**

A pre-post enhancement was found in the trained parameter: power in the individual upper alpha band averaged across parietooccipital locations. The NF group showed an average increase of 56% in task-related activity after the NF training. Resting state activity was also increased for the NF group, with an average increase of 22%, but failed to reach statistical significance in comparison to the control group. No significant changes were found for the control group. In addition to that, upper alpha power in both resting state and task-related activity showed an increase with the number of sessions, as well as an increase (pre vs. post) within each session. Bruder et al. (2008) showed alpha power differences between responders and non-responders to selective serotonin reuptake inhibitor (SSRI), with responders showing greater alpha power in resting state at study entry, specifically measured at occipital locations. While these findings show the potential utility of the present NF protocol in improving the responsiveness to antidepressant medication, it should be confirmed in future studies.

Some NF studies assess the effects of the training not only in the trained parameter, but typically in a small number of predetermined frequency bands (Escolano et al., 2011; Zoefel et al., 2011; Nan et al., 2012). Here we extended that analysis to assess the power changes in all sensors in the (1–30 Hz) frequency range by a clustering analysis, obtaining sensor x frequency maps of the power changes. We believe that the present analysis can offer a clearer insight of the electrophysiological effects. This analysis was separately applied for the two recording conditions: resting state and task-related activity. Since stronger effects were found in task-related activity in comparison to resting state, we adapted the *q* threshold to each recording condition to get clearer sensor x frequency maps. Please note that *q* threshold does not determine the type I error at cluster-level, which was set to (α = 0.05) for both conditions. Significant clusters were found for the NF group only, showing a power increase after the NF training. The strongest effect in resting state activity appeared in the lower part of alpha (8–10 Hz). Task-related activity showed stronger effects in the (4.5–20 Hz) range, covering theta, alpha, and the lower part of beta. These effects were apparent in anterior, central, and posterior locations. Our results show the common finding that the effects of the NF training on the spectral power are not spatially or spectrally restricted to the trained locations and frequency bands, instead the trainingengendersprofoundchangesof thehomeostaticproperties of the brain involving several locations and frequencies (Hughes and John, 1999). The strong effect in task-related activity illustrates the importance of recording EEG in several conditions to provide additional information of the underlying brain processes. This is in contrast to the common practice to study only the resting state, either eyes closed or eyes open.

Alpha asymmetry scores did not differ between groups at study entry and it did not change pre-post the study for any group. Note that this study can-not assess the commonly found frontal asymmetry in depression since it involves the comparison with a control group of non-depressive participants (Davidson, 2004). We investigated the initial alpha asymmetry scores for null scores, measured in five areas of the scalp (prefrontal, frontal, central, parietal, and occipital). Strong asymmetry scores were found although they failed to reach statistical significance after strict control of the type I error (Bonferroni correction). Here we summarize the significant results when not correcting for the multiple areas (see **Figure 6**). Significant scores were found in posterior areas (parietal and occipital). Left-lateralized activation (more alpha power over right locations) was found in resting state in the parietal area at a trend level for both the NF group [*t*(39) = 1.76, *p* = 0.086] and the control group [*t*(19) = 1.86, *p* = 0.077]. These results are in line with previous studies (Kemp et al., 2010). Interestingly, the opposite effect was found in task-related activity. Right-lateralized activation (more alpha power over left locations) was found in task-related activity in the occipital area at a trend level for the NF group [*t*(39) = −1.98, *p* = 0.054], significant for the control group [*t*(19) = −2.71, *p* = 0.014]. Please note that EEG data was recorded using a single earlobe reference, but re-referenced offline to Cz. The most common approach to measure brain asymmetry to date is the use of computer-averaged ears/mastoids reference, or Cz reference (Davidson, 1998; Coan and Allen, 2004; Allen and Cohen, 2010). Although there is no agreement about a preferred montage, reference placement might be a critical issue (Hagemann et al., 2001; Allen et al., 2004b; Davidson, 2004), with unilateral references being discouraged. Thus, the alpha asymmetry effects herein reported should be taken with caution and confirmed in future studies using one of the aforementioned recording approaches.

The analysis at the brain source level (using sLORETA) revealed significant changes in current density after the NF training for the NF group only. The stronger effect was found in task-related activity for the alpha band (8–12 Hz), localized in the subgenual ACC (sgACC, BA 25) extending to the entorhinal cortex (BAs 28, 34), ventromedial prefrontal cortex (BA 32) and orbitofrontal cortex (BA 11). Notice that we are controlling for the familywise type I error rate using a conservative method for correcting also for the multiple frequency bands (*t*-max permutation tests were further corrected using a Bonferroni correction for the multiple bands). Strong effects were also found

**FIGURE 6 | Initial alpha asymmetry scores measured in prefrontal (FP: FP2−FP1), frontal (F: F4−F3), central (C: C4−C3), parietal (P: (P7+P3)/2−(P8+P4)/2), and occipital (O: O2−O1) areas of the scalp.** Positive asymmetry scores denote left over right activation, i.e., more alpha power in right hemisphere locations. <sup>∗</sup> significant effect (*p* < 0.05), ± statistical trend (*p* < 0.1). Note that statistical results are not corrected for the multiple areas.

in other brain structures although they failed to reach statistical significance after such a strict control of the type I error. Here we summarize the significant results when not correcting for the multiple frequency bands, which appeared for the NF group only. Regarding the resting state activity, a current density increase appeared in the pregenual ACC, pgACC (BA 24; XYZ = −5, 35, 10; *t* = 3.25) measured in alpha band. Regarding the task-related activity, a current density increase appeared in the sgACC (BA 25; XYZ = 0, 5, −5; *t* = 3.03) measured in delta (1–4 Hz), in the pgACC (BA 24; XYZ = −5, 30, 0; *t* = 3.01) in theta (4.5–7 Hz), and in the pgACC (BA 24; XYZ = −5, 25, 15; *t* = 3.27) in beta1 (12–15 Hz).

## **4.3. CORRELATION ANALYSIS: EEG vs. BEHAVIORAL VARIABLES**

A strong correlation was found between the processing speed (elapsed time variable in the PASAT test) and the power in taskrelated activity measured in the beta frequency band. The power EEG analysis revealed that higher power in beta2 band (15–20 Hz) in parietal locations positively correlated with slower processing speed at study entry. We also found that the power increase in beta band (specifically in each one of the three beta sub-bands analyzed, ranging from 12 to 30 Hz) in prefrontal locations positively correlated with the improvement in processing speed (pre vs. post NF training). These results suggest that beta band is related to processing speed in different ways according to the area of the scalp involved. Furthermore, a strong correlation was found at the brain source level: the increase in current density measured in beta3 band (20–30 Hz) positively correlated with the improvement in processing speed (pre vs. post NF training). This correlation was localized in the pgACC (BA 24) and sgACC (BA 25) extending to the ventromedial prefrontal cortex (BA 32) and further into the orbitofrontal cortex (BA 11) and frontopolar cortex (BA 10).

Beta band activity has been traditionally linked to somatosensory and motor functions but its functional role is not well understood to date (see Engel and Fries, 2010, for a review). Regarding its perceptual-cognitive role, it has been recently associated with the active continuation of the current cognitive set in tasks involving a strong top-down component (Engel and Fries, 2010). According to this theory, the brain makes predictions in beta oscillations about what it will encounter in the internal and external environment, and updates this by measuring prediction errors, which are encoded by gamma oscillations (Arnal and Giraud, 2012). A large body of research has highlighted the relation of the ACC in error monitoring and prediction of errors (Rushworth and Behrens, 2008; Holroyd et al., 2009). The improvement in processing speed after NF training may be thus explained by this relation.

# **4.4. THE INVOLVEMENT OF THE ANTERIOR CINGULATE CORTEX**

The ACC is known to be linked to cognitive and emotional processes (Bush et al., 2000). Based on cytoarchitecture, lesion, and electrophysiological studies the ACC has been divided into two major functional sub-divisions: a cognitive and an affective sub-divisions (Bush et al., 2000). The cognitive division is localized in the dorsal ACC (areas 24 , 32 ), and the affective one is localized in the rostral-ventral ACC (rostral areas 24 and 32, and ventral areas 25 and 33). Functional differences in the ACC between depressive and non-depressive subjects have been repeatedly reported (Mayberg, 1997; Pizzagalli et al., 2001; Davidson et al., 2002). On one hand, decreased activation has been reported in the dorsal ACC (dorsal region of area 32; areas 24 , 32 ). On the other hand, increased pre-treatment activation has been found in responders (vs. non-responders) to antidepressant medication in the rostral and ventral ACC (including pregenual areas 24 and 32), and it has been suggested as a predictor of treatment response (Mayberg et al., 1999; Pizzagalli et al., 2001). Pizzagalli et al. (2001) compared depressive participants showing better response to nortriptyline treatment to responders showing worse response. Participants showing a better response had higher pretreatment theta (6.5–8 Hz) activity localized in the rostral ACC (BA 24, 32), estimated using sLORETA. In this line, the current study showed a current density increase in the rostral ACC (BA 24; XYZ = −5, 30, 0; *t* = 3.01) measured in the theta band (4.5– 7 Hz), suggesting that the present NF protocol may increase the response to antidepressant medication. Regarding the correlation between the cognitive and EEG variables, the pregenual ACC (BA 24) and subgenual ACC (BA 25) were positively correlated with the improvement in processing speed. Interestingly, the subgenual ACC has not been traditionally linked to cognitive processing. However, recent studies demonstrate that there is an overlap of autonomic, sensorimotor, affective, and cognitive processing in the midcingulate gyrus (Beissner et al., 2013) suggesting that the functional-anatomical parcellation of the cingulate gyrus is not as simple as has been assumed. The present study suggests that the subgenual ACC, apart from its well-known involvement in autonomic (Beissner et al., 2013) and emotional (Mayberg et al., 1999; Bush et al., 2000) regulation, it also is implicated in cognitive processing. Thus analogous to the midcingulate gyrus also the subgenual ACC may be involved in multiple overlapping functions.

# **4.5. LIMITATIONS**

Due to the novelty and the exploratory character of the study, the control group designed in the present study was not optimal. On one hand, the number of subjects in the NF and control group was not balanced. We decided to use an allocation ratio 2:1 between the NF and control group. On the other hand, participants were not randomly assigned to the experimental condition nor they were blinded with respect to the experimental condition. Nonetheless, the control group used in the present study can account for practice effects in the cognitive assessments and for electrophysiological changes before and after the study. Regarding the demographic characteristics, around 90% of the participants of the study followed a stable pharmacological antidepressant treatment during the study, 75% of whom consisted either on selective serotonin reuptake inhibitors (SSRI), benzodiazepines, or serotonin-norepinephrine reuptake inhibitors (SNRI). Also, 80% of the participants presented comorbidity with anxiety. Further research is needed to elucidate to what extent the obtained results can be translated to a drug-free population or to a population without anxiety symptoms. The averaged severity of depression was "moderate" for both the NF and control group according to the BDI-II. However, the averaged PHQ-9 scores were slightly lower for the NF group ("moderate") than for the control group ("moderately severe"). Nonetheless, there were no significant differences between groups in severity of depression as measured by either BDI-II or PHQ-9. The current study estimated the effects at brain source level using sLORETA (MNI-152 template). The spatial resolution and precision of sLORETA method could be improved by computing individual head models based on magnetic resonance imaging (MRI). A comparison of the cognitive performance with healthy subjects would have been an interesting analysis to estimate the significance of the observed effects. This analysis should be taken into account for future studies. Finally, although we have stated that the present NF protocol has the potential to improve depressive symptoms, such hypothesis can-not be corroborated in the present study since clinical variables were not assessed after NF training. Due to the positive cognitive effects, the present NF protocol should be evaluated in future studies using stricter control conditions such as active control conditions (e.g., psychotherapy) or sham feedback. However, a sham feedback control condition may lead to ethical concerns when effective standard treatments are available (La Vaque and Rossiter, 2001).

# **ACKNOWLEDGMENTS**

This research has been partially supported by Spanish Ministry of Science projects HYPER-CSD2009-00067 and DPI2009-14732- C02-01, DGA-FSE, grupo T04. Author Marco Congedo was partly supported by the European project ERC-2012-AdG-320684- CHESS.

# **REFERENCES**


therapeutic response to a selective serotonin reuptake inhibitor antidepressant: pre-and post-treatment findings. *Biol. Psychiatry* 63, 1171–1177. doi: 10.1016/j.biopsych.2007.10.009


Consortium for Brain Mapping (ICBM). *Philos. Trans. R. Soc. Lond. B Biol. Sci.* 356, 1293–1322. doi: 10.1098/rstb.2001.0915


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 14 April 2014; accepted: 13 August 2014; published online: 02 September 2014.*

*Citation: Escolano C, Navarro-Gil M, Garcia-Campayo J, Congedo M, De Ridder D and Minguez J (2014) A controlled study on the cognitive effect of alpha neurofeedback training in patients with major depressive disorder. Front. Behav. Neurosci. 8:296. doi: 10.3389/fnbeh.2014.00296*

*This article was submitted to the journal Frontiers in Behavioral Neuroscience.*

*Copyright © 2014 Escolano, Navarro-Gil, Garcia-Campayo, Congedo, De Ridder and Minguez. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Volitional control of the anterior insula in criminal psychopaths using real-time fMRI neurofeedback: a pilot study

#### **Ranganatha Sitaram1,2,3\*, Andrea Caria<sup>1</sup> , Ralf Veit <sup>1</sup> , Tilman Gaber <sup>1</sup> , Sergio Ruiz 1,4 and Niels Birbaumer 1,5**

1 Institute of Medical Psychology and Behavioral Neurobiology, University of Tübingen, Tübingen, Germany

<sup>2</sup> Department of Biomedical Engineering, University of Florida, Gainesville, FL, USA

<sup>3</sup> Sri Chitra Tirunal Institute of Medical Sciences and Technology, Thiruvananthapuram, Kerala, India

<sup>4</sup> Departamento de Psiquiatría, Escuela de Medicina, Centro Interdisciplinario de Neurociencias, Pontificia Universidad Católica de Chile, Santiago, Chile

<sup>5</sup> Ospedale San Camillo, Istituto di Ricovero e Cura a Carattere Scientifico (IRCCS), Venezia, Italy

#### **Edited by:**

Carmen Sandi, Ecole Polytechnique Federale De Lausanne, Switzerland

#### **Reviewed by:**

Maria Laura Blefari, Ecole Polytechnique Federale de Lausanne (EPFL), Switzerland Surjo R. Soekadar, University Hospital of Tübingen, Germany

#### **\*Correspondence:**

Ranganatha Sitaram, Department of Biomedical Engineering, University of Florida, P.O. Box 116131, Gainesville, FL 32611-6131, USA e-mail: ranganatha.sitaram@ bme.ufl.edu

This pilot study aimed to explore whether criminal psychopaths can learn volitional regulation of the left anterior insula with real-time fMRI neurofeedback. Our previous studies with healthy volunteers showed that learned control of the blood oxygenationlevel dependent (BOLD) signal was specific to the target region, and not a result of general arousal and global unspecific brain activation, and also that successful regulation modulates emotional responses, specifically to aversive picture stimuli but not neutral stimuli. In this pilot study, four criminal psychopaths were trained to regulate the anterior insula by employing negative emotional imageries taken from previous episodes in their lives, in conjunction with contingent feedback. Only one out of the four participants learned to increase the percent differential BOLD in the up-regulation condition across training runs. Subjects with higher Psychopathic Checklist-Revised (PCL:SV) scores were less able to increase the BOLD signal in the anterior insula than their lower PCL:SV counterparts. We investigated functional connectivity changes in the emotional network due to learned regulation of the successful participant, by employing multivariate Granger Causality Modeling (GCM). Learning to up-regulate the left anterior insula not only increased the number of connections (causal density) in the emotional network in the single successful participant but also increased the difference between the number of outgoing and incoming connections (causal flow) of the left insula. This pilot study shows modest potential for training psychopathic individuals to learn to control brain activity in the anterior insula.

**Keywords: real-time fMRI, brain-computer interface, neurofeedback, criminal psychopathy, antisocial behavior, anterior insula**

#### **INTRODUCTION**

Psychopathy is a personality disorder described by a constellation of affective, interpersonal and behavioral characteristics such as callousness, a marked lack of empathy, egocentricity and impulsivity. Psychopaths engage in more criminal behavior and institutional misconduct than their non-psychopathic counterparts. Central to psychopathy is the deficient processing of emotions. These include shallowness and profound lack of remorse or empathy. Hare et al. (1991) subsumed those features under the term "emotional detachment". Lykken (1957), using questionnaires and electrodermal responses, investigated the hypothesis that psychopaths fail to develop anxiety. Lykken found reduced anxiety levels in the subjective evaluations and low electrodermal responses to conditioned stimuli that were previously associated with shock in the autonomic indices. Cleckley (1951) suggested that psychopaths exhibit discordance in the verbal and experiential components of emotion. Empirical evidence indicates that psychopathic individuals have less intense aversive emotional reactions to many everyday situations than do non-psychopaths (Day and Wong, 1996). Other investigators suggested that the inability of psychopaths to anticipate the negative consequences of their behavior results from an insufficient capacity to develop anticipatory fear (Hare, 1978). Hence, psychopathy may be characterized by a faulty modulation of associative links between external stimuli and internal reactions (Patrick, 1994; Patrick et al., 1994). Neuroimaging studies investigating the affective processing of psychopathy will potentially lead to an understanding of neural mechanisms and elements that maintain this disorder (Porter, 1996). We have reported that the absence of conditioned fear in psychopathic individuals is reflected in a virtually complete lack of activation of the fear circuitry in the brain (i.e., insula, anterior cingulate, amygdala, orbital frontal cortex) (Birbaumer et al., 2005).

The amygdala plays a central role in emotional processing, particularly in fear conditioning (Kim and Jung, 2006). However, fear retention is not necessarily at the same site as fear association, and hence it is unclear whether the amygdala is the permanent storage site for long-term fear memory. Fear retention is abolished after amygdala lesion 1 day after, but not after many days, of inhibitory avoidance training, suggesting that long-term fear memory is not stored in amygdala (Liang et al., 1982; McGaugh et al., 1982). Electrical stimulation of the amygdala can have positive and negative emotional effects (Aggleton, 2000), and amygdala is active in conditions involving both positive and negative stimuli. In addition to the amygdala, a network of structures that includes insula, anterior cingulate gyrus and medial orbitofrontal cortex is suggested as important in identifying the emotional significance of the stimulus, and regulate the affective state (Adolphs, 2003a,b; Phillips et al., 2003). The insula has afferent and efferent connections to the medial and orbitofrontal cortices, anterior cingulate and amygdala (Augustine, 1996). Stein et al. (2007) maintain that the insula may have been relatively neglected compared with amygdala in the literature.

Insula activation is involved in several types of emotional processes, including differential positive vs. negative emotional processing (Büchel et al., 1998; Morris et al., 1998a,b), pain perception (Gelnar et al., 1999; Peyron et al., 2000), anticipation and viewing of aversive images (Simmons et al., 2004; Phan et al., 2006), and the making of judgments about emotions (Gorno-Tempini et al., 2001). Blair et al. (2006) propose that psychopathic individuals receive markedly reduced augmentation of the representation of the conditioned stimulus (CS) from the reciprocal connections of the amygdala and insula. Response to the CS will be impaired in individuals with psychopathy relative to controls (i.e., a weaker representation should be less able to control behavior). On the other hand, if the CS functions only as a distracter to ongoing behavior, performance will be superior in individuals with psychopathy relative to comparison individuals (i.e., a weaker representation will be a less of a competitor for the stimulus that should be controling behavior). Furthermore, insula is instrumental in the detection and interpretation of certain internal bodily states (Critchley et al., 2002, 2004), and is part of an interconnected subcortical network involved in empathic behavior (Decety et al., 2013).

Recent studies have provided new information on insula's role in criminal and psychopathic behavior. de Oliveira-Souza et al. (2008) observed bilateral gray matter reductions in mid-anterior insula of community patients high on psychopathy scores compared with healthy individuals. Tiihonen et al. (2008) compared violent offenders with healthy subjects showing that the first group, and specially those with a diagnosis of psychopathy, had reductions of focal gray matter volumes in right insula. Similarly, Cope et al. (2012) examined a large sample of individuals from community correction centers observing that psychopathic traits were negatively associated with gray matter volumes in right insula. Schiffer et al. (2011) observed that violent offenders displayed smaller gray matter volume in the left insula, compared with non-offenders. Congruently with previous reports, left insula was the area of greatest difference between psychopath and non-psychopath prison inmates (psychopaths displayed significant insula cortical thinning) in a recent study by Ly et al. (2012). These findings are in line with a series of functional MRI studies that have shown hypoactivity of insula cortex on psychopathic individuals during classical fear conditioning as compared with healthy subjects (Veit et al., 2002; Birbaumer et al., 2005), that could act as a marker of psychopathy.

Until present it is unclear whether the defective brain fear circuitry of psychopaths can be modified at all. The stability of the psychopathic personality trait and its strong genetic determination (Tielbeek et al., 2012; Yildirim and Derksen, 2013) would argue against the possibility to self-regulate brain activity in fearrelated regions. On the other hand, social and developmental factors also play a fundamental role in psychopathic behavior (Viding, 2004; Moffitt, 2005). These environmental factors suggest that modification of the fear circuitry is possible. In view of the above line of argument, we asked the following questions: could psychopaths be trained to regulate the BOLD activity in the insula, and does volitional increase of BOLD in insula have any effect on emotional processing? In this pilot study, we also wanted to test whether training to regulate the insular cortex changes the functional connectivity of the emotional network. We hypothesized that with learned increase of activity in the insula, connectivity to the other areas of the fear circuitry will improve.

We implemented a Functional Magnetic Resonance Imaging based Brain-Computer Interface (fMRI-BCI) for this study. An fMRI-BCI is a non-invasive system that can be used for on-line neurofeedback of the BOLD signal to learn regulation of localized changes in brain activity, and to study related behavioral effects of regulation of specific brain regions and their connectivity (Sitaram et al., 2007, 2009; Weiskopf et al., 2007; Caria et al., 2012; Birbaumer et al., 2013; Ruiz et al., 2013a, 2014; Sulzer et al., 2013). In a series of studies on healthy individuals, we investigated the self-regulation of anterior insula and its behavioral effects (Caria et al., 2007, 2010; Veit et al., 2012). The present pilot study employs a similar paradigm of feedback-guided regulation of left anterior insula and a subsequent evaluation of emotional and neutral pictures. Statistical parametric mapping and region of interest (ROI1) analyses were carried out to assess the brain activation during regulation, picture presentation and rating conditions. Statistical analysis was also carried out on the valence and arousal ratings of pictures to test for the influence of regulation of insular activity on picture processing.

For analyzing the functional connectivity of the emotional network due to regulation training, we used Granger Causality Modeling (GCM). Granger Causality Modeling is a method of vector autoregressive modeling using the theory of Granger causality to analyze directed influences (Goebel et al., 2003; Roebroeck et al., 2005; Seth, 2005, 2010; Abler et al., 2006). The method works by using temporal information in one or more time-series of brain regions to predict signal time courses in another region. These predictions can be used to arrive at temporally directed influences rather than only correlations in activity between brain regions. Directed influences of one neural system on another, called effective connectivity (Büchel and Friston, 2000; Friston and Büchel, 2000), could be generated for one or more regions of interest to develop a circuit diagram that could replicate the observed timing relationships between brain regions. We tested whether different stages of self-regulation training would indicate a change in the functional connectivity of the network.

# **METHODS**

## **PARTICIPANTS**

Four psychopaths with criminal records participated in the study. Mean age of the psychopaths was 31.5 years (SD = 3.5 years, see **Table 1**). The criminal psychopaths were sexual offenders out on bail and waiting for their trial or out of jail and on parole, all of whom were screened from a larger sample using the Psychopathy Checklist: Screening Version (PCL:SV, Hart et al., 1995) and Levenson Self-Report Psychopathy Scale (LSRP; Levenson et al., 1995; Miller et al., 2008). The screening version of the PCL consists of 12 items and was developed to measure psychopathy in civic or forensic settings. Participants' PCL:SV scores, including, emotional detachment aspects (PCL-SV F1) and antisocial behavior aspects (PCL-SV F2); and the LSRP scores, including, interpersonal and affective aspects (LSRP F1), and the social deviance aspects (LSRP F2) are listed in the **Table 1**. These values are much lower than the scores for the American population of psychopaths but are in accordance with the lower values for the German norms (Ullrich et al., 2003). The PCL:SV and LSRP scores were obtained by structured interviews conducted by the certified clinical psychologist of this study (author Niels Birbaumer). Written instructions were provided to all participants and informed written consent was obtained. The ethics committee of the Faculty of Medicine of the University of Tübingen approved the study.

# **fMRI DATA ACQUISITION**

Functional images were acquired on a 3.0 T whole body scanner, with standard 12-channel head coil (Siemens Magnetom Trio Tim, Siemens, Erlangen, Germany). A standard echo-planar imaging sequence was used (EPI; TR = 1.5 s, matrix size = 64 × 64, effective echo time TE = 30 ms, flip angle α = 70◦ , bandwidth = 1.954 kHz/pixel). Sixteen slices (voxel size = 3.3 × 3.3 × 5.0 mm<sup>3</sup> , slice gap = 1 mm), AC/PC aligned in axial orientation were acquired. For superposition of functional maps upon brain anatomy a high-resolution T1-weighted structural scan of the whole brain was collected from each subject (MPRAGE, matrix size = 256 × 256, 160 partitions, 1 mm<sup>3</sup> isotropic voxels, TR = 2300 ms, TE = 3.93 ms, TI = 1100 ms, α = 8◦ ). In order to reduce movement artefacts, two foam cushions were used to immobilize the participant's head.

### **LOCALIZATION OF REGION OF INTEREST**

A localizer run was used to delineate the target ROI1, the left anterior insula. A block-based paradigm was used for the localizer run consisting of four blocks (22.5 s each) during which the subjects had to use mental imagery to recall emotionally relevant personal experiences alternating with five baseline blocks (22.5 s each) during which they had to count in a reverse order from number 100. No verbal cues for the direction of emotional valence of mental strategies and imagery were given. Region of Interest, consisting of a rectangular area encompassing 5 × 5 voxels (∼20 × 20 mm) on a single slice (5 mm), was selected based on the activation maps generated on-line during the task by means of Turbo-BrainVoyager (Goebel, 2001) and saved for the following training runs. The reference ROI2 was a large background ROI1 selected from a slice positioned distant from ROI1 encompassing a complete slice, selected with the intent to cancel global effects and to average out any unspecific activations. The ROI2 was chosen closer to the motor cortical surface to subtract movement related activations and further away from the anterior insula and other subcortical emotional brain regions.

# **EXPERIMENTAL PROTOCOL**

#### **Feedback training**

Each participant was conducted through three different types of protocols in this chronological order: a pretest on the first day; four feedback runs per day for 1–3 days depending on the availability of the subject; and a final day of a post-test. Each feedback run consisted of seven baseline blocks and six up-regulation blocks, each block of 30 s duration, presented in an alternating manner. During the up-regulation subjects had to increase the BOLD response in the ROI1 (left anterior insula) and during the baseline block return it to original value by watching the feedback presented in the form of an animated graphical thermometer. Normalized average BOLD signal from the left anterior insula was used to generate animated images of the varying thermometer bars. The feedback signal was computed as (BOLDupreg – BOLDbaseline)ROI1- (BOLDupreg – BOLDbaseline)ROI2. Where BOLDupreg and

**Table 1 | The table reports for each participant his/her name, age, psychopathy score based on Psychopathy Checklist: Screening Version (PCL:SV, Hart et al., 1995) for factors emotional detachment (F1) and antisocial behavioral aspect (F2), and Levenson Self-Report Psychopathy Scale (LSRP, Levenson et al., 1995) for interpersonal and affective factor (LSRP F1), the social deviance factor (LSRP F2) and the total LSRP score**.


BOLDbaseline constituted the BOLD signal during the current scan of up-regulation, and the average BOLD signal from the previous baseline block, respectively. The up-regulation and baseline blocks were cued with red and blue colored backgrounds, respectively. Each run of the feedback training took 6.75 min to complete. At the end of each training run monetary reward was computed and presented to the subject proportional to the aggregate valid differential BOLD increase or decrease at every time point with respect to the previous time points. For computing monetary reward, percent BOLD change was counted as valid only if there was a BOLD increase but not decrease during the up-regulation block, and a percent BOLD decrease but not increase during the baseline block. Each valid percent increase and decrease of BOLD earned 10 European cents. Maximum reward for a feedback run was limited to 10 Euros.

### **Pre-test and post-test**

The pre-test and post-test were similar in structure and only differed in the stimulus material used. The intent of the tests was to measure the effect of volitional regulation of the left anterior insula on aversive and neutral picture evaluation. Each run of the pre-test or post-test consisted of five alternating up-regulation and baseline runs, totally lasting 9 m. Each run consisted of a 30 s up-regulation or baseline block performed by watching the feedback of the moving thermometer bars as described before, followed by a 9 s emotional picture presentation block, that in turn was followed by a 12 s picture rating block (**Figure 1A**).

During picture presentation blocks, one emotional or neutral picture from the International Affective Picture System (IAPS; Lang et al., 2008, NIMH Center for the Study of Emotion and Attention, see **Table 2**) was presented. The pictures presented to the participants consisted of 20 aversive and 20 neutral pictures from IAPS. Valence and arousal ratings for aversive pictures were based on ratings from a large representative reference sample (Lang et al., 2008, NIMH Center for the Study of Emotion and Attention) with mean values of 3.26 ± 0.78 SD and 4.93 ± 0.47 SD, respectively. Valence and arousal ratings for neutral pictures had mean values of 4.84 ± 0.32 SD and 2.33 ± 0.40 SD, respectively. Pictures were pseudo-randomized such that no significant difference in standard ratings of valence and arousal was present between pictures after up-regulation and baseline blocks and between runs.

During the evaluation blocks subjects had to rate the picture for valence and arousal using a button-based control device inside the MRI scanner. Pictures were rated in terms of subjective emotional valence and arousal using the

(30 s) alternating with six baseline blocks (30 s) both followed by an International Affective Picture System (31) picture presentation block (9 s) and a rating block (12 s). During rating blocks, participants were shown the presented in close succession. Subjects were provided with two buttons allowing movements of the cursor in the left and right direction.

(IAPS = International Affective Picture System).


**Table 2 | Valence and arousal ratings of emotion and neutral pictures shown during pre- and post-tests (Lang et al., 2008, NIMH Center for the Study of Emotion and Attention)**.

Self-Assessment Manikin (SAM; Bradley and Lang, 1994). The SAM is a non-verbal pictorial assessment for measuring pleasure, aversion and arousal associated with a person's affective reaction to a variety of stimuli. Valence and arousal dimensions vary along a 9-point scale. Before the experiment, participants were briefed about the experimental tasks, SAM ratings, and were also trained on how to rate the pictures using the two buttons. Positioning a red outline on the desired number of the SAM scale selected the appropriate subjective rating (**Figure 1B**).

#### **OFF-LINE DATA ANALYSIS**

#### **SPM analysis of brain images**

Off-line image post-processing and data analysis were performed using SPM2 statistical parametric mapping software package (Wellcome Department of Imaging Neuroscience, London), while BrainVoyager QX was used for ROI analysis. During signal preprocessing, the functional EPI volumes were realigned spatially, normalized into Montreal Neurological Institute (MNI) space, and smoothed spatially (9-mm Gaussian kernel) and temporally (0.0039 Hz, 1/(2.5 times the duration of the upregulation and baseline block)) to remove high frequency artefacts. Hemodynamic response amplitudes were estimated using standard regressors, constructed by convolving a boxcar function, representing the block duration, with a canonical hemodynamic response function using standard SPM2 parameters. Motion parameters were also included into the general linear model (GLM) as covariates to take account for variance caused by head motion. Signal change during increase blocks with respect to the decrease blocks was evaluated by SPM2. Areas showing training related changes were analyzed with *t*-test comparisons of BOLD magnitude over runs.

# **Region of interest analysis**

Hypothesis driven ROI analysis was performed using the ROI previously selected for each subject during the training. Region of Interest time series underwent the same preprocessing and model estimation using the GLM for whole brain analysis. The percent signal change during up-regulation blocks with respect to the baseline blocks was calculated for each run separately, and then also averaged across subjects. The training effect was evaluated by computing paired *t-*test on all subjects of percent signal changes in the target ROI run by run. Activation maps produced by offline analysis matched and validated activations maps produced in realtime by Turbo-BrainVoyager.

# **Analysis of picture ratings**

Ratings of the IAPS pictures presented after increase blocks were compared with the ratings of the pictures presented after baseline blocks across runs. Local brain activity was also compared between pictures presented after increase blocks with respect to those presented after baseline blocks across runs.

# **Effective connectivity analysis**

Granger Causality Modeling (Granger, 1969, 1980) is a method originally developed in economics for causal interaction between multiple events from time-series data. Recently, GCM has been applied in conjunction with Vector Autoregressive Models (VAR) to fMRI data also (Roebroeck et al., 2005; Abler et al., 2006) to investigate directed influences between neuronal populations. In this study, GCM analysis was carried out to evaluate the network dynamics during self-regulation during three different training runs: a run of weak regulation (2nd training run), a run of intermediate regulation (5th training run), and a run of strongest regulation (11th training run). Time-series of ROIs, known to be involved in self-induced or stimulus-induced emotions from literature (Phan et al., 2002) and also passed the height threshold of *P* < 0.05 (Bonferroni corrected) and a cluster threshold of 50 voxels were used as input to the GCM analysis for each stage of regulation. We implemented a multivariate GCM by adapting the Causal Connectivity Matlab (Mathworks Inc., USA) Toolkit from Seth (2005) to work with fMRI signals and our design protocols. Multivariate GCM was applied to multiple time-series of selected ROIs (varying from 5–10) at three different stages of regulation under consideration.

Two important measures of connectivity, namely, *causal density* and *causal flow* were adapted from Seth (2005) and Seth and Edelman (2007) for comparison of functional connectivity across feedback training runs.

**Causal Density**: The *causal density* (*cd*) of a functional network is defined as the fraction of interactions among ROIs that are causally significant. Causal density is given by the relation *cd = gc*/(*n*(*n* − 1)), where *gc* is the total number of causal connections observed and *n* is the network size. A set of unconnected ROIs will have low *cd*.

**Causal Flow**: The *causal flow* (*cf*) of an ROI *i* in Grangercausality graph is defined as difference between the number of outgoing connections and the number of incoming connections. An ROI with highly positive *cf* exerts a strong causal influence over the network and so acts as a *causal source*. An ROI with a negative *cf* can be called a *causal sink* of the network.

# **RESULTS**

Participants reported using the following mental imageries for up-regulation of BOLD signal in the anterior insula: fight with the landlord, death of parents, experience in jail, memories of grand mother, negative memories of a stay in a detention center and pronouncement of judgment in the courtroom. We used the percent BOLD difference between up-regulation and baseline rather than actual values in order to rule out effects of baseline drift and intersubject differences. Only one participant (Subject-AK) learned to increase the BOLD activity in the anterior insula with training (**Figure 2A**), while the other participants did not increase their activation levels and in some cases even reduced them (subjects RS and GM). Mean BOLD change in the left anterior insula in the up-regulation condition across the training runs for all participants correlated negatively with the participants' PCL:SV ratings (Pearson Correlation = −0.7, **Figure 2B**). Only subject AK had the most consistent increase of BOLD in the up-regulation blocks, and also underwent the most number of training runs (12 runs in 4 days) compared to (4 runs in 1 day) other participants (**Figure 2B**). To correct for the difference in the training runs, we recalculated the correlation using only data from first four training runs, the corrected Pearson Correlation Coefficient is −0.67.

Because, only subject AK showed consistent activation increases in the up-regulation blocks, further analysis on behavioral data, and effective connectivity from fMRI data were performed in this participant alone. Whole brain analysis for participant AK showed an increased activation cluster in the left anterior insula in the last run when the subject had learned to regulate strongly in comparison to the first day run. The Talairach coordinate for the activated cluster of the left anterior insula for subject AK was (−37, 27, 0; **Figure 3**).

For subject AK, mean subjective ratings of valence and arousal of aversive images from the IAPS, after all the up-regulation conditions, decreased from the pretest to the posttest, and became closer in value to the healthy subject ratings reported by Lang et al. (2008) (**Figure 4A**). This indicates that in the pretest the participant rated aversive images to be more positive in emotional valence and more arousing compared to the standard healthy ratings for the same images. However, during the posttest (conducted after neurofeedback training), the same participant rated the images as being less positive (more negative) and less arousing, and similar to the IAPS healthy ratings.

In the successful subject AK, percent BOLD increase and the *causal density* (*cd*; the number of interactions that are causally significant in the brain network involved in up-regulation) increased proportionately, from the early stage to the final stage of training, in the subject AK (**Figure 4B**). A high correlation coefficient of 0.987 was observed between the percent BOLD and the causal density.

**Figure 5** shows results of connectivity analysis for three different strengths of regulation in the successful subject AK. Left column of the figure contains schematic depictions of the directed influences among different ROIs; the right column shows bar charts of causal flow (*cf*) for the ROIs.

**Connectivity during the early stage of training**: **Figures 5A,D** show the brain network and the causal flow, respectively, during the early stage (2nd training run) of the subject AK. Regulation at this stage is driven by the posterior part of the brain, mainly involving the posterior cingulate, which has significant directed influences to left insula, anterior cingulate cortex (ACC) and hippocampus. The posterior cingulate is the major causal source at this stage followed by the left medial prefrontal cortex. At this stage one can see a very sparse network involving interactions among very few ROIs in the emotional network indicated by the low causal density.

**Connectivity during the mid stage of training**: **Figures 5B,E** show the brain network and the causal flow, respectively, during mid stage of training (5th training run) of the subject AK. Regulation at this stage continues to be driven by the posterior cingulate, which has significant directed influences to left insula, ACC, medial prefrontal cortex and hippocampus. In addition, the hippocampus directs its influence to the left insula. The posterior cingulate is still the major causal source followed by hippocampus at this stage. The causal density of the network has slightly increased compared to the early stage of training.

**Connectivity during the final stage of training**: **Figures 5C,F** show the brain network and the causal flow, respectively, during the final stage of training (11th training run) of the subject AK.

Now, the anterior portion of the brain drives the network, with left insula taking a major share. Left insula is the predominant causal source with directed influences to superior medial frontal cortex, right insula, medial prefrontal cortex and posterior cingulate. Right insula and the posterior cingulate are the major causal sinks. At this stage, a great number of ROIs in the network (high causal density) have been recruited into the regulation function, mainly mediated by the left insula, superior medial frontal cortex and hippocampus. The anterior cingulate now has reciprocal connections with hippocampus. New influences from midtemporal gyrus and amygdale towards insula are also observed.

# **DISCUSSION**

This pilot study explored the possibility of training criminal psychopaths to volitionally control the BOLD signal in the left anterior insula with the help of an fMRI Brain-Computer Interface developed in our laboratory (Sitaram et al., 2009, 2011). Our previous studies with healthy volunteers (Caria et al., 2007) had shown that learned control of the anterior insula was specific to the region, and not a result of general arousal and global unspecific brain activation, as demonstrated by a control group trained with a non-contingent feedback. From a follow-up study (Caria et al., 2010), we had reported that regulation of left anterior insula modulates the emotional response specific to aversive pictures but not neutral picture stimuli. Both studies had shown that mental imagery alone is not sufficient, and that real-time feedback of the BOLD signal extracted from the target region enhances participants' ability to achieve regulation. These studies naturally led to the question as to whether patients suffering from clinical conditions characterized by emotion deficits could learn to volitionally regulate anterior insula. In the present study, four criminal psychopaths underwent rtfMRI neurofeedback training. Three psychopathic criminals could only complete four runs of training and were not very successful in learning to control the BOLD signal in the insula. Only one participant (AK) participated

in extending neurofeedback training (12 in total) and also learned to successfully control the insula.

In comparison to the low success rate of psychopathic criminals to up-regulate the anterior insula in this study, an earlier study in our lab in 15 healthy participants showed significant increase in the BOLD signal in the anterior insula with five neurofeedback training runs (Caria et al., 2007). Linear regression across all runs showed significant increase of activity in the anterior insula [*y* = 0.174 + 0.127, *P* < 0.012]. Percent BOLD signal change in the region between up-regulation and baseline averaged across all the participants resulted in a clear monotonic increase across the first three runs (repeated measures ANOVA, *F*(2,7) = 10.32, *P* = 0.001). In a subsequent study in our lab, a group of healthy individuals given real-time, veridical feedback learned to increase BOLD activity in bilateral anterior insula while two groups of healthy participants, one with sham feedback and another with no feedback did not learn to increase activity in the region (Caria et al., 2010).

In comparison, the results of the current study, although modest in the number of participants being successful, are still in line with a recent study from our laboratory in schizophrenia patients, and indicates that insula self-regulation can be achieved with fMRI-BCI, even in severe chronic brain disorders (Ruiz et al., 2013b).

A number of reasons could have potentially affected learning in the majority of our psychopathic participants: general lack of motivation to participate in the experiments in spite of the monetary reward, failure to understand and follow experimental instructions, inability to perform emotional imagery and lack of attention to the task. In more basic terms, the inability to learn up-regulation of insula could be attributed to faulty modulation of associative links between external stimuli and internal reactions (Patrick, 1994; Patrick et al., 1994). As per earlier reports, the absence of conditioned fear in psychopathic individuals and the reduced activation of the fear circuitry in the brain (i.e., insula, anterior cingulate, amygdala, orbital frontal cortex, Birbaumer et al., 2005) may be the fundamental reason for the ineffective learning.

In the present study, one psychopathic criminal learned to regulate anterior insula by employing negative emotional imageries in conjunction with contingent feedback, despite a widespread scientific view that the psychopathic brain is resistant to change. We have also extended previous experimental protocols

by providing monetary reward after every run. Monetary reward was computed proportional to the percent BOLD increase in the target region to motivate these difficult-to-recruit experimental subjects to continue feedback training. In the successful participant, continued training enhanced the percent differential BOLD in the up-regulation condition compared to the downregulation condition. Overall, subjects with higher PCL:SV scores were less successful at up-regulation than their lower PCL:SV counterparts. The severity of psychopathic features and different symptom clusters of psychopathy as measured by the PCL:SV have been linked to different emotional processing deficits (Hicks and Patrick, 2006; Hansen et al., 2008; Gao et al., 2012). The current results, although limited due to the results from a single participant, are in line with these studies and with the existing notion that psychopathy consists of a neurobiological trait deficit of aversive anticipatory conditioning responsible for socialization and regulation of negative affect and instrumental aggression (Birbaumer et al., 2005). Alternatively, as psychopathic traits have been associated with insula abnormalities (de Oliveira-Souza et al., 2008; Tiihonen et al., 2008; Schiffer et al., 2011; Ly et al., 2012), the difficulty to learn self-regulation in individuals with higher PCL:SV scores could reflect the abnormality of the insular structure, hypothesis that can be tested in future studies. Although observed in only criminal psychopath, it is nevertheless interesting that the volitional up-regulation of insula was associated with changes in the subjective ratings of valence and arousal of aversive stimuli in this participant. More tests are needed to confirm this finding. This behavioral modulation is congruent with the evidence from previous studies in healthy participants (Caria et al., 2007, 2010) indicating that insula self-regulation modifies the appraisal of emotional picture stimuli, particularly of negative valence. Considering the abnormal

**FIGURE 5 | Effective connectivity analysis of self-regulation of insula for subject AK**. Panels **(A–C)** show directed influence maps (DIMs) and panels **(D–F)** show the bar charts of the causal flow (CF; defined as the net difference between outgoing and incoming connections) in the brain regions (the emotional network) involved in the self-regulation of insula. Panels are arranged from top to bottom from the early through mid to the final stage of training: from early stage of training (top), mid stage of training (middle) to final stage of training (bottom). As the training proceeds, the number of directed influences in the emotional network also increases as shown by the DIMs. During the final stage of training, many regions in the emotional network, including medial prefrontal cortex (MPFC), right insula, anterior cingulate cortex (ACC), amygdala and posterior cingulate cortex are seen to become connected with insula. The CF diagram of the early stage shows that the activations are driven predominantly from the posterior portion of the brain (especially by the posterior cingulate cortex) and that the CF in the left insula is close to zero. However, more training, insula and the anterior regions of the brain seem to drive the network activation to a greater extent as shown by their increasing CF.

autonomic reactivity to aversive stimuli, consistently reported in literature on psychopathy (Lorber, 2004; Gao and Raine, 2010), it would be interesting to assess whether the observed changes in the appraisal of emotional stimuli are accompanied by modifications of physiological measures related to anxiety or other emotional states (e.g., electrodermal response, heart rate, and so forth).

The main purpose of the present study was not only to ascertain whether criminal psychopaths can learn to regulate the BOLD signal in the left anterior insula but also to investigate the changes in effective connectivity in the brain of criminal psychopaths due to successful regulation. In particular, we wanted to compare the effective connectivity changes with varying strength of the regulation. To this end, we employed the multivariate GCM adapted from the Causal Connectivity Toolkit from Seth (2005)to work with fMRI data. A recent study on insula self-regulation showed that fMRI-BCI training is associated with an enhancement in the numbers of effective connections in the emotional network in a successful regulation run, compared with an unsuccessful regulation run (Ruiz et al., 2013b). In the present study, multivariate GCM was applied to time-series of 8–10 ROIs from the emotional network separately for three different levels of regulation, namely, weak, intermediate and strong, determined based on the average percent BOLD increase in up-regulation blocks compared down-regulation blocks. The results show that, firstly, the level of up-regulation is proportional to the causal density of the network, in other words, the extent of "connectedness" of the functional network purportedly involved in the regulation process. Secondly, the results show that during early training, upregulation is driven mainly from the posterior cingulate, which acts as the main causal source, while right insula and the posterior cingulate are the major causal sinks. With more training, the source of the network moves towards the anterior of the brain finally settling in the left anterior insula during the final stage of up-regulation. During the final stage of training, the left insula directs its influence outwardly to superior medial frontal cortex, right insula, medial prefrontal cortex and posterior cingulate, indicated by the high value of the causal flow. In addition, the hippocampus, amygdala and midtemporal gyrus are introduced into the causal network during the final stage of training. Our results are supported by previous reports of emotional regulation and processing. Ochsner et al. (2004) have shown that up- and downregulating negative emotion recruited prefrontal and anterior cingulate regions implicated in cognitive control. Further, they reported that self-focussed regulation recruited medial prefrontal regions implicated in internally focused processing, whereas situation-focussed regulation recruited lateral prefrontal regions implicated in externally focused processing. In our study, the predominance of medial prefrontal activation and its involvement in the causal network, and in addition the self-report of regulation strategies by subjects (e.g., fight with the landlord, death of parents, experience in jail, memories of grand mother, negative memories of a stay in a detention center and pronouncement of judgment in the courtroom) indicate the employment of self-focussed imagery, and thus further corroborate Ochsner's findings of the critical role of medial prefrontal cortex in self-focussed emotional appraisal (Ochsner et al., 2004).

The modulation of brain functional connectivity and the enhanced involvement of frontal areas during the final stage of training could have important implications. In fact, several recent studies have reported abnormal neural functional connectivity in psychopathy. Particularly, a reduced functional connectivity in prefrontal-amygdala circuits have been repeatedly observed (Marsh et al., 2011; Motzkin et al., 2011; Osumi et al., 2012; Juárez et al., 2013; Contreras-Rodríguez et al., 2014). The present findings suggest that fMRI-BCI could serve to modulate abnormally activated cortical and subcortical brain circuits, including brain systems purportedly abnormal in psychopaths, e.g., "the circuitry of empathy" (Decety and Svetlova, 2012; Decety et al., 2012). Interestingly, Ly et al. (2012) reported that psychopath inmates exhibit a reduction in functional connectivity between insula and the dorsal ACC, which could be a neural correlate of the abnormalities in the flexibility of goal-directed attention observed in psychopath. In light of this, future studies for training control of functional or effective connectivity between brain regions, e.g., insula and ACC, could serve to explore specific hypothesis regarding the link between neural substrates and psychopathology (Ruiz et al., 2013a).

Our results question a position emphasizing the resistance to change of psychopathic traits and its heritability. If the brain defect in psychopathy (evident from the neuroimaging literature on classical aversive conditioning) is genetically determined, then the modification of that circuit would not be possible at all. Our pilot study indicates to the contrary that criminal psychopaths could learn to volitionally control brain activity pertaining to emotion and that might have brain and behavioral consequences. Of particular interest are the results of GCM in the course of regulation improvement. The more a psychopathic individual regains control over the deficient activity of the anterior insular cortex, the greater is the restoration of the brain network of emotion.

The study has limitations. Firstly, there are obvious challenges to recruiting this very special population of study participants, and then motivating them to undergo multiple days of neurofeedback training. Secondly, it is difficult to ascertain how much of the variability observed in the capability to self-regulate can be explained by differences in motivation, compliance with the instructions, and other factors that might be playing a role in a population with psychopathic traits. Also, as a preliminary report, no long-term behavioral effects were explored. Future studies with larger samples will serve to generalize our results.

In summary, this pilot study shows modest feasibility that criminal psychopaths can learn self-regulation of a circumscribed brain area with fMRI-BCI, leading to modifications in the functional emotional network. Considering the limited evidence of effective treatment of psychopathy (Salekin et al., 2010) this proof-of principle study suggests that fMRI-BCI could offer a new valuable tool for modulating abnormal brain activations and behavior in this severe condition.

# **ACKNOWLEDGMENTS**

This work was supported by by the Deutschen Forschungsgemeinschaft (DFG; BI 195/64-1 and and Reinhard Kosseleck support to Niels Birbaumer); SFB 437 "Kriegserfahrungen" of the DFG; the Marie Curie Host Fellowship for Early Stage Researchers Training; the Center for Integrative Neuroscience (CIN), Tübingen, Germany (Pool-Project 2011-08); Comisión Nacional de Investigación Científica y Tecnológica de Chile (Conicyt) through Fondo Nacional de Desarrollo Científico y Tecnológico Fondecyt (project nº11121153); German Center for Diabetes Research (DZDe.V.); the German Federal Ministry of Education and Research (BMBF, grant number 01GQ0831, 16SV5838K); and the Volkswagenstiftung (VW) and the Baden-Württemberg Stiftung, Germany. The authors are indebted to Dr. Michael Erb for technical assistance in data acquisition.

# **REFERENCES**


Caria, A., Veit, R., Sitaram, R., Lotze, M., Weiskopf, N., Grodd, W., et al. (2007). Regulation of anterior insular cortex activity using real-time fMRI. *Neuroimage* 35, 1238–1246. doi: 10.1016/j.neuroimage.2007.01.018

Cleckley, H. M. (1951). The mask of sanity. *Postgrad. Med.* 9, 193–197.


**Conflict of Interest Statement**: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 05 June 2014; paper pending published: 13 July 2014; accepted: 12 September 2014; published online: 14 October 2014*.

*Citation: Sitaram R, Caria A, Veit R, Gaber T, Ruiz S and Birbaumer N (2014) Volitional control of the anterior insula in criminal psychopaths using real-time fMRI neurofeedback: a pilot study. Front. Behav. Neurosci. 8:344. doi: 10.3389/fnbeh.2014.00344 This article was submitted to the journal Frontiers in Behavioral Neuroscience*.

*Copyright © 2014 Sitaram, Caria, Veit, Gaber, Ruiz and Birbaumer. This is an openaccess article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution and reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms*.

# fMRI neurofeedback facilitates anxiety regulation in females with spider phobia

Anna Zilverstand1, 2 \*, Bettina Sorger <sup>1</sup> , Pegah Sarkheil 1, 3 and Rainer Goebel 1, 4

*<sup>1</sup> Department of Cognitive Neuroscience, Maastricht University, Maastricht, Netherlands, <sup>2</sup> Department of Psychiatry, Icahn School of Medicine at Mount Sinai, New York, NY, USA, <sup>3</sup> Department of Psychiatry, Psychotherapy and Psychosomatics, RWTH Aachen University Hospital, Aachen, Germany, <sup>4</sup> Department of Neuroimaging and Neuromodeling, Netherlands Institute for Neuroscience, Amsterdam, Netherlands*

Background: Spider phobics show an exaggerated fear response when encountering spiders. This fear response is aggravated by negative and irrational beliefs about the feared object. Cognitive reappraisal can target these beliefs, and therefore has a fear regulating effect. The presented study investigated if neurofeedback derived from functional magnetic resonance imaging (fMRI) would facilitate anxiety regulation by cognitive reappraisal, using spider phobia as a model of anxiety disorders. Feedback was provided based on activation in left dorsolateral prefrontal cortex and right insula, as indicators of engagement and regulation success, respectively.

#### Edited by:

*Ranganatha Sitaram, University of Florida, USA*

#### Reviewed by:

*Seth Davin Norrholm, Emory University School of Medicine, USA Pratibha N. Reebye, University of British Columbia, Canada Frank Scharnowski, University of Geneva, Switzerland*

#### \*Correspondence:

*Anna Zilverstand, Department of Cognitive Neuroscience, Maastricht University, Oxfordlaan 55, 6229 EV Maastricht, Netherlands anna.zilverstand@gmail.com*

> Received: *09 November 2014* Accepted: *20 May 2015* Published: *08 June 2015*

#### Citation:

*Zilverstand A, Sorger B, Sarkheil P and Goebel R (2015) fMRI neurofeedback facilitates anxiety regulation in females with spider phobia. Front. Behav. Neurosci. 9:148. doi: 10.3389/fnbeh.2015.00148* Methods: Eighteen female spider phobics participated in a randomized, controlled, single-blinded study. All participants completed a training session in the MRI scanner. Participants assigned to the neurofeedback condition were instructed to shape their regulatory strategy based on the provided feedback. Participants assigned to the control condition were asked to adapt their strategy intuitively.

Results: Neurofeedback participants exhibited lower anxiety levels than the control group at the end of the training. In addition, only neurofeedback participants achieved down-regulation of insula activation levels by cognitive reappraisal. Group differences became more pronounced over time, supporting learning as a mechanism behind this effect. Importantly, within the neurofeedback group, achieved changes in insula activation levels during training predicted long-term anxiety reduction.

Conclusions: The conducted study provides first evidence that fMRI neurofeedback has a facilitating effect on anxiety regulation in spider phobia.

Keywords: fMRI, real-time, self-regulation, neurofeedback, spider phobia, anxiety, regulation, cognitive reappraisal

# Introduction

Interest in novel treatment approaches for patients with anxiety disorders is high. Anxiety disorders are the most common mental health condition, with a year-prevalence of 12–18% (Wittchen and Jacobi, 2005; Kessler et al., 2011, 2012). Moreover, 16–47% of these patients cannot be treated successfully with a currently standard treatment such as cognitive behavior therapy (Ost, 2008). Further integration of cognitive regulation strategies into the treatment of anxiety disorders has therefore been suggested (Kamphuis and Telch, 2000; Amstadter, 2008; Farmer and Kashdan, 2012). "Cognitive reappraisal," the reinterpretation of the meaning of a stimulus, is an effective emotion regulation strategy, with beneficial long-term impact on anxiety symptoms (Kamphuis and Telch, 2000; Sloan and Telch, 2002; Amstadter, 2008; Farmer and Kashdan, 2012). This method targets negative, anxiety-provoking beliefs, which undermine regulation and prevent an adaptive response to the perceived threat (Gross, 1998; Amstadter, 2008). Spider phobics, similar to patients with other anxiety disorders, hold these beliefs (Arntz et al., 1993), and are expected to benefit from training reappraisal. The aim of this study was to investigate if providing neurofeedback during cognitive reappraisal would facilitate regulation success in spider phobia as a specific form of anxiety disorders.

Neurofeedback training based on functional magnetic resonance imaging (fMRI) is increasingly gaining interest as a novel approach in treating neurological and psychiatric disorders. This method suggests that the presentation of feedback derived from patients' current neural activation can train the voluntarily regulation of selected brain processes. The goal is to achieve a normalization of deviant brain processes, and thus improve the related behavioral symptoms. Previous studies found that neurofeedback is an efficient tool in shaping mental strategies toward a given goal (DeCharms et al., 2005; Caria et al., 2007; Linden et al., 2012; Scheinost et al., 2013; Young et al., 2014). Exploratory investigations have also indicated a benefit of fMRI neurofeedback training in clinical populations with chronic pain, tinnitus, Parkinson's disease, stroke, and mood disorders (DeCharms et al., 2005; Haller et al., 2010; Subramanian et al., 2011; Linden et al., 2012; Sitaram et al., 2012; Young et al., 2014). Neurofeedback training methods have not been applied in patients with anxiety disorders, but it has been demonstrated that subclinical levels of anxiety can be successfully reduced by learning self-regulation of select brain activation levels (Scheinost et al., 2013).

Spider phobia, as other anxiety disorders, is characterized by an exaggerated fear response when encountering the feared object, in this case spiders. This strong fear response is accompanied by hyperactivation of a network of brain regions involved in anxiety expression (the anxiety expression network), such as the amygdala and insula (Etkin and Wager, 2007). Both amygdala and insula have been proposed to belong to a core anxiety network implicated across different anxiety disorders (Etkin and Wager, 2007). While the amygdala has been linked to initial automatic fear processing during fear expression (Carlsson et al., 2004; Straube et al., 2006), the insula represents sustained anxious emotion (Somerville et al., 2013). Successful fear regulation in healthy subjects is characterized by down-regulation of this anxiety expression network (Delgado et al., 2008), and reduced activation levels in this network have been linked to a positive treatment response (Schienle et al., 2007). Beyond the anxiety network, a reduced regulatory capacity during anxiety provocation was shown in spider phobics, marked by hypoactivation of a frontal regulatory network (New et al., 2009; Manber-Ball et al., 2013). This frontal network encompasses cingulate and prefrontal cortices, such as dorsolateral prefrontal cortex (dlPFC), and is known to be activated during the regulation of negative affect in healthy participants (Ochsner et al., 2012). Engagement of the dlPFC during cognitive reappraisal is delayed in patients with anxiety disorders, with the delay predicting levels of anxiety (Goldin et al., 2009). Furthermore, dlPFC activation levels during fear regulation are inversely associated with the severity of anxiety and functional impairment (New et al., 2009; Manber-Ball et al., 2013). Also, an increase in dlPFC activation levels predicts treatment success (Hauner et al., 2012). In healthy participants, dlPFC has been implicated in safety learning and successful anxiety regulation (Delgado et al., 2008; Pollak et al., 2010).

The neurofeedback training implemented in this study provided patients with a novel dual feedback display. Participants received feedback on both their current activation levels of the insula (sustained anxious emotion) and the dlPFC (engagement in regulation) during anxiety regulation. Neurofeedback participants were asked to continuously improve their regulation strategy according to the feedback, while a non-feedback control group was asked to learn based on intuition. We expected reduced insula activation in combination with high dlPFC activation in the neurofeedback group in comparison to the control group. Additionally, we hypothesized that this normalization of brain activation patterns in the neurofeedback group would predict reduced immediate- and long-term subjective levels of spider fear. A link between successful selfregulation of brain activation levels and long term behavioral change would provide first evidence that neurofeedback may be an efficacious tool for enhancing anxiety regulation.

# Materials and Methods

# Participants

Eighteen women were recruited through public advertisement at Maastricht University. They were screened for high spider fear [Spider Phobia Questionnaire (SPQ) Score ≥ 14, (Klorman et al., 1974)] and diagnosed with spider phobia according to the criteria of The Diagnostic and Statistical Manual of Mental Disorders DSM-IV TR (American Psychiatric Association, 2000). All were free of psychotropic medication and were not affected by other current or previous neuropsychiatric comorbidities as evaluated by means of a structured clinical interview [Mini International Neuropsychiatric Interview, MINI, (Sheehan et al., 1998)]. None of the participants had previously received cognitive behavioral therapy. All participants were students, or currently employed. To balance the two experimental groups for age, self-reported use of reappraisal strategies [Emotion Regulation Questionnaire, Reappraisal score, ERQ-R, (Gross and John, 2003)], and spider fear (SPQ score), we used a restricted randomization procedure shown to be efficient for small sample sizes [sequential balancing, (Borm et al., 2005)] (**Table 1**). Participants were naïve to group assignment and goal of the study. They were informed that they were participating in a treatment study investigating a novel anxiety regulation technique. All participants were equally compensated (8 e/h) and gave their written informed

TABLE 1 | Characteristics of study participants.


*Duration, since onset of symptoms; ERQ-R, Emotion Regulation Questionnaire Reappraisal Score; SPQ, Spider Phobia Questionnaire; FSQ, Fear of Spider questionnaire; SBQ, Spider Belief Questionnaire.*

consent prior to the experiment according to the Declaration of Helsinki and approved by the local Medical Ethics Committee at Maastricht University.

# Procedure

Participants first had a 15-min practice session on how to use cognitive reappraisal during provocation of anxiety by spider photographs. An instructor (clinical psychologist) guided the participants to reinterpret a situation by "finding out calming aspects" instead of "engaging in anxiety provoking thoughts." Participants were told that the rationale was to normalize some of the most common negative beliefs held by spider phobics (Arntz et al., 1993), drawing the focus to the safety of the situation. They were asked to select from four sorts of strategies: (1) detecting the aesthetics of the spider, (2) focusing on its powerlessness, (3) changing its connotation by humanizing it, or (4) changing its context by imagining approaching it in a safe environment. Each participant was invited to write down their own personal credible version of each strategy. They were then familiarized with the MRI procedures and requested to rehearse aloud during eight practice trials (regulate trials). Last, participants were asked to practice refraining from changing their thoughts in another eight practice trials, letting thoughts occur spontaneously (watch trials).

Neurofeedback participants were introduced to the dual feedback display and explained the feedback rationale. They were instructed to adjust the reappraisal strategy based on the provided feedback throughout the experiment. They were told that the goal was to achieve high prefrontal activation ("reappraisal activation") and reduced insular activation ("anxiety activation"). Participants were asked to primarily consider the feedback from the regulatory network, if dual feedback information was challenging. The control group was presented a visually similar display, and was instructed that it indicated a short break in-between trials. Control participants were asked to adapt their strategy based on intuition throughout the session. All participants were told that experiencing high anxiety levels may be an essential part of the regulation process, and is generally not harmful. They were reminded that they could stop at any time, asked to pay attention, and to refrain from any movements in the scanner. Immediately before the imaging session, all participants completed the Questionnaire of Current Motivation [QCM, (Rheinberg et al., 2001)], which measures individual differences in current motivation and expectation of success.

The 50-min imaging session started with one 5-min anatomical imaging run, followed by four 11-min functional imaging runs. Participants performed the practiced task during all four functional runs, alternating regulate and watch trials (presented in a blocked design, e.g., 4 watch trials, 4 regulate trials, 4 watch trials, 4 regulate trials, counterbalanced order). Data from the first functional run were used for delineation of the dlPFC and insula target regions (localization run). Neurofeedback was presented from the second to fourth functional run (neurofeedback training). To keep the training challenging throughout the experiment, the presented stimuli were selected to be increasingly anxiety provoking with each run (**Figure 1**). All stimuli were selected based on a behavioral pilot study with spider phobics (Supplementary Figure 1**)**, and presented only once per condition. Each trial started with a 1.5-s cue (pictogram: watch or regulate), followed by 1-s fixation and the 12.5-s active trial period of anxiety regulation during presentation of the spider photograph (**Figure 2**). Participants then rated their subjective anxiety on a 5-point Likert scale from 0 = "not fearful at all" to 4 = "extremely fearful" using a button box (Current Designs, Philadelphia, PA, USA). The feedback display was presented to the neurofeedback group after regulate trials, 2.5 s after the trials elapsed. During watch trials, and in the control group the "break display" was shown. All displays were presented using Presentation (Version 16; Neurobehavioral Systems, Albany, USA). Between trials there was a jittered resting period of 8.75 ± 2.5-s. We chose to present intermittent feedback to avoid cognitive overload and distraction, improve signal to noise ratio of the feedback signal, and accommodate hemodynamic delay (Stoeckel et al., 2014). Intermittent feedback paradigms have been empirically demonstrated to be effective in shaping neural activity and learning (Bray et al., 2007; Johnson et al., 2012).

After the training session participants were asked to indicate which reappraisal strategy they believed to be the most successful one ("which strategy would you recommend?"), to rate on a 7 point Likert scales how helpful the reappraisal instruction (both groups) and the provided neurofeedback (only experimental group) were, if neurofeedback was helpful in selecting a reappraisal strategy (only experimental group), how comfortable they were in the scanner environment (both groups), and to indicate their willingness to come back for another session (both groups).

# MRI Imaging

Images were acquired at Maastricht Brain Imaging Centre (Maastricht University) on a 3T scanner (Tim Trio/upgraded to Prisma Fit, Siemens Healthcare, Germany). The functional echoplanar imaging (EPI) sequence was optimized for imaging of limbic and prefrontal regions (Weiskopf et al., 2007; Morawetz et al., 2008): repetition time = 1250 ms, echo time = 25 ms, flip angle = 67◦ , slice thickness = 2.5 mm, 20% gap, in-plane = 3 × 3 mm, slice angle of 25–30◦ , grappa acceleration = 2. We compromised for coverage of parietal cortex to achieve higher sampling rate for real-time imaging analysis. Heart and breathing rates were monitored using Siemens pulse oximeter

FIGURE 1 | fMRI study design. The stimuli used were selected to be increasingly anxiety provoking with each run, based on a behavioral pilot study (Supplementary Figure 1). Participants from the

neurofeedback group received feedback during the three experimental runs, after the individual target regions had been defined based on the localization run.

and breathing chest band (recording the first 5 min of each 11 min functional run). Anatomical images were collected with a magnetization-prepared rapid acquisition gradient echo (3D MPRAGE) sequence: repetition time = 1900 ms, echo time = 2.52 ms, flip angle = 9 ◦ , voxel size 1 × 1 × 1 mm<sup>3</sup> , with duration 4:26 min.

# Real-time Imaging Analysis

During the imaging session all functional images were analyzed with Turbo-BrainVoyager (Version 3.0; Brain Innovation, Maastricht, Netherlands). The images were pre-processed using motion correction, drift confound predictors, and high-pass filtering with a general linear model (GLM) Fourier basis set (2 cycles). An incremental GLM was computed using two task predictors (watch, regulate) convolved by a standard two-gamma hemodynamic response function, as well as predictors for events of no interest. Functional maps were thresholded (t = 3, cluster threshold = 4 voxels). Target regions were individually defined based on the contrasts watch vs. resting (insula) and regulate vs. resting (dlPFC). The cluster closest to the target coordinates was manually selected. The target coordinates were defined unilaterally based on previous research: x = 37, y = 11, z = 3 in the right insula and x = −43, y = 28, z = 30 in the left dlPFC (Etkin and Wager, 2007; Delgado et al., 2008; Ochsner et al., 2012). During experimental runs, neurofeedback values were computed by contrasting the activation increase during stimulus presentation (last 10-s) relative to a baseline previous to stimulus onset (7.5-s) (**Figure 3**). The feedback was displayed on a thermometer, which had its maximum adjusted to average activation during the localization run (max thermometer = 2∗ average activation localization run).

# Post-hoc Imaging Analysis

Functional and anatomical images were pre-processed posthoc in BrainVoyager (Version QX 2.7; Brain Innovation, Maastricht, Netherlands) as during real-time analysis. None of the participants moved more than 3.0 mm/degrees in any direction/rotation. All data was spatially normalized to Talairach space to enable comparison between participants. Beta estimates for the modeled individual blood oxygen leveldependent (BOLD) response (watch, regulate) were derived for the individually defined target regions to perform a regionof-interest analysis. Separate analysis of the localization run and experimental runs were performed in SPSS Statistics (IBM 21; SPSS Statistics; IBM, Armonk, NY, USA). The beta weights were submitted to a repeated measures GLM with linear contrasts, modeling within factors task (watch, regulate), functional run (1; 2–4), and group as a between factor. Effect sizes were estimated using partial eta squared (Cohen, 1973).

For whole-brain random-effects GLM analysis the data was spatially smoothed (FWHM 6 mm) and noise confounds were added to represent the six head motion parameters (Weissenbacher et al., 2009), a localized estimate of white matter signal for modeling scanner artifacts (Jo et al., 2010), and the ventricular signal to control for physiological artifacts (Birn et al., 2009). Whole brain analyses statistical maps were thresholded with an initial uncorrected voxel-threshold of α = 0.05, and cluster-size threshold with a false positive rate of α = 0.05, (Forman et al., 1995).

# Physiological Data

Pulse and breathing rate from each participant were computed per task condition (resting, watch, regulate) using a custom made MATLAB tool (R2010a; The Mathworks, Natick, USA). Physiological data were analyzed statistically as the imaging data.

# Behavioral Data during Training

Group differences during the training regarding motivation, expectation of success, comfort in the scanner, helpfulness of instruction and neurofeedback, and willingness to come back for additional sessions were statistically evaluated using independent

time to repetition) of stimulus presentation, relative to a 7.5-s period (6 TR)

previous to stimulus onset.

blue thermometer reflected their engagement in reappraisal thoughts, while the red thermometer indicated their anxiety level. Each thermometer was

sample t-tests. Subjective anxiety ratings collected during the imaging session were submitted to the same repeated measures GLM as the imaging data.

# Follow-up Assessment Spider Fear

To evaluate long-term changes in spider fear the participants were followed during a period of 3 months. Spider fear was measured during screening, after the MRI training session, 2 weeks, and 3 months after the training. At each time point, participants were administered two questionnaires: the Fear of Spider Questionnaire [FSQ, (Szymanski, 1995)], selected for its high test-retest stability and internal consistency (Muris and Merckelbach, 1996), and the Spider Belief Questionnaire [SBQ, (Arntz et al., 1993)], which was specifically designed to measure the changes in beliefs held by spider phobics. Questionnaire data was analyzed using a repeated measures GLM with linear contrasts, within factor time (screening, postfMRI, 2-week, 3-month,) and group as a between factor. To test for transfer from changes on a brain level to post-training behavioral change, we regressed change in spider fear (from screening to 3-month) on change in BOLD activation (from localization run to last experimental run) by simple linear regression.

# Results

# Behavioral Data at Baseline

Participants demonstrated similarly high levels of spider fear during screening (**Table 1**). Also, they had comparable levels of motivation and expectation of success prior to the training (**Table 2**).

# Localization Run

Subjective anxiety ratings demonstrated no group difference in average anxiety level at the beginning of the training (p = 0.84, **Figure 4**). Participants were comparable in ability to downregulate anxiety during the initial localization run (p = 0.86, **Figure 4**).

The average coordinates of the individually defined target regions were similar in both groups (max radial distance to intended target coordinates = 5 mm, Supplementary Figure 2, Supplementary Table 1). Average size of dlPFC and insula target regions were well matched (average size 12–15 functional



*QCM, Questionnaire of Current Motivation (fear* = *incompetence fear, challenge* = *perceived challenge, interest* = *level of interest, mastery* = *mastery confidence); 2nd session, willingness to return (Likert scale 1–7).*

voxels, Supplementary Table 1). Analysis of the right insula response showed no significant group difference for average activation (p = 0.58, **Figure 5B**), or ability to down-regulate insula activation levels during initial localization run (p = 0.11, **Figure 5B**). The response in left dlPFC indicated that both groups were highly engaged, as both achieved significant up-regulation of this region during regulate in comparison to watch trials [F(1, 16) = 33.7, p < 0.001, η 2 <sup>p</sup> <sup>=</sup> 0.68, **Figure 5A**]. There was no significant group difference regarding up-regulation (p = 0.35), or average dlPFC activation (p = 0.89).

# Neurofeedback Training

Subjective anxiety ratings demonstrated that both groups were able to regulate anxiety to a certain extent, showing reduced anxiety during regulate trials [up-regulation: F(1, 16) = 33.5, p < 0.001, η 2 <sup>p</sup> <sup>=</sup> <sup>0</sup>.68]. Neurofeedback participants exhibited lower average anxiety levels than the control group, an effect which increased over time as stimuli became more challenging [time<sup>∗</sup> group interaction: F(1, 16) = 8.1, p < 0.05, η 2 <sup>p</sup> <sup>=</sup> <sup>0</sup>.34]. While control participants demonstrated a marked increase in anxiety over time [F(1, 8) = 33.3, p < 0.001, η 2 <sup>p</sup> <sup>=</sup> <sup>0</sup>.81], this increase was attenuated in neurofeedback participants, who showed a non-significant trend [F(1, 8) = 4.5, p = 0.07, **Figure 4**].

Analysis of the imaging data demonstrated that neurofeedback participants in comparison to the control group had significantly lower insula activation levels during regulate trials, but not during watch trials [group<sup>∗</sup> condition interaction: F(1, 16) = 7.8, p < 0.05, η 2 <sup>p</sup> <sup>=</sup> <sup>0</sup>.33; regulate trials: <sup>F</sup>(1, 16) <sup>=</sup> <sup>7</sup>.7, p < 0.05, η 2 <sup>p</sup> = 0.33; watch trials: <sup>F</sup>(1, 16) <sup>=</sup> <sup>3</sup>.2, <sup>p</sup> <sup>=</sup> <sup>0</sup>.09, **Figure 5B**]. Post-hoc within group analysis also demonstrated a significant reduction of insular activation levels over time in neurofeedback participants [F(1, 8) = 7.1, p < 0.05, η 2 <sup>p</sup> <sup>=</sup> <sup>0</sup>.47], but not in control participants (p = 0.33), as in the analysis of subjective ratings. Across participants, insula activation level (single trial betas) and subjective anxiety ratings (single trial ratings) were moderately correlated during both regulate and watch trials (both: r = 0.29, p < 0.01). Finally, analysis of insula activation levels revealed significantly better down-regulation during regulate trials in neurofeedback in comparison to control participants [F(1, 16) = 7.8, p < 0.05, η 2 <sup>p</sup> <sup>=</sup> <sup>0</sup>.33, **Figure 5B**]. Post-hoc within group tests demonstrated that the ability to down-regulate insula activation levels was significant in the neurofeedback group [F(1, 8) = 6.7, p < 0.05, η 2 <sup>p</sup> <sup>=</sup> <sup>0</sup>.46], but not the control group (p = 0.31). The whole-brain analysis further corroborated that neurofeedback participants achieved greater capacity for down-regulation within a network of brain regions involved in anxiety expression, including the right insula (Supplementary Table 2, Supplementary Figure 3). There was no significant group difference for average dlPFC activation level (p = 0.53) or up-regulation in dlPFC during regulate trials (p = 0.52).

# Physiological Data

The physiological data analysis showed no significant difference between groups (breathing: p = 0.36; pulse: p = 0.45), and

no differences in physiology during regulation in comparison to watch trials (breathing: p = 0.26; pulse: p = 0.36; group task interaction: breathing: p = 0.56; pulse: p = 0.46). Average breathing rate of all participants was 18 breaths/min and average pulse rate was 66 beats/min.

# Training Evaluation

Both groups demonstrated high willingness to return for a second session after the training (**Table 2**). While participants from the neurofeedback group felt slightly less comfortable in the scanner than control participants, this difference was not significant (**Table 3**). Both groups reported that the reappraisal instruction facilitated anxiety regulation (**Table 3**). Neurofeedback participants indicated that neurofeedback was useful both in general, as well as specifically for selecting the reappraisal strategy (**Table 3**). While participants in the control group found focusing on the aesthetics of the spider and humanizing most successful, the neurofeedback participants chose emphasizing the spider's powerlessness and humanizing as the two most powerful reappraisal strategies (**Table 4**).

# Follow-up Assessment of Fear

When assessed at follow-up, both groups achieved a significant long-term decrease of spider fear, with group differences being attenuated over time [reduction in spider fear: FSQ: F(1, 16) = 23.0, p < 0.001, η 2 <sup>p</sup> <sup>=</sup> <sup>0</sup>.59, SBQ: <sup>F</sup>(1, 16) <sup>=</sup> <sup>35</sup>.1, <sup>p</sup> <sup>&</sup>lt; <sup>0</sup>.001, <sup>η</sup> 2 <sup>p</sup> <sup>=</sup> 0.690, **Figure 6**]. Importantly, this long-term reduction in spider fear (screening to 3-month follow-up) correlated with the ability to down-regulate insula activation during neurofeedback training (localization run to last experimental run) in neurofeedback participants (FSQ: r = 0.64, p < 0.05; SBQ: r = 0.57, p = 0.05, **Figure 7**) but not in control participants (FSQ: r = 0.26, p = 0.49; SBQ: r = 0.13, p = 0.73). Individual differences in efficiency of regulation of brain activation levels therefore predicted change in individual long-term improvement.

# Discussion

We investigated the effect of fMRI neurofeedback training on brain regions involved in fear processing and symptom reduction in patients with spider phobia. Our results demonstrate that neurofeedback participants exhibited lower levels of anxiety than control participants at the end of training. Second, neurofeedback participants, compared to control participants, achieved downregulation of a region important for anxiety expression (insula), which in turn correlated with improvements in long term anxiety symptoms in these participants.

All participants maintained high prefrontal activation levels during reappraisal, indicating recruitment of regions supporting cognitive reappraisal (Delgado et al., 2008; Ochsner et al., 2012). However, only the neurofeedback group showed a concurrent attenuation of the response in the insula, which grew stronger over time, as expected during successful anxiety regulation (Schienle et al., 2007; Hauner et al., 2012). Decrease of insula activation levels has been shown to be a valid predictor of long term reduction of spider fear (Schienle et al., 2007; Hauner et al., 2012). Neurofeedback participants hence demonstrated the expected modification of brain activation pattern, suggesting the efficiency of cognitive reappraisal strategies for anxiety regulation. Accordingly, achieved attenuation of insula activation levels was accompanied by a reduction of subjective anxiety levels in neurofeedback participants relative to controls. Second, only neurofeedback participants achieved down-regulation of insula activation levels by cognitive reappraisal during regulation in comparison to watch trials. Capacity to down-regulate has been linked to safety learning and successful regulation in

healthy subjects (Delgado et al., 2008; Pollak et al., 2010). Group differences in achieved down-regulation of insula activation levels were not reflected in subjective anxiety ratings, nor physiological control data. A possible explanation is that subjective ratings measured on a five-point Likert scale, as well as heart and breathing rate measured during scanning may not be sensitive enough indicators for capturing subtle differences in regulation success. It has previously been shown that heart rate is not a sensitive measure of anxiety regulation even in much larger samples (Aldao and Mennin, 2012; Cristea et al., 2014), and breathing rate is generally not strongly correlated with anxiety levels (Prigatano and Johnson, 1974; Sarlo et al., 2002). Importantly however, observed individual differences in down-regulation of insula activation levels were predictive of long-term changes in fear. While a sustained group difference in fear could not be shown, individually achieved downregulation of insula predicted fear reduction 3 months after the training. This demonstrates that achieved self-regulation of insula during training was indeed relevant for later behavioral improvement.

Generally, the presented findings add to accumulating evidence that regional changes in brain activation levels can be a

#### TABLE 3 | Training evaluation.


*Participants rated helpfulness of the instruction and neurofeedback, and how comfortable they were in the scanner environment (Likert scale 1–7).*

#### TABLE 4 | Evaluation reappraisal strategies.


*Percentage of participants finding a reappraisal strategy successful ("which would you recommend?"), several options could be named.*

valid indicator of therapeutic change (Schienle et al., 2007; Goldin et al., 2009; New et al., 2009; Hauner et al., 2012; Manber-Ball et al., 2013). Observed group differences could not be attributed to differences in engagement or compliance. Participants showed similar baseline levels of subjective anxiety, right insula response, and left dlPFC response, as well as baseline ability to regulate anxiety. Also, both groups reported equal levels of motivation, and expectation of success prior to the training, reported a high level of comfort in the scanner, high helpfulness of the reappraisal instruction, and indicated a comparable desire to return for future sessions.

fMRI neurofeedback training has been previously conceptualized as a method that combines principles of cognitive-behavioral therapy with brain stimulation approaches (Linden et al., 2012). Within this framework, the advantage of neurofeedback training in comparison to physical brain stimulation is that voluntary self-regulation is a self-controlled process, and therefore more accessible in the long run. The assumed mechanism in neurofeedback training is learning. Feedback is expected to facilitate learning through at least two mechanisms: "explicit representational learning" of the strategy and "implicit reinforcement learning" after successful trials (Goebel et al., 2010; Weiskopf, 2012; Sulzer et al., 2013). Additionally, it has been suggested that learning during neurofeedback training may be enhanced by increasing the individuals' self-efficacy (Sarkheil et al., 2015). The presented data support a facilitating effect of neurofeedback for learning of fear regulation, as group differences emerged gradually and became more pronounced over time. The current study therefore corroborates previous studies showing that healthy participants can learn to self-regulate activation levels in various brain regions (Caria et al., 2012; Weiskopf, 2012), including the insula (Caria et al., 2007, 2010). The presented results also show for the first time that patients with high levels of anxiety can achieve selfregulation of insula activation levels when guided by feedback. Furthermore, our data supports previous research demonstrating that cognitive strategies can be successfully shaped by neural feedback, leading to symptom reduction in chronic pain patients (DeCharms et al., 2005), depressed patients (Linden et al., 2012), and participants with subclinical levels of anxiety (Scheinost et al., 2013). A previous study with patients with subclinical levels of contamination anxiety provided participants with feedback on activation levels of a brain region implicated in anxiety provocation (orbitofrontal cortex) during anxiety regulation. The neurofeedback group achieved a sustained reduction of anxiety in comparison to a sham feedback control group. The presented data further substantiate these results, showing that neurofeedback can enhance learning of anxiety regulation.

In the current study feedback was presented intermittent, using a novel dual feedback display. Intermittent feedback paradigms have been previously applied in healthy participants, but not in patients (Bray et al., 2007; Johnson et al., 2012; Stoeckel et al., 2014). The rationale for presenting dual intermittent neurofeedback was to provide patients with a richer representation of their current brain processing than possible with single region neurofeedback. Different to newly emerging methods for network-based connectivity neurofeedback, which capture the interaction between brain regions (Ruiz et al., 2014; Zilverstand et al., 2014), dual neurofeedback is not a direct measure of brain processing between two select brain regions. It is however a method of maximizing relevant information content of the feedback signal, as it allows to simultaneously target several aspects of a complex behavior through training. While a dual neurofeedback display may be challenging for certain groups of patients, the participants in the presented study reported that the feedback provided was helpful in selecting the reappraisal strategy. The presented results show that the approach is feasible, and may be used in clinical populations. While the current results confirm that behavioral effects can be achieved within a single session of neurofeedback training (Sulzer et al., 2013), patients groups with more severe anxiety disorders may benefit from receiving multiple sessions of training (Scheinost et al., 2013).

A limitation of the current study is the modest sample size. To increase homogeneity of the sample only females with spider phobia were recruited, and the generalization of the results to males remains to be determined. The lack of a sham feedback group may also be seen as a limitation. However, previous research found that sham feedback may induce a negative performance bias, which can limit performance of the control group (Johnson et al., 2012; Stoeckel et al., 2014). A non-neurofeedback control group with blinding of participants therefore seemed the strictest design choice available. The presented data confirmed that motivation and expectation effects were well controlled for.

FIGURE 6 | Spider fear. Long-term development of spider fear, as assessed with the Fear of Spider Questionnaire (A) and the Spider Belief Questionnaire (B), is depicted. While neurofeedback participants

demonstrated less anxiety after scanning (post-fMRI), this group difference was not significant and was washed out during the follow-up period (2-week, 3-month).

In summary, the conducted study provides first evidence that dual intermittent neurofeedback has a facilitating effect on anxiety regulation in spider phobia. Our results support the idea that self-supervising anxiety regulation by neurofeedback is feasible and can facilitate anxiety regulation. We therefore suggest that neurofeedback training may be incorporated as a therapeutic tool in future clinical trials. Because of common cognitive-behavioral trajectories and neurophysiological pathways, we believe that the presented approach could be extended to a broader range of anxiety disorders.

# Acknowledgments

We would like to thank Valentin Kemper and Federico De Martino for their support with optimizing fMRI data acquisition, Michael Lührs, Joel Reithler, and Jan Zimmermann for providing custom data analysis tools, and Scott Moeller for his helpful comments regarding the manuscript. The authors gratefully acknowledge the support of the Brain Gain Smart Mix Program of The Netherlands Ministry of Economic Affairs and The Netherlands Ministry of Education, Culture and Science (grant number: SSM06011) and funding from the European Community's Seventh Framework Programme FP7/2007-2013 under grant agreement numbers 290011 ("ABC" Initial Training Network), 269853 (Advanced Investigator Grant) and 602186 ("BrainTrain" Health-Innovation).

# References


# Supplementary Material

The Supplementary Material for this article can be found online at: http://journal.frontiersin.org/article/10.3389/fnbeh. 2015.00148/abstract

magnetic resonance imaging (fMRI): use of a cluster-size threshold. Magn. Reson. Med. 33, 636–647. doi: 10.1002/mrm.1910330508


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2015 Zilverstand, Sorger, Sarkheil and Goebel. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Learned self-regulation of the lesioned brain with epidural electrocorticography

#### **Alireza Gharabaghi 1,2\*, Georgios Naros 1,2 , Fatemeh Khademi 1,2 , Jessica Jesser 1,2 , Martin Spüler <sup>3</sup> , Armin Walter <sup>3</sup> , Martin Bogdan3,4 , Wolfgang Rosenstiel <sup>3</sup> and Niels Birbaumer 5,6,7**

<sup>1</sup> Division of Functional and Restorative Neurosurgery and Division of Translational Neurosurgery, Department of Neurosurgery, Eberhard Karls University Tuebingen, Tuebingen, Germany


<sup>7</sup> DZD, Eberhard Karls University Tuebingen, Tuebingen, Germany

#### **Edited by:**

Christoph M. Michel, University of Geneva, Switzerland

#### **Reviewed by:**

John H. Gruzelier, Goldsmiths, University of London, UK Tomas Ros, University of Geneva, Switzerland

#### **\*Correspondence:**

Alireza Gharabaghi, Division of Functional and Restorative Neurosurgery and Division of Translational Neurosurgery, Department of Neurosurgery, Eberhard Karls University Tuebingen, Otfried-Mueller-Str. 45, 72076 Tuebingen, Germany e-mail: alireza.gharabaghi@unituebingen.de

**Introduction**: Different techniques for neurofeedback of voluntary brain activations are currently being explored for clinical application in brain disorders. One of the most frequently used approaches is the self-regulation of oscillatory signals recorded with electroencephalography (EEG). Many patients are, however, unable to achieve sufficient voluntary control of brain activity. This could be due to the specific anatomical and physiological changes of the patient's brain after the lesion, as well as to methodological issues related to the technique chosen for recording brain signals.

**Methods**: A patient with an extended ischemic lesion of the cortex did not gain volitional control of sensorimotor oscillations when using a standard EEG-based approach. We provided him with neurofeedback of his brain activity from the epidural space by electrocorticography (ECoG).

**Results**: Ipsilesional epidural recordings of field potentials facilitated self-regulation of brain oscillations in an online closed-loop paradigm and allowed reliable neurofeedback training for a period of 4 weeks.

**Conclusion**: Epidural implants may decode and train brain activity even when the cortical physiology is distorted following severe brain injury. Such practice would allow for reinforcement learning of preserved neural networks and may well provide restorative tools for those patients who are severely afflicted.

#### **Keywords: electrocorticography, neuroprosthetics, epidural implant, brain-machine interface, neurofeedback, cortical lesion, stroke**

#### **INTRODUCTION**

Specific feedback and reward of brain activity allows learning of self-regulation strategies. Operant conditioning of electroencephalography (EEG) and of blood-oxygen-leveldependent (BOLD) signal activity has been applied to reduce disorder-specific symptoms in a variety of neurological and neuropsychiatric conditions (Wyckhoff and Birbaumer, 2014). When neurofeedback is coupled to external devices such as brain-machine interfaces (BMI), the volitional control of brain activity can often be attained, opening up novel training opportunities for the very severely brain-injured and even paralyzed (Buch et al., 2008, 2012; Ang et al., 2011, 2014; Gomez-Rodriguez et al., 2011; Ramos-Murguialday et al., 2012, 2013); first results using EEG-based BMI were promising (Ang et al., 2011, 2014; Ramos-Murguialday et al., 2013). Some -even healthy- participants, however, fail to achieve volitional control of brain activity (Vidaurre and Blankertz, 2010) because of subject-specific anatomical (Halder et al., 2011; Buch et al., 2012; Várkuti et al., 2013) and physiological (Blankertz et al., 2010; Grosse-Wentrup et al., 2011; Vukeli´c et al., 2014) limitations of the brain, or methodological issues of brain signal recording (Leuthardt et al., 2009). In the context of rehabilitation, additional neurophysiological considerations might contribute to limitations of EEG-based BMI: previous approaches have chosen those frequency bands and algorithms which differentiated best between "motor imagery" and "rest", e.g., the mu/alphaband and/or modified common spatial filter algorithms to optimize the selection of temporo-spatial discriminative EEG characteristics (Buch et al., 2008, 2012; Ang et al., 2011, 2014; Ramos-Murguialday et al., 2013). Although even larger groups of stroke patients have participated in BMI training with this approach, a more restricted feature space, e.g., perturbations in the beta-band over selected sensorimotor electrode contacts, might be preferred as a reinforced therapeutic target for restorative purposes (Gharabaghi et al., 2014a,b), despite the fact that they might be less optimal from classification purposes, e.g., to differentiate movement-related brain states in stroke patients (Gomez-Rodriguez et al., 2011; Rossiter et al., 2014).

In general, EEG-based approaches have a characteristically low spatial resolution and a low signal-to-noise ratio because of signal attenuation caused by the skull, possible contamination by muscle artifacts and external electrical activity. These approaches might therefore be specifically challenged in cases of an intentionally limited feature space due to therapeutic purposes. Moreover, they often require a relatively long period of training before subjects can gain real-time control of devices (Birbaumer et al., 1999; Leuthardt et al., 2009; Gharabaghi et al., 2014b).

By contrast, electrocorticographic (ECoG) neurofeedback approaches may be able to surmount such difficulties thanks to their proximity to the neural signal source. We recently proposed a new approach which is less invasive than the classical implanted approaches with *subdural* grids (Yanagisawa et al., 2011, 2012; Wang et al., 2013) or even brain penetrating electrodes (Hochberg et al., 2012; Collinger et al., 2013). This novel approach entailed the application of *epidural* ECoG to decode volitional brain activity in patients with locked-in syndrome suffering from amyotrophic lateral sclerosis (Bensch et al., 2014), with chronic pain as a result of upper limb amputation (Gharabaghi et al., 2014c), and with hemiparesis following subcortical hemorrhagic stroke (Gharabaghi et al., 2014b). In all of these cases, however, most of the cortical tissue of the patients was preserved.

Essential questions with regard to the clinical usefulness of implantable brain-computer interfaces based on epidural ECoG remain unanswered. For instance, would this technique also be applicable in patients with extended cortical lesions? Are these patients able to learn consistent online-control of brain activity? Would high intensity neurofeedback training in these patients be possible? Would ECoG neurofeedback be applicable in patients who are not using volitional control of their brain oscillations with a standard EEG-based approach?

We therefore investigated a brain-machine interface based on epidural ECoG and examined its practicability for neurofeedback training in a patient with an extended ischemic lesion of motor cortical areas who did otherwise not adequately engage in voluntary modulation of brain activity based on EEG recordings.

# **METHODS**

#### **PATIENT**

The patient, a 52-year-old man, had suffered an ischemic stroke of the right hemisphere with extended cortical lesions (see **Figure 1**) 13 years prior to implantation. This caused a persistent severe hemiparesis and he no longer had control of his left upper extremity (Medical Research Council motor scale < 2).

Several months before surgery, the patient underwent twenty sessions of EEG-based BMI neurofeedback similar to the training described earlier (Ramos-Murguialday et al., 2012; Vukeli´c et al., 2014) with the same study design that was later used for ECoG-based BMI neurofeedback (see Section Experimental Procedure and **Figure 2**). Offline evaluation of the EEG data revealed artifacts in the recorded brain signals induced by muscle contraction, i.e., showing EEG amplitudes which exceeded the mean cortical activity by at least two standard deviations. For each feedback electrode (FC4, C4 and CP4) we calculated, separately for the "move" and "rest" period of each trial, the percentage of artifacted samples per session and compared their evolution over time with the respective BMI performance evaluated by the area under the recipient operating characteristics curve (AUC).

Several months later, the patient participated in a different, long-term study for motor cortex stimulation with epidural implants simultaneously with rehabilitation training to improve upper limb motor function following the stroke. The study protocol, approved by the ethics committee of the Medical Faculty of the University of Tuebingen, also involved a fourweek evaluation period immediately subsequent to implantation, with electrodes externalized with percutaneous extensions to assess the patient's cortical physiology for optimization of stimulation. The data shown below is derived from this period.

Following implantation of the electrode array, i.e., several months after the preoperative evaluation with EEG, the patient was subjected to several different experiments for parameter selection and optimization of motor cortex stimulation (not part of the present report) which included altogether 30 ECoG-based neurofeedback sessions with a mean of ∼108 feedback trials per session. Due to their heterogeneity these sessions are not suited to evaluate the evolution of BMI performance during this period, however, they may serve as a valuable source of information for estimating the influence of muscle artifacts, which were visually detected during offline analysis, and the feasibility and reliability of ECoG-based neurofeedback.

# **EPIDURAL ELECTROCORTICOGRAPHY**

The epidurally implanted 4 × 4 electrode array consisted of four electrode leads for chronic application (Resume II, Medtronic, Minneapolis, USA) with four platinum iridium electrode contacts, each (4 mm diameter, 10 mm center-to-center distance) covering parts of the right primary motor, somatosensory cortex and premotor cortex. During the evaluation period, the electrode grid was externalized with percutaneous extensions which were connected to a recording and processing unit and a robotic hand orthosis. A monopolar amplifier (BrainAmp MR plus, BrainProducts, Munich, Germany) with 1 kHz sampling rate and a high-pass filter (cutoff frequency at 0.16 Hz) and a lowpass filter (cutoff frequency at 1000 Hz) was used for ECoG recording. Online processing of brain signals was performed using the BCI 2000 framework (Schalk et al., 2004) extended with custom-built features to control an electromechanical hand orthosis (Amadeo, Tyromotion GmbH, Graz, Austria). The data was collected batch-wise, i.e., every 40 ms, the recording computer received a batch of data that contained 40 samples per channel (Walter et al., 2012; Gharabaghi et al., 2014a). The reference electrode was chosen from the contacts on the somato-sensory

**FIGURE 1 | Lesion mask: Normalized lesion mask displayed on MNI (Montreal neurological institute) brain in standard space (Fonov et al., 2009)**.

cortex, i.e., medio-posterior or latero-posterior corner of the grid.

#### **EXPERIMENTAL PROCEDURE**

We used closed-loop, orthosis-assisted opening of the paralyzed left hand which was triggered online by ipsilesional oscillatory brain activity during cued kinesthetic motor imagery of hand opening (Walter et al., 2012; Gharabaghi et al., 2014a). Each session contained 4–16 runs (average 10.86 ± 4.5 runs). Each of the runs had a duration of circa 3 min and consisted of 11 trials. Each trial began with a preparation phase of 2 s, followed by a 6 s movement imagination phase and an 8 s rest phase (see **Figure 3**). Preparation, imagination and rest phases were instigated by a recorded female voice that gave the commands "left hand", "go" and "rest" respectively.

A hand orthosis passively opened the affixed left hand as soon as motor imagery-related event-related desynchronization (ERD) in the beta-band (17–23 Hz) was identified during the movement imagination phase. An epoch was regarded as ERD-positive only when the output of the classifier exceeded a threshold. The latter and the electrode selection were determined individually from three training runs before the test sessions (Walter et al., 2012; Gharabaghi et al., 2014a). The spectral power was calculated using an autoregressive model with an order of 16 (McFarland and Wolpaw, 2008) over a normalized 500 ms sliding window shifting every 40 ms. In order to sidestep a noisy control signal

**FIGURE 3 | Lesion size in percentage of affected cortical AAL (=automated anatomical labeling) region (Tzourio-Mazoyer et al., 2002): Affected cortical regions are named according to the AAL brain atlas labels: PreCG = precentral gyrus, IFGoperc = pars opercularis of inferior frontal gyrus, MFG = middle frontal gyrus, PoCG = postcentral gyrus, SFGdor = superior frontal gyrus, dorsolateral, IFGtriang = pars triangularis of inferior frontal gyrus, SMA = supplementary motor area, IPL = inferior parietal lobule, SMG = supramarginal gyrus**.

for the orthosis, i.e., giving robust and harmonic feedback, we initiated or discontinued orthosis-assisted movement only when five consecutive 40 ms epochs (i.e., 200 ms) where classified as ERD-positive or negative, respectively.

#### **PERFORMANCE EVALUATION**

To determine the patient's ability to modulate his brain activity contingent on the BMI feedback task, we determined the percentage of trials with orthosis movement (i.e., ERD) and the average time with orthosis movement (i.e., ERD) divided by the total feedback duration phase (Gharabaghi et al., 2014b,c).

We also measured a baseline condition to supervise spontaneous perturbations of brain activity which could cause fluctuations in the online performance during the feedback task, i.e., could start the orthosis movement independent of motorimagery. This baseline condition entailed several ECoG recordings which were taken while the patient rested, i.e., one run with eyes open and one run with eyes closed before each session throughout the whole study period. All in all, we recorded approximately 20 min of such spontaneous baseline ECoG activity for offline analysis, segmented it into trials of the same structure and processed it in the same way as in the feedback sessions (Gharabaghi et al., 2014b,c). For statistical analysis, we used the Matlab toolbox (Wilcoxon rank-sum test) to compare the distribution of performance values per run in each feedback session with the distribution of performance values for the baseline data.

#### **IMAGING EVALUATION**

Before implantation magnetic resonance imaging (MRI) was performed on a 3.0-Tesla Siemens Trio Scanner (TR 1.95 s, TE 2.26 ms, 176 slices of 1 mm slice thickness). For lesion segmentation MRIcron software<sup>1</sup> was used to manually delineate the lesion. The anatomical image and the mask were normalized to MNI space using SPM 8 (Statistical Parametric Mapping, The Wellcome Department of Imaging Neuroscience, Institute of Neurology, University College London, UK). The overlap of the Automated Anatomical Labeling (AAL) atlas regions and the normalized lesion mask were calculated.

## **RESULTS**

Lesion segmentation revealed that extended parts of the right hemisphere were affected by the stroke, in particular the primary motor and somatosensory cortex with 45% and 33% lesion size and higher motor areas with 35% (middle frontal gyrus) and 22% (superior frontal gyrus) lesion size with respect to the AAL atlas. The basal ganglia were not affected by the lesion (**Figure 3**).

EEG analysis of the non-invasive training showed a systematic change of the number of muscle artifacts. In the course of the training, there was an increase of artifacted samples in the "rest" period of each trial and a decrease in the respective "move" periods. The patient learned to increase and decrease muscle tension in the rest period and in the move period of each trial, respectively (see **Figures 4A,B**).

These changes correlated significantly (*p* < 0.05) with the BMI performance for all channels and both conditions (rest and move), i.e., channel FC4 *r* = 0.8905 for rest and *r* = −0.8254 for move; channel C4: *r* = 0.7045 for rest and *r* = −0.8447 for move; channel CP4: *r* = 0.8878 for rest and *r* = −0.8386 for move (Pearsons correlation coefficient). As a result of the increasing difference between the rest and move condition, there was an increase of BMI control (see **Figure 4B**), i.e., the increased baseline activity in "rest" made it easier to reach the desynchronization threshold in the "move" period for controlling the BMI. Thus, the patient did not volitionally control his oscillatory brain activity for the neurofeedback training.

In contrast, ECoG analysis of the implant based training showed no systematic change in the number of muscle artifacts. Due to the low distance of the two recording channels, the number of artifacted samples was identical. In the course of the training, there was a fluctuating amount of artifacted samples both in the "rest" period and in the "move". Similar to the EEG experiment there were more artifacts in the rest period, but showed no evolution over time. Thus, although muscle tension was not completely eliminated, it did not influence the volitional control of oscillatory brain activity (see **Figure 5**).

Accordingly, in the ECoG-based approach, the patient modulated his motor-imagery related ERD contingent on the BMI feedback task, i.e., initiated the orthosis movement in a mean of 90.49 ± 13.73% of all trials (baseline condition: 32.72 ± 9.77%), thus retaining significant control of brain activity throughout the whole study period (see **Figure 6**).

In fact, he controlled the orthosis movement (i.e., ERD) for a mean of 37.15 ± 15.27% of the feedback duration in each trial. Thus, his performance in this online closed-loop paradigm was constant and significantly higher than in the baseline condition (14.52 ± 7.30%) throughout the study period (see **Figure 7**).

# **DISCUSSION**

The patient presented here—with an extended ischemic lesion of the cortex—learned control of high intensity neurofeedback training based on self-regulation of brain oscillations recorded from the epidural space by ECoG. Although the ECoG based approach enabled the patient to maintain consistent control of his sensorimotor rhythms in the beta-band in an online closed-loop paradigm throughout the study period, his performance in controlling the neurofeedback device in ∼30–40% of the feedback duration was—while significantly better than baseline (∼15%)—nonetheless markedly lower than comparable ECoG-based (Gharabaghi et al., 2014b) or EEG-based (Ramos-Murguialday et al., 2013) approaches in other similarly affected patients who had attained control rates of ∼50–60% of the feedback duration. These variations in performance might be explained by physiological and morphological differences: The respective patients showed strikingly different baseline conditions, i.e., spontaneous perturbations of brain activity in the betaband could start the orthosis movement independent of motorimagery during ∼15% vs. ∼30% of the feedback period in the present and in previous cases (e.g., Gharabaghi et al., 2014b), respectively. These physiological baseline differences could be explained by the different lesion characteristics, namely extended cortical vs. circumscribed subcortical lesions, respectively. Since this brain activity is known to originate from primary motor and

<sup>1</sup>http://www.mccauslandcenter.sc.edu/mricro/mricron/install.html

before grid implantation: Green and red lines indicate "Go" and "Rest" cues during each trial, respectively. Arrows highlight muscle artifacts during the run. From the first to the last session the number of the artifacts in the rest period of each trial increased. **(B)** Percentage of artifacted samples during the

the course of twenty sessions. As a result of the increasing difference of artifacts in the rest and the move condition, there was an increase of BCI control measured by the area under the recipient operating characteristics curve (AUC).

somatosensory as well as from secondary motor areas, the most plausible explanation for the decrease of spontaneous perturbations in the presented case is that they have been affected by the lesion. Our results are in line with recent findings that movementrelated beta desynchronization in the contralateral primary motor cortex was found to be significantly reduced in stroke patients compared to controls, while within this patient group, smaller desynchronization has been seen in those with more motor impairment (Rossiter et al., 2014). Moreover, these observations support our general strategy, applied in the present case as well, to choose beta-band desynchronisation as a therapeutic target for restorative interventions in severely affected stroke patients (Gharabaghi et al., 2014a,b).

An intriguing insight gained in this study was that the epidural ECoG technique enabled the patient to engage in feedback exercises based on voluntary modulation of brain activity despite the fact that he did otherwise not use properly a standard EEG-based approach. Interestingly enough, prior to using the implanted brain interface, the patient learned to increase and decrease muscle tension in the rest period and in the move period of each trial, respectively, for BMI control. This alternative conditioning probably occurred because the extent of his **course of thirty sessions**.

own voluntary modulation of brain activity was too insignificant to be detected by EEG whereas the muscle contractions could sufficiently be detected and were reinforced by feedback and reward. This alternative control strategy applied by the patient was unexpected. The participants in this study and in previous studies with healthy subjects (Vukeli´c et al., 2014) and similarly severely affected stroke patients (Ramos-Murguialday et al., 2012) were instructed to avoid blinking, chewing, head and body compensation movements. Along with visual inspection and feedback by an experienced examiner this approach proved to be a sufficient method to prevent alternative BMI control in the past. Moreover, the examiners were prepared to detect compensatory movements *during* the "move" phase of the feedback task as this is the most commonly observed strategy to pretend volitional modulation of ERD, and not *before* the actual task in the "rest" phase. Therefore, increasing baseline activity in "rest" through elevated muscle tension and concurrent reduced muscle tension in the "move" period, have in future to be considered as subtle bypassing strategies to reach the desynchronization threshold more easily.

For this purpose, online detection of EMG contamination with dedicated spectral and topographical analyses might be necessary to prevent alternative BMI control in future. Previous work in this field was conducted without such precautions most probably due to the fact that lower frequency bands were applied for BMI control, which are usually less affected by muscle artifacts (Goncharova et al., 2003). However, due to their relevance for sensorimotor control (Kilavik et al., 2013; Brittain et al., 2014), motor learning (Herrojo Ruiz et al., 2014) and corticospinal excitability (Takemi et al., 2013) as well as due to their correlation with the extent of functional impairments after stroke (Rossiter et al., 2014), higher frequency bands in the beta

range might be considered in future more often as therapeutic targets for restorative EEG neurofeedback and motor rehabilitation (Gharabaghi et al., 2014a,b), necessitating the consideration of even subtle EMG contamination as observed in the presented case. EMG artifact detection may include relatively simple methods such as rejection of EEG segments that exceed a predefined amplitude threshold or more sophisticated methods such as factor decomposition using principal component or independent component analysis with or without source reconstruction algorithms (Goncharova et al., 2003; Hipp and Siegel, 2013). In any case,

applicable approaches need to work even with only few available channels within a narrow frequency band and have to provide real time processing and low computational complexity (Tiganj et al., 2010).

Should EMG artifacts turn out to be too difficult to mitigate (yet not explicitly addressed by this study) or should the targeted physiological brain state, e.g., motor imagery-related beta-band desynchronisation, be too weak to be robustly detected in the EEG of severely affected stroke patients, implantable approaches might provide an alternative. In this context, the ECoG approach has two advantages over EEG: On account of its proximity to the neural signal source, it surmounts difficulties related to signal attenuation caused by the skull. It is also less susceptible to contamination by muscle artifacts, and, in this case, benefits from the signal attenuation caused by the skull. In this vein, simultaneously recorded ECoG and EEG activity in motor cortical areas revealed that invasively measured signals had a twenty to hundred times better brain signal quality than signals that were acquired noninvasively (Ball et al., 2009).

The technique presented here is limited by the necessity to connect the intracranial implant to an external online processing framework for recording and neurofeedback training via extension leads which are externalized through the skin (Gharabaghi et al., 2014b,c). Future applications of this brain selfregulation approach will require wireless devices capable of fast and reliable information transfer (Borton et al., 2013; Piangerelli et al., 2014). This would facilitate the application of this intervention on a day-patient basis or even in the patient's home environment.

However, before drawing definite conclusions regarding effectiveness of various neurofeedback approaches, future studies need to directly compare ECoG-based techniques to EEG-based methods which control for EMG artifacts. This research needs to consider further aspects such as direct and indirect costs, complications, learning curve, motivation, applicability for longterm use and the possibility of performing training independent of professional support. Based on the respective findings, patients with different impairment levels might then be referred to the specific treatment modality best suited for the individual pathophysiological state.

In conclusion, epidural implants could provide reliable feedback interfaces for brain self-regulation in patients in whom noninvasive approaches fail on account of signal attenuation caused by the skull or due to the underlying pathophysiology. This could establish them as valuable tools in the context of reinforcement learning in a variety of neurological and neuropsychiatric conditions.

# **ACKNOWLEDGMENTS**

Alireza Gharabaghi is supported by grants from the German Research Council, Deutsche Forschungsgemeinschaft [DFG GH 94/2-1, DFG EC 307], and from the Federal Ministry for Education and Research [BFNT 01GQ0761, BMBF 16SV3783, BMBF 03160064B, BMBF V4UKF014]. Fatemeh Khademi is supported by the Graduate Training Centre of Neuroscience, International Max Planck Research School, Tuebingen, Germany. Niels Birbaumer is supported by the Deutsche Forschungsgemeinschaft (Reinhard Koselleck Project) and by Brain Products GmbH, Munich, Germany. We thank Ramin Azodi Avval for his support with the figures and Dr. Mathias Vukeli´c and Vladislav Royter for fruitful discussions.

#### **REFERENCES**


**Conflict of Interest Statement**: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 30 June 2014; accepted: 24 November 2014; published online: 09 December 2014*.

*Citation: Gharabaghi A, Naros G, Khademi F, Jesser J, Spüler M, Walter A, Bogdan M, Rosenstiel W and Birbaumer N (2014) Learned self-regulation of the lesioned brain with epidural electrocorticography. Front. Behav. Neurosci. 8:429. doi: 10.3389/fnbeh.2014.00429*

*This article was submitted to the journal Frontiers in Behavioral Neuroscience*.

*Copyright © 2014 Gharabaghi, Naros, Khademi, Jesser, Spüler, Walter, Bogdan, Rosenstiel and Birbaumer. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution and reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms*.

# A subject-independent pattern-based Brain-Computer Interface

Andreas M. Ray 1 †, Ranganatha Sitaram1, 2, 3 \* † , Mohit Rana1, 4, Emanuele Pasqualotto<sup>5</sup> , Korhan Buyukturkoglu<sup>1</sup> , Cuntai Guan<sup>6</sup> , Kai-Keng Ang<sup>6</sup> , Cristián Tejos <sup>7</sup> , Francisco Zamorano8, 9, Francisco Aboitiz <sup>10</sup>, Niels Birbaumer 1, 11 and Sergio Ruiz 1, 10 \*

1 Institute of Medical Psychology and Behavioral Neurobiology, Medical Faculty, University of Tübingen, Tübingen, Germany, 2 Institute for Medical and Biological Engineering, Schools of Engineering, Medicine and Biology, Pontificia Universidad Católica de Chile, Santiago de Chile, Chile, <sup>3</sup> Department of Psychiatry and Section of Neuroscience, School of Medicine, Pontificia Universidad Católica de Chile, Santiago de Chile, Chile, <sup>4</sup> Graduate School of Neural and Behavioral Sciences, University of Tübingen, Tübingen, Germany, <sup>5</sup> Institut de Recherche en Sciences Psychologiques, Université Catholique de Louvain, Louvain-la-Neuve, Belgium, <sup>6</sup> Neural and Biomedical Technology Department, Institute for Infocomm Research, Singapore, Singapore, <sup>7</sup> Department of Electrical Engineering and Biomedical Imaging Center, Pontificia Universidad Católica de Chile, Santiago, Chile, <sup>8</sup> División de Neurociencia, Centro de Investigación en Complejidad Social, Facultad de Gobierno, Universidad del Desarrollo, Santiago, Chile, <sup>9</sup> Unidad de Imágenes Avanzadas, Clínica Alemana, Universidad del Desarrollo, Santiago, Chile, <sup>10</sup> Departamento de Psiquiatría, Centro Interdisciplinario de Neurociencia, Escuela de Medicina, Pontificia Universidad Católica de Chile, Santiago, Chile, <sup>11</sup> Ospedale San Camillo, Istituto di Ricovero e Cura a Carattere Scientifico, Venezia, Italy

While earlier Brain-Computer Interface (BCI) studies have mostly focused on modulating specific brain regions or signals, new developments in pattern classification of brain states are enabling real-time decoding and modulation of an entire functional network. The present study proposes a new method for real-time pattern classification and neurofeedback of brain states from electroencephalographic (EEG) signals. It involves the creation of a fused classification model based on the method of Common Spatial Patterns (CSPs) from data of several healthy individuals. The subject-independent model is then used to classify EEG data in real-time and provide feedback to new individuals. In a series of offline experiments involving training and testing of the classifier with individual data from 27 healthy subjects, a mean classification accuracy of 75.30% was achieved, demonstrating that the classification system at hand can reliably decode two types of imagery used in our experiments, i.e., happy emotional imagery and motor imagery. In a subsequent experiment it is shown that the classifier can be used to provide neurofeedback to new subjects, and that these subjects learn to "match" their brain pattern to that of the fused classification model in a few days of neurofeedback training. This finding can have important implications for future studies on neurofeedback and its clinical applications on neuropsychiatric disorders.

Keywords: neurofeedback, BCI, subject-independent classification, emotion imagery, common spatial patterns

# Introduction

A variety of studies using Brain-Computer Interfaces (BCI) and neurofeedback have demonstrated that individuals can be trained to gain control of different brain signals. Researchers have anticipated that long-term BCI training may lead to neuroplastic changes, potentially opening up new treatment approaches for certain psychiatric disorders, e.g., depressive disorders,

#### Edited by:

Carmen Sandi, École Polytechnique Fédérale de Lausanne, Switzerland

#### Reviewed by:

Yijun Wang, University of California, San Diego, USA Serafeim Perdikis, École Polytechnique Fédérale de Lausanne, Switzerland

#### \*Correspondence:

Ranganatha Sitaram, Institute for Medical and Biological Engineering, School of Engineering, Pontificia Universidad Católica de Chile, Ingeniería Biológica y Médica, Vicuña Mackenna 4860, Hernán Briones, Piso 2, Macul 782-0436, Santiago, Chile rasitaram@uc.cl Sergio Ruiz, Departamento de Psiquiatría, Centro Interdisciplinario de Neurociencia, Escuela de Medicina, Pontificia Universidad Católica de Chile, Street Marcoleta 391, 2 Piso, 32349 Santiago, Chile sruiz@uc.cl

† These authors have contributed equally to this work.

Received: 08 November 2014 Accepted: 22 September 2015 Published: 20 October 2015

#### Citation:

Ray AM, Sitaram R, Rana M, Pasqualotto E, Buyukturkoglu K, Guan C, Ang K-K, Tejos C, Zamorano F, Aboitiz F, Birbaumer N and Ruiz S (2015) A subject-independent pattern-based Brain-Computer Interface. Front. Behav. Neurosci. 9:269. doi: 10.3389/fnbeh.2015.00269 schizophrenia, and attention deficit hyperactivity disorder (Strehl et al., 2006; Choi et al., 2011; Linden et al., 2012; Ruiz et al., 2013a,b; Young et al., 2014). Many of these studies are based on the idea that patients can be trained to correct their abnormal brain activation to produce healthy brain activation, aided by the feedback of their own brain activity.

While earlier BCI studies have mostly focused on modulating specific brain regions or signals, new developments in pattern classification of brain states are enabling real-time decoding and modulation of an entire functional network (Cox and Savoy, 2003; Peltier et al., 2009; Hollmann et al., 2011; Laconte, 2011; Sitaram et al., 2011; Rana et al., 2013; Sato et al., 2013; Niazi et al., 2014; Ruiz et al., 2014b). However, a major methodological concern arises from these approaches: the prior studies have focused on building pattern classifiers to decode subject-specific brain patterns, and it is not clear if a general approach could be developed such that a classifier trained on brain signals from a group of individuals could be used to distinguish between any two given specified brain states. Progress in this type of classifier using a fused model, created by combining data from several subjects, hereafter called subject-independent pattern classifier, is increasingly considered to be necessary. The goal is to decode, in real-time, from an individual's brain signals without having to first train the pattern classifier on his subject-specific data (Rana et al., 2013; Ruiz et al., 2014b). One potential application of such a technique involves training neuropsychiatric patients to correct and reverse their abnormal patterns of brain activity: instead of providing feedback of their own brain activity (purportedly abnormal due to the presence of an active neuropsychiatry disorder), future neurofeedback experiments could reinforce patients whenever they are able to emulate patterns of activity similar to those of healthy brains. A system that is capable of providing neurofeedback in this manner has to fulfill two basic requirements: (1) The feature extraction method must consider information from spatial, temporal and spectral domains to be able to encapsulate the entire functional network. (2) The feature extraction method has to be generalizable to multiple subjects in order to be able to construct a fused classification model.

It is to be noted that there have been a few published reports of subject-independent pattern classification, although they have been limited to event related potentials and were not demonstrated to be generalized to other brain states. A previous attempt to create a subject-independent classification model from P300 event related potentials for a BCI spelling application was reported in Lu et al. (2009). More recently, similar results were reported by Jin et al. (2013) and Kindermans et al. (2014). The above studies aimed to build a subject-wide model of ERP signals to minimize calibration time for new users of the BCI speller application. Although the results were promising, the reported methods were limited to event related signals with very specific spatial and temporal characteristics which might not be suitable for a more general application of classifying between any two arbitrary brain states.

Another study by Fazli et al. (2009) used Common Spatial Patterns (CSP) to extract features from the EEG data of eventrelated synchronization and desynchronization during motor imagery. Multiple linear classifier were combined to form a merged classifier that enabled classification on new data without calibration.

These reports show that group-based, real-time classification on signals with well-known characteristics is possible. However, in a paradigm that reinforces brain-pattern matching by enabling group-based neurofeedback of more general patterns of activity in functional networks, the characteristics of the signal in terms of spatial and spectral distribution are not always know in advance. However, the fact that CSP is a data-dependent method makes it an interesting candidate for such an approach (Krusienski et al., 2012). CSP constructs spatial filters from the data that, when projected, identify important parts of the data, i.e., regions on the scalp that contain information for the discrimination of two classes (see Koles et al., 1990; Koles, 1991; Müller-Gerking et al., 1999; Ramoser et al., 2000; Blankertz et al., 2008). The above-mentioned study by Fazli et al. shows that CSP can be used in a group-based setting, however, the efficacy was only shown in offline tests. An example for the use of the CSP method in a single-trial BCI setting is given in Guger (Guger et al., 2000). Here, motor imagery data is used to train subject-specific classifiers based on CSP, and were also successfully tested in real-time experiments.

Toward our long-term goal of developing a BCI for training neuropsychiatric patients to produce brain-patterns similar to those of healthy subjects as a treatment approach, two intermediate steps have to be taken. The first step is designing and building a technically reliable system that fulfills all the technical requirements. Secondly, to iron-out all technical issues, we plan to test the system at this stage only in a healthy participant population. The current study is targeted to achieve the above two aims, by demonstrating that a subject-independent Common Spatial Pattern classifier can be used in a neurofeedback paradigm for brain pattern matching.

As per our aims, the current study is divided into two experimental stages (**Figure 1**). In the first set of experiments, a subject-independent classifier was trained from EEG data collected from a group of healthy individuals who were instructed to perform positive emotional imagery and motor imagery. The motor imagery task was chosen as a contrasting brain state to the positive emotional state in a two-class Support Vector Machine (SVM) classifier. For training the two-class SVM classifier, EEG features were obtained by spatially filtering the band-separated EEG signals by the method of CSPs (Ang et al., 2012).The CSPs of the trained classifier represented the healthy brain patterns pertaining to the two states, i.e., positive emotion and motor imagery.

Once the classifier was proven to be feasible for robust classification of positive emotion and motor imagery in different participants, a second stage experiment tested whether the classifier could decode brain states and provide feedback in real-time. Five new individuals participated in a neurofeedback experiment, in which they were trained to replicate the common brain states of the participants of the first experiment, aided by the feedback provided by the subject-independent classifier.

The result of this study shows that the classification of these brain states can be performed in real-time and used as a

neurofeedback system, and demonstrates the technical feasibility of the subject-independent pattern classification approach.

# Materials and Methods

# Real-time Pattern Classification

A real-time pattern classifier is constructed in two stages: An offline stage in which the classifier is trained on previously recorded data, and an online stage in which the trained classifier is used to provide feedback to the participant in real-time. In the offline stage, a classification model is trained by extracting relevant features from the data. These features capture the most discriminative characteristics of the different classes of data. In the online part, the model is applied to new data in real-time. The class-label that is predicted by the classifier at every time point is converted to feedback by the neurofeedback system.

In our implementation, the classification system is built to discriminate between two classes using trial-by-trial EEG data. The approach is based on the Filter Bank Common Spatial Patterns algorithm (Ang et al., 2008, 2012; **Figure 2**). In the first step, a filter bank is created by repeatedly band-pass filtering the raw EEG data using Chebyshev type 2 filters. In the next step, the data in each frequency band is spatially filtered by the method of CSP. This procedure yields features for each of the bands. These features are finally evaluated by computing their mutual information with the label of the data from each trial, i.e., the class-label, and thus the most discriminative pairs of CSP and frequency bands are selected. The selected features are then used for the classification. A more detailed description of this method is given in the following sections.

# Feature Extraction

The algorithm of CSP can be understood as a method that creates weight maps of the channels of the EEG signal. The weight maps reflect the importance of the signal content of the channels for separating the conditions encoded in the data (Blankertz et al., 2008). The maps are essentially spatial filters that are projected onto data. By projection of these filters, the data is transformed to maximize the ratio of the variance of the EEG amplitudes between the two conditions. Therefore, the variance of the filtered signal can be used as a discriminative feature for a classification task.

The approach is based on the simultaneous diagonalization of two matrices as described in Fukunaga (1972). CSP has been used for feature extraction in EEG data classification for the first time in a study by Koles et al. (1990), and in several studies that followed (e.g., Müller-Gerking et al., 1999; Ang et al., 2008, 2012).

The decomposition of one trial of EEG data can be described as,

$$Z = P^T E.$$

Where E represents a single trial of band-pass filtered EEG time series. Z denotes the EEG time series E after spatial filtering by the CSP projection matrix P T . This projection matrix is computed by solving the Eigen-decomposition problem,

$$\mathbf{S}\_{\mathbf{x}} = U\psi\_{\mathbf{x}}U^T; \mathbf{x} \in \{a, b\},$$

where S<sup>x</sup> originates from transformations of the sample covariance matrices of the trials of the two classes in the EEG data. U is the matrix of eigenvectors of S<sup>x</sup> and ψ<sup>x</sup> are

the eigenvalues. The index x may be substituted by a and b, representing the labels of the two classes. A brief explanation of the transformations is given below; a formal description can be found in Fukunaga (1972). Koles and Lazar (Koles et al., 1990) describe the use of this approach in an experimental setting with EEG data.

For obtaining S<sup>x</sup> the sample covariance matrix of the data of each condition is calculated, normalized with respect to its trace and averaged over trials. These matrices are combined to form a composite covariance matrix that is factored into its eigenvectors. These eigenvectors are used to formulate a whitening transformation that renders the composite covariance matrix isotropic. The same transformation is applied to the individual trial-averaged, normalized sample covariance matrices of the EEG data of each class. It has been shown that after this application the two transformed matrices share the same eigenvectors, and the sum of their eigenvalues λ is 1 (Fukunaga, 1972).

Using the eigenvectors, the projection matrix can be computed as,

P = UW

where W denotes the aforementioned whitening matrix. The first and last m rows of the projection matrix P are used to decompose the EEG into CSP.

After that the classification features F are extracted from the decomposed EEG time series Z by computing and normalizing its variances by the first m and last m rows of Z,

$$F = \log\left(\frac{\nu ar\,(Z)}{\sum\_{i=1}^{2m} \nu ar\,(Z)}\right).$$

# Automatic Feature Selection Based on Mutual Information

After the extraction step, a set of pattern features exists for each band in the filter bank. The optimal subset of these features is selected from the whole set of features by a mutual informationbased algorithm. Mutual information is an information theoretic quantity that measures the mutual dependence of two random variables (Cover and Thomas, 2006). In the training phase of our classification system, the two random variables are: (1) the variable that represents the extracted features, and (2) the variable that represents the class for every trial, for the whole duration of the EEG acquisition.

Mutual information can be formulated as:

$$I(F, a) = H\left(a\right) - H(a|F)$$

where F denotes the set of features, and ω denotes the class labels (Ang et al., 2012). H(ω) is the entropy defined as:

$$H(\boldsymbol{\alpha}) = -\Sigma\_{\boldsymbol{\alpha}=1}^2 \boldsymbol{p}\left(\boldsymbol{\alpha}\right) \log\_2 \boldsymbol{p}(\boldsymbol{\alpha}).$$

The conditional entropy is given by

$$H\left(\phi|F\right) = -\Sigma\_{\phi=1}^2 \not\!p\left(\phi|F\right)\log\_2 p(\phi|F)$$

Bayes rule is used to compute the conditional probability of the class given the features:

$$\operatorname{p}\left(\boldsymbol{\alpha}|\boldsymbol{F}\right) = \frac{\operatorname{p}\left(\boldsymbol{F}|\boldsymbol{\alpha}\right)\operatorname{p}\left(\boldsymbol{\alpha}\right)}{\operatorname{p}\left(\boldsymbol{F}\right)}.$$

After computing the mutual information for all the available features and the class-labels of the corresponding data trials, they are sorted by descending value of mutual information, and the k best ones are selected.

#### Support Vector Machine

Classification is the task of estimating the class label of a given data sample on the basis of a trained model. From the data one or more characteristic features are extracted. The model for classification is built by feeding a set of labeled training features into the classifier.

A SVM is a type of classifier that maximizes the separation between two classes of data by finding a hyperplane that separates a high dimensional feature space into two subspaces of distinct classes. The estimation of the class label is carried out by computing the signum of the decision function y i that is defined as,

$$\mathbf{y}^{i} = \mathbf{w}^{T}\mathbf{x}^{i} + b$$

The hyperplane is optimal if the objective function L is minimized

$$L = \frac{1}{2}\boldsymbol{\w}^T \boldsymbol{\omega} + \mathcal{C} \sum\_{i=1}^N \boldsymbol{\xi}^i$$

under the constraint

$$\gamma^i \ge 1 - \xi^i \text{ with } \xi^i \ge 0$$

The N feature vectors x i are weighted by the vector w. Here, b is a constant bias value. A margin of ξ around the hyperplane is allowed to account for misclassification of each index i of the feature vector if a dataset cannot be separated without classification error. The slack variables ξ can be weighted by C.

The hyperplane is a linear decision boundary in a feature space that optimally separates points of one class-label from the other. It is defined by the closest feature vectors called support vectors, from which the name of the classification system SVM is derived (Schölkopf et al., 1999; Laconte et al., 2005). Once the hyperplane is found, the unknown class-label of a new feature point can be easily predicted based on its position with respect to the decision boundary.

#### Experimental Setup and Data Flow

Our algorithm is implemented in the Matlab interpreter language (Matlab Inc., Natick MA, US). It works on EEG signals in a trialby-trial manner, i.e., the data is recorded continuously and then split into a number of trials (e.g., 64) of equal duration (e.g., 5 s) for each condition.

The general setup of the hardware for conducting our experiments can be seen in **Figure 3**. The data acquisition system was connected to a computer for collecting data offline, storing it and training the classification models. This setup was expanded for the real-time experiments by adding another computer for the presentation of stimuli and neurofeedback. Thereby, data acquisition and processing and stimulus presentation could be uncoupled.

In both, the offline and real-time setups, the BCI2000 software was used to retrieve the data from the acquisition system and the Fieldtrip Buffer was used to access the buffered signal in Matlab (Schalk et al., 2004; Oostenveld et al., 2011).

The experiments for our study were conducted at the Institute for Medical Psychology and Behavioral Neurobiology at the University Clinic of Tübingen and the Escuela de Medicina, Universidad Católica, Santiago de Chile as part of a research collaboration between the two institutions. Two comparable hardware and software setups were used for our experiments depending on the institution where the experiments were conducted. The EEG signals where recorded from 28 channels and the EOG was captured with four channels. In both experimental series the signal was sampled at 500 Hz. For further information on the implementation of the classification system and the experimental setup please see the Supplementary Material, Section Methods.

#### Classification Parameters

For both, offline classification and real-time neurofeedback sessions we used the same parameters of the classification software. The filterbank ranged from 0 to 36 Hz and consisted of 6 bandpass frequency filters, each one with a bandwidth of 6 Hz. We chose m = 2 as the number of spatial filters to use for the CSP algorithm in accordance to prior studies with CSP (Müller-Gerking et al., 1999; Ang et al., 2012). The number of features to be extracted from the data was k = 4. We tested different values for k with our data, and the results showed that the average classification accuracy over all subjects did not differ significantly, but the standard deviation of the classification results was lowest when k = 4.

#### Generation of Visual Feedback

Real-time feedback of the brain states was given by changing the bars of a graphical thermometer (**Figure 4B**, second block) in proportion to the output of the online SVM classifier. The feedback was initialized with the baseline value, which was represented by 10 blue bars that reached up to the dashed red line in the middle of the thermometer. In each update interval one bar was added to or removed from the thermometer, according to the sign of the output of the SVM, i.e., the classification result. The bars above the baseline value were colored in red. The letter besides the thermometer indicated the type of imagery (positive emotion or motor imagery).

#### Bias Correction of the SVM

Systemic changes, e.g., alignment of the EEG cap and slight differences in recording impedance, as well as mood and

FIGURE 3 | The general setup of hardware and software used for our experiments. The color-coding of the flow-chart indicates the experiment type. Components shown in a green color were used in the offline setup. Blue marks components that have been used in the real-time setting. Components with both colors were used in both experimental setups. All software components surrounded by a solid line were installed on the data processing computer, the ones surrounded by a dashed line were run on the computer for stimulus presentation.

neurofeedback training.

concentration level of the subject potentially introduce a bias of the classifier toward one condition (Sitaram et al., 2011). We corrected for this bias by first collecting EEG signals during a baseline period when the participant was instructed to focus on a fixation cross, and not move or perform any mental imagery. We then subtracted the mean of the SVM values during the baseline period from the SVM values in the real-time feedback.

# Experimental Paradigm

The experiment was divided into two stages: In stage one, data for training the classification model was recorded and later used to test the classification system offline. In stage two, a fused classification model was built from the data of stage one. The model was used to provide neurofeedback to two new healthy subjects. In the first stage, data from 27 healthy female participants between the age of 18 and 29 (average age: 24.6) was recorded. For the first 23 subjects, EEG data was recorded with the BrainAmps system. The EEG data of the last four subjects was recorded with the NuAmps system. Each subject was seated in a comfortable chair in front of a screen for presentation of stimuli.

Ethics approval was given by the Medical Faculty of the University of Tübingen and the Medicine Department of Universidad Católica de Chile. Informed consent was obtained from all subjects before the experiment.

# Offline Classifier Training

Each recording session used a block-design protocol (**Figure 4A**). The core components of the experimental paradigm were two types of imagery. In one condition cued by the letter "H," participants were asked to perform mental imagery of a happy situation. In the other condition cued by the letter "M," the task was to perform motor imagery. One session comprised five runs of 4 min each. During these 4 min, 4 different stimuli were shown repeatedly for 5 s each. A pause of 2 s was introduced between two stimuli. The order of the stimuli was deterministic. The first stimulus was a previously selected image from the International Affective Picture System (IAPS, Lang et al., 1997), the second one was the letter "H" representing the word "Happy," followed by the letter "R" for "Rest," and finally the letter "M" for "Motor." The letters were presented in a size that was easily readable for the participants. They performed happy imagery during the presentation of the letter "H," counted backwards during the letter "R," and performed motor imagery during the letter "M." There were eight regulation trials for each condition per run, totaling 80 regulation trials for one session.

Participants were instructed well in advance to identify several emotional episodes from their personal lives, so that they could use those episodes during happy imagery. Also, before the experiment, participants were asked to identify one image from the IAPS that best epitomized "happiness." To remind and strengthen their emotional recall strategies, the pre-selected IAPS images were included in the block design preceding the happy imagery block, as a reminder of the specific type of emotion imagery to employ.

For the motor imagery block, subjects were instructed to perform kinesthetic motor imagery of an action involving opening and closing of both hands repeatedly (e.g., squeezing a small ball). The hand movement was shown to the participant prior to the experiment, and was practiced several times under supervision from the researcher.

# Online Classification and Neurofeedback

In the second stage of the experiment, a fused classification model was built from the data of all the 16 subjects with a mean classification accuracy of 75% or greater. During preprocessing, the EEG data was normalized to a range between −1 and 1. The 1.5% largest and the 1.5% smallest values in the EEG signal were identified and considered as outliers and replaced by the new maximum or minimum value of the normalized range, 1 or −1. Our signal showed that, on subject average, amplitude values of about −100 and 100µV lay above the 1.5% percentile and below the 98.5% percentile. The amplitudes of the EEG for waking adults usually lie between 10 and 100µV (Niedermeyer and Silva, 2005). The normalization was necessary to even out absolute differences of the values in the data between subjects and the two different acquisition systems.

The data from the 16 subjects was then concatenated and a classification model was created according to the aforementioned methodology. The total number of trials used for creating the model was 1280 (80 ∗ 16).

For the feedback training runs, we recruited five healthy subjects who had not been part of the first set of experiments. The new subjects were instructed to learn to control the feedback signal (Instruction: "Make the feedback thermometer move up!"). They were notified that the feedback signal comes from a classifier that contains information from other subjects' brain states, and that the matching of their own brain activity/state with the classifier information would allow them to control the feedback signal. As the original subjects performed happy and motor imagery, the new subjects were also instructed that it might be easier to match the original brain states performing those mental actions (cued by the letter H and M in the experimental paradigm).

The real-time feedback paradigm consisted of the following blocks (**Figure 4B**): In the very beginning of each session a "+" sign was shown (fixation period). Subjects were instructed to fixate on the plus sign, and to avoid moving or blinking. In the first regulation block, the letter "H" for the happy imagery was shown right next to the feedback thermometer. The second regulation showed the letter "M," indicating the motor imagery. Between each of these blocks, the sign "−" was presented to indicate a resting period. During this period, subjects were allowed to blink, relax and get ready for the next task block. After the first four out of the eight regulation blocks an additional fixation period "+" was shown. Thus, the classification system could readjust the bias of the SVM for the second half of the run. Each of the blocks was presented for 20 s, except for the resting periods which lasted 10 s. A whole real-time testing run lasted 5 min. The update interval for the feedback was 1 s. The data that was used to classify and adjust the feedback accordingly comprised the data of the last second. According to the sampling frequency of 500 Hz, 500 data points were used.

Data from the subjects was measured in three sessions comprising four runs each. Therefore, the number of regulation trials per session was 32 and the total number of regulation trials was 96. The three sessions were conducted on three different days across 2 weeks.

# Results

# Offline-classification Results

In order to analyze and assess the quality of the data of the 27 participants, we performed visual inspection of the EEG signals in the Brainvision Analyzer software (Brain Products GmbH, Gilching, Germany). EEG signals from all subjects were of good quality, and hence no part of the data was rejected.

**Figure 5A** shows a bar graph of the classification accuracies of all 27 participants in a 10 times 10-fold cross-validation test: In this test the 80 raw EEG trials were randomly permuted for each participant. In 10 repetitions, 90% of the trials were used for feature extraction and classifier training, and the remaining 10% were used for classifier testing. This procedure was repeated 10 times for the data of each participant. The trials were randomly permuted to generate a new composition of the folds in each repetition. The classification accuracies indicate the number of times that the classifier predicted the label of the testing trials correctly, averaged over all the permutations. The standard deviation of the accuracy of the 10 repetitions of the crossvalidation was within the bounds shown by black lines above and

#### FIGURE 5 | Continued

marked by a dashed gray line. The standard deviation of the 10 repetitions are shown with black indicators. The dotted gray line marks the 75% level. The blue asterisks indicate the subjects that were chosen for the subject-independent classification model. (B) Sensitivity (light gray) and specificity (dark gray) of the classifier, averaged over all permutations of the cross-validation test for the individual subjects. (C) Boxplots of the results of the randomization test. The boxes show the first and the third quartile of the classification accuracies of the randomly labeled data. The short blue lines show the median. 99.9% of the data lie within the bounds indicated by the whiskers. The blue markers report the results of the cross-validation with correctly labeled data. The dashed gray line marks the 50% mean accuracy level and the dotted gray line the 75% mean accuracy level.

below each of the bars. The average classification accuracy for all the cross-validation runs of all the participants was 75.30%. For Subjects 11 and 22, the classification accuracy is below chance, indicating that the classifier was not able to devise a good model. **Figure 5B** shows sensitivity and specificity of the classifier for the cross-validation test. To ascertain the statistical significance of the results of the offline classification experiment, a randomization test was carried out. In this test, for each subject 1001 repetitions of the 10-times 10-fold classification with our system were executed. The labels of the trials in the data were randomly permuted in each repetition. The results of this test showed that the individual classification accuracy of these subjects with correctly labeled data lie outside of the 99.9% percentile of the results of the randomization test (**Figure 5C**).

In order to assess the validity and robustness of the fused model, another cross-validation analysis, with 20 folds repeated 10 times, was performed with the concatenated data from the 16 chosen subjects. The classification accuracy averaged over the 10 repetitions was 61.5% with a standard deviation close to zero.

Furthermore, a leave-one-subject-out-analysis has been carried out with the data of the selected subjects. Sixteen models have been trained on the data of 15 subjects and tested on the data of the remaining one. The bar graph in **Figure 6** shows the classification accuracies of the analysis when the data was tested on the subject indicated on the x-axis, i.e., the data of this subject had no influence on the model. The classification accuracy averaged over all the 16 tests was 66.2% with a standard deviation of 13.2.

**Figure 7A** shows the CSP extracted from the fused model used in the neurofeedback sessions. The left topographic plot represents the pattern for the "happy" condition, and the right one shows the pattern for the "motor" condition. The black dots indicate EEG channels. These patterns show the channels of the EEG that contain the most discriminating information for the classification of the two brain states. The darker the gray value is in the map, the more prominent is the ratio of variance in the corresponding channel. Therefore, these patterns can be interpreted as weight-maps indicating the contribution of the individual channels toward classification. The left topographic map shows greater involvement of the frontal cortex in the "happy" as compared to the "motor" condition. The right plot, on the other hand, indicates that the data from the channels over the motor cortex were more important to classify this condition. **Figure 7B** shows the most prominent CSP, i.e., the first and last column of the inverse projection matrix P, averaged over all the subjects with a classification accuracy of 75% or greater (n = 16). To generate the plots, for each subject, the absolute values of the CSPs of each cross-validation run were averaged. Then, the values

of these composite individual CSPs were scaled to a range of 0–1 to ensure that the data of each subject contributes equally to the final averaged plot.

The classifier selected bands two and three of the filter bank for feature extraction, i.e., the band from 6 to 12 Hz and from 12 to 18 Hz.

# Online-classification and Neurofeedback Results

The results of the real-time feedback sessions are presented in **Figure 8**. The first healthy subject showed an increase of the classification accuracy of 62.2% averaged over the four runs of the first session on day 1 to 71.6% averaged over the four runs of the session of the second day. One week later, in the third session, the results were still stable with an average classification accuracy of 72.0%. The second subject achieved an average classification accuracy of 58.5% on the first day. On the second day the average classification accuracy increased to 63.0% and on the third day to 68.0%. The third and fourth subjects started with a classification accuracy close to chance. Their classification accuracies increased to 61.9 and 57.41%, respectively, due to subsequent training. The fifth subject was able to increase his/her average classification accuracy from 61.8% on the first day to 70.2% after the final training session.

The above values were computed by counting how many labels were estimated correctly by the classifier. As the feedback interval was 1 s, there were 152 data samples in each run for which the classifier estimated the class labels.

Sensitivity and specificity of the classifier are shown in the Supplementary Figure 1.

For further investigation of potential learning effects, we attempted to quantify the similarity of the CSP of all subjects throughout the course of the neurofeedback training. The CSPs for each run of each subject were extracted from the EEG data recorded during the sessions. The two most prominent CSP vectors were combined and their similarity with the CSPs extracted from the fused classification model were computed. The methods used for comparing the CSPs were: correlation coefficient, mutual information and Euclidean distance. All three methods produced similar results, and hence for conciseness we will only present the results of the correlation method.

**Figure 9** shows the dynamics of the two most prominent CSP over the course of the experiment for the first subject. The figure plots the similarity index against the number of runs. Two topographical plots show the CSPs for data of runs with the lowest and the highest similarity values.

Linear regression models between the similarity indices and the classification accuracies were created for all subjects. **Table 1** shows the r 2 -values of the regression, giving an estimate of how well the similarity of the pattern predicts the classification accuracy.

# Discussion

The aim of the present study was to demonstrate a new method for real-time subject-independent pattern classification and neurofeedback of brain states from EEG signals and to assess the feasibility of this approach as a precursor to its application in a planned, future study on patients with difficulties achieving a healthy affective state (e.g., depressive disorders).

The present implementation of the classification system is based on the application of spatial filters on EEG data in different frequency bands (Ang et al., 2008). Spatial filtering is achieved by the method of CSP and the best features, i.e., the most discriminative pairs of CSPs and frequency bands are selected by a Mutual Information-based approach (Ang et al., 2012). The procedure was applied and tested in an offline experiment with 27 healthy subjects and subsequently in a series of sessions with five subjects who received real-time neurofeedback.

The result of the offline experiment with a mean classification accuracy of 75.30% for the 27 subjects shows that the classification system at hand can reliably decode the two types of imagery, i.e., emotional imagery and motor imagery. The spatial patterns yielded by the algorithm and averaged over subjects indicate that the channels in the frontal regions are important for discrimination of the happy imagery condition, whereas for the motor imagery condition the channels above the central brain areas are important. The findings for the motor condition are in line with previous studies on classification of motor imagery (Müller-Gerking et al., 1999). The importance of frontal regions for the emotional imagery condition is concordant with the extensive literature signaling the involvement of frontal areas in emotion processing and regulation (Phan et al., 2002; Ochsner et al., 2004; Kohn et al., 2014).

To choose the subjects to include in the fused classification model incremental training and testing of the model with different thresholds of the individual mean classification accuracy (i.e., 60, 65, 75, 80, 90%) were carried out. In each iteration subjects with an individual mean classification accuracy of above the threshold value were in included. A threshold value of 75% was the best choice for having enough number of training samples for generalization, i.e., all trials from 16 subjects, while maintaining good data quality, which means sufficient information in the data for reliable classification. The classification results for these subjects with the correctly labeled data lay outside of the 99.9% percentile of the distribution of the classification test with randomly permuted class labels, which shows that the classification results are very significant and suited to be used for the second stage of the experiment.

The CSP plot that was generated for **Figure 7B** shows very similar patterns to the one that we extracted from the subject-independent classification model (**Figure 7A**). In addition to the online classification results, this shows that removing outliers and normalizing and concatenating the EEG data before applying the classification algorithm is a viable method for creating the classification model.

Additional offline analysis was carried out to ascertain the performance of the classifier. Although, the subject-independent

offline cross-validation analysis showed a relatively low classification accuracy of 61.5%, the low standard deviation of the mean classification accuracies during the cross-validation test indicate stable performance.

Besides, we have also included the data from the "Rest" trials in another round of analyses. The classification system was able to distinguish "Happy" vs. "Rest" trials and also "Rest" vs. "Motor" trials with an average accuracy of 69.5 and 78%, respectively (Please see Section Classification analysis Including the "Rest" Trials of the Supplementary Material).

The leave-one-subject-out-analysis showed performance comparable to the subject-independent offline cross-validation analysis. The average classification accuracy for 11 subjects lay well above chance, for some even close to 90%. However, for five subjects, namely S7, S14, S20, S23, and S24, the system was not able to achieve above chance accuracy. Subjects S7 and S23 were among the best in the cross-validation analysis carried out for the individual subjects. One interpretation of this result might be that these subjects exhibited a very clear and distinct brain activation pattern that was easily classifiable individually but that was quite different from that of the other subjects to be included in a fused classifier. The individual classification accuracies and their high statistical significance in terms of the randomization test with permuted class labels may neither be the only nor the best criterion for the selection of subjects for the fused classification model. A method that selects subjects according to comparison of their individual CSP could be devised. However, such a method has pitfalls of its own that need to be addressed, as we explain in Section Online-classification and Neurofeedback Results and in the following paragraph.

Another possibility to find the best fused model from our data would be to run a large series of tests that builds many of such models derived from subgroups of our population of healthy subjects and tests and validates them against the remaining data. However, this kind of "brute-force" analysis requires substantial computing power and time, a potential topic of a future investigation.

The online experiment in the five new subjects demonstrates that mental states can be decoded from brain activity in real-time to provide neurofeedback. The results from the three sessions of Subjects 1, 2, and 5, which were distributed across 2 weeks, show a reliable classification accuracy that increases in the beginning and becomes stable in the last session. The upwards trends for these subjects shown in **Figure 8** indicate a learning effect of the training with the classifier. Subjects 3 and 4 were not able to improve the classification accuracy throughout the course of the training to the level of the other subjects. Nonetheless, the absolute increase on average for Subject 3 is 10% points, which is a considerable increment. The results of Subject 4 show a general upwards trend. However, this increase is not significant. The reasons for that could be that some subjects need longer to find good strategies to match the modeled brain patterns and some are simply not able to perform the necessary imagery in a consistent and persistent manner. In spite of the general upward trend the observation can be made that the classification accuracies are not increasing monotonically over time. This fact can be attributed to session-to-session variability in the internal state of the subject, i.e., levels of concentration, fatigue and motivation but also to trying different imagery strategies. As one of the external factors, noise from the EEG may corrupt the data. Both lead to variations in the performance of the classifier. Furthermore, there

CSP of a particular run and the CSP of the model, against the number of runs. Four topographical plots show the CSPs for data of runs with the lowest and the highest similarity values for visual comparison with the classification model seen in Figure 7A. The topographical plots marked by "H" represent the pattern for the happy condition, the ones marked by "M" the pattern for the motor condition.

#### TABLE 1 | The correlation coefficient (CC) between the subject's Common Spatial Patterns in all the runs and the CSPs of the classification model has been computed.


Regression analysis has been carried out to quantify the relationship between the similarity value (CC) and the classification accuracy for all the runs of the individual subjects. The r 2 -values are shown here. With one exception, the motor condition in Subject 3, they are all very low, indicating no (linear) relationship between the similarity index and the classification accuracy.

is numerous prior evidence that learning to control brain signals by neurofeedback is not necessarily a monotonically increasing process (Gruzelier et al., 1999; Subramanian et al., 2011; Linden et al., 2012).

The block length for the imagery trials has been increased from 5 s in the offline case to 20 s in the online case as suggested in the literature (Ruiz et al., 2013b; Sulzer et al., 2013).

The results of the investigation of the dynamics of the individual CSPs of each subject do not show a clear positive correlation of the similarity indices and the classification accuracy. For all of them the goodness-of-fit of the linear regression model is low. There could be several reasons for this to happen. Although computing the similarity measures with linearly increasing random noise showed the expected result of almost linear decrease of the similarity indices (see Supplementary Figure 2), these measures might still not be optimal to quantify similarity of CSPs in a useful way as they firstly might not fully capture the information conveyed in the patterns and secondly might not be able to account for variations in the patterns, e.g., slight spatial shifts.

Furthermore, it is important to understand that the CSP show the regions that exhibit the most discriminative information of the data used to compute them. In the optimal case, "pattern matching" could be measured by comparing the CSPs extracted from the data during the real-time experiments to the ones from the model because the patterns extracted from the subjects' data and the patterns of the model would be very similar. However, under the conditions of real-time experiments this might be an overoptimistic expectation. There are factors that influence the data and, therefore, change the subject's CSPs. Consider the occurrence of movement artifacts, for example. Especially when only looking at a few data samples the respective CSPs might be influenced by artifacts that exhibit high variances (Blankertz et al., 2008). The classifier, however, going a step further by extracting (weighted) features from the EEG transformed by the filter matrices of the fused model, could still prevail and extract the relevant information, as it relies on the transformation matrix of the model and its weights learned in prior. For these reasons the expectation that a high classification accuracy inevitably leads to an increased similarity of the subjects' CSPs to the model CSP cannot always be fulfilled.

Moreover, CSP show regions of high information content for the discrimination of two conditions (Müller-Gerking et al., 1999). That means that patterns for one condition change when the brain-state of the other condition changes. As an example, let us assume that during a run of neurofeedback the subject is able to exhibit the desired brain activations for one of the conditions. For the other condition, however, she is exploring a new mental strategy. This would lead to high classification accuracies at least for the first condition because the classifier is extracting features based on the group model. The CSPs, on the other hand, may look completely different to the ones of the model, even though the classifier can classify the brain state successfully.

Considering the above, we propose that the classification accuracy is a better measure than the similarity of CSP for assessing the performance and the learning effect of the subjects. Our results from the online training showed that new subjects are able to "match" the brain patterns of a fused classifier based on the brain signals of a different group of individuals, aided by the feedback provided by the subject-independent classifier. Future experiments could investigate if prolonged training leads to further increase and more stable classification accuracies and patterns. More robust and meaningful similarity measures for CSP could also be investigated.

Our results have important implications for future experiments on BCIs and its potential clinical applications. For example, for depressive disorders, in which current treatments are commonly based on antidepressants and/or psychotherapy (Lam et al., 2009; Parikh et al., 2009; Patten et al., 2009; Gelenberg et al., 2010), several attempts have been made using neurofeedback. However, the majority of previous neurofeedback studies have attempted to train patients to achieve a healthy brain state aided by the feedback of their own brain activity. As an example, the most used protocols of EEG based neurofeedback in depression have focused on Alpha band (and its inter-hemispheric asymmetry) and Theta/Beta ratio within the left prefrontal cortex (Choi et al., 2011; Dias and van Deusen, 2011; Escolano et al., 2014) in an effort to correct abnormal patterns of brain electrical signals. These systems have been built upon the idea that a patient can learn by practice to consciously generate healthy brain states. However, patients could have difficulties finding healthy patterns of brain activity. A subject-independent classifier, which provides neurofeedback information of a healthy pattern of brain states, could offer a novel alternative for patients suffering from brain disorders characterized by an abnormal mood or affect (e.g., depression).

Furthermore, many neurofeedback studies have provided feedback of neural information coming from a few sources or circumscribed brain areas (for example: Ruiz et al., 2013b; Sitaram et al., 2014; Young et al., 2014). The pattern classification system allows the feedback of distributed patterns of activity of the brain, accounting for the coordinated action of multiple networks, such as schizophrenia (e.g., Gaspar et al., 2009; Fitzsimmons et al., 2013; Ruiz et al., 2013a, 2014a) and autistic disorders (e.g., Just et al., 2012; Maximo et al., 2014).

Besides that, different patients might have different brain alterations although sharing the same clinical diagnosis. Hence, the use of a particular brain pattern or signal coming from the patients' brain activity for neurofeedback might not be appropriate for all patients. A subject-independent classifier that offers the patient a healthy brain state to "match" by

# References


neurofeedback, can offer an interesting alternative for both the problem of the heterogeneity of brain abnormalities among patients, and the involvement of distributed brain regions in neuropsychiatric disorders. Future studies should explore in detail the clinical benefits of this new approach.

# Acknowledgments

The authors are grateful to Comisión Nacional de Investigación Científica y Tecnológica de Chile (Conicyt) through Fondo Nacional de Desarrollo Científico y Tecnológico, Fondecyt (project n◦ 11121153), Vicerrectoría de Investigación de la Pontificia Universidad Católica de Chile (Proyecto de Investigación Interdisciplina N◦ 15/2013) the Medical Faculty of the University of Tübingen for their support through fortüne funding (project n◦ 2114-1-0), the ERA-Net (European Research Area)—New INDIGO project funded by the BMBF (project n ◦ 01DQ13004), and the Koselleck Award of the Deutsche Forschungsgemeinschaft (DFG) and the Open Access Publishing Fund of University of Tübingen. Furthermore, the authors would like to express their gratitude to Prof. Steffen Gais from the Institute for Medical Psychology and Behavioral Neurobiology at University of Tübingen for allocating substantial computational resources to our project over a period of several weeks.

# Supplementary Material

The Supplementary Material for this article can be found online at: http://journal.frontiersin.org/article/10.3389/fnbeh. 2015.00269


down- and up-regulation of negative emotion. Neuroimage 23, 483–499. doi: 10.1016/j.neuroimage.2004.06.030


Subramanian, L., Hindle, J. V., Johnston, S., Roberts, M. V., Husain, M., Goebel, R., et al. (2011). Real-time functional magnetic resonance imaging neurofeedback for treatment of Parkinson's disease. J. Neurosci. 31, 16309–16317. doi: 10.1523/JNEUROSCI.3498-11.2011

Sulzer, J., Haller, S., Scharnowski, F., Weiskopf, N., Birbaumer, N., Blefari, M., et al. (2013). Real-time fMRI neurofeedback: progress and challenges. Neuroimage 76, 386–399. doi: 10.1016/j.neuroimage.2013.03.033

Young, K. D., Zotev, V., Phillips, R., Misaki, M., Yuan, H., Drevets, W. C., et al. (2014). Real-time FMRI neurofeedback training of amygdala activity in patients with major depressive disorder. PLoS ONE 9:e88785. doi: 10.1371/journal.pone.0088785

**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2015 Ray, Sitaram, Rana, Pasqualotto, Buyukturkoglu, Guan, Ang, Tejos, Zamorano, Aboitiz, Birbaumer and Ruiz. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Resting state functional connectivity predicts neurofeedback response

#### **Dustin Scheinost <sup>1</sup> , Teodora Stoica<sup>1</sup> , Suzanne Wasylink<sup>2</sup> , Patricia Gruner <sup>2</sup> , John Saksa<sup>2</sup> , Christopher Pittenger 2,3,4 and Michelle Hampson<sup>1</sup>\***

<sup>1</sup> Magnetic Resonance Research Center (MRRC), Department of Diagnostic Radiology, Yale School of Medicine, New Haven, CT, USA

<sup>2</sup> Department of Psychiatry, Yale School of Medicine, New Haven, CT, USA

<sup>3</sup> Department of Psychology, Yale University, New Haven, CT, USA

<sup>4</sup> Child Study Center, Yale School of Medicine, New Haven, CT, USA

#### **Edited by:**

Sergio Ruiz, Pontificia Universidad Catolica de Chile, Chile

#### **Reviewed by:**

Gillipsie Minhas, Post Graduate Institute of Medical Education and Research, India (in collaboration with Akshay Anand) Akshay Anand, Post Graduate Institute of Medical Education and Research, India Alicia Izquierdo, University of California, Los Angeles, USA Ralf Veit, Eberhard Karls-University, Germany

#### **\*Correspondence:**

Michelle Hampson, Magnetic Resonance Research Center (MRRC), Department of Diagnostic Radiology, Yale School of Medicine, The Anlyan Center, N121, 300 Cedar Street, New Haven, CT 06520-8043, USA e-mail: michelle.hampson@yale.edu Tailoring treatments to the specific needs and biology of individual patients—personalized medicine—requires delineation of reliable predictors of response. Unfortunately, these have been slow to emerge, especially in neuropsychiatric disorders. We have recently described a real-time functional magnetic resonance imaging (rt-fMRI) neurofeedback protocol that can reduce contamination-related anxiety, a prominent symptom of many cases of obsessive-compulsive disorder (OCD). Individual response to this intervention is variable. Here we used patterns of brain functional connectivity, as measured by baseline resting-state fMRI (rs-fMRI), to predict improvements in contamination anxiety after neurofeedback training. Activity of a region of the orbitofrontal cortex (OFC) and anterior prefrontal cortex, Brodmann area (BA) 10, associated with contamination anxiety in each subject was measured in real time and presented as a neurofeedback signal, permitting subjects to learn to modulate this target brain region. We have previously reported both enhanced OFC/BA 10 control and improved anxiety in a group of subclinically anxious subjects after neurofeedback. Five individuals with contamination-related OCD who underwent the same protocol also showed improved clinical symptomatology. In both groups, these behavioral improvements were strongly correlated with baseline whole-brain connectivity in the OFC/BA 10, computed from rs-fMRI collected several days prior to neurofeedback training. These pilot data suggest that rs-fMRI can be used to identify individuals likely to benefit from rt-fMRI neurofeedback training to control contamination anxiety.

**Keywords: neurofeedback, real-time fMRI, resting state connectivity, obsessive-compulsive disorder, orbitofrontal cortex**

# **INTRODUCTION**

Dysregulation of anxiety is a core component of many neuropsychiatric conditions. Obsessive-compulsive disorder (OCD) is characterized by intrusive obsessions, which are often associated with anxiety, and with repetitive compulsions that seek to control that anxiety (Jenike, 2004). One common presentation of OCD is characterized by extreme contamination anxiety, often triggered by thoughts or images of, or contact with, potential contaminates such as dirt, body secretions, or mold (Bloch et al., 2008). Improving control of contamination anxiety is a key step in improving the quality of life for many individuals with OCD.

Treatments for contamination anxiety and for OCD exist, but none are universally effective (Franklin and Foa, 2011). For individuals who do not respond to standard behavioral and pharmacological treatments, interventions that more directly modulate the specific brain regions whose dysfunction is implicated in the disorder may be of benefit. In extreme cases, this modulation is sometimes done through invasive procedures such as deep brain stimulation (Greenberg et al., 2010). Targeted brain modulation using neurofeedback via real-time functional magnetic resonance (rt-fMRI) may prove to be an alternative (Scheinost et al., 2013).

rt-fMRI neurofeedback involves monitoring the blood oxygenation level dependent (BOLD) signal, a measure of brain activity, and providing immediate feedback to the subject showing them how specific brain activity patterns are changing over time. This form of feedback can facilitate learned control over brain activity and associated behaviors (Sulzer et al., 2013; Ruiz et al., 2014). Neurofeedback training has shown promise as a potential treatment in several clinical disorders including addiction (Hanlon et al., 2013; Li et al., 2013), tinnitus (Haller et al., 2010), stroke (Sitaram et al., 2012), depression (Linden et al., 2012; Young et al., 2014), Parkinson's Disease (Subramanian et al., 2011), and schizophrenia (Ruiz et al., 2013a,b). rt-fMRI neurofeedback can produce changes in brain function (Hampson et al., 2011; Harmelech et al., 2013) and related behaviors (Shibata et al., 2011). In individuals with significant but subclinical contamination anxiety, we have shown that neurofeedback of activity in the orbitofrontal cortex (OFC) and anterior prefrontal cortex Brodmann area (BA) 10 can reorganize functional brain networks associated with anxiety and reduce the anxiety produced by contamination-related stimuli (Scheinost et al., 2013).

Clinically, a trial of an intervention that ultimately proves ineffective carries substantial cost, in both time, resources, and ongoing patient suffering. How best to match an individual to an intervention is therefore a crucially important question. Predictors of response that can help with treatment selection in neuropsychiatric conditions such as OCD would be of enormous clinical value but have been slow to emerge.

Here, we ask whether resting-state fMRI (rs-fMRI) can predict response to neurofeedback training and, thus, potentially guide treatment selection in the future. Previous research suggests that imaging-based biomarkers can be used to predict performance with a brain-computer interface (Halder et al., 2013). rs-fMRI, in particular, provides a great opportunity for identifying biomarkers to aid clinical decisions, given that it can be collected in clinical populations without requiring any task performance and yet provides a wealth of information about brain function (Constable et al., 2013; Lee et al., 2013). To investigate whether brain connectivity at rest can predict reduction in contamination anxiety induced during a neurofeedback intervention, we correlated a voxel-wise measure of functional connectivity, computed from rs-fMRI collected prior to neurofeedback training, with behavioral response to the neurofeedback intervention in a cohort of healthy subjects with subclinical contamination anxiety (Scheinost et al., 2013). We then examined whether a similar relationship existed in a small cohort of patients.

# **METHODS**

Data were from studies performed at Yale University School of Medicine, New Haven, CT. All protocols were reviewed and approved by Human Research Protection Program at Yale University. Written informed consent was obtained. All scans were obtained and analyzed at Yale University.

# **SUBJECTS**

Two cohorts of subjects were used in this study. The first cohort has been described previously (Scheinost et al., 2013) and consisted of 10 subjects without any clinical diagnosis of OCD, but with high levels of contamination anxiety. Only the 10 subjects who received true neurofeedback in our previous study—not the 10 who received sham neurofeedback in the control condition—are included in the present analysis. The second cohort consisted of five OCD patients with moderate symptom severity (**Table 1**) and prominent contamination obsessions.

# **NEUROFEEDBACK TRAINING**

Healthy subjects and OCD patients received neurofeedback training following a previously detailed protocol (Hampson et al., 2012a). Of the five OCD patients, the first two underwent only a single neurofeedback session, without pre- or post-neurofeedback resting-state scans. All other individuals participated in four separate MRI scanning sessions, spaced several days apart. In the first session, rs-fMRI data were collected and a functional localizer was used to identify the target area of the OFC/BA 10 region to be used for neurofeedback. The second and third sessions involved rt-fMRI neurofeedback training based on the target OFC/BA 10 region. A final session (not of relevance to this work) involved collecting post-intervention rs-fMRI data. The rs-fMRI data were always collected before any other functional scans in a given session to avoid possible effects of previous task on the rs-fMRI data.

The overlap of the target area for feedback for all 15 subjects is shown in **Figure 1**. Overlap was calculated by (1) smoothing the target region of each individual with a 6 mm Gaussian smoothing kernel to account for differences in functional anatomy and registration errors; (2) warping the target regions to a common reference; and (3) averaging across subjects the likelihood of a voxel being included in the target region.

# **BEHAVIORAL MEASURES**

Behavioral measures of control over contamination anxiety (for the first cohort) and clinical measures (for the second cohort) were collected before and after neurofeedback training. The pre-intervention assessment data was collected before the first neurofeedback session (second overall imaging session), immediately prior to the start of neurofeedback training. The postintervention assessment data was collected several days after the completion of neurofeedback training, either in a separate session with no imaging (for the first two OCD patients) or in conjunction with the fourth fMRI session (for all other patients). Finally, midpoint assessment data was collected in between the first and second neurofeedback sessions for the subjects who received two sessions of neurofeedback.

For the healthy subjects, with subclinical contamination anxiety we used a behavioral measure designed to assess the subjects' ability to control their anxiety. Subjects were instructed to try to control their anxiety while viewing 25 contamination-related images and to indicate their experienced anxiety for each image on a 1–5 scale. A rating of one indicated the least anxiety and a rating of five indicated the most anxiety. The ratings for the 25 contamination-related images were then averaged yielding a single measure of anxiety. Different sets of images were used before and after the intervention, but the sets were designed to induce similar levels of contamination related anxiety and piloted to verify that they were balanced in this respect (Hampson et al., 2012a).

For the patients, we administered a modified version of the Yale–Brown Obsessive Compulsive Scale (Y-BOCS), in which they were instructed to report on their symptoms over the last 3 days, rather than over the past week as in the traditional Y-BOCS (Goodman et al., 1989a,b). The Y-BOCS ranges from 0–40, with higher scores representing more severe symptoms, and measures the frequency, intrusiveness, and distress associated with obsessions and compulsions. Scores in the mid-twenties, as these patients had (**Table 1**), correspond to moderate to severe disease.


#### **Table 1 | Clinical characteristics and symptom improvement in five OCD patients who underwent rt-fMRI biofeedback**.

MDD—major depressive disorder. Panic D/O—panic disorder, with agoraphobia. GAD—generalized anxiety disorder. SUD—substance use disorder (in remission). BDD—body dysmorphic disorder. <sup>∗</sup> taken occasionally, as needed. † rs-fMRI data collected prior to neurofeedback and used in connectivity analysis; see Figure 2.

For both groups, change in behavior measures were calculated as score prior to neurofeedback minus score after neurofeedback, such that a positive change indicates an improvement in anxiety.

#### **IMAGING PARAMETERS**

All imaging was done on a 1.5-T Siemens Sonata scanner (Siemens Medical Systems, Erlangen, Germany). A sequence designed to optimize signal in the OFC was used for all functional data collection (repetition time = 2000 ms, echo time = 30 ms, flip angle = 80, bandwidth = 2604, 200 mm field of view for 3.1 mm isotropic voxels, 31 axial-oblique slices covering almost the whole cerebrum and most of the cerebellum). Two 5 min resting data runs were collected.

#### **RESTING-STATE CONNECTIVITY**

Images were preprocessed using a previously detailed pipeline (Hampson et al., 2012b). All images were slice time and motion corrected using SPM.<sup>1</sup> Unless otherwise specified, all further analysis was performed using BioImage Suite (Joshi et al., 2011). Several covariates of no interest were regressed from the data including linear and quadratic drift, six rigid-body motion parameters, mean cerebrospinal fluid (CSF) signal, mean whitematter signal, and mean global signal. The data were low-pass filtered via temporal smoothing with a 0 mean unit variance Gaussian filter (approximate cutoff frequency = 0.12 Hz). Finally, a gray matter mask was applied to the preprocessed data so that only voxels in the gray matter were used in subsequent calculations. After preprocessing, all resting-state runs were concatenated and the connectivity for each voxel was then calculated in each subject's individual brain space.

The gray and white matter and CSF masks were defined on a template brain (Holmes et al., 1998), and warped to individual subject space using a series of transformations, described below.

<sup>1</sup>http://www.fil.ion.ucl.ac.uk/spm/

The gray matter mask was dilated to ensure full coverage of the gray matter after warping into individual subject space. Regions that were not included in all subjects' data (for e.g., the bottom of the cerebellum) were excluded from analysis. Likewise, the white matter and CSF masks were eroded to ensure only pure white matter or CSF signal were regressed from the data.

Global functional connectivity of each voxel was measured from rs-fMRI data using the network theory measure *degree* (Bullmore and Sporns, 2009) as previously described (Martuzzi et al., 2011). The BOLD time course for each voxel was correlated with every other voxel in the gray matter. Two voxels were considered connected if correlation of their timecourses was greater than *r* = 0.25; the *degree* of each voxel was defined as the number of such connections. The process was repeated for every voxel in the gray matter. Each subject's degree map was normalized by subtracting the mean across all voxels and dividing by the standard deviation across all voxels. This normalization has been shown to reduce the impact of confounds related to motion (Yan et al., 2013).

To facilitate comparisons of imaging data, all degree maps were spatially smoothed with a 6 mm Gaussian filter and then warped to a common template space through the concatenation of a series of linear and non-linear registrations, as previously described (Scheinost et al., 2013). All transformations were computed using the intensity-based registration algorithms in BioImage Suite (Papademetris et al., 2004).

### **EVALUATING THE RELATIONSHIP BETWEEN RESPONSE TO INTERVENTION AND rs-fMRI DATA**

To identify which brain regions predicted response to neurofeedback training, we related the rs-fMRI data acquired before any neurofeedback training with changes in the behavioral measure of control over contamination anxiety (for the healthy subjects) and changes in clinical severity (for the patients). For the healthy subjects, we performed a data-driven, wholebrain analysis by correlating the change in control of anxiety with the *degree* maps in a voxel-wise manner. Significance was assessed at a *p* < 0.05 level after correcting for multiple comparisons across the gray matter via AFNI's AlphaSim program. From this voxel-wise analysis, we defined a region of interest (ROI) that showed significant effects in the healthy subjects to explore whether this finding translated to the smaller cohort of OCD patients. For the three OCD patients on whom preneurofeedback rs-fMRI was collected, *degree* averaged over all voxels in this ROI was extracted and related to changes in Y-BOCS scores.

# **RESULTS**

# **IMAGING PREDICTORS OF BEHAVIOR IN SUBCLINICALLY ANXIOUS SUBJECTS**

As reported previously (Scheinost et al., 2013), healthy subjects with subclinical contamination anxiety showed a significant (*p* < 0.05) increase in control over anxiety after neurofeedback training. Whole-brain connectivity analysis revealed a single significant cluster (*p* < 0.05 corrected; MNI coordinate of peak voxel: 0, 66, −4, max *t*-value = 5.84, cluster size = 5857 mm<sup>3</sup> ) in which *degree* prior to neurofeedback training was significantly correlated with improved control over anxiety (**Figure 2A**). This cluster was located in the OFC/BA 10 target region. Subjects with the highest connectivity in this region prior to neurofeedback training exhibited the most improvement in post-treatment anxiety. A scatterplot of the average connectivity change in this region vs. the change in control of anxiety is shown in **Figure 2B**. As the choice of threshold used to consider whether two voxels are connected can impact connectivity results (Scheinost et al., 2012), we repeated this analysis over a range of thresholds (0.10 < *r* < 0.65). This produced no qualitative change in the findings. Additionally, as motion has been shown to confound functional connectivity results, average frame to frame displacement was calculated for each group (Van Dijk et al., 2012). Motion was not correlated with improved control of anxiety (*r* = 0.18, *p* > 0.60) and adding motion as a covariate in the group analysis did not change the presented results.

#### **CLINICAL IMPROVEMENT AFTER NEUROFEEDBACK IN SUBJECTS WITH OCD**

Five patients with moderate-to-severe OCD and prominent contamination symptoms underwent one or two sessions of neurofeedback (**Table 1**). All five tolerated the procedure well and exhibited reduced symptoms, as evaluated by the Y-BOCS several days after the last neurofeedback session. Average symptom improvement was 20%. Of the OCD patients, the three with the greatest symptom improvements also had a co-diagnosis or a history of major depressive disorder (MDD). Demographic and clinical details are given in **Table 1**.

### **IMAGING PREDICTORS OF CLINICAL IMPROVEMENT**

Next, we tested whether a similar relationship between connectivity and behavioral improvements would be found in OCD patients. Pre-neurofeedback rs-fMRI was not measured on the first two subjects; this analysis was therefore performed only on the three subjects who underwent the full two-session neurofeedback protocol. To maximize power in this very limited dataset, we used the OFC/BA 10 region defined in the first cohort as an *a priori* ROI, Average *degree* in this ROI prior to neurofeedback training was related to clinical improvement for the three patients. Consistent with the pattern seen in the healthy subjects, a strong linear relationship was observed (*r* = 0.99). Thus, in both groups, increased connectivity in the OFC/BA 10 measured from rs-fMRI data collected prior to neurofeedback training was associated with greater behavioral improvements.

# **DISCUSSION**

Advances in understanding individual differences motivate a new approach to health care in which treatment is tailored to the specific needs and biology of an individual patient. This "personalized medicine" approach has been endorsed by the National Institute of Mental Health, NIMH,<sup>2</sup> but its adoption depends critically on our ability to identify which patients are likely to respond to which interventions. rs-fMRI holds great promise as a tool for providing this information. It is easy to collect, does not require patients to perform any difficult tasks, and yet is a rich source of potentially clinically relevant information about brain function (Constable et al., 2013; Lee et al., 2013).

In a pilot study, we demonstrate, for the first time, that rsfMRI can be a useful tool to predict response to neurofeedback training via rt-fMRI. After receiving two sessions of neurofeedback training, healthy subjects showed improved control over anxiety and OCD patients showed a reduction in OCD symptom severity. For both groups, these behavioral improvements were strongly correlated with the pre-intervention level of whole-brain connectivity in the anterior prefrontal cortex.

The resting state functional connectivity analysis used in this study was unbiased by *a priori* expectations regarding regions of interest. Therefore, it is striking that the region that emerged from our whole-brain analysis as most relevant for predicting improvements in contamination anxiety was in our target area of the OFC/BA 10. Taken together with a large body of data highlighting the importance of the OFC and anterior prefrontal cortex in obsessive-compulsive symptoms (Swedo et al., 1992; Chamberlain et al., 2008; Menzies et al., 2008; Harrison et al., 2009; Sakai et al., 2011; Anticevic et al., 2014; Beucke et al., 2013), this gives us confidence that we are targeting a biologically relevant brain area.

Notably, OFC/BA 10 connectivity predicted the response to the intervention in both healthy subjects and OCD patients, suggesting a shared neurobiological mechanism for improved control over contamination anxiety across groups. It is possible that the phenomenon of contamination anxiety is a dimensional construct, differing in a quantitative rather than a qualitative sense in patients when compared to healthy individuals. Supporting this view are previous reports of OFC/BA 10 activations to contamination related imagery in both healthy subjects and OCD patients (Mataix-Cols et al., 2003, 2004).

To the extent that neurobiology of a phenomenon is shared across patients and healthy subjects, interventions developed in the healthy group are likely to translate into the patient population. In this particular intervention, based on our preliminary patient data, translational potential appears high. A variety of other applications of rt-fMRI neurofeedback trainings have been developed in healthy populations (Hampson et al., 2011; Shibata et al., 2011; Chiew et al., 2012; Garrison et al., 2013). It will be interesting to see how well the findings from these studies translate into clinical populations. If the dimensional approach implicit in NIMHs Research Domain Criteria<sup>3</sup> is an accurate description of pathological brain dysfunction, many of these studies may successfully translate into the respective patient groups.

An important consideration for predictive validity is the reliability of rs-fMRI. Overall, graph theory measures have been shown to be reliable (Telesford et al., 2010; Braun et al., 2012) and, in particular, voxel-wise degree has shown good test-retest reproducibility across different sites and scanners (Tomasi and Volkow, 2010). While generally reliable, a variety of factors can reduce the predictive power of rs-fMRI. Medications and other drugs such as caffeine can alter connectivity patterns (Rack-Gomer et al., 2009; Martuzzi et al., 2010). Sleep also changes connectivity patterns (Tagliazucchi et al., 2013) which can be an issue if subjects are falling asleep and not reporting it. Finally, factors related to subject comfort such as hunger may reduce data quality and prediction accuracy due to motion artifacts and effects on subject compliance. The degree to which all these variables are controlled is likely to affect the power of future studies to identify clinically relevant biomarkers that predict treatment response.

The major limitation of this pilot study is the small number of subjects, particularly in the patient group, in which we only had three subjects with resting data. Although the finding in the healthy subject group is statistically significant, the finding in the patient group must be considered preliminary. However, the tight correspondence between connectivity and intervention response in our modest clinical sample, and its similarity to the relationship seen in healthy subjects, are promising. Future studies are needed

<sup>2</sup>http://www.nimh.nih.gov/about/strategic-planning-reports/index.shtml

<sup>3</sup>http://www.nimh.nih.gov/research-priorities/rdoc/nimh-research-domaincriteria-rdoc.shtml

to rigorously examine whether this biomarker is an effective predictor of response in the clinical group. A large study that can examine possible modulating variables would be particularly valuable. For example, the data in our small sample suggest that patients with a current co-diagnosis or a history of MDD show the greatest improvement in clinical symptoms, but we were unable to investigate this given our limited data in the patient group. A study with the power to test that possibility could yield interesting insights.

# **CONCLUSION**

These pilot data provide evidence that rs-fMRI connectivity can be used to identify individuals likely to benefit from rt-fMRI neurofeedback interventions for training control over contamination anxiety. Specifically, we have identified a biomarker that may be useful in developing personalized treatment programs in patients with OCD. More generally, these findings illustrate the potential utility of rs-fMRI data for identifying biomarkers of treatment response and thereby facilitating a personalized medicine approach to treating mental illness.

#### **ACKNOWLEDGMENTS**

This research was supported by NIH grant R21 MH090384, The Taylor Foundation for Chronic Disease, and this publication was made possible by CTSA Grant Number UL1 TR000142 from the National Center for Advancing Translational Science (NCATS), a component of the National Institutes of Health (NIH). Its contents are solely the responsibility of the authors and do not necessarily represent the official view of NIH. We thank H. Sarofin for her technical assistance and E. Billingslea for her assistance with coordinating OCD patient recruitment, characterization, and logistics.

## **REFERENCES**


**Conflict of Interest Statement**: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 23 April 2014; accepted: 08 September 2014; published online: 24 September 2014*.

*Citation: Scheinost D, Stoica T, Wasylink S, Gruner P, Saksa J, Pittenger C and Hampson M (2014) Resting state functional connectivity predicts neurofeedback response. Front. Behav. Neurosci. 8:338. doi: 10.3389/fnbeh.2014.00338*

*This article was submitted to the journal Frontiers in Behavioral Neuroscience*.

*Copyright © 2014 Scheinost, Stoica, Wasylink, Gruner, Saksa, Pittenger and Hampson. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution and reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms*.

# Internet addictive individuals share impulsivity and executive dysfunction with alcohol-dependent patients

# **Zhenhe Zhou\*, Hongmei Zhu, Cui Li and JunWang**

Department of Psychiatry, Wuxi Mental Health Center, Wuxi, China

#### **Edited by:**

Niels Birbaumer, University of Tuebingen, Germany

#### **Reviewed by:**

Matthew O. Parker, Queen Mary University of London, UK Matthias Brand, University Duisburg-Essen, Germany

#### **\*Correspondence:**

Zhenhe Zhou, Department of Psychiatry, Wuxi Mental Health Center, 156 QianRong Road, Wuxi City, Jiangsu Province 214151, China e-mail: zhouzhenhe1970@gmail.com

Internet addiction disorder (IAD) should belong to a kind of behavioral addiction. Previous studies indicated that there are many similarities in the neurobiology of behavior and substance addictions. Up to date, although individuals with IAD have difficulty in suppressing their excessive online behaviors in real life, little is known about the patho-physiological and cognitive mechanisms responsible for IAD. Neuropsychological test studies have contributed significantly to our understanding of the effect of IAD on the cognitive function. The purpose of the present study was to examine whether Internet addictive individuals share impulsivity and executive dysfunction with alcohol-dependent individuals. Participants include 22 Internet addictive individuals, 22 patients with alcohol dependence (AD), and 22 normal controls (NC). All participants were measured with BIS-11, go/no-go task, Wisconsin Card Sorting Test, and Digit span task under the same experimental condition. Results showed that Barratt impulsiveness scale 11 scores, false alarm rate, the total response errors, perseverative errors, failure to maintain set of IAD and AD group were significantly higher than that of NC group, and hit rate, percentage of conceptual level responses, the number of categories completed, forwards scores, and backwards scores of IAD and AD group were significantly lower than that of NC group, however, no differences in above variables between IAD group and AD group were observed.These results revealed that the existence of impulsivity, deficiencies in executive function and working memory in an IAD and an AD sample, namely, Internet addictive individuals share impulsivity and executive dysfunction with alcohol-dependent patients.

**Keywords: Internet addiction disorder, alcohol dependence, impulsivity, executive function, working memory**

# **INTRODUCTION**

Internet addiction disorder (IAD) originates from the phenomenon of the Internet now being a part of the common person's daily life. It is well known that the Internet provides persons with the abilities to easily acquire information, learn new knowledge, gain and maintain relationships, and even make money. In short, the Internet has been instrumental in improving people quality of life. IAD is defined as a person's inability to control his or her use of the Internet, which eventually leads to psychological, social, school, and work difficulties or dysfunction in an individual's life (Young and Rogers, 1998; Davis, 2001). Because IAD is recognized internationally and is known to be linked with academic and social dysfunction, it has been increasingly recognized as a mental disorder. Recent investigations of its high prevalence in youth populations, combined with evidence that IAD is a maladaptive behavior with potentially serious occupational and mental health consequences, support the validity of the diagnosis (Ko et al., 2012). A previous study that investigated deficient inhibitory control in persons with IAD using a go/no-go task by event-related potentials (ERPs) indicated adult individuals with IAD were more impulsive than controls and shared neuropsychological and ERPs characteristics of compulsive–impulsive spectrum disorder (Zhou et al., 2010). Another study using the cue-related go/no-go switching task showed that individuals with IAD present cognitive biases

toward information related to Internet gaming and poor executive functioning skills (lower mental flexibility as well as response inhibition) (Zhou et al., 2012). Impairments in executive functioning, including response monitoring, have been suggested as a hallmark feature of impulse control disorders. The error-related negativity (ERN) reflects person's ability to monitor behavior. A recent study examines whether individuals with IAD display response monitoring functional deficit characteristics in a modified Eriksen flanker task (Zhou et al., 2013). In the study, subjects and controls completed the modified Eriksen flanker task while measured with ERPs. Results showed that the mean ERN amplitudes of total error response conditions at frontal electrode and central electrode sites of subjects were reduced compared with controls. These results indicated that individuals with IAD display response monitoring functional deficit characteristics and share ERN characteristics of individuals with compulsive–impulsive spectrum disorder. Subtypes of IAD include excessive gaming, sexual preoccupations, and e-mail/text messaging. Three subtypes share the common components, i.e., preoccupation, mood modification, unplanned use, withdrawal, tolerance, and functional impairment (Block, 2008). By using the Diagnostic and Statistical Manual of Mental Disorders (Fourth Edition, DSM-IV) criteria, some scholars suggest that IAD is an impulse disorder or at least related to impulse control disorder (Beard and Wolf, 2001; Shaw and Black, 2008).

Behavioral addiction is a form of addiction not caused by the usage of drugs. It consists of a compulsion to repeatedly engage in an action until it causes negative consequences to the person's physical, mental, and social well-being. Behavior persisting in spite of these consequences can be taken as a sign of addiction (Potenza, 2006; Parashar and Varma, 2007). According to above interpretation, IAD should belong to a kind of behavioral addiction. The drug based reinforcement and reward based learning processes are the most important mechanism of addictions. Impulsivity is considered as the tendency to act prematurely without foresight (Dalley et al., 2011). According to both animal and human studies, there are two forms of impulsivity: one depends on the temporal discounting of reward; another on motor or response disinhibition (Buckholtz et al., 2010). Barratt impulsiveness scale 11 (BIS-11) is considered more of a trait measure of impulsivity (Patton and Stanford, 1995). The go/no-go task is used for operational measures of impulsivity. Studies displayed that impulsivity is commonly associated with addiction to drugs from different pharmacological classes (Dick et al., 2010; Ersche et al., 2011; Molander et al., 2011; Economidou et al., 2012).

Executive function and working memory are critical features of cognition. The Wisconsin Card Sorting Test (WCST) is a neuropsychological test of "set-shifting,"i.e., the ability to display flexibility in the face of changing schedules of reinforcement (Monchi et al., 2001). WCST is employed to assess the "frontal" lobe functions including strategic planning, organized searching, utilizing environmental feedback to shift cognitive sets, directing behavior toward achieving a goal, and modulating impulsive responding. Because of its reported sensitivity to frontal lobe dysfunction, WCST has been considered a measure of executive function. Individuals with substance dependence present working memory impairments as well as executive dysfunctions, which include reasoning, problem solving, inhibitory controlling, and decisionmaking (Crean et al., 2011; Hanson et al., 2011; Kiluk et al., 2011; Thoma et al., 2011; Yücel et al., 2012). Working memory refers to a brain system that provides temporary storage and manipulation of the information necessary for such complex cognitive tasks as language comprehension, learning, and reasoning. Digit span task (forwards/backwards) from the Wechsler Adult Intelligence Scale was used to index the maintenance and manipulation of verbal information in working memory (Baddeley, 1992).

Many studies displayed that individual with IAD present executive dysfunctions and reward/punishment sensitivities. For example, a study, which used a gambling task to simulate extreme win/lose situations to find the reward/punishment sensitivities after continuous wins and losses, showed that higher superior frontal gyrus activations after continuous wins for IAD subjects than for normal controls (NC). The brain activities in IAD subjects were not disturbed by their losses. In addition, IAD participants showed decreased posterior cingulate activation compared to NC after continuous losses. These results indicated that IAD subjects showed enhanced sensitivity to win and decreased sensitivity to lose (Dong et al., 2013a). Studies on neuroimaging indicated that individuals with IAD present executive dysfunctions including attentional selections and decision-making (Sun et al., 2009; Pawlikowski and Brand, 2011; Dong et al., 2013b).

Diminished control is a core defining concept of substance dependence or addiction. The concept of behavioral addictions has some scientific and clinical heuristic value, but remains controversial. Several behavioral addictions, such as pathological gambling, pathological kleptomania, and pathological shopping, have been hypothesized as having similarities to substance addictions. Additionally, these behavioral addictions are classified as impulse control disorders, a separate category from substance use disorders. However, not all impulse control disorders should be considered behavioral addictions (Grant et al., 2010). For example, intermittent explosive disorder is a behavioral disorder characterized by extreme expressions of anger, often to the point of uncontrollable rage, that are disproportionate to the situation at hand. Impulsive aggression is unpremeditated, and is defined by a disproportionate reaction to any provocation, real or perceived. Intermittent explosive disorder does not share characters of behavioral addictions. Previous studies indicated that there are many similarities in the neurobiology of behavior and substance addictions (Leeman and Potenza, 2012). Behavioral and substance addictions have many similarities in natural history, adverse consequences, and phenomenology. Individuals with behavioral addictions and those with substance use disorders both score high on self-report measures of impulsivity and sensation-seeking and generally low on measures of harm avoidance (Lejoyeux et al., 1997; Kim and Grant, 2001; Grant and Kim, 2002). Prevalence studies showed that individuals with IAD or substance dependence display common characteristics including high novelty-seeking behavior and low reward dependence (Ko et al., 2012). Adolescents with alcohol dependence were more likely to have IAD and show certain psychosocial characters including high behavior activation, low self-esteem, low family function, and life satisfaction (Ko et al., 2008). Substance dependence has been associated with sensation-seeking (Sargent et al., 2010), which has also been positively correlated with IAD (Chiu et al., 2004; Mehroof and Griffiths, 2010).

In research work, Internet Addiction Test (IAT, Young, 1999), Diagnostical Questionnaire (DQ, Young, 1996), the modified Diagnostic Questionnaire for Internet Addiction (YDQ, Beard and Wolf, 2001), and the Compulsive Internet Use Scale (CIUS, Meerkerk et al., 2009) are usually used as diagnosis instruments. It is becoming a common opinion that the IAT is not completely reliable and valid psychometric instrument nowadays. DQ is a significant contribution in providing a concrete basis for establishing problematic Internet use. However, there is limited research on Internet addiction including a representative sample to use as a comparison for those being diagnosed. As a result, no reliable and valid diagnostic criteria have been determined. Although YDQ does not solve all of the previously mentioned problems, it may help strengthen Young's proposed criteria (Beard and Wolf, 2001). CIUS showed good factorial stability across time and across different samples and subsamples. The internal consistency is high, and high correlations with concurrent and criterion variables demonstrate good validity (Meerkerk et al., 2009).

Being a non-invasive method, neuroimaging plays important roles in the investigation of neurobiological mechanism and adequate treatments of IAD and drug abuse. Until now, there are several neuroimaging studies on IAD. Studies indicated that individuals with IAD shared impulsivity features of individuals with substance dependence (Dong et al., 2011, 2012, 2014; Yuan et al., 2011; Zhou et al., 2013).

Up to date, although individuals with IAD have difficulty suppressing their excessive online behaviors in real life, little is known about the patho-physiological and cognitive mechanisms responsible for IAD (Weinstein and Lejoyeux, 2010). Neuropsychological test studies have contributed significantly to our understanding of the effect of IAD on the cognitive function. Under the same experimental condition to assess impulsivity and executive function of IAD and substance dependence (such as alcohol dependence) may not only help guide decisions as to whether or not IAD should be grouped together with substance use disorders, but also play important roles in the investigation of neurobiological mechanism and adequate treatments of IAD. In this study, participants are individuals with IAD, patients with alcohol dependence (AD) and NC. All participants were measured with BIS-11, go/no-go task, WCST, and Digit span task under the same experimental condition. The purpose of the present study was to examine whether Internet addictive individuals share impulsivity and executive dysfunction with alcohol-dependent individuals.

# **MATERIALS AND METHODS**

#### **TIME AND SETTING**

The experiment was completed in the Department of Psychology and the Department of Psychiatry at Wuxi Mental Health Center, China, from May 2011 to October 2013.

# **DIAGNOSTIC APPROACHES AND PARTICIPANTS IAD group**

The diagnostic criteria of IAD group included: (i) met the criteria of the modified Diagnostic Questionnaire for Internet Addiction (YDQ) (Beard and Wolf, 2001), i.e., individuals who answered "yes" to questions one through five and at least any one of the remaining three questions were classified as suffering from IAD; (ii) whose age were more than 18 years old; (iii) did not meet criteria of any DSM-IV axis I disorder or personality disorders by administering a structured clinical interview (Chinese version); (iv) were not smokers; and (v) had not a diagnosis of alcohol or substance dependence, neurological disorders, all kinds of head injury, or systemic disease that might affect the central nervous system. The duration of the disorder was confirmed via a retrospective diagnosis. Subjects were asked to recall their life-style when they were initially addicted to the Internet. In order to confirm that they were suffering from Internet addiction, we retested them with the criteria of the modified YDQ. The reliability of these self-reports from the IAD subjects were confirmed by talking with their parents via telephone. The IAD subjects spent 11.20 ± 1.81 h/day on online activities (including gaming, Internet shopping, pornography, Internet social interaction, virtual society, and obtaining information). The days of Internet use per week was 6.41 ± 0.6. We verified this information from the roommates and co-workers of the IAD subjects that they often insisted being on the Internet late at night, disrupting others' lives despite the consequences. IAD group was recruited from Psychology Department of Wuxi Mental Health Center. They have regulated sleep patterns and did not ingest large quantities of caffeinated and energetic drinks by medical staffs' management. Twenty-two subjects were recruited as IAD group.

### **AD group**

The diagnostic criteria of AD group included: (i) met the criteria of DSM-IV for alcohol dependence; (ii) no medication was received before 2 weeks; (iii) were not smokers; and (iv) had not a diagnosis of comorbid psychiatric illness (with the exception of depression in the alcohol-dependent group), history of head injury or neurological disorder. Alcohol-dependent subjects were in-patients at Psychiatry Department of Wuxi Mental Health Center. Sobriety at time of testing was confirmed by breath alcohol readings ≤0.01 mg/l. All subjects were abstinent for >1 week. The mean duration of abstinence was 15 days.

#### **NC group**

The controls were selected from citizens lived in Wuxi city, Jiangsu Province, China through local advertisement. Controls were excluded from the study if they were smokers; or had a diagnosis of alcohol or substance dependence, neurological disorders, all kinds of head injury, or systemic disease that might affect the central nervous system. Twenty-two healthy persons were recruited as NC group. Referred from a previous IAD study (Ko et al., 2009a), we chose NC who spent <2 h/day on the Internet. The NC were tested with the YDQ criteria modified by Beard and Wolf to certificate they were not suffering from IAD. All participants were Chinese.

All participants underwent a clinical assessment by a psychiatric residency to collect information on medication, sociodemographic data, and to confirm/exclude an IAD and AD diagnosis. In this study, we gave all participants a written informed consent to participate and all were paid. The protocol for the research project was approved by the Ethics Committee of Wuxi Mental Health Center, China.

The demographic characteristics of the sample are detailed in **Table 1**.

#### **TASKS AND PROCEDURE**

All participants completed the Hamilton Depression Scale (HAMD) (version of 17 items) (Hamilton, 1967) to measure depressive symptoms and BIS-11 to measure impulsivity. BIS-11 is a questionnaire on which participants rate their frequency of several common impulsive or non-impulsive behaviors/traits on a scale from 1 (rarely/never) to 4 (almost always/always). BIS-11 includes 30 items and is divided into three subscales including attentional key, motor key, and non-planning key, to determine overall impulsiveness scores, all items are summed,with higher scores indicating greater impulsivity. The AD group completed the Severity of Alcohol Dependence Questionnaire (SADQ) (Stockwell et al., 1983).

The Neuropsychological tests included the following measures.

# **Go/no-go task**

E-Prime software 2.0 (Psychology Software Tools Inc., Sharpsburg, NC, USA) was used for the go/no-go task. The task, referred from pervious study (Zhou et al., 2010), involved the serial presentation on a computer screen of eight different two-digit numerical stimuli (four go stimuli and four no-go stimuli), displayed white on

**Table 1 | Demographic characteristics and clinical data of the sample**.


IAD, internet addictive individual group; AD, alcohol-dependent individual group; NC, normal control group; M, male; F, female; SD, standard deviation; SADQ, severity of alcohol dependence questionnaire; HAMD, Hamilton depression scale; NS, not significant.

black background (1.5 cm × 1.5 cm in size). A total of 160 stimuli were presented in 20 blocks. Each block included eight trials, and pseudo-randomly presented with no more than three consecutive trials with either a go or no-go stimulus so that withholding a response involved overcoming an established response tendency. The go stimuli in any blocks were "08," "63," "74," and "25"; the no-go were "58," "19," "14," and "79." Subjects were told that the task involved learning when to go (bar press as quickly as possible) or not to go (withhold response) and that responses after some numbers would result in winning money (\$0.16 per trial) but responses after others would result in losing money (\$0.16 per response). The response window was 1000 ms and the inter-trial interval (ITI) was 1500 ms. Reward contingencies (green background with +\$0.16 in white) or punishment contingencies (red background with −\$0.16 in white) were presented on the computer screen for 1000 ms immediately after a response (within the 1500 ms ITI). The experiment included a practice phase and a recording phase. The practice phase consisted of 16 go and no-go trials. The percentage of hits and reactive time (RT) to go stimuli and the percentage of false alarms to no-go stimuli were used for analysis. When the button was pressed within 200–1000 ms after the presentation of a go stimulus, the response was confirmed as correct. Lack of a response in this latency window was defined as a miss, whereas responses made within this window to no-go stimuli were defined as false alarms. False alarms were defined for each modality separately. The percentage of correct responses to go stimuli was confirmed as 100 × N (target detections) divided by the total number of go stimuli. The percentage of false alarms to no-go stimuli was confirmed as 100 × N divided by the sum of no-go stimuli presented. RT was measured from the onset of the go stimulus to the button press.

### **Wisconsin card sorting test**

TheWCST (Beijing Ka YipWise Development Co., Ltd, computerized versionVI) was present graphically on a computer screen. The WCST entailed matching stimulus cards with one of four category cards, in which the stimuli were multidimensional according to color, shape, and number, each dimension determining a sorting rule. By trial and error, the participant has to decide a preordained sorting rule given just the feedback ("Right" or "Wrong") on the screen after each sort. After 10 consecutive correct sorts the rule changed. There were up to six attempts to derive a rule, providing five rule shifts in the following sequence (color – shape – number – color – shape number), with each rule attainment referred to as "completing a category." Participants were not informed of the correct sorting principle and that the sorting principal shifts during the measurement; measuring continues until all 128 cards were sorted and irrespective of whether the participant achieved completes all the rule shifts. Two types of errors were possible, perseverative errors,in which the participant made a response in which they persist with a wrong sorting rule, and non-perseverative errors. In this study, five main types of WSCT were used for analysis: (i) the total response errors; (ii) perseverative errors; (iii) percentage of conceptual level responses; (iv) the number of categories completed; and (v) failure to maintain set.

## **Digit span task**

Wechsler Adult Intelligence Scale-Revised China (WAIS-RC, Beijing KaYipWise Development Co., Ltd, computerized version) was used for measurement of Digit span task. All participants are given sets of digits to repeat initially forwards then backwards. This is a test of immediate auditory recall and freedom from distraction. The participant was told to listen carefully because he or she will say a series of numbers and ask him or her to repeat them back in the same order. The first series is three numbers, such as "3, 9, 2." Each number is said in a monotone voice, one second apart. The person repeats those numbers back. The next step is to speak a series of four numbers, such as, "4, 7, 3, 1." Again, the individual repeats those back. Continue in the same manner by increasing the series of numbers to five and asking the participant to repeat the numbers back.

#### **STATISTICAL ANALYSIS**

Data were analyzed using SPSS (SPSS, Chicago, IL, USA). Sex ratio among IAD group, AD group, and NC group were analyzed with χ 2 tests. Comparisons of years of addiction between IAD group and AD group were done using independent-sample *t*-tests. Comparisons of HAMD scores, BIS-11 scores, data of go/no-go task, WSCT, and Digit span task among IAD group, AD group, and NC group were done using one-way analysis of variance (ANOVA). Least square difference (LSD) tests were performed as *post hoc* analyses if indicated. Alpha values of 0.05 were considered significant throughout.

# **RESULTS**

### **COMPARISONS OF BIS-11 SCORES AMONG IAD GROUP, AD GROUP, AND NC GROUP**

Using attentional key scores, motor key scores, non-planning key scores, and BIS-11 total scores as dependent variable, respectively, a one-way ANOVA revealed a significant main effect of Group (IAD group, AD group, and NC group). *Post hoc* LSD tests showed that attentional key scores, motor key scores, non-planning key scores, and BIS-11 total scores of IAD and AD group were significantly higher than that of NC group (for attentional key scores, *p* = 0.038 and 0.028, respectively; for motor key scores, *p* = 0.030 and 0.036, respectively; for non-planning key scores, *p* = 0.017 and 0.049, respectively; for BIS-11 total scores, *p* = 0.022 and 0.035, respectively), while above four main type data were not significantly different between IAD and AD group (all *p* > 0.05) (**Table 2**).

#### **COMPARISONS OF RTs, HIT RATE, AND FALSE ALARM RATE AMONG IAD GROUP, AD GROUP, AND NC GROUP**

Using RTs as dependent variable, a one-way ANOVA revealed no main effect of Group (IAD group, AD group, and NC group). Using hit rate and false alarm rate as dependent variable, respectively, a one-way ANOVA revealed a significant main effect of Group (IAD group, AD group, and NC group). *Post hoc* LSD tests showed that false alarm rate of IAD and AD group were significantly higher than that of NC group, and hit rate was significantly lower than that of NC group (for false alarm rate, *p* = 0.027 and 0.034, respectively; for hit rate, *p* = 0.017 and 0.020, respectively), while false alarm rate and hit rate was not significantly different between IAD and AD group (all *p* > 0.05) (**Table 3**).

#### **COMPARISONS OF WSCT DATA AMONG IAD GROUP, AD GROUP, AND NC GROUP**

Using total response errors, perseverative errors, percentage of conceptual level responses, the number of categories completed, and failure to maintain set as dependent variable respectively, a one-way ANOVA revealed a significant main effect of Group (IAD group, AD group, and NC group). *Post hoc* LSD tests showed that total response errors, perseverative errors, and failure to maintain set of IAD and AD group were significantly higher than that of NC group, and percentage of conceptual level responses and the number of categories completed of IAD and AD group were significantly lower than that of NC group (for total response

#### **Table 2 | BIS-11 scores [mean (SD)] in IAD group (n** = **22), AD group (n** = **22), and NC group (n** = **22)**.


IAD, internet addictive individual group; AD, alcohol-dependent individual group; NC, normal control group; SD, standard deviation.

errors, *p* = 0.041 and 0.022, respectively; for perseverative errors, *p* = 0.039 and 0.040, respectively; for failure to maintain set, *p* = 0.024 and 0.027, respectively; for percentage of conceptual level responses, *p* = 0.011 and 0.021, respectively; for the number of categories completed, *p* = 0.043 and 0.0391, respectively), while above five main type data were not significantly different between IAD and AD group (all *p* > 0.05) (**Table 4**).

# **COMPARISONS OF DIGIT SPAN TASK SCORES AMONG IAD GROUP, AD GROUP, AND NC GROUP**

Using forwards scores and backwards scores as dependent variable, respectively, a one-way ANOVA revealed a significant main effect of Group (IAD group, AD group, and NC group). *Post hoc* LSD tests showed that forwards scores and backwards scores of IAD and AD group were significantly lower than that of NC group (for forwards scores, *p* = 0.016 and 0.025, respectively; for backwards scores, *p* = 0.017 and 0.041, respectively), while above two main type data were not significantly different between IAD and AD group (all *p* > 0.05) (**Table 5**).

**Table 3 | RTs, hit rate, and false alarm rate [mean (SD)] in IAD group (n** = **22), AD group (n** = **22), and NC group (n** = **22)**.


IAD, internet addictive individual group; AD, alcohol-dependent individual group; NC, normal control group; SD, standard deviation.



IAD, internet addictive individual group; AD, alcohol-dependent individual group; NC, normal control group; SD, standard deviation.



IAD, internet addictive individual group; AD, alcohol-dependent individual group; NC, Normal control group; SD, standard deviation.

### **DISCUSSION**

This study is the first to test impulsivity, executive function, and working memory between Internet addictive individuals and with alcohol-dependent patients under the same experimental condition. In this study, impulsivity was measured with BIS-11 and a go/no-go task, executive function was assessed with WCST and working memory was tested with Digit span task. Our results indicate the existence of impulsivity in an IAD group and an AD group, deficiencies in executive function and working memory in an IAD and an AD sample.

Internet addiction disorder and alcohol dependence involve continued use of alcohol and Internet, respectively despite negative consequences, i.e., loss of behavioral control of alcohol and Internet use. Impulsivity refers to premature, unduly risky, and poorly conceived actions. Dysfunctional impulsivity includes deficits in attention, lack of reflection, or insensitivity to consequences, all of which may occur in addiction (Evenden, 1999; de Wit, 2009).

A recent study using traditional neuropsychological tests including the Stroop and computerized neuropsychological tests showed that IAD group exhibited more trait impulsivity than the healthy control group, Furthermore, IAD group performed more poorly than the healthy control group in a computerized stop signal test, and no group differences appeared for other neuropsychological tests, which indicated that individuals with IAD exhibited impulsivity as a core personality trait and in their neuropsychological functioning (Choi et al., 2014). Many studies displayed that alcohol-dependent patients present neurocognitive deficits in memory, learning, visuospatial functions, psychomotor speed processing, executive functions and decision-making, and the cognitive alterations are directly related to compliance with treatment and maintenance of withdrawal (Parsons, 1998). In our study, there were significant differences in BIS-11 scores among IAD group, AD group, and NC group; however, no differences in BIS-11 scores between IAD group and AD group were observed. Simultaneously, in go/no-go task, there were significant differences in false alarm rate and hit rate among IAD group, AD group, and NC group, and no differences in false alarm rate and hit rate between IAD group and AD group were observed. Above two tests indicate that both IAD and AD are more impulsive than controls, and Internet addictive individuals share impulsivity with alcohol-dependent patients.

Executive functions include abstract thinking, motivation, decision-making, planning, attention to tasks, and inhibition of impulsive responses. Although WCST exists some acknowledged

weaknesses in interpretation of the profiles, i.e., difficulties in task performance could be caused by set-shifting, poor abstraction and conceptualization, or attentional problems, this procedure integrates multiple measurements of executive processes and is the most widely reported neuropsychological task. WCST commonly was used for neuropsychological measure of cognitive flexibility (or set-shifting ability). Our study results displayed that the total response errors, perseverative errors, and failure to maintain set of IAD and AD group were significantly higher than that of NC group, while above three main type data were not significantly different between IAD and AD group. Additionally, percentage of conceptual level responses and the number of categories completed of IAD and AD group were significantly lower than that of NC group, while above two main type data were not significantly different between IAD and AD group. These results indicate that both Internet addictive individuals and alcohol-dependent patients present the same property of executive dysfunctions. Many previous neuropsychological researches indicated that Internet-related cues interfere with control processes mediated by the prefrontal cortex and prefrontal brain areas, and Internet-related stimuli interfere with decision-making and other prefrontal functions, such as working memory and further executive functions (Brand et al., 2014). Our results support that the reductions of prefrontal control processes play a major role in developing and maintaining an addictive use of the Internet.

Working memory is the system that actively holds multiple pieces of transitory information in the mind, where they can be manipulated. Working memory is generally used synonymously with short-term memory, and it depends on how the two forms of memory are defined (Cowan, 2008). The cognitive processes needed to achieve this include the executive and attention control of short-term memory, which permit interim integration, processing, disposal, and retrieval of information (Rouder et al., 2011). This study results showed that by measurement of Digit span task, there were significant differences in forwards scores and backwards scores among IAD group, AD group, and NC group. Forwards scores and backwards scores of IAD and AD group were significantly lower than that of NC group, however, forwards scores and backwards scores were not significantly different between IAD and AD group. These results manifest that Internet addictive individuals share impairment of working memory with alcohol-dependent patients.

In conclusion, the results of this study clearly show that the existence of impulsivity, deficiencies in executive function, and working memory in an IAD and an AD sample, namely, Internet addictive individuals share impulsivity and executive dysfunction with alcohol-dependent individuals. Understanding the biological effects and characters of cognitive function of IAD on the human brain may provide insight into the pathogenesis of IAD and treatment. Up to date, although there is much argument on the diagnostic definition of IAD, numerous neuroimaging studies had highlighted structural and functional abnormalities in individuals with IAD similar to other type of addictive disorders, such as substance addiction and behavioral addiction (Fischl and Dale, 2000; Ko et al., 2009b). Our study using neuropsychological test proved that cognitive dysfunction in individuals with IAD similar

to alcohol-dependent individuals. Neurocognitive assessment may be a useful tool for the detection and assessment of the progress of these alterations, as well as for the cognitive rehabilitation and psychosocial reinsertion of individuals with IAD.

A limitation of this study is that this study used the modified Diagnostic Questionnaire for Internet Addiction scores of higher than six as an indicator of IAD. Although this questionnaire is a frequently used instrument for assessing IAD, its validity as a diagnostic instrument has been questioned (Beard, 2005). Future studies may utilize other measures of assessing diagnostic criteria or severity of IAD to assess impulsivity, executive function, and working memory between Internet addictive individuals and alcohol-dependent patients. Additionally, this study results are preliminary because of the small sample size. Further studies with larger sample sizes are needed to replicate these findings.

# **ACKNOWLEDGMENTS**

This study was supported by the Nature Science Foundation of Jiangsu Province, China (No. BK2007024).

# **REFERENCES**


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 29 April 2014; accepted: 07 August 2014; published online: 25 August 2014. Citation: Zhou Z, Zhu H, Li C and Wang J (2014) Internet addictive individuals share impulsivity and executive dysfunction with alcohol-dependent patients. Front. Behav. Neurosci. 8:288. doi: 10.3389/fnbeh.2014.00288*

*This article was submitted to the journal Frontiers in Behavioral Neuroscience. Copyright © 2014 Zhou, Zhu, Li and Wang . This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Winning the game: brain processes in expert, young elite and amateur table tennis players

#### **Sebastian Wolf <sup>1</sup>\* † , Ellen Brölz <sup>2</sup> , David Scholz <sup>3</sup> , Ander Ramos-Murguialday3,4 , Philipp M. Keune5,6 , Martin Hautzinger <sup>1</sup> , Niels Birbaumer 3,7,8 and Ute Strehl <sup>3</sup>**

<sup>1</sup> Faculty of Science, Institute of Clinical Psychology and Psychotherapy, University of Tuebingen, Tuebingen, Germany


<sup>4</sup> TECNALIA, Health-Technologies, San Sebastian, Spain


<sup>8</sup> German Center for Diabetes Research, Tuebingen, Germany

#### **Edited by:**

Carmen Sandi, Ecole Polytechnique Federale de Lausanne, Switzerland

#### **Reviewed by:**

Leonardo G. Cohen, National Institutes of Health, USA Sabrina Schneider, University of Tuebingen, Germany

#### **\*Correspondence:**

Sebastian Wolf, Faculty of Science, Institute of Clinical Psychology and Psychotherapy, University of Tuebingen, Schleichstr. 4, Tuebingen, 72076, Germany e-mail: sebastian.wolf@

# uni-tuebingen.de

†This work is part of the dissertation of Sebastian Wolf that has been published in German 2013 online at the publication server of the University of Tuebingen (TOBIAS-lib).

This study tested two hypotheses: (1) compared with amateurs and young elite, expert table tennis players are characterized by enhanced cortical activation in the motor and fronto-parietal cortex during motor imagery in response to table tennis videos; (2) in elite athletes, world rank points are associated with stronger cortical activation. To this aim, electroencephalographic data were recorded in 14 expert, 15 amateur and 15 young elite right-handed table tennis players. All subjects watched videos of a serve and imagined themselves responding with a specific table tennis stroke. With reference to a baseline period, power decrease/increase of the sensorimotor rhythm (SMR) during the pretask- and task period indexed the cortical activation/deactivation (event-related desynchronization/synchronization, ERD/ERS). Regarding hypothesis (1), 8–10 Hz SMR ERD was stronger in elite athletes than in amateurs with an intermediate ERD in young elite athletes in the motor cortex. Regarding hypothesis (2), there was no correlation between ERD/ERS in the motor cortex and world rank points in elite experts, but a weaker ERD in the fronto-parietal cortex was associated with higher world rank points. These results suggest that motor skill in table tennis is associated with focused excitability of the motor cortex during reaction, movement planning and execution with high attentional demands. Among elite experts, less activation of the fronto-parietal attention network may be necessary to become a world champion.

**Keywords: ERD/ERS, EEG, sensorimotor rhythm, elite athletes, table tennis, motor efficiency, motor skill**

# **INTRODUCTION**

Motor efficiency plays a crucial role in determining athletic skill, characterized by automaticity, speed and accuracy. Experts in a highly reactive sport, such as table tennis, move rapidly, effortlessly and smoothly. Motor efficiency can be achieved through intensive training, which leads to improved perception, focus, anticipation, planning and fast responses (Yarrow et al., 2009). These skills are especially important in a fast pace sport like table tennis, where athletes have to process many cues simultaneously in order to react appropriately. Studies of self-paced movements using electroencephalography (EEG) found a frequency increase and a desynchronization with a reduction in amplitude in the 8–15 Hz frequency band recorded from sensorimotor areas (sensorimotor rhythm, SMR). A desynchronization of the SMR prior to and during a self-paced movement is called movement eventrelated desynchronization (ERD) and is linked to activation of the central motor system. An increase in SMR is referred to as event-related synchronization (ERS) and is associated with inhibition of the motor cortex (Pfurtscheller et al., 1996; Pfurtscheller and Lopes Da Silva, 1999; Klimesch et al., 2007, 2011). Synchronization in the SMR rhythm (ERS) is often observed at electrodes recording from task irrelevant brain areas during cognitive (Worden et al., 2000; Sauseng et al., 2005) or motor tasks (Alegre et al., 2004): For example, after movement onset, the contralateral motor cortex desynchronizes while the ipsilateral cortex and the surrounding areas synchronize. This pattern has been named focal ERD surrounded by ERS (Pfurtscheller and Neuper, 1994, 2006). However, recent findings in healthy participants (Ramos-Murguialday and Birbaumer, under review) and in stroke patients (Antelis et al., 2012) observed ERD in contralateral and ipsilateral electrodes. ERD can be divided into two functionally and topographically different components: 8–10 Hz ERD and 10–12 Hz ERD. The low-frequency ERD occurs especially when movements become more complex and is widespread over the entire scalp in all cortical areas involved in a motor task including primary sensorimotor, premotor and parietal areas. 8–10 Hz ERD seems to be further related to attentional processes during motor tasks (Klimesch, 1999; Pfurtscheller and Lopes Da Silva, 1999; Pfurtscheller et al., 2000; Klimesch et al., 2007). The high-frequency ERD is restricted to sensorimotor areas and seems to reflect processing specific to simple motor-tasks (Klimesch, 1999; Pfurtscheller and Lopes Da Silva, 1999; Pfurtscheller et al., 2000; Klimesch et al., 2007).

In the context of athletic performance several authors stated that elite athletes require less brain activation (weaker ERD) in task relevant brain areas compared to novices, which had led to the formulation of the "neural efficiency" hypothesis in the athletic context (Babiloni et al., 2010). In line with these assumptions, the EEG literature emphasizes that low-frequency and high-frequency ERD is weaker in elite athletes. Babiloni et al. (2009) for example found that elite rhythmic gymnasts showed weaker low- and high frequency alpha ERD compared to non-gymnasts in occipital and temporal areas and in the dorsal pathway while observing rhythmic gymnastics videos. A similar study found, that low- and high-frequency alpha ERD was weaker in dorsal and fronto-parietal pathways in elite karate athletes compared to amateurs and non-athletes while watching and judging karate videos (Babiloni et al., 2010). Karate and fencing athletes showed weaker low-frequency alpha ERD at left central, right central, mid-parietal, and right parietal areas and weaker highfrequency alpha ERD at right frontal, left central, right central, and mid-parietal areas during a monopodalic upright standing task compared to non-athletes (Del Percio et al., 2009b). Elite pistol shooters showed weaker low- and high-frequency alpha ERD than non-athletes over the whole scalp in a shooting task and best shots were correlated with right parietal and left central highfrequency ERS (Del Percio et al., 2009a). Furthermore, compared to amateur rifle shooters, elite shooters showed stronger occipital alpha ERS during a pre-shot period and stronger high-frequency alpha ERS over left central electrodes (Haufler et al., 2000). In sum, these EEG studies indicate that skilled athletes show less cortical activation (weaker low- and high frequency ERD) when performing tasks specific to their discipline.

However, some EEG literature emphasizes, that athletic expertise is associated with increased brain activation in task relevant brain areas (stronger ERD). High-frequency alpha ERD was stronger in successful compared to unsuccessful putts over the frontal midline and the arm and hand region of the right primary sensorimotor area in expert golfers (Babiloni et al., 2008). Research on motor related ERD in the athletes has mainly focused on self-paced sports like shooting, golf or gymnastics. However, highly reactive sports are different from self-paced sports: They require the perception and integration of quickly changing visual, auditory and tactile cues. Thus highly reactive sports are similar to complex procedural multisensory integration motor tasks. There is evidence indicating that these tasks produce increased brain activation in premotor, motor and visual areas with the development of expertise or automaticity (Weisberg et al., 2007; Waldschmidt and Ashby, 2011). Karni et al. (1995) used fMRI to show stronger activity in the primary motor cortex after learning a complex motor task. An EEG study with karate and fencing athletes during an upright standing task, which required the integration of visual cues for body sway indicated stronger highfrequency ERD at right ventral centro-parietal electrodes in elite compared to novice athletes (Del Percio et al., 2007). Overall, evidence whether expertise in sports is correlated with reduced or increased cortical activation in task relevant brain areas is still ambiguous, indicating a need for larger and controlled studies to clarify this issue.

We are not aware of any published study examining athlete's cortical activation patterns in table tennis athletes and suggest, that demands of table tennis and complex motor learning tasks are highly similar. Therefore we expect stronger activation in motor and fronto-parietal areas in expert table tennis players. We further assume, that attentional processes play a more important role in highly reactive sports than in self-paced sports and that movements in table tennis are much more complex. We hypothesize that table tennis athletes show predominantly ERD in the low-frequency alpha band (8–10 Hz), that seem to reflect general motor attentional processes and occurs predominantly in complex motor tasks (Klimesch, 1999; Pfurtscheller et al., 2000; Klimesch et al., 2007).

Most EEG studies looking at athletes have focused on differences between experts and novices, who are entirely unfamiliar with the specific athletic discipline. Thus, it is still unclear if the observed differences in cortical activation represent a psychophysiological mechanism induced by the amount of training that underlies superior performance in sports. To assure that the activation differences are dependent on expertise, we added young elite athletes as a group with an intermediate skill level and assumed that the cortical activation lies in between experts and amateurs. We developed a research design to compare different skill level groups from the same discipline and attempted to answer the following questions:


# **MATERIAL AND METHODS**

#### **PARTICIPANTS**

A total of 60 table tennis players were recruited from the German Table Tennis Association (DTTB, experts: *N* = 16), clubs from regional district leagues around Tuebingen, Germany (amateurs: *N* = 19) and from the squad of the Table Tennis Association of Baden-Wuerttemberg (TTBW), Germany (young elite athletes: *N* = 25). All young elite athletes were the best upcoming teen table tennis players in Baden-Württemberg with a realistic chance to become an elite expert in their career. Inclusion criteria were: right-handedness, 14–36 years of age, no medical disorders, no use of medication or drugs and no pregnancy. Of the recruited participants, 14 experts (5 female), 15 amateurs (4 female) and 15 young elites (6 female) met all criteria and were included in the analysis. There were no differences in the distribution of gender between groups (*c* 2 (2,44) = 0.62, *p* = 0.73) and no difference in age between experts (*M* = 23.8, *sd* = 4.86) and amateurs (*M* = 22.8, *sd* = 4.16). Naturally, young elite athletes (*M* = 14.9, *sd* = 0.96) were younger than experts (*p* < 0.001) and amateurs (*p* < 0.001). Subjects signed an informed written consent and received monetary compensation for participation. All experiments reported here were approved by the local ethics committee and are in accordance with the Declaration of Helsinki.

#### **EXPERIMENTAL PARADIGM**

We chose a motor imagery paradigm to minimize electromyographic (EMG) artifacts in the EEG data. Cortical activation patterns of real motor action and kinesthetic motor imagery, are similar (Porro et al., 1996; Roth et al., 1996; Jeannerod, 2001; Naito et al., 2002; Ehrsson et al., 2003; Neuper et al., 2005), although activation is stronger during active movements (Pfurtscheller and Neuper, 1997; Jeannerod and Frak, 1999). There is also evidence, that cortical activation during the preparatory period of motor imagery is similar to the activation preceding real movements (Kranczioch et al., 2009) although the preparation for real movements induces larger activations (Ramos-Murguialday and Birbaumer, under review). The mere observation of movements appears to activate the same sensorimotor areas that are involved in motor preparation and motor programming (Grezes and Costes, 1998; Babiloni et al., 2009, 2010; Halder et al., 2011). Based on these findings, we assume motor imagery to be a reliable and valid experimental paradigm to test our hypothesis. Participants were instructed to imagine themselves (kinesthetic motor imagery) playing table tennis, i.e., reacting to an opponent serving a ball presented in different video formats (for detailed information about the task please refer to paragraph 2.4 "task" and **Figure 2**).

### **PROCEDURE**

For an overview of the general procedure, please refer to **Figure 1**. Data was collected at a training center in Faak, Austria (male experts), at a training center in Oberhof, Germany (female experts), at the Institute of Medical Psychology, University in Tuebingen, Germany (amateurs) and at the Landessportschule in Tailfingen, Germany (young elite athletes).

EEG was recorded continuously while subjects sat on a chair facing the wall, on which all instructions and the table tennis videos were projected. At the beginning of the experiment subjects watched a video of Prof. Birbaumer, explaining the importance of the study in order to standardize the introduction and to induce motivation. Following the introduction, the player's level of arousal was self-rated using the Self Assessment Mannequin (SAM; Bradley and Lang, 1994). Participants received detailed instructions projected onto the wall. They were then presented with a total of 40 video-clips of a table tennis player serving a ball. In each of the clips, they had to imagine the score to be 10:10 in the last set of an important match and react to the serve with a specific table tennis stroke: a forehand top spin. In order to further enhance ecological validity and to meet the degree of mental pressure experienced in a real competition situation, the athletes were told the imagined forehand topspins of all participants will be ranked in order of intensity and quality based on the EEG recording. Immediately after this pressure induction the level of arousal was self-rated again using the SAM questionnaire. There was a break (duration was chosen by the participant) after the first 20 videos and a 5 s break after each video. During the 5 s break the sentence "10:10 last set" was displayed and subjects were instructed to look at this sentence in order to prevent distraction and to ensure a competitive mindset. After the task, subjects were given questionnaires to assess demographic and control variables.

All experiments were carried out in a quiet room using the same EEG equipment, software and set up (beamer, notebooks, screen, distance from the participant to the screen). All instructions for the task were presented with E-Prime (Science Plus Group BV, Groningen, Netherlands). Start and end triggers of the videos were sent from E-Prime via a trigger interface (NeXus Trigger Interface, Mind Media, Herten, Holland) to a Lenovo Thinkpad T500 Notebook on which all EEG data were recorded.

### **TASK**

Each video trial lasted 7 s (see **Figure 2**). Between the 3rd and the 4 th second the player in the video throws up the ball (the ball leaves the passive hand of the opponent), to perform the serve. We termed the first 3 s of the video to comprise the "preparatory phase" in which the return movement is planned, because height, speed, rotation and direction of the arriving ball can only be anticipated when the opponent's specific wrist-, arm- and shoulder movements, racket angle and body orientation can be perceived (Singer et al., 1996).

Seconds 4, 5 and 6 of each trial comprise the "movement phase", during which we assume the subject imagines the execution of the forehand topspin. Four videos with different serves were presented 10 times in random order. Each serve was either directed more to the middle or to the right side of the table and was served with back- or sidespin. The player in the video served the ball in a way (length, height, rotation) to make a forehand top spin the most appropriate return option. This was confirmed by several expert trainers, who evaluated all videos and consistently regarded a forehand top spin as a suitable return. The forehand topspin was chosen, because it is one of the most important and profitable table tennis strokes.

To ensure ecological task validity and motivation of the participants, skill level and gender of the player in the video were adapted for each group. The videos of the young elite table tennis players were further adapted for younger (under 15 years of age) and older (under 18 years of age) participants. This resulted in 8 different video sets comprising each of four different videos in which the ball was presented to two directions (forehand side or middle side of the table) and with two different rotations (sidespin or underspin).

All videos were recorded in a coliseum, standardized in length, height of table, direction, rotation and sequence of the serve, the table set-up and were presented without sound. The only differences between videos were the table tennis player's clothes, age, gender and handedness: only male experts watched a lefthanded opponents. Female experts, all amateurs and all young elite athletes watched right-handed table tennis players.

# **EEG**

EEG was recorded from 21 electrodes using a 32-channel system with sintered ring-electrodes with carbon coating and active shielding (Nexus 32, Mind Media B.V., Herten, Holland) according to the 10–20 system (Jasper, 1958). A common average reference was used to minimize biases introduced by single

electrode referencing (Qin et al., 2010). DC offset was kept below 25 µV during the whole recording. The signal was amplified with a NeXus 32 amplifier (Mind Media B.V., Herten, Holland) using a 24 bit A/D conversion, and was digitized at a rate of 2048 Hz. Data was exported in EDF+ format and imported to Brain Vision Analyzer (Brain Products, Gilching, Germany). A band pass filter (0.1 and 50 Hz) was applied and the data was down-sampled to 512 Hz. Additionally a 50 Hz notch filter was used to remove 50 Hz noise. For ocular artifact control, we used the independent component analysis (ICA) algorithm infomax (Bell and Sejnowski, 1995; Delorme et al., 2007). By using this algorithm we avoid rejecting large portions of EEG signal due to eye blinks, thus reducing the amount of brain activity subtracted from the measurements (Vigário, 1997; Iriarte et al., 2003). EEG data were then semiautomatically screened for artifacts by the following criteria: maximal voltage step of 50 µV/ms, maximal amplitude of ± 100 µV, values greater than 200 µV per 200 ms interval, activity below 0.5 µV in a 100 ms period. This semiautomatic mode allowed for additional visual inspection and the possibility to reject additional artifacts missed by the software. Artifact free, 15- s long segmented EEG data (see **Figure 2**) were then exported into MATLAB for further analysis. There were no significant differences between groups in the amount of artifact-free segments included for further analysis (*F*(2,41) = 2.46, *p* = 0.10).

#### **EEG AND STATISTICAL ANALYSIS: ERS/ERD**

Event related spectral perturbation (ERSP) is a measure of a signal's time-frequency composition changes in a particular frequency range in reference to a baseline time interval. ERSP was computed using the MATLAB toolbox EEGlab (Delorme and Makeig, 2004) using Morlet Wavelets for spectral wavelet transformation. *ERSP*% values were transformed in log-units and converted to decibel unit (dB), by multiplying the log ratio with the factor 10 (Grandchamp and Delorme, 2011). Negative *ERSP*log values which are shown green and red in the plots indicate a decrease in power from baseline which is further called ERD. Positive values plotted in blue indicate increase in power which implies ERS. The Morlet transforms used 3 cycles at the lowest frequencies, 30 at the highest, a time window of 2229 ms and overlap of 27 ms. 300 linear-spaced frequencies from 1.5 Hz to 30 Hz and 400 time points were generated.

The "video-end" trigger marked the end of each video in the EEG recording and was used to extract relevant epochs from the signal. Since each video lasted 7 s, time 0 ms depicts exactly the start of a video. As a baseline interval for the calculation of the ERSP values, a 1-s time window from −2500 ms to −1500 ms before the start of the video was used. The result of these calculations were one ERSP matrix per electrode per participant. Statistical analysis was performed on time-frequencyaveraged ERSP values representing larger areas of interest in the frequency range 8–10 Hz from −1000 ms to 7000 in timeintervals of 1000 ms. This was done for all electrodes separately. The 1000 ms before the start of the video marked a baseline period, whereas the first 3 s of the task marked the preparatory phase and seconds four to six marked the motor execution phase (see **Figure 2**). The 1-s segments were further included in statistical ANOVA analysis as variables. For correlation analysis the mean ERD/ERS values for each phase (baseline vs. preparatory vs. execution) were computed. In order to find outliers within

the groups (amateurs, young elite and experts) ERD/ERS values were pooled by group and second in the preparatory phase (second 1 until second 3). ERD/ERS values in three electrode groups (P3, PZ, P4; F3, FZ, F4; C3, CZ, C4) were averaged for each of the three participant group over the preparatory phase resulting in 27 samples. Mean and standard deviation were calculated for each of these samples. Participants who differed more than three standard deviations from the mean were considered to be outliers and excluded from statistical analysis of EEG data. One outlier was found and excluded from the amateurs group.

#### **QUESTIONNAIRES**

To assess subjective arousal, the SAM questionnaire was used (Bradley and Lang, 1994) as a non-verbal pictorial assessment technique that directly measures arousal along a 9-point scale. Participants rated the videos based on perceived quality of the serve (complexity, rotation, variability and realism) of the opponent. Additionally participants rated the quality and intensity of their own motor imagination. To assess general commitment to the study, the subjects rated motivation, interest in the study, professionalism of the researchers and perceived relevance of the research project. Participants were also asked about the relevance of the forehand top spin within their own playing system and the perceived ability of their own forehand topspin. Rankings by the International Table Tennis Federation served as an objective performance measure for experts<sup>1</sup> .

#### **RESULTS**

#### **ERD/ERS AT THE MOTOR CORTEX**

To test hypothesis (1), that table tennis experts show stronger 8–10 Hz ERD in the motor cortex at the end of the movement period, we analyzed differences in ERD in the motor cortex (C3, CZ, C4) over time between amateurs, young elite athletes and experts. Visual inspection and frequency analysis indicate ERD predominantly in the 8–10 Hz frequency band as expected (see **Figure 3**). An ANOVA of 8–10 Hz ERD/ERS (dB values) with time (−1000 ms to 6000 ms in 1-s bins), electrode position (C3 vs. Cz vs. C4) and group (experts vs. amateurs vs. young elite athletes) as independent factors showed a significant main effect of time (*F*(2.89,115.63) = 17.50, *p* < 0.001, *partial* η <sup>2</sup> = 0.30), a significant main effect of group (*F*(2,40) = 4.309, *p* < 0.05, *partial* η <sup>2</sup> = 0.177) and a significant interaction of time and group (*F*(5.78,115.63) = 2.781, *p* < 0.05, *partial* η <sup>2</sup> = 0.012)<sup>2</sup> . Bonferroni corrected *post hoc* tests of the main effect of time with aggregated

<sup>1</sup>Reliable and valid performance indices for young elite and amateur players cannot be provided because there are no consistent ranking systems for these athletes.

<sup>2</sup>All results have been corrected for violations of sphericity (Greenhouse-Geisser).

db values over groups and electrode positions showed significant decreases in db values from second -1 (baseline) to second 2 (*p* < 0.001) and second 3 (*p* < 0.01) of preparatory and all other time points of the movement phase (*p* < 0.001). From second 1 of the preparatory phase there were significant decreases to second 2 (*p* < 0.001) of preparation and second 4 (*p* < 0.005), 5 (*p* < 0.001) and 6 (*p* < 0.001) of the movement phase. Further decreases have been shown from the last second of preparation (second 3) to the last second of the movement (second 6, *p* < 0.05) and from first (second 4) to the last second of movement (second 6, *p* < 0.05). The *post hoc* tests of the main effect of group should be regarded with caution because of the significant interaction between time and group. Bonferroni corrected *post hoc* tests with mean db values over all positions and time segments showed significant stronger ERD in experts compared to amateurs (*p* < 0.05) and no differences between experts and young elite athletes (*p* <= 0.69) and between young elite and amateur athletes (*p* = 0.26). We conducted ANOVAs of aggregated db values (mean of db values at C3, Cz, C4) at each time segment to explain the significant interaction of time and group and to assess differences in ERD between groups in the end of the motor execution phase (main hypothesis). When adjusting for multiple comparisons (Bonferroni), only the differences between groups at second 5 of the motor execution phase were significant (*F*(2,40) = 5.94, *p* < 0.01, *partial* η <sup>2</sup> = 0.23). Further Bonferroni corrected t-tests showed significant stronger ERD in experts compared to amateurs (*p* < 0.01) with a strong effect size (*d* = 1.13), but no differences between experts and young elite athletes (*p* = 0.55) and no differences between young elite athletes and amateurs (*p* = 0.12). Since we a priori stated the strongest differences at the last second of the movement phase (second 6), we also present ANOVA results for this time segment without Bonferroni correction. There was a significant main effect of group (*F*(2,40) = 4.16, *p* < 0.05, *partial* η <sup>2</sup> = 0.17). Experts showed significantly stronger ERD than amateurs (*p* < 0.05) with a slightly stronger effect size (*d* = 1.17) compared to second 5. There were marginal significant differences between young elite athletes and amateurs (*p* = 0.08) and no differences between experts and young elite athletes (*p* = 0.1).

#### **ERD/ERS AT THE FRONTO-PARIETAL CORTEX**

To test the second part of hypothesis (1) of increased frontoparietal activation in experts, we conducted an ANOVA of 8– 10 Hz ERD/ERS with time (−1000 ms to 6000 ms in 1-s bins), region (frontal vs. parietal), position (F3/P3 vs. Fz/Pz vs. F4/P4) and group (experts vs. amateurs vs. young elite athletes) as independent factors. As at the motor cortex, there was a significant main effect of time (*F*(2.06,82.72) = 23.14, *p* < 0.001, *partial* η <sup>2</sup> = 0.37), but no significant effects of group (*F*(1,40) = 2.87, *p* = 0.07, *partial* η <sup>2</sup> = 0.13) and no significant interaction between time and group (*F*(4.14,82.72) = 2.18, *p* = 0.08, *partial* η <sup>2</sup> = 0.01). But there was a significant interaction between time, region and group (*F*(8.29,165.88) = 3.01, *p* < 0.01, *partial* η <sup>2</sup> = 0.13). There were no significant interactions between group, time, region and electrode position (left, central, right). Bonferroni corrected *post hoc* tests of the main effect of time with aggregated db values over groups and electrode positions showed significant decreases in db values from second -1 (baseline) to second 1 (*p* < 0.05), second 2 (*p* < 0.001), second 3 (*p* < 0.005) of preparatory and all other time points of the movement phase (*p* < 0.001). From second 1 of the preparatory phase there were significant decreases to second 2 (*p* < 0.001) of preparation and second 4 (*p* < 0.005), 5 (*p* < 0.001) and 6 (*p* < 0.001) of the movement phase. Further decreases have been shown from the last second of preparation (second 3) to seconds 5 (*p* < 0.01) and 6 ( *p* < 0.01) of motor execution and from first (second 4) to the last second of movement (second 6, *p* < 0.05). To test the hypothesis of increased fronto-parietal activation in experts in the end of the motor execution phase, we report the *post hoc* tests of the three-way interaction time x region x group and focus on differences between groups in the last second of the movement phase (second 6). ANOVA at this time point with group (expert vs. amateur vs. young elite) and region (frontal vs. parietal) as independent factors showed no significant main effect of group (*F*(2,40) = 2.90, *p* = 0.07, *partial* η <sup>2</sup> = 0.13), a significant main effect of region (*F*(1,40) = 30.71, *p* < 0.001, *partial* η <sup>2</sup> = 0.43) and a significant interaction between region and group (*F*(2,40) = 3.49, *p* < 0.05, *partial* η <sup>2</sup> = 0.15). The main effect of region indicates significant stronger ERD at the parietal (*M* = −3.43, SD = 3.77) than at the frontal cortex (*M* = −1.65, SD = 2.06). There were no significant differences between groups at the frontal cortex (*F*(2,40) = 2.69, *p* = 0.08, *partial* η <sup>2</sup> = 0.12) nor at the parietal cortex (*F*(2,40) = 3.07, *p* = 0.06, *partial* η <sup>2</sup> = 0.13), but there was a significant main effect of group (*F*(2,40) = 3.49, *p* < 0.05, *partial* η <sup>2</sup> = 0.15) when comparing the contrasts of frontal and parietal ERD (frontal ERD/ERS minus parietal ERD/ERS). Young elite athletes showed significant stronger parietal ERD (relative to frontal) compared to amateurs (*p* < 0.05). There were no significant differences in the parietal ERD (relative to frontal ERD) between experts and amateurs nor between experts and young elite athletes. The results indicate that there is ERD also in the fronto-parietal cortex in experts, amateurs and young elite athletes, but experts did not show significant stronger ERD at frontal or parietal electrodes.

## **ACTIVATION OF THE MOTOR CORTEX AND PREDICTION OF WORLD RANK**

There were no correlations between 8–10 Hz ERD at the motor cortex and world rank in experts. However there were significant correlations between 8–10 Hz ERD within the task in frontal and parietal electrodes and world rank points: The weaker the ERD (the stronger the ERS) the more world rank points. The correlations were not different for left, central or right electrode positions. However, they were higher in the preparatory phase at parietal electrodes and higher in the movement phase at frontal electrodes (see **Table 1** and **Figure 4**). There were no correlations in the baseline period.

#### **CONTROL ANALYSIS: OCCIPITAL CORTEX**

To demonstrate, that the stronger ERD in experts compared to amateurs at the motor cortex (hypothesis 1) relies on differences in cerebral motor cortical processes and not on differences in visual attention processes, we conducted an ANOVA of 8–10 Hz ERD/ERS at the occipital cortex at the last second of the movement phase (second 6) with electrode position (O1, O2)

**Table 1 | Correlation coefficients between mean frontal (F3, Fz, F4), central (C3, Cz, C4) and parietal (P3, Pz, P4) ERD/ERS and world rank points in experts for baseline (second -1), preparatory (mean second 1 to second 3) and movement (mean second 4 to second 6) period.**


Positive correlation coefficients imply that weaker ERD (or stronger ERS) is correlated with more world rank points in experts. <sup>∗</sup>p < 0.05; <sup>+</sup>p = 0.05; ++p < 0.07.

and groups (experts, amateurs, young elite athletes) as independent factors. As expected, there were no significant differences between groups (*F*(2,40) = 2.23, *p* = 0.12, *partial* η <sup>2</sup> = 0.10) and no significant interaction between group and electrode site (*F*(2,40) = 2.06, *p* = 0.14, *partial* η <sup>2</sup> = 0.09).

# **CONTROL ANALYSIS: QUESTIONNAIRES**

To investigate whether the pressure induction was successful, we conducted an ANOVA to assess differences in subjective arousal (SAM) between groups at the different measurement times. ANOVA indicate a strong main effect of time (*F*(2,80) = 14.747,

**Pz, P4) ERD/ERS and world rank points for the preparatory und movement period during the task, where subjects watched 40 table tennis videos and imagined a specific table tennis stroke**. Seconds 1 to 3 comprise the preparatory period and seconds 4 to 6 comprise the

*p* < 0.001, *partial* η <sup>2</sup> = 0.27). However neither significant differences between groups nor an interaction of time and skill level emerged, indicating that there were no differences in total arousal between groups nor between groups at the different measurement points. *Post hoc* analyzes indicate a strong increase of arousal after the pressure induction (*p* < 0.005) but no significant increase in arousal after the first 20 video trials (*p* = 0.12), verifying, that all participants experienced an increase in performance pressure. There were no significant differences between groups in the perception of the expertise of the player in the videos (*F*(2,40) = 1.83, *p* = 0.17, *partial* η <sup>2</sup> = 0.08). There were no differences between groups in the perceived quality of the serve (*F*(2,40) = 0.464, *p* = 0.63, *partial* η <sup>2</sup> = 0.02). Overall, amateurs (*M* = 2.79, SD = 0.82) rated their imagination quality slightly more vivid than experts (*M* = 2.45, SD = 0.81) and young elite athletes (*M* = 2.2, SD = 0.79), but these differences were not statistically significant (*F*(2,40) = 0.13, *p* = 0.18, *partial* η <sup>2</sup> = 0.08). Young elite athletes showed the highest commitment (*M* = 3.3, SD = 0.51), similar to amateurs (*M* = 3.29, SD = 0.31) and higher than experts (*M* = 2.87, SD = 0.58). These differences between groups were significant (*F*(2,40) = 3.72, *p* < 0.05, *partial* η <sup>2</sup> = 0.16), however bonferroni corrected *post hoc* tests showed no significant differences between experts and amateurs (*p* = 0.08), between amateurs and young elite athletes (*p* = 0.1) and only marginal significant differences between experts and young elite athletes (*p* = 0.06). Neither the relevance of the forehand topspin (*F*(2,40) = 1.32, *p* = 0.29, *partial* η <sup>2</sup> = 0.06) nor the perceived expertise of the forehand topspin differed significantly between groups (*F*(2,40) = 0.45, *p* = 0.64, *partial* η <sup>2</sup> = 0.02). We further assessed, if the commitment to the study, arousal during the task and the imagination quality is correlated with ERD in the motor and fronto-parietal cortex. We correlated mean motor and mean fronto-parietal ERD during

the preparatory phase and the movement phase with these three control variables. Only the commitment to the study showed small positive correlation coefficients with motor ERD, indicating a slight association between higher commitment to the study and weaker ERD at the motor cortex. However, after controlling for multiple comparisons (Bonferroni), these correlations did not reach significance and were thus not included as a covariates into the main analysis.

# **DISCUSSION**

We tested the hypothesis that expert compared to amateur table tennis athletes exhibit stronger activation (indexed by stronger 8– 10 Hz alpha ERD) in the motor and fronto-parietal cortex during a motor imagery task, in which athletes had to react to a table tennis stroke presented in different video clips. We expected an intermediate activation level in a third group with an intermediate skill level (young elite athletes). To our knowledge to this date there is no study assessing differences in ERD between groups of different skill levels in high reactive sports like table tennis. Since elaborated attentional skills and multisensory integration are important features of motor skills in table tennis, we argued that there are differences in motor and fronto-parietal cortex activation. We further assumed ERD in the low-frequency 8– 10 Hz band reflecting motor and motor attentional processes during complex motor tasks. To demonstrate that ERD in the motor cortex predicts performance in elite athletes, we assumed a positive correlation between the activation of the motor cortex (stronger ERD) and world rank points.

The results demonstrate a significantly stronger 8–10 Hz ERD in experts compared to amateurs in the motor cortex (C3, Cz, C4) at the end of the motor execution phase with an intermediate 8–10 Hz ERD in young elite athletes. There was a trend towards stronger 8–10 Hz ERD in the frontal and parietal cortex in experts and young elite athletes compared to amateurs, however these differences were not significant. Contrary to our expectation, less activation or even cortical inhibition in the fronto-parietal cortex was correlated with more world rank points during motor preparation and execution. This association was task-dependent, since there was no correlation between 8–10 Hz ERD in the baseline period and world rank points.

The data suggests that 8–10 Hz ERD in the motor cortex might reflect a physiological mechanism of motor skill in table tennis. SMR and alpha rhythms are generated by thalamo-cortical and cortico-cortical loops that control the access and retrieval of stored sensorimotor and cognitive information (Pfurtscheller and Lopes Da Silva, 1999) and the functional meaning of low frequency 8–10 Hz ERD seems to be related to general motor attention and readiness during motor tasks (Klimesch, 1999; Pfurtscheller and Lopes Da Silva, 1999; Pfurtscheller et al., 2000; Klimesch et al., 2007). The present results indicate that high motor skills in table tennis imply selective engagement of thalamocortical and cortico-cortical loops for motor-attentional performance: excitability of the motor cortex during motor reaction, planning and execution with high attentional demands and, once elite expertise is reached, less activation of the fronto-parietal attention network. Motor cortical excitability indicates higher focal neuronal firing, which is often observed after motor learning (Koeneke et al., 2006b; Cirillo et al., 2011).

Our finding of a stronger activation of the motor cortex contradicts several studies showing that elite athletes (shooters, karateka, gymnasts, kendoists) require "less" brain activation (weaker ERD) in task relevant brain areas compared to novices during sport specific tasks (Kita et al., 2001; Di Russo et al., 2005; Del Percio et al., 2008, 2009a; Babiloni et al., 2009). These authors state a more efficient cortical function in more skilled individuals by means of less cortical activation (weaker ERD) when skills are highly elaborated and become automatic ("neural efficiency hypothesis"). These assumptions are also based on studies showing a reduction of brain activity during cognitive tasks with the development of expertise (Grabner et al., 2006; Dunst et al., 2014) and on motor skill learning tasks showing a reduction in brain activity in the motor cortex from pre- to posttraining measurements indicating an optimization of cortical resources (Haufler et al., 2000; Jäncke et al., 2006; Koeneke et al., 2006a). The association of less fronto-parietal activation (weaker ERD) with higher world rank points in elite experts in our study can be interpreted in terms of the "neural efficiency hypothesis": the better the performance of elite table tennis players the fewer attentional resources they require for motor tasks with high attentional demands. Less attentional resources and executive control processes are often observed also in slow motor learning tasks (for a review see Dayan and Cohen, 2011).

The stronger activation of the motor cortex in elite experts compared to amateurs in our study resembles an EEG study with karate and fencing athletes that found stronger ERD in elite compared to novice athletes during a sport specific visual integration task (Del Percio et al., 2007) and functional imaging studies with musicians that showed stronger activations in the contralateral primary sensory cortex in expert violinists compared to amateurs (Lotze et al., 2003). Further, our results are in line with studies looking at cortical activation patterns of complex motor learning tasks (Karni et al., 1995, 1998; Hazeltine et al., 1997; Rauch et al., 1997; Seidler et al., 2005) and complex motor visual multisensory integration tasks (Waldschmidt and Ashby, 2011), that showed increased cortical activity in task-relevant areas, when expertise or automaticity was developed (for reviews see Dayan and Cohen, 2011; Chang, 2014). Several motor learning studies using fMRI or PET showed an initial decrease of activity at the primary motor region (M1) in the first motor learning stage, further an increase in activation of M1, supplementary motor area (SMA), whole motor cortex and other subcortical areas after 3–5 weeks of training when motor skills become implicit and automatic (Karni et al., 1995; Inoue et al., 2000; Floyer-Lea and Matthews, 2004, 2005; Xiong et al., 2009). Contradictory evidence also exists (Hlustík et al., 2004; Pollok et al., 2014). A recent experimental motor learning study in monkeys showed reduced metabolic activity at M1 after training only when the motor task was fully self-generated and not visually guided (Picard et al., 2013). We argue, that table tennis is highly similar to complex motor tasks that require perception, visual attention and multisensory integration. Therefore it differs from the sport disciplines such as rifle shooting, golf and gymnastics, which have been the focus of investigations using EEG and analyzing ERD/ERS during motor actions in elite experts. These sport disciplines are rather characterized by simple, self-generated movements with low attentional demands. We argue, that the stronger and focused activation of the motor cortex in table tennis athletes found in our study is a physiological mechanism in highly developed motor skills in complex motor tasks with high attentional demands that require anticipation, multi-sensory integration and fast reactions.

Further, our study extends recent neuroimaging findings on motor expertise in action observation (for a review see Turella et al., 2013). There is strong evidence that elite experts of several disciplines like archery (Kim et al., 2011), badminton (Wright et al., 2010) or basketball (Abreu et al., 2012) showed stronger activation in brain regions typically involved in action observation during observation of sport specific movements. In two FMRI studies (Calvo-Merino et al., 2005, 2006), Calvo-Merino et al. (2005) could demonstrate that ballet dancers and capoeira fighters showed stronger activation for the observation of trained in comparison to untrained dance styles in brain regions (bilateral premotor cortex, bilateral superior parietal lobule, and anterior intraparietal sulcus, left ventral premotor cortex and left superior temporal sulcus) typically involved in action observation (Rizzolatti and Craighero, 2004; Caspers et al., 2010; Rizzolatti and Sinigaglia, 2010). There were no such differences in nondancers. Calvo-Merino et al. (2006) further showed, that activity within the left premotor cortex and the bilateral anterior intraparietal sulcus was higher when observing actions within the observer's motor repertoire emphasizing an effect of motor expertise rather than visual familiarity with the dance styles presented. The result of a stronger engagement of the motor cortex during action observation and motor imagery in our study is thus in line with the results of a stronger engagement of task relevant brain areas in highly-skilled athletes or dancers during action observation of skilled and familiar motor actions in their discipline.

## **LIMITATIONS**

It should be noted, that due to the constraints of EEG (spatial resolution) and the small number of electrodes, we cannot exactly conclude that the source of the 8–10 Hz ERD is the motor cortex (SMR). Nevertheless, the differences between experts and amateurs were only significant in the motor cortex with no differences between groups at the occipital cortex. Consequently, we argue that the source of the ERD is the cerebral motor system. Also, the EEG data does not allow us to analyze localized functional topographical details in motor or parietal cortex compared. However, we decided to use EEG in order to get exquisite temporal resolution needed for the specific task demands of table tennis which comprises short reaction times and fast movements. We also aimed to extend the existing literature of ERD/ERS in elite athletes to indicate that ERD serves as a physiological mechanism of elaborated motor skills in table tennis. Another apparent limitation is the difference in age between young elite athletes, experts and amateurs. There is some evidence that movement related ERD in young children (around 7 years of age) is weaker compared to adults during motor tasks (Pangelinan et al., 2011). Thus we included only children that most probably already reached puberty (14 years of age and older). However, we still cannot rule out the possibility, that the age of the young elite athletes might influenced our results. Another limitation is the small sample size (*N* = 14) especially for the correlation analysis. Thus also these analysis should be regarded as preliminary which needs replication. These results should therefore be interpreted with caution. However, the strong effect sizes of the differences between experts and amateurs at the motor cortex and the high correlation coefficients underpin the quality of the results.

# **CONCLUSIONS**

In conclusion, the present study shows that 8–10 Hz ERD is stronger in elite table tennis players compared to amateurs at the motor cortex and a weaker 8–10 Hz ERD in the fronto-parietal cortex is associated with more world rank points in experts. These results suggest that high motor skills in table tennis are associated with focused excitability of the motor cortex during reaction, movements planning and execution with high attentional demands. Among elite experts however, less activation of the fronto-parietal attention network may be necessary to become a world champion.

#### **AUTHOR AND CONTRIBUTORS**

We declare that all authors substantially contributed to the conception or design of the work, or the acquisition, analysis, or interpretation of data for the work and drafting the work or revising it critically for important intellectual content. All authors further approved the version to be published and agreed to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. More specifically, authors contributed with the following focuses:

Sebastian Wolf: Design of the work, application for funding, data acquisition, data processing, data analysis, drafting the work, revision.

Ellen Broelz: data acquisition (amateurs, young elite athletes), interpretation of the work, drafting.

David Scholz: data acquisition, data processing, data analysis, drafting (figures) and ERD/ERS.

Ander Ramos Murguialday: conceptualizaion, data processing (MATLAB Code), data analysis, drafting, revision.

Philipp M. Keune: conceptualization, data analysis, drafting.

Martin Hautzinger: design of the work, data interpretation, drafting, revision.

Niels Birbaumer: design of the work, data interpretation, drafting, revision.

Ute Strehl: design of the work, data interpretation, drafting, revision.

## **ACKNOWLEDGMENTS**

We thank the German Table Tennis Association (DTTB) and the Table Tennis Association of Baden-Württemberg (TTBW) for their participation. Especially we thank the chief trainers Dirk Schimmelpfennig (DTTB), Soenke Geil (TTBW) and Frank Fuerste (TTBW) for their support to contact the table tennis athletes. Moreover, we would particularly like to thank Benjamin Wesa for his perseveringly support during the data acquisition phase. We are deeply saddened that he died in an accident in the end of the data acquisition phase. Helena Schütze and Benedikt Joost contributed significantly with their patient and flexible support during data acquisition. We further thank all table tennis players that served as a video model: Qianhong Gotsch, Peter Franz, Silke Fürste, Harald Maier, Katharina Sabo, Carolin Reisig and Michael Klyeisen. The project was funded by the German research association (DFG, BI 195/67-1). The authors also acknowledge the support by the DFG and the Open Access Publishing Fund of Tuebingen University. The DFG had no involvement in study design, the collection, analysis and interpretation of data, in the writing of the report nor in the decision to submit the article for publication.

#### **REFERENCES**


**Conflict of Interest Statement**: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 13 August 2014; accepted: 08 October 2014; published online: 27 October 2014*.

*Citation: Wolf S, Brölz E, Scholz D, Ramos-Murguialday A, Keune PM, Hautzinger M, Birbaumer N and Strehl U (2014) Winning the game: brain processes in expert, young elite and amateur table tennis players. Front. Behav. Neurosci. 8:370. doi: 10.3389/fnbeh.2014.00370*

*This article was submitted to the journal Frontiers in Behavioral Neuroscience*.

*Copyright © 2014 Wolf, Brölz, Scholz, Ramos-Murguialday, Keune, Hautzinger, Birbaumer and Strehl. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution and reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms*.

# Lateralization of music processing with noises in the auditory cortex: an fNIRS study

# *Hendrik Santosa1, Melissa Jiyoun Hong2† and Keum-Shik Hong1,3\**

*<sup>1</sup> Department of Cogno-Mechatronics Engineering, Pusan National University, Busan, South Korea*

*<sup>2</sup> Department of Education Policy and Social Analysis, Columbia University, New York, NY, USA*

*<sup>3</sup> School of Mechanical Engineering, Pusan National University, Busan, South Korea*

#### *Edited by:*

*Ranganatha Sitaram, University of Florida, USA*

#### *Reviewed by:*

*Lutz Jäncke, University of Zurich, Switzerland Adam Michael Stewart, University of Pittsburgh, USA*

#### *\*Correspondence:*

*Keum-Shik Hong, School of Mechanical Engineering, Pusan National University, 2 Busandaehak-ro, Geumjeong-gu, Busan 609-735, South Korea e-mail: kshong@pusan.ac.kr*

#### *†Present address:*

*Melissa Jiyoun Hong, FIRST 5 Santa Clara Country, 4000 Moorpark Ave., San Jose, CA 95117, USA*

The present study is to determine the effects of background noise on the hemispheric lateralization in music processing by exposing 14 subjects to four different auditory environments: music segments only, noise segments only, music + noise segments, and the entire music interfered by noise segments. The hemodynamic responses in both hemispheres caused by the perception of music in 10 different conditions were measured using functional near-infrared spectroscopy. As a feature to distinguish stimulus-evoked hemodynamics, the difference between the mean and the minimum value of the hemodynamic response for a given stimulus was used. The right-hemispheric lateralization in music processing was about 75% (instead of continuous music, only music segments were heard). If the stimuli were only noises, the lateralization was about 65%. But, if the music was mixed with noises, the right-hemispheric lateralization has increased. Particularly, if the noise was a little bit lower than the music (i.e., music level 10∼15%, noise level 10%), the entire subjects showed the right-hemispheric lateralization: This is due to the subjects' effort to hear the music in the presence of noises. However, too much noise has reduced the subjects' discerning efforts.

**Keywords: functional near-infrared spectroscopy (fNIRS), auditory cortex, lateralization, background noise, music processing**

# **INTRODUCTION**

Asymmetry in functional responses in the right and left hemispheres of a human brain has been observed (Toga and Thompson, 2003). Notable brain asymmetries include the dominance of the auditory cortex in the left hemisphere for speech processing (particularly, the left-side planum temporale region for consonant-vowel syllables, see Jancke et al. (2002), the theory of asymmetric sampling in time (Giraud et al., 2007), and that in the right hemisphere for music processing (Bever and Chiarello, 1974; Tervaniemi and Hugdahl, 2003). This asymmetric response is observed even in infants: speech (Dehaene-Lambertz et al., 2002) and music (Perani et al., 2010). However, Shtyrov et al. (1998) showed that, in a noisy environment, the involvement of the left auditory cortex in discerning speeches considerably decreases, while that of the right hemisphere increases. This reveals that the existence of background noise in spoken words diminishes the dominant role of the left hemisphere in the speech processing (i.e., non-lateralization) (Wong et al., 2008).

Music processing is one of the most complex cognitive activities that the human brain performs. The detailed mechanism of music processing is not well understood yet. Even Bever and Chiarello (1974) had shown that the right lateralization in music processing is even higher among musicians than non-musicians, but still there are contradicting results among different genders, musicians, and non-musicians (Schuppert et al., 2000; Ono et al., 2011). To the best of our knowledge, the influence of background noise in music processing has not been fully investigated yet. The objectives of this study are first to investigate whether, in the presence of noise, the right-lateralization of music processing (in contrast to the speech processing) is destructed or not and second to characterize the noise levels in the music to any differences in the hemodynamic responses between the right and left auditory cortices.

In a live music environment, it will be interesting to categorize the level of noise that would not disturb the music. Under the assumption that the right-hemispheric laterization exists (for some people), it will become the case that the introduced noise does not destroy the right laterization in enjoying the music. Three distinguishing features of a sound are its intensity, frequency, and perception duration, which may elicit different responses to different humans. The study of audition (i.e., hearing a sound transmitted as acoustic waves) involves the examination of the sensory responses to the surrounding environment. To measure sensory responses, various modalities have been pursued in the past: For instance, electroencephalography (EEG) for detecting the language processing (Sinai and Pratt, 2003; Zaehle et al., 2007; Kuhnis et al., 2013b) and the music processing (Meyer et al., 2006; Headley and Pare, 2013) in the auditory cortex, and for detecting the P300 signals in the motor cortex (Turnip et al., 2011; Turnip and Hong, 2012; Khan et al., 2014; Soekadar et al., 2014) magnetoencephalography (MEG) for the auditory cortex (Shtyrov et al., 2012) and the motor cortex (Boulenger et al., 2012), functional magnetic resonance imaging (fMRI) for the auditory cortex (Warrier et al., 2009; Wong et al., 2010) and the frontal cortex (May et al., 2014; Zhou et al., 2014), the positron emission tomography (Tervaniemi et al., 2000), and functional near-infrared spectroscopy (fNIRS) for the motor cortex (Hu et al., 2013) and for the prefrontal cortex (Guhn et al., 2014).

fNIRS is a non-invasive method that measures the absorbed quantity of near-infrared light in the 650∼950 nm wavelength range. fNIRS can detect the hemoglobin concentration changes in response to neural activities. For brain imaging, fNIRS offers higher temporal resolution than fMRI and higher spatial resolution than EEG. Particularly, while fNIRS can be used in a natural environment (particularly important in music processing), fMRI should be used in its designated place (or at least a simultaneous hearing and measurement cannot be done without an earphone). Recent fNIRS studies include the hemodynamics analyses in the prefrontal cortex (Hu et al., 2012; Naseer et al., 2013; Santosa et al., 2013; Bhutta et al., 2014), the motor cortex (Aqil et al., 2012a; Kamran and Hong, 2013, 2014), the visual cortex (Hong and Nguyen, 2014), the motor imagery (Naseer and Hong, 2013), and the somatosensory cortex (Hu et al., 2011). Particularly, fNIRS is suitable for studying the auditory cortex because of its non-invasiveness, mobility, cost, and most importantly silence of the equipment. On the other hand, fMRI is relatively problematic in studying the auditory cortex because its measurements are accompanied by acoustic noise resulting from slice selection pulses, cryogen pumping, and magnetic resonance gradient interference (Gaab et al., 2007). To solve the noise problem in fMRI, many schemes have been employed to shield the subject from the machine's acoustic noise, although none of these methods have been proven effective: A few examples are sealing the gradient coil in a vacuum chamber (Katsunuma et al., 2002), using an active noise cancelation (McJury et al., 1997), and utilizing a low-noise gradient-coil design (Mansfield et al., 2001). The noise from an fMRI machine can even interfere with the stimuli designed to evoke neuronal activation: Fuchino et al. (2006) have shown that the level of oxy-hemoglobin in the sensorimotor cortex decreased with the increase of fMRI acoustic noise.

Conducting experiments using fNIRS, on the other hand, is much quieter than using an fMRI system, which makes fNIRS a much more suitable device for experiments related to audio stimulation. Another advantage of fNIRS is that it can detect two main chromophores: oxy-hemoglobin (HbO) and deoxyhemoglobin (HbR), while fMRI can detect only the BOLD (blood oxygenation level dependent) signal (Plichta et al., 2006). When neuron fires, HbO decreases while HbR increases instantaneously (but it will bring more blood to the area, which will cause the increase of HbO). A drawback of fNIRS is the penetration depth of the light, which is limited to the cortical surface. However, even the relatively poor spatial resolution of fNIRS compared to fMRI (Kovelman et al., 2009) can be ignored due to its other strengths, the relatively faster time resolution than fMRI, low cost, portability, and quietness. While fNIRS enables testing under more relaxed conditions for the subjects, its results on the auditory cortex are relatively rare. The present study is to determine the effects of background noise in music processing: Fourteen healthy subjects participate, and the activations in the auditory cortices are recorded with fNIRS. Specifically, four different sound environments involving ten different conditions are designed for this experiment: music segments only, noise segments only, music segments including noise, and the entire music with noise segments. To investigate this, we examined the hemodynamic responses from 44 channels for three noise categories in both hemispheres.

Our interest exists in finding the (subjective) level of noise that does not disturb listening to music as well as hemispheric lateralization. The following questions will be pursued: (i) Are there more hemodynamic changes in the right auditory cortex than the left auditory cortex, when people hear music. Instead of listening to continuous music, music segments will be exposed to the subjects so that they can focus on. (ii) Can noise alone cause a similar behavior like music segments, or will it bring any difference? (iii) What would be the level of noise that distort the hearing status? (iv) What would be the noise enterance effects when listening to the entire music. As a feature to tell any difference, the gap between the mean value and the minimum value of the hemodynamic responses caused by various conditions will be used.

# **MATERIALS AND METHODS**

#### **SUBJECTS**

Fourteen subjects (age: 28 ± 5 years; 7 males and 7 females, 12 right-handed and 2 left-handed) participated in the experiment, see **Table 1**. In this study, the handedness was obtained by asking the subjects about a better performance for use of a hand. Thirteen subjects knew (understood) the music and three subjects were musicians. The definition of musician in this work is whether they are able to play the music with piano or not. The response between musicians and non-musicians is known distinctly different (Kraus and Chandrasekaran, 2010; Kuhnis et al., 2013a, 2014). All musicians (1 male and 2 females; primary musical instrument: piano; mean age 25 ± 2 years) started their musical training between 6 and 12 years. They had more than 16 ± 2 years of musical training and practiced their musical instrument for 1.5 h/day when they started to learn piano. We selected the same number of subjects between male and female to examine a possible gender difference in the hemispheric lateralization (Shirao et al., 2005). All of them had normal hearing and none had a history of any neurological disorder. To reduce noise and artifacts, the subjects were asked to remain relaxed with closed eyes by enjoying the music and to avoid motions during the experiment (i.e., head movement, eye blinking, etc.). During the experiment, the subjects were asked to listen the auditory stimuli attentively (not passively), since selective attention is important to the activation pattern in the auditory cortex (Jancke et al., 1999). Moreover, musicians were asked to imagine playing the music with a piano. The work was approved by the Institutional Review Board of Pusan National University. The subjects were informed about the experimentation and written consents were obtained, which was conducted in accordance with the ethical standards encoded in the latest Declaration of Helsinki.

#### **AUDIO STIMULI**

**Figure 1** shows the experimental paradigm used in this work. The audio stimuli (music) was *Für Elise* composed by Ludwig van Beethoven. The entire length of the music was 165 s. One

**Table 1 | 14 Subjects in experiment.**


*\*They can play the music.*

experiment consists of four stages involving ten conditions. The first 200 s after the pre-initiation trial constitutes Stage 1 (S1), which examines the basic condition (that is, a music hearing). In S1, the subjects are exposed to 25 s rest and 15 s music for 5 times (it is noted that the pre-initiation trial is not included in the analysis). The objective of S1 is to examine which side (right or left) in the brain is more active upon a musical stimulus. Stage 2 (S2) is the next 200 s after S1 that involves two different noise levels. Therefore, S2 constitutes three conditions: no noise (NN), mid-level noise (MN), and high-level noise (HN). The objective of S2 is to examine whether the noise alone can cause a similar response like the music in S1. Then, the subsequent 200 s after S2 becomes Stage 3 (S3), in which mixed music and noise segments are repeated 5 times. The objective of S3 is to find out whether a music + noise segment will induce more efforts in the brain than the cases of music or noise segments alone. Distinguishing NN, MN, and HN, 3 conditions are examined in S3. Finally, the interruption of noise when hearing the music has been mimicked in Stage 4 (S4). For comparison purposes, similar noise conditions like S2 and S3 are made in S4, yielding another 3 conditions. The noises introduced to the music appear in a pseudo random order. The durations of music and noise segments in S1∼S4 are the same. The shaded boxes in S3 and S4 indicate the time periods where noises enter the music. The total experimental time was 14 min and 50 s. The music and noises were digitally mixed using the Adobe Audition software (a WAV-format file: 16 bit, 44,100 Hz, stereo). The same earphone (Sony MDR-NC100D; digital noise canceling earbuds) with the same sound level setting was used across the subjects.

#### **NOISE LEVELS**

The *y*-axis in **Figure 1** denotes the relative amplitude of the audio signal (its max. scale has 30,000 sample values, which corresponds to 100%) in the Adobe Audition software. In this work, the highlevel noise is defined to be the white noise whose amplitude reaches 20% of the full scale. The mid-level noise means 10% in the full scale. To provide variability, the noises were introduced in a pseudo-random order (15 s for each stimulus). The average amplitude of the music was about 10–15% in the *y*-axis.

#### **fNIRS DATA AND PROCESSING**

**Figure 2** shows the optodes configuration of the fNIRS system (DYNOT: DYnamic Near-infrared Optical Tomography; NIRx Medical Technologies, Brooklyn, NY) for imaging the auditory cortices in the left and right hemispheres. The distance between an emitter and a detector is 23 mm. The data were acquired at a sampling rate of 1.81 Hz and for two wavelengths (760 and 830 nm). A total of 22 channels were measured from 8 emitters (black circles) and 7 detectors (white circles) in both hemispheres. All the lights in the room were turned off during the experiment to minimize signal contamination from the ambient light sources. The optodes were placed on the scalp above the left and right auditory cortices. Ch. 16 in both left/right hemispheres was set to coincide with the T3/T4 locations, respectively, in the International 10–20 system. Since the optodes configuration in **Figure 2** covers the entire auditory cortex, the averaged value from the 22 channels is used in the analysis.

Relative changes of the concentrations of HbO and HbR were computed using the modified Beer-Lambert Law as follows,

$$
\Delta A\_i(\lambda, k) = \left[ a^{\rm HbO}(\lambda) \Delta c\_i^{\rm HbO}(k) + a^{\rm HbR}(\lambda) \Delta c\_i^{\rm HbR}(k) \right] \, d\_l l\_i, (1)
$$

where -*A* (·, *k*) is the absorbance variation at time *k*; the subscript *i* denotes the channel number; λ is the wavelength of the laser source; *a*HbO and *a*HbR are the absorption coefficients of HbO and HbR, respectively; *c*HbO and *c*HbR are the concentration changes of HbO and HbR, respectively; *d* is the differential path length factor (in this study, constant values *d* = 7.15 for λ = 760 nm and *d* = 5.98 for λ = 830 nm were used for all the channels), and *l* is the distance between a source and a detector.

To analyse the fNIRS data, the open-source software NIRS-SPM (Ye et al., 2009) was utilized in our own Matlab® (Mathworks, Natick, MA) code. The respiration and cardiac noises contained in the hemodynamic responses were removed by a low-pass filter of a cut-off frequency 0.15 Hz.

#### **CLASSIFICATION FEATURE**

The difference between the mean and the minimum value of a given hemodynamic response has been used as a feature to classify whether the hemispheric lateralization occurred or not. **Figure 3** depicts a typical activated hemodynamic response obtained by convolving a stimulus pattern and the impulse hemodynamic response function as

$$h\_{\mathsf{M}}(k) = \sum\_{n=-\infty}^{\infty} B\alpha(k)\, h(k-n),\tag{2}$$

where *h*<sup>M</sup> (*k*) denotes the "modeled" hemodynamic response to be used as a reference signal, *Box*(*k*) is the box-type stimulus pattern (in this paper, the 15 s activation period) and *h*(*k*) is the impulse hemodynamic response function adopted from the SPM8 (Wellcome Trust Centre for Neuroimaging, London, UK)

(Friston et al., 2008). For example, the curve in **Figure 3** shows that the mean value is 0.36μM, the minimum value is -0.12μM, and therefore the difference is 0.48μM. The reason for using the gap between the mean and the minimum value of the hemodynamic response as a classification index in this work is that, with the current experimental paradigm in **Figure 1**, the hemodynamic response may not come back to the baseline value in a short time interval after each stimulus. Actually, it was so and the baseline drifting has been compensated by the mean-min difference.

#### **STATISTICAL ANALYSIS**

The estimation of the cortical activation is the most important factor in the analysis of fNIRS data. Previous studies showed that the activation could be statistically estimated by fitting the measured response to a regression model (Hu et al., 2010; Aqil et al., 2012b). Let **h***<sup>j</sup> <sup>i</sup>* <sup>∈</sup> *RN*×<sup>1</sup> be the measured fNIRS data at the *<sup>i</sup>*-th channel, the superscript *j* denote the *j*-th stimulus (i.e., the total 20 stimuli obtained by 5 stimuli/condition × 4 conditions), and *N* be the size of fNIRS data for each stimulus (in this study, *N* = 72 for the period of 15 s stimulus and 25 s rest using the sampling frequency of 1.81 Hz). Then, the general linear regression model is formulated as follows,

$$\mathbf{h}\_{i}^{j} = \boldsymbol{\phi}\_{i}^{j}\mathbf{h}\_{\mathcal{M}} + \boldsymbol{\psi}\_{i}^{j} \cdot \mathbf{1} + \mathbf{e}\_{i}^{j},\tag{3}$$

where **<sup>h</sup>**<sup>M</sup> <sup>∈</sup> *RN*×<sup>1</sup> is the modeled hemodynamic response obtained by (2), **<sup>1</sup>**<sup>∈</sup> *<sup>R</sup>N*×<sup>1</sup> is a column vector of ones for correcting the baseline, φ is the unknown coefficient that indicates the activity strength of the modeled hemodynamic response, ψ is the coefficient to compensate the baseline drift of the signal, and **<sup>ε</sup>** <sup>∈</sup> *RN*×<sup>1</sup> is the error term in the regression model. In this paper, the coefficient φ is estimated by using the *robustfit* function available in Matlab™ as follows,

$$\left( \begin{bmatrix} \hat{\phi}\_i^j\\ \hat{\psi}\_i^j \end{bmatrix}, \text{stat} \right) = robust \text{fit} (\{ \mathbf{1} \ \mathbf{h}\_{\mathbf{M}} \}, \mathbf{h}\_i^j), \tag{4}$$

**FIGURE 2 | Optodes configuration.** Numbers represent the measurement channels: The channel number 16 coincides with the T3/T4 locations in the International 10–20 System.

where <sup>φ</sup><sup>ˆ</sup> *<sup>j</sup> i* and <sup>ψ</sup><sup>ˆ</sup> *<sup>j</sup> <sup>i</sup>* denotes the estimate of <sup>φ</sup> *<sup>j</sup> <sup>i</sup>* and <sup>ψ</sup> *<sup>j</sup> i* , respectively, and *stats* denotes the statistical data including the *t*-value, *p*-value, standard errors, etc.

The idea is to test the null hypothesis that the estimated parameter <sup>φ</sup><sup>ˆ</sup> *<sup>j</sup> <sup>i</sup>* is equal to zero or not. Furthermore, if <sup>φ</sup><sup>ˆ</sup> *<sup>j</sup> <sup>i</sup>* is positive, the particular activation is assumed to be active, and if it is negative, the particular activation is not active for the *j*-the stimulus at the *i*-th channel, in which the *t*-value test has been used. In this paper, the *t*-value was computed using the following equation,

$$t\_i^j = \frac{\hat{\phi}\_i^j}{SE(\hat{\phi}\_i^j)},\tag{5}$$

where *SE* is the standard error of the estimated coefficient. We used two criteria to assess the selection reliability of a particular activation for further analysis. They were *t j <sup>i</sup>* > 0 and *p j <sup>i</sup>* < α, where *p* denotes the *p*-value (in this work, α = 0.05 was set). Alternatively, it could be done by checking *t j <sup>i</sup>* > *t*crt, where *t*crt denotes the critical *t*-value that depends on the degree of freedom (i.e., 71, which is *N* -1). In this case, *t*crt = 1.994.

#### **RESULTS**

The hemodynamics changes in the auditory cortex upon the occurrence of noises in music processing have been examined. Using the paradigm in **Figure 1**, four different stages involving ten conditions in combination with three different noise levels were tested. The three-digit numbers in **Table 2** indicate the differences between the mean and the minimum value of the HbO concentration change in both left and right auditory cortices for 14 subjects (Subject 1∼14). S1∼S4 represent four different experimental stages in **Figure 1**. It is reminded that only those hemodynamic responses whose *p* < 0.05 and *t* > 0 were counted for averaging over 22 channels. It is also noted that (i) the data outside the 3 standard deviations were excluded (i.e., 9 cases in bold-italic in **Table 2** were outliers, see Subject 5 and Subject 12) and (ii) if a right-left difference is not significant (i.e., difference < 0.1σ), it was considered to be the same (they were marked in italic font). **Table 3** summarizes **Table 2**. The following observations are made:


were exposed to the subjects. Considering that the sound level of the music was about 10∼15%, the mid-level noise and music segments showed the highest level of rightlateralization over the subjects (see the second row in S3). Actually, removing two undistinguishable subjects (Subject 11, Subject 14), all twelve subjects out of 14 subjects showed the higher activation in the right auditory cortex. This is due to that the subjects tried to hear the music in the presence of noise (recall that the noise level was 10%). But, if the noise level became higher than the sound level of music (i.e., the high-noise condition, 20%), the number of subjects who showed the higher activation in the right auditory cortex had reduced to 10. Another interesting observation was found from the first row in S3, which provided no sound in the middle of music + noise segments. No sound itself had shown almost the same level of right-lateralization in terms of number of subjects as the case of music + high-noise condition.

iv) The entire music with noise segments (S4): The final stage was to investigate the effect of noise interruptions in hearing music. When there was no noise (see the first row in S4), the tendency of the right-hemispheric lateralization was exactly the same as the case of S1 (i.e., 75%). Also, even upon the interruptions of noise, the tendency was similar.

For the 140 cases (i.e., 14 subjects × 10 conditions), 9 cases were outliers, 11 cases showed no significant difference, 32 cases showed the higher left auditory cortex, and finally 88 cases showed the higher right auditory cortex. Therefore, the overall lateralization across all the experimental conditions was about 73.3% (i.e., 88/120); specifically, for female 81.1% (i.e., 43/53) and for male 67.2% (i.e., 45/67). It is also noted that 20 cases were excluded.

Now, for analyzing the tendency/strength of lateralization, only the cases showing that the right hemisphere is more active than the left hemisphere were further considered (bold font). **Table 4** shows the average and its standard deviation of such cases. That is, the value 2.080 in S1 is the average of 1.453, 2.596, 2.458, 1.704, 2.494, 2.394, 2.357, 2.013, and 1.253 from **Table 2**. The following observations are made. (i) Comparing S1 and S4 in **Table 4**, the background noise intensifies the hemispheric lateralization (i.e., the Right Avg—Left Avg value has increased from 0.311 for music only condition to 0.533 for the mid-noise condition and 0.743 for the high-noise condition, respectively). (ii) As seen in the last column of S2, the difference in the mid-noise condition (i.e., 0.536) is larger than the other two cases (i.e., 0.315 and 0.403). This may reveal the fact that too much noise does destroy the subject's discerning efforts for music in the presence of noise. (iii) In S3, even if there is no sound, the difference in the right auditory cortex was higher than that of the left (i.e., 0.566). This reflects that, in the presence of noise, the subjects tried to listen to the music and this effort was continued even if there is no sound. A similar effort from the subjects was also shown in the mid- and high-noise conditions. Overall, the discerning efforts for music from noises has been seen, which is consistent throughout all three conditions.

**Table 2 | Differences between the mean and the minimum value of the HbO concentration (unit:** μ**M, scale-up by 10−4).**




*The numbers in the parenthesis indicate the number of females (overall: 73.3%, female: 81.1%, male: 67.2%).*

**Table 4 | Average of only those cases where Right** *>* **Left from Table 2.**


*Avg and Std denote average and standard deviation, respectively.*

## **DISCUSSION**

This was the first fNIRS study to examine whether auditorycortex activation by background-noise and music stimuli could change the hemispheric lateralization in both hemispheres. To investigate the effects of noise on music processing, the participants were subjected to various levels of noises in different conditions in the experiment. It was found that exposure to any noise compared to no noise led to higher auditory-cortex activation in the right hemisphere. This result, in fact, is in line with the previous MEG data on speech stimuli (Shtyrov et al., 1998), wherein instead the left hemisphere was dominant in the perception and production of speech. However, when high-level noise was introduced, activation was observed in both hemispheres resulting in a lack of lateralization in auditory-cortex activation. This clearly demonstrates that auditory-cortex activation by music stimuli with noise changes the hemispheric lateralization when a sufficiently loud distraction is introduced to reduce the specialized function of the brain. Similar results from the previous studies on speech stimuli show that lateralization occurs in different responses for noisy condition.

fNIRS have its advantages over fMRI. Previous studies have employed fMRI to determine the effects of various stimuli on neuronal activation in the auditory cortex (Scarff et al., 2004). The drawback of this modality is the problematic effect of acoustic noise from baseline activity in the auditory cortex. Whereas fMRI can measure the changes in magnetic susceptibility in the blood, fNIRS can directly measure hemodynamic changes, HbO and HbR. fMRI signals are physiologically ambiguous, because they reflect changes in cerebral blood flow, cerebral blood volume, and oxidative metabolism. Many studies have investigated the correlation between fMRI and fNIRS signals (Yuan and Ye, 2013). By contrast, the present study focused on the HbO values, as they offer results that are more direct than HbR (Plichta et al., 2006).

We employed the fNIRS technique directly because of its inherent advantages in the analyses of sound-evoked activation: indeed, as a brain-imaging tool, it works non-invasive and silent. Lateralization means a more blood flow in one hemisphere in fNIRS. With this, we assume that more neuronal activity has been made in that region (but, this might be caused by different capillaries in that region or from other aspect, which deserves further neurophysiological inverstigation). We obtained all of the music stimuli results by examining the gap between the mean value and the minimum value of the HbO taken from individual channels covering the auditory cortex. This study observed a pattern of HbO activation that supports the view of functional lateralization in the auditory cortex. Since a variety of light penetrates through the scalp, we can get reliable experimental results by using the difference between the mean and the minimum value of the HbO with two criteria (*t* and *p* values) for statistical analyses. A possible explanation for the difference between S3 and S4 could be due to the influence of emotional behavior on the classical music. Music may contain features that evoke positive (or negative) emotional responses. Hemispheric lateralization increases when noise appeared in S4, which was different from S3.

Another possible explanation for the differences between the results of S3 and S4 could be the full-length music played in the background while introducing the different level of noise. *Für Elise* composed by Ludwig van Beethoven is a commonly known music that everyone can recognize and possibly imagine. Prior research has shown that music perception and music imagination has similarities in brain activation and neural mechanism (Zatorre et al., 1996). If a continuous music was cut off due to noise for a short period of time, the noise level should not make a difference if the subject can imagine the missing portion of the music. These findings further support the idea of the specialization in the auditory cortex evoked by audio stimuli.

# **CONCLUSIONS**

The main finding of this study was that background noise affects the hemodynamic response in the activation of the auditory cortex in music processing. Under various noise levels throughout the four different stages involving ten conditions of the experiment, the involvement of the right hemisphere's auditory cortex was higher than that of the left hemisphere (overall 73.3%, specifically, 67.1% for male and 81.1% for female). Particularly, the subjects listened to the music segments, the lateralization was 75%. Also, if only noise segments were heard, the lateralization was about 65%. But if the music was mixed with noise, such lateralization tendency became intensified. Particularly, the music + noise segments condition revealed 100% rightlateralization when the music level was 10∼15% and the noise level was 10%. However, the noise level became too big (i.e., noise 20%, music 10%), the lateralization had reduced. This is an indication that too much noise diminishes the human's efforts to discern the music from surrounding noises. The obtained results support the theory that the brain is divided into compartments specializing in specific functions.

#### **AUTHOR CONTRIBUTIONS**

Hendrik Santosa performed the experiment and carried out the data processing, Melissa Jiyoun Hong provided suggestions to improve the manuscript, and Keum-Shik Hong supervised the research and corrected the entire manuscript. All of the authors read and approved the final manuscript.

### **ACKNOWLEDGMENTS**

This work was supported by the National Research Foundation of Korea (grant no. NRF-2012-R1A2A2A01046411 and NRF-2014- R1A2A1A10049727).

#### **REFERENCES**


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 11 July 2014; accepted: 16 November 2014; published online: 09 December 2014.*

*Citation: Santosa H, Hong MJ and Hong K-S (2014) Lateralization of music processing with noises in the auditory cortex: an fNIRS study. Front. Behav. Neurosci. 8:418. doi: 10.3389/fnbeh.2014.00418*

*This article was submitted to the journal Frontiers in Behavioral Neuroscience.*

*Copyright © 2014 Santosa, Hong and Hong. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Combined neuromodulatory interventions in acute experimental pain: assessment of melatonin and non-invasive brain stimulation

Nádia Regina Jardim da Silva<sup>1</sup> , Gabriela Laste<sup>1</sup> , Alícia Deitos <sup>1</sup> , Luciana Cadore Stefani 1,2 , Gustavo Cambraia-Canto<sup>1</sup> , Iraci L. S. Torres 1,3 , Andre R. Brunoni <sup>4</sup> , Felipe Fregni <sup>5</sup> and Wolnei Caumo1,2,6 \*

<sup>1</sup> Post-Graduate Program in Medical Sciences, School of Medicine, Universidade Federal do Rio Grande do Sul (UFRGS), Porto Alegre, Brazil, <sup>2</sup> Pain and Anesthesia in Surgery Department, School of Medicine, Universidade Federal do Rio Grande do Sul, Porto Alegre, Brazil, <sup>3</sup> Pharmacology Department, Instituto de Ciências Básicas da Saúde, Universidade Federal do Rio Grande do Sul, Porto Alegre, Brazil, <sup>4</sup> Berenson-Allen Center for Noninvasive Brain Stimulation, Department of Neurology, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, USA, <sup>5</sup> Service of Interdisciplinary Neuromodulation, Department and Institute of Psychiatry, University of São Paulo, São Paulo, Brazil, <sup>6</sup> Pain and Palliative Care Service at Hospital de Clínicas de Porto Alegre (HCPA), Laboratory of Pain and Neuromodulation at UFRGS, Porto Alegre, Brazil

#### Edited by:

Niels Birbaumer, University of Tuebingen, Germany

#### Reviewed by:

Kerstin Luedtke, Universitätsklinikum Hamburg-Eppendorf, Germany Manfred Hallschmid, University of Tuebingen, Germany

#### \*Correspondence:

Wolnei Caumo, Pain and Palliative Care Service at Hospital de Clínicas de Porto Alegre (HCPA), Laboratory of Pain and Neuromodulation at UFRGS, Cel Corte Real, 295 - Bairro Petrópolis Porto Alegre, Rua Ramiro Barcelos, 2350 - CEP 90035-003, Bairro Rio Branco - Porto Alegre – RS, Brazil Tel: (55) 51- 3359.8083, Fax: (55) 51- 3359.8083 caumo@cpovo.net

> Received: 11 November 2014 Accepted: 11 March 2015 Published: 31 March 2015

#### Citation:

da Silva NRJ, Laste G, Deitos A, Stefani LC, Cambraia-Canto G, Torres ILS, Brunoni AR, Fregni F and Caumo W (2015) Combined neuromodulatory interventions in acute experimental pain: assessment of melatonin and non-invasive brain stimulation. Front. Behav. Neurosci. 9:77. doi: 10.3389/fnbeh.2015.00077 Transcranial direct current stimulation (tDCS) and melatonin can effectively treat pain. Given their potentially complementary mechanisms of action, their combination could have a synergistic effect. Thus, we tested the hypothesis that compared to the control condition and melatonin alone, tDCS combined with melatonin would have a greater effect on pain modulatory effect, as assessed by quantitative sensory testing (QST) and by the pain level during the Conditioned Pain Modulation (CPM)-task. Furthermore, the combined treatment would have a greater cortical excitability effect as indicated by the transcranial magnetic stimulation (TMS) and on the serum BDNF level. Healthy males (n = 20), (aged 18–40 years), in a blinded, placebo-controlled, crossover, clinical trial, were randomized into three groups: sublingual melatonin (0.25 mg/kg) + a-tDCS, melatonin (0.25 mg/kg) + sham-(s)-tDCS, or sublingual placebo+sham-(s) tDCS. Anodal stimulation (2 mA, 20 min) was applied over the primary motor cortex. There was a significant difference in the heat pain threshold (◦C) for melatonin+a-tDCS vs. placebo+s-tDCS (mean difference: 4.86, 95% confidence interval [CI]: 0.9 to 8.63) and melatonin+s-tDCS vs. placebo+s-tDCS (mean: 5.16, 95% CI: 0.84 to 8.36). There was no difference between melatonin+s-tDCS and melatonin+a-tDCS (mean difference: 0.29, 95% CI: −3.72 to 4.23). The mean change from the baseline on amplitude of motor evocate potential (MEP) was significantly higher in the melatonin+a-tDCS (−19.96% ± 5.2) compared with melatonin+s-tDCS group (−1.36% ± 5.35) and with placebo+s-tDCS group (3.61% ± 10.48), respectively (p < 0.05 for both comparisons). While melatonin alone or combined with a-tDCS did not significantly affect CPM task result, and serum BDNF level. The melatonin effectively reduced pain; however, its association with a-tDCS did not present an additional modulatory effect on acute induced pain.

Keywords: tDCS, TMS, CPM, pain threshold, melatonin, clinical trial

**Trial registration**: current controlled trial is registered at clinical trials.gov upon under number: NCT02195271.

# Introduction

Transcranial direct current stimulation (tDCS) is capable of modulating pain systems. Several studies have shown that tDCS applied on the primary motor cortex (M1) and/or the prefrontal cortex (among others) shows clinically significant pain reduction in various chronic pain syndromes, such as fibromyalgia (Fregni et al., 2006; Lefaucheur et al., 2008; Valle et al., 2009; Mendonca et al., 2011; Mylius et al., 2012; O'Connell et al., 2014; Vaseghi et al., 2014), Phantom pain (Bolognini et al., 2013), trigeminal neuralgia (Hagenacker et al., 2014), chronic migraine (Dasilva et al., 2012), low back pain (O'Connell et al., 2013) and Myofascial Pain Syndrome (Sakrajai et al., 2014) tDCS was shown to have a benefit on decreasing pain scores. However, these positive results should not be considered in isolation as some of these studies have methodological shortcomings that may bias these results, leading for instance to false positive findings. The proposed mechanism for tDCS on pain rests on its polarity-dependent shifts of the resting membrane potential and consequent cortical and subcortical modulation (Simis et al., 2014). Hence, one strategy to optimize the analgesic effects of active (a)-tDCS is its combination with pharmacological interventions (Brunoni et al., 2011c).

Which has shown advantages such as the augmentation of its clinical effects, as was observed when combined with sertraline for major depression (Brunoni et al., 2013). In pain, a case report of tDCS combined with D-cycloserine (an N-methyl-D-aspartate agonist) suggested its beneficial clinical effects (Antal and Paulus, 2011).

Pre-clinical evidence have demonstrated melatonin effects on inflammatory (Laste et al., 2013) and neuropathic pain (Ambriz-Tututi and Granados-Soto, 2007), and clinical trials in acute (Caumo et al., 2007, 2009) and chronic human pain (Citera et al., 2000; Hussain et al., 2011; Schwertner et al., 2013; Vidor et al., 2014). Melatonin modulates pain systems such as the gamma-aminobutyric acid (GABAergic) and opioidergic systems (Ambriz-Tututi and Granados-Soto, 2007; Zurowski et al., 2012). Its long-term use in endometriosis and fibromyalgia improves pain and decreases the levels of serum brain-derived neurotrophic factor (BDNF; Schwertner et al., 2013; de Zanette et al., 2014). Furthermore, an experimental study showed that melatonin constrained the synaptic plasticity in a concentration-dependent manner (≥1 nM) (Wang et al., 2005), affecting networks that are not directly influenced by tDCS, such as subcortical pain circuits. The concurrent use of tDCS and conditioned pain modulation (CPM), which modulates the descending pain control systems, also show a synergistic effect (Reidler et al., 2012); thus, it is conceivable that such a combination would potentiate melatonin's effects on pain.

Plasticity in both excitatory and inhibitory circuits in the human motor cortex is regulated by homeostatic metaplasticity (Murakami et al., 2012). Therefore, in this explanatory trial, we tested the hypothesis that compared to the control condition and melatonin alone, a-tDCS combined with melatonin would have a greater effect on pain modulatory effect, as assessed by quantitative sensory testing (QST) and by the pain level during the CPM-task. Furthermore, the combined treatment would have a greater cortical excitability effect as indicated by the transcranial magnetic stimulation (TMS) and on the serum BDNF level.

# Material and Methods

# Study Design, Setting, and Participants

All volunteers provided written informed consent before participating in this study, and the protocol was approved by the Research Ethics Committee at the Hospital de Clínicas de Porto Alegre (Institutional Review Board IRB -13-0155) according to the Declaration of Helsinki. The volunteers were recruited from the general population by advertisement postings in the universities, on the internet, and in public places in the Porto Alegre area. Subjects were considered eligible to participate if they were male, right-handed, and between 19 and 40 years of age, and were screened for eligibility by phone. They answered a structured questionnaire that assessed the following variables: current acute or chronic pain conditions, use of analgesics in the past week, rheumatologic disease, clinically significant or unstable medical or psychiatric disorder, history of alcohol or substance abuse in the past 6 months, neuropsychiatric comorbidity, and use of psychotropic drugs. Subjects responding affirmatively to any of these questions, and those with contra-indications for TMS (Rossi et al., 2009) were excluded. Subjects with Beck Depression Inventory (BDI; Warmenhoven et al., 2012) scores higher than 13 were also excluded (Beck et al., 1996). We include males only to exclude the influence of the cyclical fluctuation of gonadal steroids during the menstrual cycle on pain threshold and in the cortical excitability parameters (Smith et al., 2002; Stefani et al., 2012).

# Sample Size

The number of subjects in each study group was determined according to parameters of a previous study (Stefani et al., 2012). A priori estimate indicated that in a superiority test from a crossover design, a sample size of 20 subjects divided into three groups with a 2:2:1 ratio, in three sessions, to test for difference between interventions groups on mean of 2.5◦C (SD 3◦C) for the heat pain threshold (HPT), with a variation coefficient of 0.5, superiority margin 0.22 and to achieves 80% power at a 5% significance. We estimated a sample size for a large effect size using the Analysis of variance. To be an incomplete blocks crossover trial each subject received some interventions but not all subjects received interventions in the third session. This mean that the allocation in a cross-over manner in the first and second sessions was a-tDCS+melatonin (n = 8), s-tDCS+melatonin (n = 8) and in s-tDCS+placebo (n = 4), respectively. In the third session a-tDCS+melatonin (n = 4), s-tDCS+melatonin (n = 4) and in s-tDCS+placebo (n = 2), respectively. The estimative was determined using the Power Analysis and Sample Size Software PASS version 13 (NCSS Statistical Software, Kaysville, Utah).

#### da Silva et al. Effects of melatonin and tDCS on pain

# Interventions

The intervention involved one dose of sublingual melatonin (Sigma Chemical, Germany; batch-by-batch certificates of analysis for authenticating the purity of each batch provided): 0.25 mg/kg (maximum dose 20 mg), or placebo (Stefani et al., 2013). This solution was combined with 0.5 mL of 10% glucose solution. The placebo was an equivalent volume of 10% glucose solution. The tDCS was introduced 10 min after administer the melatonin to conclude the session (20 min) 30 min after administer the melatonin, since in previous study we demonstrated that at this time we get a serum peak of melatonin when it is used sublingual (Stefani et al., 2013).

tDCS is a therapeutic tool that is relatively inexpensive, non-invasive, painless, safe, its shape can be simulated (sham) and used efficiently for double-blinded studies. In this study, the anode was positioned over the left M1, and the cathode was positioned on the right supraorbital region. The rubber electrodes were inserted into a 35-cm<sup>2</sup> sponge (moistened with NaCl). The current was 2 mA, and the attachment of electrodes to the scalp was maintained by an elastic band (Vandermeeren et al., 2010). The stimulation time was 20 min, consistent with previous studies (Valle et al., 2009; Knotkova et al., 2013). For the sham conditions, the device was turned off after 1 min of starting the stimulation, which is a reliable blinding method (Brunoni et al., 2011b), capable of mimicking the common adverse effects induced by the real stimulation (Brunoni et al., 2011a). The evaluators and subjects were blinded to the treatment; contact between participants was avoided to enhance study blinding.

# Randomization

The randomization was generated by a computer with a fixed block size of 5. Twenthy subjects were randomly allocated to receive three sequences of treatment (melatonin+active(a)-tDCS, melatonin+sham(s)-tDCS, and s-tDCS+placebo). An allocation of 2:2:1 in favor of the melatonin treatment to maximize allocation to the experimental group and to improve the experimental power. To be an incomplete blocks crossover trial each subject received some interventions but not all subjects received interventions in the third session. This mean that the allocation in a cross-over manner in the first and second sessions was a-tDCS+melatonin (n = 8), s-tDCS+melatonin (n = 8) and in s-tDCS+placebo (n = 4), respectively. In the third session a-tDCS+melatonin (n = 4), s-tDCS+melatonin (n = 4) and in s-tDCS+placebo (n = 2), respectively. The experimental desing and interventions in each session is presented in the **Figure 1**. Before the recruitment phase, opaque envelopes containing the protocol materials were prepared. Each opaque envelope was sealed and numbered sequentially. The opaque envelopes were opened by the nurse who administered the medications only after gaining subjects' informed and signed consent.

# Blinding

To control for possible measurement bias participants were instructed to discuss all aspects related to their tDCS treatment only with their treating physician (rather than the research personnel). During the sham stimulation, subjects underwent tDCS experiences that were comparable to the active stimulation. Individuals other than those responsible for administering the interventions were blinded to the allocated interventions. Further, to assess whether blinding was effective, at the end of the experiment we asked participants to guess whether they had received active or sham tDCS and to rate their confidence on the answer on a Likert scale with five categories (no confidence to completely confident). This scale was used to assess the blinding about both interventions (tDCS and sublingual).

# Outcomes

The primary outcome was the HPT as assessed by QST. The secondary outcomes were the excitability of the cortical spinal system indexed by motor-evoked potentials (MEPs), pain reduction on the Numerical Pain Scale (NPS0−10) during the CPM task, and other cortical excitability parameters (intracortical facilitation (ICF), current silent period (CSP), intracortical inhibition (ICI)), and serum BDNF.

The method of limits with a computer Peltier-based device thermode (30 × 30 mm) was used to assess the heat pain threshold (HPT; Schestatsky et al., 2011). The thermode was attached to the skin on the ventral aspect of the mid-forearm, and the temperature was increased at a rate of 1◦C/s, from 32◦C to a maximum of 52◦C, which primarily stimulates C-nociceptive afferents (Backonja et al., 2013). Participants were asked to press a button as soon as the sensation of heat began (heat detection threshold) and as soon as the stimulation became painful (HPT). Three assessments were performed with an interstimuli interval of 40 s. Each subject's HPT was defined as the mean painful temperature of the three assessments. The position of the thermode was slightly altered between trials (although it remained on the left ventral forearm) to avoid either sensitization or response suppression of the cutaneous heat nociceptors. The same equipment was used to determine the maximum tolerated temperature, where volunteers pressed a button to stop the temperature increase. If 52◦C was achieved before reporting pain, the device cooled down automatically and the pain threshold was considered unknown.

To test the CPM, we used the term CPM rather than diffuse noxious inhibitory control/DNIC because of the recent recommendations of Yarnitsky et al. (2010), we used the protocol of Tousignant-Laflamme et al. (2008) and the guidelines for the cold-pressor task (CPM-TASK) as an experimental pain stimulus (von Baeyer et al., 2005). The CPM-TASK activates the diffuse noxious inhibitory control-like effect (CPM) because it is a strong nociceptive stimulus that takes place over a lengthy time span (Willer et al., 1989) and is applied over a large body surface area (Marchand and Arsenault, 2002). The CPM-TASK allows us to modify the endogenous pain-modulating system. To quantify the CPM, we evaluated the pain intensity of three tonic heat pain (HPT) test stimuli separated by a CPM-TASK. Although the HPT might lead to habituation and sensitization according to the dual process theory, cold water to zero is a reliable stimulus to induce CPM (Tousignant-Laflamme et al., 2008).

CPM-Task: The cold-pressor task was used as a conditioning stimulus to elicit a strong and prolonged pain sensation to trigger the CPM. The CPM-TASK consisted of immersing the nondominant hand in cold water (zero to 1◦C) for 1 min. During the last 30 s of the cold-water immersion, the HPT procedure was administered over the right forearm (dominant forearm). The temperature was held constant during the experiment for each subject. The HPT that elicited pain ratings of 6/10 on the Numerical Pain Scale [(NPS) 0/10] (HPT60) was used for the first HPT before the CPM-TASK (HPT0). After a short break, the HPT0 was applied at the volar region. Following HPT0, the CPM-TASK was used to trigger the CPM. One minute after the CPM-TASK, we applied the second HPT (HPT1). We quantified the amount of the CPM by subtracting the mean pain rating of HPT1 from the first HPT0 before the CPM-TASK (HPT1); negative values indicate inhibitory CPM. This test was applied after measuring the cortical excitability parameters.

Cortical excitability parameters were registered through surface electromyography recordings, which were gathered at the contralateral right first dorsal interosseous muscle using Ag/AgCl electrodes. First, the resting motor threshold (RMT) was determined by obtaining five MEPs with a peak-to-peak amplitude of 50 µV out of 10 consecutive trials using the minimum output of the TMS device. Next, 10 MEPs were recorded with an intensity of 130% of the individual RMT. The CSPs were assessed during muscle activity measured by a dynamometer to be approximately 20% of the maximal force. Accordingly, 10 CSPs were recorded using an intensity of 130% of the RMT. The short-interval ICI (SICI), using an inter-stimulus interval of 2 ms was also assessed. The first conditioning stimulus was set at 80% of the RMT, whereas the second test stimulus was set at 100% of the individual MEP intensity. The ICF was assessed with an interstimulus interval of 12 ms. Paired-pulse TMS was conducted in a randomized order for a total of 30 trials (10 for each SICI, ICF, and control stimuli). Off-line analyses included collecting the amplitudes of all MEP, SICI, and ICF values, as well as the duration of the CSPs. The corresponding units for these parameters are mV for MEP, ratio to MEP for SICI and ICF, and ms for CSP (Pascual-Leone et al., 1994).

The serum BDNF concentration was determined using an enzyme-linked immunosorbent assay kit (Chemicon/Millipore, catalog n◦ CYT306). The serum was frozen at −80◦C until the assays were performed.

# Other Instruments and Assessments

Pain catastrophizing thinking was assessed using the validated Brazilian-Pain Catastrophizing Scale (Sehn et al., 2012). Depression symptoms were screened using the BDI (Warmenhoven et al., 2012). Anxiety was measured with the State-Trait Anxiety Inventory (STAI), adapted to Brazilian Portuguese (Kaipper et al., 2010). Demographic data were gathered using a standardized questionnaire. The clinical assessment of sedation was determined by simultaneous recording using a visual analog scale (VAS0–10) ranging from zero (sleepiness) to 10 (completely awake). To assess safety, we used the Systematic Assessment for Treatment with tDCS questionnaire based on previously reported adverse events (Brunoni et al., 2011b).

# Statistical Analyses

The differences among the sequence cohort were examined with the analysis of variance (ANOVA) for parametric variables, and categorical outcomes were examined by chi-square or Fisher's tests.

Continuous data were evaluated for normality using Shapiro-Wilk test. After verifying the corresponding assumptions the results were evaluated using the absolute mean variation for HPT of delta values (post-treatment minus pre-treatment). We analyzed the data using a mixed ANOVA model in which the independent variables were the cohort time of session (time), treatment (placebo+s-tDCS, melatonin+s-tDCS, and melatonin+a-tDCS), the interaction term time vs. the treatment group, and subject identification.

The results were evaluated using the absolute mean variation for MEPs of the percentage of variation [(post-treatment−pretreatment)/post-treatment] × 100. The HPT was adjusted by the sleepiness score assessed by a VAS0–10. All analyses were performed with two-tailed tests at the 5% significance level. All analyses were adjusted for multiple comparisons using Bonferroni test. However due to the excessive number of outcomes, some of our results should be considered exploratory and thus need to be replicated in confirmatory trials. An intention-to-treat analys was planned according to the last observation carried forward through the time points if we had oberved dropouts. The analyses were performed with SPSS version 20.0 (SPSS, Chicago, IL).

# Results

# Subject Characteristics

Twenty healthy subjects were randomized, with the ratio of 2:2:1 to the three interventions, in three sessions to participate in the three sequences of treatment (**Figure 1**). The demographic and psychological characteristics of the subjects according to the sequence allocation were comparable and are shown in **Table 1**. All subjects completed the course protocol for which they had been randomized. There was no carry over effect, tested by comparison of pre-treatment assessments (p > 0.05). Although participants correctly guessed the intervention used in the transcranial stimulation (tDCS) and sublingual, when the question was about the level of certain of their assigned intervention group, only melatonin was guessed correctly but tDCS not. A maximum of 23% of the subjects in each group correctly guessed the activetDCS condition; the level of confidence in the intervention was moderate to high in more than 75% of the individuals in all groups, and the percentages of answers between groups, for each item were similar without statistically significant differences (p > 0.05, for both measures). Importantly, our results would not change if we exclude subjects with higher incidence of adverse effects.

The incidence of reported side effects presented a similar distribution between groups. Headache, neck and scalp pain, skin redness, mood changes, and difficulties in concentration were reported by <15% of subjects. Burning and itching were reported by more than 25% of the subjects. Tingling was the most common side effect reported, with an incidence higher than 30%. The scores on the VAS0–10 (higher score less sleepiness) showed that placebo+s-tDCS groups 9.62 ± 0.52 induced lower sleepiness than the active arms (melatonin+a-tDCS 5.62 ± 1.31 and melatonin+s-tDCS 5.93 ± 1.43; p < 0.01 for each comparison vs. placebo+s-tDCS), although there was no difference between the two active tDCS groups (p = 0.9). The VAS0–10 scores for sleepiness vs. group comparison did not demonstrate a significant interaction (F(2,46) = 0.18; p = 0.84). Additionally, there was no statistically significant effect of sleepiness score on HPT (β = 0.31, t = 1.16; 95% confidence interval [CI]: −0.23 to 0.85; p = 0.25).

# Treatment Effects on the HPT (Primary Outcome) and on the Descending Modulatory System (Secondary Outcome)

In the incomplete factorial analysis, there were two factors: (atDCS and s-tDCS) and melatonin (real or placebo). The analysis showed no significant interaction between tDCS and melatonin on HPT (F(2,46) = 0.3; p = 0.95), but a significant main effect for treatment was observed (F = (2,46) = 3.94; p = 0.02); (**Table 2**). The differences mean in the HPT tests are presented in (**Figure 2**).

The function of the descending modulatory system was assessed using the CPM task. Although all the interventions improved the pain reduction during the CPM task, there were no differences in their effectiveness between them (p > 0.05). The reduction in pain scores on the NPS0-10 during the CPM task was 48.41% (HPT0 = 5.04 ± 1.06; HPT1 = 2.60 ± 1.27) in the melatonin+active-tDCS group, 37.88% (HPT0 = 4.25 ± 1.37; HPT1 = 2.64 ± 1.55) in the melatonin+sham-tDCS group, and 33.74% (HPT0 = 4.83 ± 1.06; HPT1 = 3.2 ± 1.42) in the placebo+sham-tDCS group. These results reveal that the interventions did not change the descending modulatory system as assessed by the CPM task.

# Effect on the Neurophysiological Outcomes (Secondary), as Indicated by the TMS Cortical Excitability Parameters: MEPs, ICI, ICF, CSP, and BDNF

Similar analyses showed significant main effects of the intervention group for MEPs (F(2,46) = 11.55; p = 0.03). There was

#### TABLE 1 | Values are given as the mean (±SD) or as a frequency according to the sequence cohort (n = 20).


SD = standard deviation, tDCS = transcranial direct current stimulation.

#### TABLE 2 | The mean delta score (SD) (post-treatment values minus pre-treatment values) of the heat pain thresholds and motor-evoked potentials (n = 20).


SD = standard deviation, CI = confidence interval, tDCS = transcranial direct current stimulation.

\*Percentage represents the percent change, calculated as [(post-intervention–pre-intervention)/post-intervention] × 100.

<sup>U</sup>p value represents the results from the mixed-model analysis of variance <sup>×</sup> group interaction (for the main analysis) and for the factorial analysis.

Different superscripts (a and b) indicate significant differences among intervenios groups according to the Bonferroni test.

significant difference in MEP amplitude between the treatment group melatonin+a-tDCS and the melatonin+s-tDCS group (−19.96% ± 5.2 vs. −1.36% ± 5.35; mean difference: −18.60%, 95% CI: −42.44 to −7.12; p = 0.03) and melatonin+a-tDCS and the placebo+s-tDCS group (−19.96% ± 5.2 vs. 3.61% ± 10.48; mean difference: −23.57%, 95% CI: −39.68 to −1.2; p = 0.01). However, there was no significant difference in MEP amplitude between the melatonin+s-tDCS and the placebo+s-tDCS group (−1.36% ± 5.35 vs. 4.31% ± 10.56; mean difference: −5.67%, 95% CI: −39.68 to −1.2; p = 0.48). The differences between the groups in the percentage of variation before and after treatment are shown in **Figure 3**.

The MEP differences (mean ± SD) before and after treatment, irrespective of the session sequence, are presented in **Table 2**. Melatonin alone did not result in any significant MEP changes.

The effects of the interventions on the secondary outcomes related to cortical excitability are presented in **Table 3**. The interventions did not induce significant changes on the other cortical excitability parameters (ICF, ICI, and CSP). No significant difference between the treatment groups was observed for the serum BDNF levels at baseline, which had great variability (**Table 1**). From the baseline level, the serum BDNF level demonstrated a mean decrease of 10.96% in the placebo+s-tDCS group, whereas the melatonin+s-tDCS and melatonin+a-tDCS groups presented mean reductions of 12.79% and 6.09%, respectively (**Table 3**).

#### TABLE 3 | Outcomes related to cortical excitability and serum BDNF (20 subjects) and the number of assessment.


Mean (SD) pre vs. post-intervention.

SD = standard deviation, tDCS = transcranial direct current stimulation.

Standardized mean difference (SMD) [(pre minus post)/baseline standard deviation]. The size effect was interpreted as follows: small, 0.20; moderate, 0.50–0.60 and large, 0.80.

All comparisons between the melatonin+tDCS, melatonin+sham-tDCS, and placebo+sham-tDCS groups were performed using a mixed analysis of variance model (p > 0.05 for all comparisons). Different superscripts a indicate absence of significant differences among treatment groups according to the Bonferroni test.

# Discussion

current stimulation.

The main findings of this study confirm that melatonin significantly affects the pain pathways, which are not changed by the concurrent tDCS stimulation. Furthermore, this effect does

correction for multiple post hoc comparisons. tDCS = transcranial direct

not seem to be associated with changes in cortical excitability. This finding contrasts to our initial hypothesis that melatonin combined with tDCS would improve pain control, considering that treatment with tDCS (Valle et al., 2009; Mendonca et al., 2011) or melatonin alone demonstrated an effect on pain in preclinical (Laste et al., 2012a,b), experimental (Stefani et al., 2013), and clinical studies (Caumo et al., 2002, 2007, 2009; Schwertner et al., 2013; Vidor et al., 2013, 2014) One possible explanation for this result is that melatonin induced maximum homeostatic control to modulate the painful stimuli via neurobiological systems that are common targets for both interventions (i.e., melatonin induced a ceiling effect on pain). This hypothesis is biologically plausible and is supported by pre-clinical evidence indicating that the GABAergic (Wilhelmsen et al., 2011), opioid, and glutamatergic systems (Mantovani et al., 2006) act as targets for both melatonin and tDCS.

Another explanation for the lack of an effect when tDCS was combined with melatonin may be that melatonin blocked the effects of tDCS. It has been shown that pharmacological agents (Liebetanz et al., 2002) such as benzodiazepines, are capable of partially blocking the clinical effects of tDCS (Brunoni et al., 2013). Thus, the increased excitability of GABA-A and GABA-B circuits in M1 might increase the inhibitory tone, which is responsible for the general occlusion of the subsequent induction of long-term potentiation- and long-term depressionlike plasticity (Castro-Alamancos and Borrell, 1995; Castro-Alamancos et al., 1995; Hess et al., 1996). Therefore, it is plausible that the failure of additive effect of a-tDCS+melatonin is explained by a similar response, because of melatonin action on GABA-A receptor (Coloma and Niles, 1988; Niles and Peace, 1990). It is also possible that the lack of interaction effect is a result of metaplasticity, i.e., when two plasticity protocols are used together, the effect of the first one modulates that of the second (Murakami et al., 2012). Other mechanism to explain this finding is the depotentiation, which refers to two protocols that when used alone do not induce changes in the excitability, but when used together cancel out the effect of a preceding potentiation protocol to achieve homeostasis (Froc et al., 2000; Yashiro and Philpot, 2008; Müller-Dahlhaus and Ziemann, 2014). Accordingly, the tonic depression of the nociceptive threshold may result from the activation of pro-nociceptive areas of the brain or from inhibition of the endogenous pain inhibitory system (Burkey et al., 1999).

Other explanations for these results, that challenge our hypothesis, are evidences of previous clinical studies, which demonstrated that the behavioral data did not indicate a pain-reducing effect of anodal stimulation (Grundmann et al., 2011; Jürgens et al., 2012; Luedtke et al., 2012). Interestingly, previous studies on experimental pain using the same stimulation paradigm also showed inconclusive effects of tDCS on psychophysical variables (Grundmann et al., 2011; Jürgens et al., 2012; Luedtke et al., 2012). These inconclusive effect of change on cortical nociceptive processing, as a response to heat pain was also reported in other recent study, which did not found neither cathodal nor anodal tDCS effect over the left M1 (1 mA, 15 min) (Ihle et al., 2014).

The effect of a-tDCS on neurophysiological outcomes (such as evoked potentials) demonstrated in the present study, were also reproduced in the majority of trials after tDCS (Matsunaga et al., 2004; Csifcsak et al., 2009; Luedtke et al., 2012). Perhaps the psychophysical variables depend on a range of different pathways because evaluation of pain is a more complex process than mere somatosensory processing in evoked potentials. Higher stimulation intensities, longer stimulation duration, or repeated stimulation sessions may be required to produce a statistically significant experimental pain reduction that matches the effect observed in chronic clinical pain studies. Although M1 excitability is a reliable marker for indexing the effects of interventions on pain (Volz et al., 2013; Dall'Agnol et al., 2014; Vidor et al., 2014), this marker seems to be more specific for chronic pain than for acute experimental pain. It also suggests that melatonin modulation on pain does not involve a direct effect on M1, while tDCS does (Reidler et al., 2012; Knotkova et al., 2013).

In addition, we have shown that melatonin's effect on pain is not mediated by descending pain control systems. In fact, in this study, using the CPM task, the pain score on the NPS0−<sup>10</sup> was reduced by more than 30% in all of the treatment groups including the control group. This is consistent with previous studies demonstrating an approximated CPM effect of 29% (Pud et al., 2005, 2009; Niesters et al., 2013). The conditioning stimulus used in this study (hand placed in water at 0–1◦C for at least 30 s) was a strong, painful stimulus that depresses the nociceptive messages elicited from remote localized body areas. Here, ceiling effects were also possible, i.e., the CPM responses were at their maximum effect given the intensity of the conditioning stimulus used. However, other studies have shown that it is possible to modulate CPM using melatonin, if used in the long-term (de Zanette et al., 2014), or with tDCS alone (Reidler et al., 2012). These results, namely the lack of melatonin-induced M1 modulation and descending inhibitory pain system involvement, support to some extent the notion that acute melatonin after-effects may have limited impact on cortical and spinal systems, thus suggesting that melatonin may modulate subcortical centers. However, this hypothesis needs to be confirmed in further trials with other neurophysiological techniques or functional imaging techniques, such as quantitative electroencephalography, near-infrared spectroscopy (NIRS) or functional magnetic resonance imaging (fMRI).

The a-tDCS effect on the corticospinal system is related to increased MEP amplitude (**Table 2**), an effect that is consistent with previous studies that the anodal tDCS over the M1 induced an enhancement of the corticospinal excitability (Pellicciari et al., 2013). The tDCS effect on the cortico-subcortical networks is also supported by recent evidence of a functional coupling increase on the thalamo-cortical circuits following anodal stimulation over the motor cortex (Polanía et al., 2012). We speculate that the not site-limited cortical excitability increase could be determined by a decrease of the contralateral hemisphere inhibition, mediated, at least partially, by the anodal tDCS-induced reduction of GABA concentration (Stagg et al., 2009). Also, the tDCS might induce an increased cortical evoked response with a probable concurrent involvement of the N-Methyl-D-aspartate (NMDA) receptors (Islam et al., 1995; Nitsche et al., 2003). Thus, these findings suggest that the modulatory effects produced by a-tDCS were not limited to the targeted cortical area but also occur at distant interconnected sites including spinal tract. Given the results, it is likely that the tDCS does not have a direct excitatory or inhibitory effect but mostly a modulation role, presumably expressed as to changes in the excitability of cortical circuits.

The present findings showed that the use of melatonin alone or with tDCS did not induce changes in the serum BDNF levels. Although it is widely distributed in the CNS, the BDNF has been used as a possible neuroplasticity marker that is modulated by rTMS (Dall'Agnol et al., 2014), tDCS (Brunoni et al., 2014), or melatonin treatment (Schwertner et al., 2013; de Zanette et al., 2014), particularly when assessed in the long-term treatment of chronic pain. However, in this study, serum BDNF was measured in healthy subjects and only a short time after one intervention session. Another possible explanation of the lack of changes in serum BDNF in the present study is that the intervention effect was not sufficient to induce a level of neuroplasticity detectable on serum BDNF. These hypotheses are plausible considering that BDNF is produced in the CNS and transported through the blood-brain barrier via saturable systems (Poduslo and Curran, 1996; Pan et al., 1998; Asmundson et al., 1999). Although the study demonstrated that the CNS contributes to 70−80% of the circulating BDNF (Rasmussen et al., 2009), this measure may underestimate its real level in the CNS, since it was demonstrated that in healthy subjects it can be 14-fold the BDNF level in the plasma (Yoshimura et al., 2014). These findings are important for understanding the physiological mechanisms and the pharmacological and non-pharmacological substrates of the combined effect of melatonin and active-tDCS on pain. However, for a better comprehension this effect further studies are needed in patients with chronic pain, which show amplified sensory pain.

It is important to emphasize that we chose a crossover design as to have a single-subject design, in which the subjects serve as their own control. This design is sensitive to individual organism's differences allowing better assessment of causal relationship between the independent and dependent variables (Xeniditis et al., 2001; Dallery et al., 2013). Whereas it reduces the between subjects comparison, it is an ideal strategy to validate results because subjects have significant variability when assessing outcomes related to behavior and physiological parameters. In addition this design also helps with controlling between-subject differences in the effects of stimulation as recent evidence based computational models suggest that inter individual differences in head anatomy may affect the distribution of the electric field in the brain and that a uniform dose of stimulation for all patients may not be the most efficient procedure (Datta et al., 2012). Given the costs associated with individual modeling required to customize the stimulation on an individual basis, our design controls for this issue as we compare the same subjects before and after each intervention (Datta et al., 2012).

Several issues concerning the design of our study must be address: **First**, the absence of a group of placebo plus a-tDCS is a limitation of our study. However, previous studies showed the effectiveness of a-tDCS in increasing on sensory and pain thresholds in healthy individuals and pain levels in patients with chronic pain (Vaseghi et al., 2014). Other possibility to explain this finding is that s-tDCS potentiates the mechanisms involved in placebo analgesia as suggested by a recent study (Dossantos et al., 2014). **Second**, even though the tDCS is an efficient technical solutions to conduct blinded studies of both the patients and experimenters (Gandiga et al., 2006) the efficacy of patient blinding has been questioned especially be present at stimulation intensities of 2 mA compared with lower intensities (O'Connell et al., 2012). However, it is improbable that the unblinding change the directions of our conclusions, because the findings did not change when analyzing only subjects that did not guess the allocation group. Accordingly, as neurophysiological studies have shown a stimulation shorter than 3 min induce no significant after–effects (Nitsche and Paulus, 2000). **Third**, we included only males subjects, because an enhanced pain response in females has been attributed to physiological and psychological variables, including mechanisms of endogenous inhibition, the capability to endure pain, genetic factors, pain expectation and personality traits (Keefe et al., 2000; Wiesenfeld-Hallin, 2005). In this context, the gender may be an important confounding factor because female are more prone to activation upon negative emotional responses (i.e., stress, fear, and anxiety) and higher trait-anxiety is associated with an imbalance between excitatory and inhibitory descending systems of the corticospinal tract (Vidor et al., 2014). Another factor to consider is the hormonal variation throughout the menstrual cycle. Finally, even after the adjustment for multiple comparisons the effect of melatonin was significantly on heat pain threshold and the a-tDCS on MEP (see **Table 2**). We agree that our study, as the majority of similar studies has also an exploratory nature and thus it is possible that our study has increased type I and type II error. Hence the results of secondary outcomes should be interpreted as explanatory.

The melatonin effectively reduced pain; however, its association with a-tDCS did not present an additional modulatory effect on acute induced pain. Melatonin effects on induced acute pain did not seem to be mediated by cortical or brainstem modulation given the lack of results from the cortical excitability and descending pain control systems. Although the a-tDCS changed the cortical-spinal excitability assessed by MEP this effect not changed the CPM. In fact, these findings might have physiological implications to support an understanding of the maximum homeostatic physiological control when are used combined interventions which have common targets to modulate the painful stimuli.

# Author Contribution

AB participated in the sequence alignment and drafted the manuscript. MT participated in the sequence alignment. ES participated in the design of the study and performed the statistical analysis. FG conceived the study, participated in its design and coordination and helped drafting the manuscript. Nadia Regina Jardim da Silva: nadia.jardimsilva@gmail.com; (UFRGS); FG Gabriela Laste: gabrielalaste@gmail.com; (UFRGS); MT Alícia Deitos: aliciadeitos@gmail.com; (UFRGS); MT Gustavo Cambraia-Canto: guscanto91@gmail.com; (UFRGS); MT Luciana Stefani: lustefani@terra.com.br; (UFRGS); ES Iraci Torres: iracitorres@gmail.com; (UFRGS); AB Andre R Brunoni: brunoni@usp.br; (São Paulo); AB Felipe Fregni: Fregni.Felipe@mgh.harvard.edu; (Harvard Medical School); ES Wolnei Caumo: caumo@cpovo.net. (UFRGS). Responsible for maintaining the study records; FG.

# Acknowledgments and Disclosures

The present research was supported by the following Brazilian agencies: Research grant: National Council for Scientific and Technological Development-CNPq (I.L.S. 302345/2011-6 Torres and W. Caumo WC-301256/2013-6). Brazilian Innovation Agency (FINEP) process number—1245/13. Post-doctoral

# References


grant: Committee for the Development of Higher Education Personnel—CAPES - PNPD/CAPES, GL, process number (No: 71/2013). Assistance, medicines, equipment, and administrative support: Postgraduate Research Group at the Hospital de Clínicas de Porto Alegre, Number: 13-0155. The institutions (HCPA, UFRGS) received support from the following governmental. Brazilian agencies: The Foundation of Support of Research at Rio Grande do Sul (FAPERGS), National Council for Scientific and Technological Development-CNPq, and Committee for the Development of Higher Education Personnel.


**Conflict of Interest Statement**: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2015 da Silva, Laste, Deitos, Stefani, Cambraia-Canto, Torres, Brunoni, Fregni and Caumo. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution and reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.