Edited by: Leonardo G. Cohen, National Institutes of Health, USA
Reviewed by: Gregor Thut, University of Glasgow, UK; Giuseppe Sciamanna, University of Rome Torvergata, Italy
*Correspondence: Christoph Zrenner
This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution and reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
Closed-loop neuroscience is receiving increasing attention with recent technological advances that enable complex feedback loops to be implemented with millisecond resolution on commodity hardware. We summarize emerging conceptual and methodological frameworks that are available to experimenters investigating a “brain in the loop” using non-invasive brain stimulation and briefly review the experimental and therapeutic implications. We take the view that closed-loop neuroscience in fact deals with two conceptually quite different loops: a “brain-state dynamics” loop, used to couple with and modulate the trajectory of neuronal activity patterns, and a “task dynamics” loop, that is the bidirectional motor-sensory interaction between brain and (simulated) environment, and which enables goal-directed behavioral tasks to be incorporated. Both loops need to be considered and combined to realize the full experimental and therapeutic potential of closed-loop neuroscience.
Much has been learned about the mechanics of the brain by treating it as a “black box”, placed in a controlled laboratory environment and stimulated by the experimenter in an open-loop fashion using a pre-defined stimulus protocol to determine its input-output characteristics and how these may be modulated. This approach has been fruitful also in the field of non-invasive brain stimulation (NIBS) and transcranial magnetic stimulation (TMS) in particular, enabling significant advances in the understanding of the functional and pharmacological basis of cortical dynamics (Rothwell et al.,
However, in spite of the significant advances of the past 30 years, TMS has yet to realize its full potential, especially with regard to therapy (Lefaucheur et al.,
Moreover, there are compelling conceptual considerations that motivate a “closed-loop” approach: the brain is not a black box; it is not a mere transducer of input to output signals but a generator of behavior—and behavior requires an environment, that is, an entailment between an output of the brain and a (feedback) input to the brain.
In the following, we discuss the conceptual context of closed-loop methods in general, the possibilities and technical challenges of using EEG and TMS to implement a closed-loop set-up, and the implications of this approach to neurophysiology. We review some of the current research and present preliminary technical results from our own lab supporting the expectation that brain-state dependent brain-stimulation will be significantly more effective at modulating neural pathways than current open-loop protocols.
An experiment may be considered a “closed-loop” experiment when actions (output from the brain) have consequences (future input to the brain). This is the natural state of affairs for an organism roaming through the environment; indeed, the output of the brain is relevant
It is of course possible to “close the loop” in the laboratory by establishing a causal relationship between the measured output of the brain and the stimulus generator. Indeed, there are two quite distinct ways in which this can be done: In the simpler case, the stimulus is applied as a function of the simultaneously measured instantaneous brain state (Figure
The more complex closed-loop scenario (Figure
This third scenario is qualitatively different from the second scenario in that any causal explanation of the dynamic interaction between brain and environment can no longer be limited to the state and transition dynamics of the brain alone, but must take into account the state of the environment, too. Indeed, whereas the complete state of the brain is necessarily not accessible by the experimenter, we can present the subject with a simple constructed or simulated environment to which we do have comprehensive access and which can be experimentally controlled.
This is also the key difference to the ethological approach of just observing an animal in its natural environment: in the fully closed-loop setting as described above, the experimenter is able to access and interfere with the flow of information between neural system and environment on the one hand and the state and transition dynamics of the environment on the other. Instead of observing the output in response to different inputs, the “behavior” of the neural system now includes both the brain and the environment and can be studied in environments following different equations of motion (state transition laws). Whereas in the open-loop stimulus-response setting, the scope of what may be experimentally observed is limited to a single variable (the “response”), in the closed-loop setting the possible state space trajectories that the neural system may take in conjunction with the environment are not so constrained.
There is large body of work creating an artificial connection between the brain and the natural environment through a brain-computer interface (Buch et al.,
The insight that a closed-loop experimental approach may enable us to learn new things about the brain that we cannot learn from a “black box” approach is of course not a new one, as both experimental work
TMS is an old technique but it remains the only way to non-invasively excite a specific population of cortical neurons with a spatial resolution of millimeters and a temporal resolution of microseconds (Barker et al.,
The “true” brain state is not accessible, it would constitute a long vector describing the state and activity of each nerve cell and synapse; in practice, the dimensionality is already reduced (information is lost) by the EEG or MEG recording and then further as this is projected onto some quantifiable correlate of a physiological relevant process. In the following description, we focus on what is perhaps the most salient feature of brain state as measured by EEG, i.e., spontaneous oscillatory activity of neuronal populations (Buzsáki and Draguhn,
Spatially, brain states can be observed locally, i.e., in the activity of a specific brain area (Gharabaghi et al.,
Closing the loop not only on spectral power but also on instantaneous phase is methodologically more challenging because it requires a real-time signal processing stage with a time-resolution of milliseconds, however, this has recently become possible (see the next section and Figure
The implementation of a closed-loop set-up requires several different stages: a brain output measurement stage, a signal processing stage, and a stimulus modulation stage (Figure
The performance of the closed-loop system can be characterized by the fundamental sample-time of the processor, the bandwidth of data acquired, the complexity of computations that can be performed within a single time-step, the overall feedback latency through the loop from signal to stimulus, as well as any jitter in that latency. Importantly, a loop latency and jitter in the order of milliseconds is required for phase-dependent stimulation of endogenous brain activity in higher frequency bands (beta, gamma).
Sources of latency include the data processing and transfer buffers at each stage, but also the phase shift inherent to signal filters. Generic general real-time signal processing systems (such as Mathworks Simulink Real-Time or National Instruments LabView) can easily process data in sample time steps <1 ms (Zrenner et al.,
Until recently, artifacts introduced by the TMS pulse in the simultaneously recorded EEG signal were a major obstacle to closed-loop EEG-TMS because of amplifier saturation. However, with the availability of high dynamic range 24 bit analog-to-digital converters, the stimulus artifact is simply captured by the amplifier and the complexity and discontinuities introduced by previously required sample-and-hold or blanking circuits is obviated. In combination with TMS compatible sintered ring electrodes the TMS artifact duration can be reduced to <10 ms (Virtanen et al.,
The most significant limitation concerns real-time analysis of the biosignal data, as it is streamed to the real-time processor. Only a sliding window of data preceding the current time point can be considered and this means that any kind of filtering will cause phase shifts or edge effects that need to be compensated for. Furthermore, since we are only considering an individual epoch in each trial, none of the standard methods for averaged data with a window extending both directions around the event of interest are available.
Closed-loop non-invasive brain stimulation with millisecond precision enables selective interference with ongoing brain activity and thereby can help to clarify important open questions in neurophysiology: How does the TMS pulse evoke cortico-spinal activity (Rossini et al.,
A pioneering application of closed-loop brain-state triggered stimulation in a “behavior-in-the-loop” setting can be found in an animal experiment by Siegle and Wilson (
Nevertheless, the implicit physical environment of a freely behaving animal is difficult to capture, the sensory and motor interaction difficult to quantify, and the experimental control and opportunities for intervention limited; we therefore expect that an explicit “task dynamics” loop, in the form of an environment simulated on a computer, will be an increasingly relevant feature of closed-loop designs. Neuronal cell cultures have long been studied as an experimental system with such a view in mind (Shahaf and Marom,
With regard to the therapy of neurological disorders due to “network dysregulation”, closed-loop paradigms can be applied in two different ways: firstly, a control-system theory approach uses the feedback stimulus as a “regulator” to re-“set” the local excitability and activity of a network (see Wallach,
For instance, in a rodent model of generalized epilepsy closed-loop transcranial electrical stimulation based on a threshold mechanism of cortical local field potentials and unitary activity was shown to reduce epileptiform spike-and-wave episodes (Berényi et al.,
Another important example for closed-loop stimulation is auditory stimulation during <1 Hz slow-wave sleep, a phase of sleep that is critical to declarative memory consolidation. It was demonstrated in healthy sleeping humans that auditory stimulation in phase with the ongoing rhythmic occurrence of slow oscillation up states measured in the EEG significantly enhanced the slow oscillation rhythm, phase-coupled sleep spindle activity, and in turn, the consolidation of declarative memory (Ngo et al.,
In summary, EEG brain-state triggered NIBS or behavior-in-the-loop set-ups will enable physicians to interfere with their patients’ ongoing brain activity with high temporal, spatial and spectral precision. NIBS or behavioral neurofeedback can thus be coupled to endogenous brain activity in functionally defined brain networks in real time. This approach has several important advantages. Firstly, neuromodulation can be personalized to individual network function, that is, inter-individual differences in the excitability and connectivity of brain networks can be taken into account. Secondly, the time-course of dynamic changes during network reorganization such as during stroke rehabilitation (Grefkes and Ward,
Finally, behavior-in-the-loop set-ups can additionally capitalize on agency and subjectivity of brain-environment interactions. For instance, NIBS can be applied through the “brain dynamics loop” in synchrony to motor-sensory feedback from the “task dynamics loop” (see Figure
Two different closed-loop interactions can be differentiated: a direct coupling to instantaneous brain states through non-invasive brain stimulation (“brain dynamics” loop), and a coupling to an environmental system that presents the brain with the opportunity to generate goal-directed behavior through the motor-sensory loop (“task dynamics” loop). These two approaches to closed-loop neuroscience are conceptually quite different but they are complementary in that they serve to induce and then interfere with specific brain states. Our assertion is that there is significant experimental and therapeutic potential in the application of brain-state dependent brain stimulation while the subject or patient is simultaneously performing a task, where both loops are interactively optimized for neuromodulatory efficacy.
All authors listed, have made substantial, direct and intellectual contribution to the work, and approved it for publication.
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
This work was supported by the Fortüne Junior Intramural research funding program at the Faculty of Medicine in Tübingen (IZKF; to CZ) and the Inter-University Center for Medical Technologies Stuttgart-Tübingen (IZST) Industry-on-Campus Project 211 (to PB and CZ) as well as the Open Access Publishing Fund of University of Tübingen.