# **CORTICO-CORTICAL COMMUNICATION DYNAMICS**

**Topic Editors Gustavo Deco, Per E. Roland and Claus C. Hilgetag**

#### *FRONTIERS COPYRIGHT STATEMENT*

© Copyright 2007-2014 Frontiers Media SA. All rights reserved.

All content included on this site, such as text, graphics, logos, button icons, images, video/audio clips, downloads, data compilations and software, is the property of or is licensed to Frontiers Media SA ("Frontiers") or its licensees and/or subcontractors. The copyright in the text of individual articles is the property of their respective authors, subject to a license granted to Frontiers.

The compilation of articles constituting this e-book, wherever published, as well as the compilation of all other content on this site, is the exclusive property of Frontiers. For the conditions for downloading and copying of e-books from Frontiers' website, please see the Terms for Website Use. If purchasing Frontiers e-books from other websites or sources, the conditions of the website concerned apply.

Images and graphics not forming part of user-contributed materials may not be downloaded or copied without permission.

Individual articles may be downloaded and reproduced in accordance with the principles of the CC-BY licence subject to any copyright or other notices. They may not be re-sold as an e-book.

As author or other contributor you grant a CC-BY licence to others to reproduce your articles, including any graphics and third-party materials supplied by you, in accordance with the Conditions for Website Use and subject to any copyright notices which you include in connection with your articles and materials.

All copyright, and all rights therein, are protected by national and international copyright laws.

The above represents a summary only. For the full conditions see the Conditions for Authors and the Conditions for Website Use.

Cover image provided by Ibbl sarl, Lausanne CH

**ISSN** 1664-8714 **ISBN** 978-2-88919-288-5 **DOI** 10.3389/978-2-88919-288-5

# *ABOUT FRONTIERS*

Frontiers is more than just an open-access publisher of scholarly articles: it is a pioneering approach to the world of academia, radically improving the way scholarly research is managed. The grand vision of Frontiers is a world where all people have an equal opportunity to seek, share and generate knowledge. Frontiers provides immediate and permanent online open access to all its publications, but this alone is not enough to realize our grand goals.

# *FRONTIERS JOURNAL SERIES*

The Frontiers Journal Series is a multi-tier and interdisciplinary set of open-access, online journals, promising a paradigm shift from the current review, selection and dissemination processes in academic publishing.

All Frontiers journals are driven by researchers for researchers; therefore, they constitute a service to the scholarly community. At the same time, the Frontiers Journal Series operates on a revolutionary invention, the tiered publishing system, initially addressing specific communities of scholars, and gradually climbing up to broader public understanding, thus serving the interests of the lay society, too.

# *DEDICATION TO QUALITY*

Each Frontiers article is a landmark of the highest quality, thanks to genuinely collaborative interactions between authors and review editors, who include some of the world's best academicians. Research must be certified by peers before entering a stream of knowledge that may eventually reach the public - and shape society; therefore, Frontiers only applies the most rigorous and unbiased reviews.

Frontiers revolutionizes research publishing by freely delivering the most outstanding research, evaluated with no bias from both the academic and social point of view.

By applying the most advanced information technologies, Frontiers is catapulting scholarly publishing into a new generation.

# *WHAT ARE FRONTIERS RESEARCH TOPICS?*

Frontiers Research Topics are very popular trademarks of the Frontiers Journals Series: they are collections of at least ten articles, all centered on a particular subject. With their unique mix of varied contributions from Original Research to Review Articles, Frontiers Research Topics unify the most influential researchers, the latest key findings and historical advances in a hot research area!

Find out more on how to host your own Frontiers Research Topic or contribute to one as an author by contacting the Frontiers Editorial Office: researchtopics@frontiersin.org

# **CORTICO-CORTICAL COMMUNICATION DYNAMICS**

Topic Editors:

**Gustavo Deco,** Universitat Pompeu Fabra, Spain **Per E. Roland,** University of Copenhagen, Denmark **Claus C. Hilgetag,** University Medical Center Hamburg-Eppendorf, German**y** 

# Table of Contents


# **SYSTEMS NEUROSCIENCE**

# Tracing evolution of spatio-temporal dynamics of the cerebral cortex: cortico-cortical communication dynamics

#### *Per E. Roland1 \*, Claus C. Hilgetag2,3 and Gustavo Deco4*

*<sup>1</sup> Department of Neuroscience and Pharmacology, Faculty of Health Sciences, University of Copenhagen, Copenhagen, Denmark*


*\*Correspondence: perrol@sund.ku.dk*

#### *Edited and reviewed by:*

*Maria V. Sanchez-Vives, ICREA-IDIBAPS, Spain*

**Keywords: action potential transmission, connectivity models, multi-area voltage sensitive dye recordings, EEG, MEG**

A considerable number of axons from neurons in one cortical area end up on other cortical areas. When one neuron in one cortical area sends an action potential to target neurons in other cortical areas, this is a realization of a cortico-cortical communication. Sensory perception, thinking, and planning of a specific behavior, all rely on the evolution of cortico-cortical communications. The action potentials change the membrane potentials in the target neurons and, in turn, may excite these neurons to produce action potentials and complex patterns of excitation and inhibition in *their* targets. We launched the *special research topic of cortico-cortical communication dynamics* to invite contributions that would cast light on such evolution of spatiotemporal action potential and membrane potential dynamics in the cerebral cortex.

The contributions were theoretical models, human EEG, and MEG data and data-driven models, and *in vivo* experimental data from animals accounting for specific aspects of cortico-cortical communication dynamics.

In a recent *in vitro* experiment, Branco et al. (2010) show that single dendrites of pyramidal layer 2–3 neurons depolarize more and have larger Ca2<sup>+</sup> influx when their depolarization progresses toward the soma, than when depolarization progresses away from the soma. Kiebel and Friston (2011) construct a (developmental) model of the pruning of single synapses and show that they can reproduce the findings of Branco et al. (2010) if the selforganizing pruning follows a Bayesian and information theory derived principle of minimization of free energy. Cortico-cortical communication dynamics can only be comprehensively studied *in vivo. In vivo*, the neurons and their dendrites are in a high conductance state (Destexhe et al., 2003), and the propagation of depolarizations to the soma and action potential generation may thus be difficult to predict (Williams and Mitchell, 2008). This does not exclude, however, that the model of Kiebel and Friston (2011) may be appropriate in early development and in the formation of cortio-cortical synapses. The pruning of synapses under development and hence the formation of the adult cortical network is the theme of the contribution of van den Bergh et al. (2012). Their model departs from a random network. This network is subsequently shaped by spontaneous ongoing spike activity. After a while the random structure disappears and many small-world sub-networks emerge. As van den Bergh et al. (2012) show, this only happens if the connectivity in the network is larger than a critical value. This is interesting as the developing brain has many cortico-cortical connections that disappear at later stages.

As pointed out in a critical review of cortico-cortical communication dynamics, there are many obstacles precluding the tracing the ms by ms evolution of the spatio-temporal dynamics of the cortex (Roland et al., 2014). Therefore examination of the spatiotemporal dynamics in biologically plausible computational models of neurons may be one way to develop experimentally testable hypotheses. Li and Zhou (2011) made a computational model of neurons in two inter-connected cortical areas. The duration of the delays in communication and the distribution of inhibition in the local network determined whether the neurons would spike in phase or in anti-phase and whether interactions between slow and fast membrane oscillations would produce anti-phase spiking. These findings are pertinent for the hypothesis on cortico-cortical communication through coherence (Fries, 2009).

Facing the obstacles of tracing the spatio-temporal dynamics of cortico-cortical communications at the cellular scale, many scientists choose to study membrane electrical activity at the scale of large neuron populations, and from EEG and MEG signals try to infer putative routes of communication. Banerjee et al. (2012) discuss these methods and point out that there is no consensus as to what constitutes a large-scale network. Further, they show how MEG measurements may be interpreted by combining the empirical analysis with large-scale models of biologically realistic membrane activity. This is what is done in the contributions by Misic et al. (2011) and Vakorin et al. (2011). Their results show that time delays and the number of connections between sources, of MEG signals or EEG signals, contribute to the relation between variance in the signals and information transfer between the sources (Misic et al., 2011; Vakorin et al., 2011).

At the mesoscopic scale one can observe changes in the membrane potentials with voltage sensitive dyes, local field potentials and combine this with recordings of action potentials from a few neurons or single neurons in experimental animals. Harvey and Roland (2013) demonstrate both forward spatiotemporal population membrane dynamics in higher visual areas that after 50 ms was followed by backward propagation of net-excitation from these areas in experiments with objects moving in the visual field. Vinnik et al. (2012) examined the communications from the auditory cortex to the hippocampus and show that the access to fire hippocampal neurons is state dependent. Sleep favors fast reactions of the hippocampal neurons to the extent as only seen for novel sounds in awake animals (Vinnik et al., 2012). Civillico and Contreras (2012) examined how the communication from the thalamus to the barrel cortex is affected by the state of the neurons in the barrel cortex. When the cortical neurons were in an up-state, the local field potentials, the membrane potential increases, and the multiunit activity evoked by a whisker stimulus was smaller than when the whisker stimulus was given just during the early transition from a down-state to an up-state (Civillico and Contreras, 2012).

If one wants to understand how the cerebral cortex works one must be able to trace the evolution of the spatio-temporal transmission of action potentials and membrane conductances down to the cellular scale. As the critical review concludes, this is not possible yet. Assume that a full connectome of the mouse cerebral cortex exists (Bohland et al., 2009). This might help in finding the target neurons in other areas for a given neuron. However, it still remains to identify that source neuron spiking in an experiment and measure the membrane potential changes induced by that neuron on each of the target neurons, as each target neuron may have 1000 other source neurons. One may argue that if this multidimensional cellular dynamics should have any impact on perception and behavior, the dynamics of action potentials and membrane potential dynamics at more coarse scales should organize to make such impacts. The contributions to this special issue are fine examples of the many contemporary attempts to advance theoretical knowledge of cortico-cortical communication dynamics, provide testable hypotheses in this field, and test these hypotheses at the microscopic, mesoscopic, and macroscopic scales.

# **REFERENCES**


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 25 September 2013; accepted: 15 April 2014; published online: 05 May 2014. Citation: Roland PE, Hilgetag CC and Deco G (2014) Tracing evolution of spatiotemporal dynamics of the cerebral cortex: cortico-cortical communication dynamics. Front. Syst. Neurosci. 8:76. doi: 10.3389/fnsys.2014.00076*

*This article was submitted to the journal Frontiers in Systems Neuroscience.*

*Copyright © 2014 Roland, Hilgetag and Deco. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Free energy and dendritic self-organization

# *Stefan J. Kiebel 1\* and Karl J. Friston2*

*<sup>1</sup> Department of Neurology, Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany*

*<sup>2</sup> The Wellcome Trust Centre for Neuroimaging, University College London, London, UK*

#### *Edited by:*

*Gustavo Deco, Universitat Pompeu Fabra, Spain*

#### *Reviewed by:*

*Robert Turner, Max Planck Institute for Human Cognitive and Brain Sciences, Germany Anders Ledberg, Universitat Pompeu*

*Fabra, Spain*

#### *\*Correspondence:*

*Stefan J. Kiebel, Department of Neurology, Max Planck Institute for Human Cognitive and Brain Sciences, Stephanstr. 1a, 04103 Leipzig, Germany.*

*e-mail: kiebel@cbs.mpg.de*

In this paper, we pursue recent observations that, through selective dendritic filtering, single neurons respond to specific sequences of presynaptic inputs. We try to provide a principled and mechanistic account of this selectivity by applying a recent free-energy principle to a dendrite that is immersed in its neuropil or environment. We assume that neurons self-organize to minimize a variational free-energy bound on the self-information or surprise of presynaptic inputs that are sampled. We model this as a selective pruning of dendritic spines that are expressed on a dendritic branch. This pruning occurs when postsynaptic gain falls below a threshold. Crucially, postsynaptic gain is itself optimized with respect to free energy. Pruning suppresses free energy as the dendrite selects presynaptic signals that conform to its expectations, specified by a generative model implicit in its intracellular kinetics. Not only does this provide a principled account of how neurons organize and selectively sample the myriad of potential presynaptic inputs they are exposed to, but it also connects the optimization of elemental neuronal (dendritic) processing to generic (surprise or evidence-based) schemes in statistics and machine learning, such as Bayesian model selection and automatic relevance determination.

**Keywords: single neuron, dendrite, dendritic computation, Bayesian inference, free energy, non-linear dynamical system, multi-scale, synaptic reconfiguration**

# **INTRODUCTION**

The topic of this special issue, cortico-cortical communication, is usually studied empirically by modeling neurophysiologic data at the appropriate spatial and temporal scale (Friston, 2009). Models of communication or effective connectivity among brain areas are specified in terms of neural dynamics that subtend observed responses. For example, neural mass models of neuronal sources have been used to account for magneto- and electroencephalography (M/EEG) data (Kiebel et al., 2009a). These sort of modeling techniques have been likened to a "mathematical microscope" which effectively increase the spatiotemporal resolution of empirical measurements by using neurobiologically plausible constraints on how data were generated (Friston and Dolan, 2010). However, the models currently used in this fashion generally reduce the dynamics of a brain area or cortical source to a few neuronal variables and ignore details at a cellular or ensemble level.

To understand the basis of neuronal communication, it may be useful to understand what single neurons encode (Herz et al., 2006). Although the gap between a single neuron and a cortical region spans multiple scales, understanding the functional anatomy of a single neuron is crucial for understanding communication among neuronal ensembles and cortical regions: The single neuron is the basic building block of composite structures (like macrocolumns, microcircuits, or cortical regions) and, as such, shapes their functionality and emergent properties. In addition, the single neuron is probably the most clearly defined functional brain unit (in terms of its inputs and outputs). It is not unreasonable to assume that the computational properties of single neurons can be inferred using current techniques such as two-photon laser microscopy and sophisticated modeling approaches (London and

Hausser,2005;Mel,2008; Spruston,2008). In short, understanding the computational principles of this essential building block may generate novel insights and constraints on the computations that emerge in the brain at larger scales. In turn, this may help us form hypotheses about what neuronal systems encode, communicate, and decode.

In this work, we take a somewhat unusual approach to derive a functional model of a single neuron: instead of using a bottom-up approach, where a model is adjusted until it explains empirical data, we use a top-down approach by assuming a neuron is a Bayes-optimal computing device and therefore conforms to the free-energy principle (Friston, 2010). The ensuing dynamics of an optimal neuron should then reproduce the cardinal behaviors of real neurons, see also Torben-Nielsen and Stiefel (2009). Our ultimate goal is to map the variables of the Bayes-optimal neuron to experimental measurements. The existence of such a mapping would establish a computationally principled model of real neurons that may be useful in machine learning to solve real-world tasks.

The basis of our approach is that neurons minimize their variational *free energy* (Feynman, 1972; Hinton and van Camp, 1993; Friston, 2005, 2008; Friston et al., 2006). This is motivated by findings in computational neuroscience that biological systems can be understood and modeled by assuming that they minimize their free energy; see also Destexhe and Huguenard (2000). Variationalfree energy is not a thermodynamic quantity but comesfrom information and probability theory, where it underlies variational Bayesian methods in statistics and machine learning. By assuming that the single neuron (or its components), minimizes variational free energy (henceforth free energy), we can use the notion of optimization to specify Bayes-optimal neuronal dynamics:in other words, one can use differential equations that perform a gradient descent on free energy as predictions of single neuron dynamics. Free-energy minimization can be cast as Bayesian inference, because minimizing free-energy corresponds to maximizing the *evidence* for a model, given some data (see **Table 1** and Hinton and van Camp, 1993; Friston, 2008; Daunizeau et al., 2009).

Free energy rests on a generative model of the sensory input a system is likely to encounter. This generative model is entailed by form and structure of the system (here a single neuron) and specifies itsfunction in terms of the inputs it should sample. Free-energy minimization can be used to model systems that decode inputs and actively select those inputs that are expected under its model (Kiebel et al., 2008). Note that the Bayesian perspective confers



attributes like expectations and prior beliefs on any system that conforms to the free-energy principle; irrespective of whether it is mindful (e.g., a brain) or not (e.g., a neuron). Using free-energy minimization, we have shown previously that many phenomena in perception, action, and learning can be explained qualitatively in terms of Bayesian inference (Friston et al., 2009; Kiebel et al., 2009b). Here, we apply the same idea to the dendrite of a single neuron. To do this, we have to answer the key question: what is a dendrite's generative model? In other words, what synaptic input does a dendrite expect to see?

Differences in the morphology and connections among neurons suggest that different neurons implement different functions and consequently "expect" different sequences of synaptic inputs (Vetter et al., 2001; Torben-Nielsen and Stiefel, 2009). Recently (Branco et al., 2010) provided evidence for sequence-processing in pyramidal cells. By using *in vitro* two-photon laser microscopy, glutamate uncaging, and patch clamping, these authors showed that dendritic branches respond selectively to specific sequences of postsynaptic potentials (PSPs). Branco et al. (2010) found PSP sequences that move inward (toward the soma) generate higher responses than "outward" sequences (**Figure 1C**): Sequences were generated by activating spines along a dendritic branch with an interval of ca. 2 ms (**Figures 1A,B**). They assessed the sensitivity to different sequences using the potential generated at the soma by calcium dynamics within the dendritic branch. In addition, they found that the difference in responses to inward and outward sequences is velocity-dependent: in other words, there is an optimal sequence velocity that maximizes the difference between the responses to inward and outward simulation (see **Figures 1C,D**). These two findings point to intracellular mechanisms in the dendritic branches of pyramidal cells, whose function

is to differentiate between specific sequences of presynaptic input (Destexhe, 2010). Branco et al. (2010) used multi-compartment modeling to explain their findings and proposed a simple and compelling account based on NMDA receptors and an impedance gradient along the dendrite. Here,we revisit the underlying cellular mechanisms from a functional perspective: namely, the imperative for self-organizing systems to minimize free energy.

In brief, this paper is about trying to understand how dendrites self-organize to establish functionally specific synaptic connections, when immersed in their neuronal environment. Specifically, we try to account for how postsynaptic specializations (i.e., spines) on dendritic branches come to sample particular sequences of presynaptic inputs (conveyed by axons). Using variational free-energy minimization, we hope to show that the emergent process of eliminating and redeploying postsynaptic specializations in real neuronal systems (Katz and Shatz,1996; Lendvai et al., 2000) is formally identical to the model selection and optimization schemes used in statistics and machine learning. In what follows, we describe the theoretical ideas and substantiate them with neuronal simulations.

#### **FREE ENERGY AND THE SINGLE NEURON**

Our basic premise is that any self-organizing system will selectively sample its world to minimize the free energy of those samples. This (variational) free energy is an information theory quantity that is an upper bound on surprise or self-information. The average surprise is called entropy; see **Table 1**. This means that biological systems resist an increase in their entropy, and a natural tendency to disorder. Crucially, surprise is also the negative log-evidence that measures the "goodness" of a model in statistics. By applying exactly the same principle a single dendrite,we will show that it can

**FIGURE 1 | Findings reported by Branco et al. (2010): Single dendrites are sensitive to the direction and velocity of synaptic input patterns. (A)** Layer 2/3 pyramidal cell filled with Alexa 594 dye; the yellow box indicates the selected dendrite. **(B)** Uncaging spots (yellow) along the selected dendrite.

**(C)** Somatic responses to IN (red) and OUT (blue) directions at 2.3 mm/ms. **(D)** Relationship between peak voltage and input velocity (values normalized to the maximum response in the IN direction for each cell, *n* = 15). Error bars indicate SEM. Reproduced from Branco et al. (2010) with permission.

explain the optimization of synaptic connections and the emergence of functional selectivity, in terms of neuronal responses to presynaptic inputs. This synaptic selection is based upon synaptic gain control, which is itself prescribed by free-energy minimization: When a synapse's gain falls below a threshold it is eliminated, leading to a pruning of redundant synapses and a selective sampling of presynaptic inputs that conforms to the internal architecture of a dendrite (Katz and Shatz, 1996; Lendvai et al., 2000). We suggest that this optimization scheme provides an interesting perspective on self-organization at the (microscopic) cellular scale. By regarding a single neuron, or indeed a single dendrite, as a biological system that minimizes surprise or free energy, we can, in principle, explain its behavior over multiple time-scales that span fast electrochemical dynamics, through intermediate fluctuations in synaptic efficacy, to slow changes in the formation, and regression of synaptic connections.

This paper comprises three sections. In the first, we describe the underlying theory and derive the self-organizing dynamics of a Bayes-optimal dendrite. The second section presents simulations, in which we demonstrate the reorganization of connections under free-energy minimization and record the changes in free energy over the different connectivity configurations that emerge.We also examine the functional selectivity of the model's responses, after optimal reconfiguration of its connections, to show the sequential or directional selectivity observed empirically. In the third section, we interpret our findings and comment in more detail on the dendritic infrastructures and intracellular dynamics implied by the theoretical treatment. We conclude with a discussion of the implications of this model for dendritic processing and some predictions that could be tested empirically.

# **MATERIALS AND METHODS**

In this section, we present a theoretical treatment of dendritic anatomy and dynamics. Following previous modeling initiatives, we consider a dendrite as a spatially ordered sequence of segments (see, e.g., Dayan and Abbott, 2005, p. 217ff). Each segment expresses a number of synapses (postsynaptic specializations) that receive action potentials from presynaptic axons. Each synapse is connected to a specific presynaptic axon (or terminal) and registers the arrival of an action potential with a PSP. Our aim is to explain the following: If a dendrite can disambiguate between inward and outward sequences (Branco et al., 2010), how does the dendrite organize its synaptic connections to attain this directional selectivity? In this section, we will derive a model that reorganizes its synaptic connections in response to synaptic input sequences using just the free-energy principle.

We start with the assumption that the dendrite is a Bayesoptimal observer of its presynaptic milieu. This means that we regard the dendrite as a model of its inputs and associate its physical attributes (e.g., intracellular ion concentrations and postsynaptic gains) with the parameters of that model. In what follows, we describe this model, its optimization and consider emergent behavior, such as directional selectivity.

To illustrate the approach, we modeled a dendrite with five segments, each of which expresses four synapses: see **Figure 2**. This means the dendrite has to deploy *T* = 20 synapses to sample five distinct presynaptic inputs in a way that minimizes its free energy or surprise. The internal dynamics of the dendrite are assumed to provide predictions for a particular sequence of synchronous inputs at each dendritic segment. In other words, each connection within a segment "expects" to see the same input, where the order of inputs over segments is specified by a sequence of intracellular predictions: see **Figure 3**.

To minimize free energy and specify the Bayes-optimal update equations for changes in dendritic variables, we require a generative model of sequential inputs over segments. To do this, we use a model based on Lotka–Volterra dynamics that generates a sequence, starting at the tip of the dendrite and moving toward

the distal end. In other words, the dendrite models its synaptic input *S*(*t*) = [*S*1,...,*S*T] *<sup>T</sup>* as being caused by a saltatory sequence of changes in (hidden) states *x*(*t*) = [*x*1,...,*x*s] *<sup>T</sup>* representing the presynaptic activity to which each segment is exposed. The speed at which the sequence is expressed is controlled by a hidden variable *v*(*t*) according to the following equations

$$\begin{aligned} s &= g(\mathbf{x}, \boldsymbol{\nu}) + \boldsymbol{\omega}\_s \\ \dot{\boldsymbol{x}} &= f(\mathbf{x}, \boldsymbol{\nu}) + \boldsymbol{\omega}\_x \\ g(\mathbf{x}, \boldsymbol{\nu}) &= \mathbf{W} \mathbf{x} \\ f(\mathbf{x}, \boldsymbol{\nu}) &= \mathbf{A}(\boldsymbol{\nu}) \boldsymbol{\sigma}(\mathbf{x}) - \frac{1}{8} \mathbf{x} + \mathbf{1}\_s \\ \boldsymbol{\sigma}(\mathbf{x}) &= \frac{1}{1 - e^{-\mathbf{x}}} \\ \boldsymbol{\omega}\_s &\sim \mathcal{N}(\mathbf{0}, \boldsymbol{\Sigma}^{(s)}) : \boldsymbol{\Sigma}^{(s)} = \operatorname{diag}(\exp(-\boldsymbol{\gamma}\_i)) \\ \boldsymbol{\omega}\_x &\sim \mathcal{N}(\mathbf{0}, \boldsymbol{\Sigma}^{(x)}) \end{aligned} \tag{1}$$

These equations model winnerless competition among the hidden states to produce sequential dynamics (called a stable heteroclinic channel; Rabinovich et al., 2006). Here, <sup>ω</sup>*s*(*t*) <sup>∈</sup> <sup>R</sup>*<sup>T</sup>* and <sup>ω</sup>*<sup>x</sup>* (*t*) <sup>∈</sup> <sup>R</sup>*<sup>S</sup>* correspond to random fluctuations on synaptic input and the hidden states respectively. These fluctuations have covariances (Σ(s),Σ(*x*) ) or precisions (Π(*s*) ,Π(*x*) ) (inverse covariances), where the precision of the *i*-th synaptic input is determined by its log-precision or gain: γ*i*. Log-precisions are a convenient way to express variances because they simplify the update equations presented below.

The mapping from hidden states (presynaptic axons) to synaptic input is parameterized by a connectivity matrix **<sup>W</sup>** <sup>∈</sup> <sup>R</sup>*T*×*<sup>S</sup>* with elements **<sup>W</sup>***ij* <sup>∈</sup> <sup>R</sup>{0, 1} that determine whether there is a synaptic connection between the *j*-th presynaptic axon and the *i*-th segment (**Figure 3**). It is this pattern of connections that determines the form of the model. The matrix **<sup>A</sup>** <sup>∈</sup> <sup>R</sup>*<sup>S</sup>* <sup>×</sup> *<sup>S</sup>* determines the sequential dynamics. For example, to generate a sequence for *S* = 4 segments, one would use:

$$A\left(\nu\right) = \begin{pmatrix} 0 & -e^{\nu} & 0 & e^{\nu} \\ e^{\nu} & 0 & -e^{\nu} & 0 \\ 0 & e^{\nu} & 0 & -e^{\nu} \\ -e^{\nu} & 0 & e^{\nu} & 0 \end{pmatrix} - \begin{pmatrix} 0 & 1 & 1 & 1 \\ 1 & 0 & 1 & 1 \\ 1 & 1 & 0 & 1 \\ 1 & 1 & 1 & 0 \end{pmatrix} \tag{2}$$

In Eq. 2, the first matrix models exhibition and inhibition between neighboring segments and determines the sequence. We use the exponential of the hidden cause to ensure the speed *e<sup>v</sup>* is positive. The second matrix encodes non-specific inhibition among segments.

Given this model of how synaptic inputs are generated, we can now associate the internal states of the dendrite with the model's parameters or variables. In other words, we assume that the intracellular dynamics of the dendrite are trying to predict the input that it receives, based on the generative model in Eq. 1: see **Figure 2B**and **Figure 3** (left panel). This means that we can cast dendritic changes as minimizing the free-energy associated with any given synaptic inputs. This provides Bayes-optimal update equations (i.e., non-linear differential equations) for each internal state. These equations describe how the internal states of a dendrite change when receiving input over time. A related application using winnerless competition in the context of sequence recognition can be found in Kiebel et al. (2009b). In the following, we will briefly describe the resulting update equations. Details of their derivation and implementation can be found in Friston et al. (2008). The key quantities involved in this scheme are described in **Table 1**.

#### **FREE-ENERGY MINIMIZATION**

Minimizing the free-energy based on the generative model (Eq. 1) involves changing the internal variables of the dendrite so that they minimize free energy. Free energy is a function of the internal variables because they constitute Bayes-optimal predictions of the hidden states and synaptic input. In the present context, there are four sets of variables that can change (μ˜ (*x*) , <sup>μ</sup>˜ (*v*) , μ(γ) , η(γ) ); these correspond to conditional expectations or predictions about hidden states and causes; predictions about synaptic log-precision or gain and predictions about existence of a synaptic input *per se* (see **Table 1**). As we will see below, optimizing these variables with respect to free energy is necessarily mediated at three distinct time-scales pertaining to (i) fast intracellular dynamics (e.g., depolarization and consequent changes in intracellular concentrations such as calcium): (ii) synaptic dynamics that change the efficacy or precision of synaptic connections and (iii) an intermittent selection and regeneration of postsynaptic specializations. Crucially, all three minimize free energy and can be expressed as a series of differential equations or update rules, as follows:

$$\begin{aligned} \dot{\tilde{\mu}}^{(\mathbf{x})} &= -\frac{\partial}{\partial \tilde{\mu}^{(\mathbf{x})}} \mathcal{F}(\tilde{\mathbf{s}}, \boldsymbol{\mu}|m) + \mathcal{D}\tilde{\mu}^{(\mathbf{x})} \\ \dot{\tilde{\mu}}^{(\mathbf{v})} &= -\frac{\partial}{\partial \tilde{\mu}^{(\mathbf{v})}} \mathcal{F}(\tilde{\mathbf{s}}, \boldsymbol{\mu}|m) + \mathcal{D}\tilde{\mu}^{(\mathbf{v})} \\ \dot{\mu}^{\prime(\mathbf{v})} &= \boldsymbol{\mu}^{\prime(\mathbf{v})} \end{aligned} \tag{3a}$$

$$\dot{\mu}^{\prime(\curlyeq)} = -\frac{\partial}{\partial \mu^{\prime(\curlyeq)}} \mathcal{F}(\tilde{s}, \mu | m) - \pi \mu^{\prime(\curlyeq)} \tag{3b}$$

$$\mathfrak{n}^{(\gamma)} = \underset{\mathfrak{n} \in m}{\arg \min} \int \mathcal{F}(\tilde{s}, \mathfrak{u} \, | \, m) \, dt \tag{3c}$$

We now consider each of these updates in turn.

#### **FAST INTRACELLULAR DYNAMICS (Eq. 3a)**

Equation 3a represents the fastest time-scale and describes the predictions of hidden states associated with each dendritic segment and the hidden cause controlling the speed of the synaptic sequence. Later, we will associate these with depolarization and intracellular concentrations within the dendrite. The dynamics of this internal states correspond to a generalized gradient descent on free energy: <sup>F</sup>(*s*˜,μ|*m*) <sup>∈</sup> <sup>R</sup>, such that when free energy is minimized they become Bayes-optimal estimates of the hidden variables. This is the basis of Variational Bayes or ensemble learning and is used widely in statistics to fit or invert generative models, see Hinton and van Camp (1993), Friston (2008), Friston et al. (2008) for details. For those readers with a time-series background, Eq. 3a has the form of a generalized Kalman–Bucy filter and is indeed called Generalized Filtering (Friston et al., 2010). The reason it is generalized is that it operates on generalized states μ˜ = [μ,μ ,μ, ...] *<sup>T</sup>* , where <sup>D</sup> is a matrix derivative operator, such that <sup>D</sup>μ˜ = [μ ,μ,μ, ...] *<sup>T</sup>* . See **Table 1** and Friston et al. (2010).

It can be seen that the solution to Eq. 3a [when the motion of the prediction is equal to the predicted motion ˙ <sup>μ</sup>˜ (*x*) <sup>=</sup> <sup>D</sup>μ˜ (*x*) ] minimizes free energy, because the change in free energy with respect to the generalized states is zero. At this point, the internal states minimize free energy or maximize Bayesian model evidence and become Bayes-optimal estimates of the hidden variables.

Gaussian assumptions about the random fluctuations in the generative model (Eq. 1) allow us to write down the form of the free energy and therefore predict the exact behavior of the dendrite. Omitting constants, the free energy according to Eq. 1 is:

$$\begin{aligned} \mathcal{F} &= \mathcal{G}(\tilde{\mu}) + \frac{1}{2} \ln|\mathcal{G}\_{\tilde{\Pi}\tilde{\mu}}| \\ \mathcal{G} &= \frac{1}{2} \tilde{\varepsilon}^{(s)T} \tilde{\Pi}^{(s)} \tilde{\varepsilon}^{(s)} + \frac{1}{2} \tilde{\varepsilon}^{(x)T} \tilde{\Pi}^{(x)} \tilde{\varepsilon}^{(x)} + \frac{1}{2} \varepsilon^{(\gamma)T} \Pi^{(\gamma)} \varepsilon^{(\gamma)} \\ &\quad - \frac{1}{2} \ln|\tilde{\Pi}^{(\nu)} \tilde{\Pi}^{(x)} \Pi^{(\gamma)}| \\ \tilde{\varepsilon}^{(s)} &= \tilde{s} - \tilde{\mathcal{g}}(\tilde{\mu}) \\ \tilde{\varepsilon}^{(x)} &= \mathcal{D}\tilde{\mu}^{(x)} - \tilde{f}(\tilde{\mu}) \\ \varepsilon^{(\gamma)} &= \mu^{(\gamma)} - \eta^{(\gamma)} \end{aligned} \tag{4}$$

In these expressions, a subscript denotes differentiation. The expression for <sup>G</sup> may appear complicated but the first three terms are simply the sum of squares of precision-weighted prediction errors. The last three equalities are prediction errors for the sensory states, hidden states, and log-precisions. The synaptic precisions; (*s*) *ii* <sup>=</sup> exp(μ(γ) <sup>i</sup> ) depend on the optimized log-precisions or gains, where (η(γ) , Π(γ) ) are the prior expectation and precision on these log-precisions. In other words, the dendrite embodies the prior belief that <sup>γ</sup> <sup>∼</sup> <sup>N</sup>(η(γ) , Σ(γ) ).

Given Eq. 4, we can now specify the dynamics of its internal states according to Eq. 3a:

$$\begin{aligned} \dot{\tilde{\mu}}^{(\mathbf{x})} &= \mathcal{D}\tilde{\mu}^{(\mathbf{x})} + \tilde{\mathbb{g}}\_{\tilde{\mathbf{x}}}^T \tilde{\Pi}^{(\mathbf{s})} \tilde{\mathbf{c}}^{(\mathbf{s})} + (\tilde{f}\_{\tilde{\mathbf{x}}}^T - \mathcal{D}^T) \tilde{\Pi}^{(\mathbf{x})} \tilde{\mathbf{c}}^{(\mathbf{x})} \\ \dot{\tilde{\mu}}^{(\mathbf{v})} &= \mathcal{D}\tilde{\mu}^{(\mathbf{v})} + \tilde{f}\_{\tilde{\mathbf{v}}}^T \tilde{\Pi}^{(\mathbf{x})} \tilde{\mathbf{c}}^{(\mathbf{x})} \end{aligned} \tag{5}$$

These equations describe the segment-specific dynamics <sup>μ</sup>˜ (*x*) and dendrite-wide dynamics <sup>μ</sup>˜ (*v*) that we associate with local depolarization, within each segment and intracellular (e.g., calcium ion concentrations) throughout the dendrite (see Discussion). The precision Π(*s*) *ii* <sup>=</sup> exp(μ(γ) *<sup>i</sup>* ) can be regarded as the gain of a synaptic connection, because it modulates the effect of presynaptic input on internal states. This leads to the next level of optimization; namely, changes in synaptic gain or plasticity.

#### **SYNAPTIC DYNAMICS (Eq. 3b)**

Equation 3b prescribes the dynamics of the log-precision parameter that we associate with synaptic efficacy or gain. Mathematically, one can see that the solution to Eq. 3b minimizes free energy, when both the change in efficacy and its motion are zero. This motion can be thought of as a synaptic tag μ(γ) at each connection that accumulates the prediction error on inputs. Under the model in Eq. 1, Eq. 3b becomes

$$\begin{aligned} \dot{\mu}^{\prime(\flat)} &= \mu^{\prime(\flat)}\\ \dot{\mu}^{\prime(\flat)} &= \frac{1}{2} \dim(\tilde{\Pi}^{(s)}) - \frac{1}{2} \tilde{\varepsilon}^{(s)T} \tilde{\Pi}^{(s)} \tilde{\varepsilon}^{(s)} - \Pi^{(\flat)} \varepsilon^{(\flat)} - \pi \mu^{\prime(\flat)} \end{aligned} \tag{6}$$

This has a simple and plausible interpretation: the log-precision or gain has a tendency to increase but is counterbalanced by the precision-weighted sum of the squared error (e.g., potential difference) due to the inputs. When the noise is higher than predicted, the level of the synaptic tag will fall and synaptic efficacy will follow. The final two terms mediate a decay of the synaptic tag that depends on its prior expectation η(γ) , see Eq. 4, where π is the (large) precision on prior beliefs that connections change slowly, see Friston et al. (2010). In the simulations, we actually update the efficacy by solving Eq. 6 after it has been exposed to a short periods of inputs (128 time-bins). Finally, we turn to the last level of optimization, in which the form of the model (deployment of connections) is updated.

#### **SYNAPTIC SELECTION (Eq. 3c)**

Equation 3c can be regarded as a form of Bayesian model selection, when the dendrite reconfigures its model in structural terms; that is, by redeploying synaptic connections through changing the matrix **W**. This is implemented using the free energy or logevidence for competing models. For each connection, two models are considered: a model *m*<sup>0</sup> with a synapse that has a low prior log-precision η(γ) <sup>0</sup> = −2 and a model *m*<sup>1</sup> in which the connection has a high prior η(γ) <sup>1</sup> = 4. If the evidence for the model with a high prior (gain) is greater, then the synapse is retained. Intuitively, this procedure makes sense as model *m*<sup>1</sup> with high prior will be better than model *m*0, if the internal states of the dendrite predicted the input (the dendrite sampled what it expected). Otherwise, if synaptic input is unpredictable (and model *m*<sup>0</sup> is better than *m*1) the synapse is removed (regresses) and is redeployed randomly to sample another input. The corresponding odds ratio or Bayes factor for this model comparison is

$$\begin{split} \frac{p(\tilde{\mathbf{s}}|m\_1)}{p(\tilde{\mathbf{s}}|m\_0)} &= \int p(\boldsymbol{\gamma}\_i|\tilde{\mathbf{s}}, m\_0) \frac{p(\boldsymbol{\gamma}\_i|m\_0)}{p(\boldsymbol{\gamma}\_i|m\_1)} d\boldsymbol{\gamma}\_i \\ \varrho(\boldsymbol{\gamma}\_i|m\_k) &= \mathcal{N}(\boldsymbol{\eta}\_k^{(\boldsymbol{\gamma})}, \boldsymbol{\Sigma}^{(\boldsymbol{\gamma})}) \end{split} \tag{7}$$

Where **s**˜ corresponds to the all the presynaptic input seen between model updates. The details of this equality are not important here and can be found in Friston and Penny (2011). The key thing is that it is a function of the conditional density of the logprecision of the *i*-th connection *p*(γ*i*|˜**s**, *m*0). The ensuing relative probabilities of models with and without a high-precision connection have the simple (sigmoid) form shown in **Figure 4**. The threshold appears at unity because it is half-way between the high

**FIGURE4|Synaptic selection function.** This sigmoidal function measures the relative probability of two models with and without a high-precision synapse. The *i-*th synaptic connection is retained when μ(γ) *<sup>i</sup>* > 1 ; i.e., there is more evidence that the synapse has high precision. If μ(γ) *<sup>i</sup>* - 1, the synaptic connection is removed and a new, randomly chosen, presynaptic target is chosen. See also Eq. 7.

(four) and low (minus two) prior expectations we allow the neuron to consider. In the present context, this means that a connection is retained when μ(γ) *<sup>i</sup>* > 1.

In summary, after the conditional log-precisions are optimized they are used to determine whether the synapse should be retained or replaced. This decision is formally identical to a model selection procedure and can be regarded as a greedy search on models *m* ⊃**W** with different patterns of connections. In practice, we update the model after every four bursts (128 time points) of input.

#### **SIMULATIONS AND RESULTS**

Synaptic inputs were generated using the Lotka–Volterra system in **Figure 3** and presented to the dendrite using simulated noise with a log-precision of two. Each simulated 128 time-bin timeseries was presented over (64 × 4) repeated trials. The internal states of the dendrite evolved according to Eq. 3a and provide (Bayes-optimal) conditional expectations about the hidden states and cause of sampled inputs. We started with an initially random deployment of connections and changed them every four trials (after which the conditional estimates of log-precision had converged, see Eq. 3b). As described above, connections were only retained if the conditional log-precision was greater than one (see **Figure 4**). Otherwise, it was replaced at random with a connection to the same or another input. This slow model optimization or reorganization (Eq. 3c) was repeated over 64 cycles.

**Figure 5** shows the results of the ensuing optimization of the connections, based upon free-energy minimization, at the three temporal scales (Eq 3). Since the optimization scheme converged after 30 iterations we only show the first 32 iterations. The middle panel shows that the negative free-energy increased as a function of pruning over cycles of reorganization. Note that this is a stochastic search, because the relocation of each connection was chosen at

random. However, it is greedy because connections are retained if they provide a better model at that point in the search. Over time, the dendrite comes to sample presynaptic inputs in exactly the way that it expects to sample them. This means that each segment samples the same presynaptic axons (**Figure 2B**) and all five segments sample presynaptic inputs in a particular temporal sequence (a sequence prescribed by intracellular dynamics that rest on the generative model in Eq. 1). As the pattern of connections is optimized, the conditional log-precisions (or the gain) of the synaptic connections increases; as shown in the left panel of **Figure 5**. Usually, but not inevitably, once a synapse has found its place, in relation to others, it retains a relatively high log-precision, and is immune from further regression. Note that the fourth segment (synapses 13–16) converged quickly on a particular axon. After this, other segments start to stabilize, as confident predictions about their inputs enable the dendrite to implicitly discriminate between a good synapse (that samples what is expected) and a bad synapse (that does not). The sequence grows slowly until all 20 synapses have found a pattern that samples the five inputs in the order anticipated. This simulation is typical of the many that we have run. Note further, there is no unique pattern of connections; the sequence could "start" at any segment because (in this example) there was no prior constraint in the generative model that the first input would be sampled at any particular segment. Examples of synaptic connections are shown in the right panel of **Figure 5** as "connectivity matrices" in the lower row for the 1st, 10th, 20th, and 32nd cycles. We see here a progressive organization from an initially random deployment to an ordered and coherent sequence that is internally consistent with the generative model.

**Figure 6** shows the conditional predictions of synaptic input and hidden states before (upper row) and after (lower row) the synaptic pattern has been established. The right panels show the location of the connections in terms of expected log-precisions. The left panels show the predictions (solid lines) and prediction errors (red dotted lines) of synaptic inputs. Note that when the dendrite can predict its inputs (lower left) the prediction error (input noise) has a log-precision of about two (which is what we used when simulating the inputs). The predictions are based upon the conditional expectations of hidden states describing the Lotka– Volterra dynamics shown in the middle panels. Here, the solid lines correspond to the conditional expectations. After optimization, the predicted hidden states become quickly entrained by the input to show the stable sequential orbit or dynamics prescribed by the dendrite's generative model. This orbit or attractor has a single control parameter *v* (Eqs 1 and 2) that the dendrite is also predicting and implicitly estimating (see below). Notice that this synchronous entrainment never emerges before reconfiguration (upper panels) and the sequence of expected hidden states is not the sequence expected *a priori*; deviations from this sequence are necessary to explain the seemingly unpredictable input. Note that there is still a winnerless competition and sequential itinerancy to the conditional (internal) dynamics, because there are strong priors on its form. However, the expected sequence has not been realized at this stage of synaptic reconfiguration.

It is worth noting that although relatively simple, this greedy search has solved a quite remarkable problem: It has identified a viable arrangement of connections in a handful of iterations, from an enormous number 5<sup>20</sup> of potential configurations. Once the pattern of connection weights has converged the dendrite has effectively acquired selectivity for the particular spatiotemporal pattern of inputs it originally expected. This is demonstrated in the next section.

# **FUNCTIONAL SPECIALIZATION**

In the simulations above, the prior on the synaptic noise had a low log-precision (minus two). The log-precision of the hidden states and cause was assumed to be 16; i.e., the precision of the hidden states was very high relative to synaptic precision. After the synaptic reorganization converged, we reduced the log-precision on the hidden states and causes to eight, to test the ability of the branch to correctly infer its inputs, even when less confident about its internal dynamics. In this context, the hidden cause reports the presence of a sequence. This is because when the hidden cause takes small values, the implicit speed of flow through the sequence becomes very small and, effectively, the sequence disappears.

**Figure 7** illustrates the selectivity of postsynaptic responses to particular sequences of presynaptic inputs, using the hidden cause as a summary of postsynaptic responses. Each row of this figure corresponds to a different trial of 128 time-bins, in which we presented presynaptic inputs with different sequences. Each sequence was created by integrating a Lotka–Volterra system using a Gaussian bump function for the cause *v*(*t*), that peaked at *v* = 1 at time-bin 64 and adding noise in the usual way. The top row of **Figure 7** shows the sequence the dendrite expects, as can be seen by the progressive temporal shift in presynaptic input over the 20 neurons (organized into five successive segments; upper left). The corresponding postsynaptic response, modeled in terms of the conditional expectation of the hidden cause, is shown on the upper right. This shows a sustained firing throughout the sequence. In contrast,if we present the identical presynaptic inputs but in a different sequence, this sustained response collapses to 0.5 (the default). This can be seen in the lower two rows for two arbitrary examples. The thin blue lines correspond to the conditional expectation of the hidden cause that controls the velocity of orbits through the stable heteroclinic channel. The gray areas correspond to 90% confidence intervals (tubes). The thick blue lines correspond to null firing and provide a reference. While the middle row contains no evidence of the expected sequence, the lower row contains two occurrences of the sequence between

time point 40–60 and time point 80 (bottom left). These chance occurrences are promptly indicated by the postsynaptic response (bottom right). These graded responses between the maximum response (top right) and the minimum response (middle right) replicate another finding of Branco et al. (2010), who reported similar graded responses to random sequences.

# **VELOCITY-DEPENDENT RESPONSES**

Branco et al. (2010)were able to demonstrate a velocity-dependent selectivity in relation to inward and outward sequences of presynaptic activation (see **Figure 1D**).We attempted to reproduce their results using the above simulations by changing the velocity of the input sequence and presenting it in the preferred and reverse order: Presynaptic inputs were presented over 64/*v* time-bins for each of four velocities, *v* ∈ {1, 2, 4, 8}. We generated two presynaptic input sequences at each velocity, one with an inward and the other with an outward direction. **Figure 8** shows the maximum response as a function of velocity for inward (red) and outward (blue) directions using the same format as **Figure 1D** (Branco et al., 2010). As with the empirical results, velocity-dependent responses are observed up to a certain ceiling. This ceiling (about *v* = 4 in these simulations) arises because the generative model cannot predict fast or high-velocity sequences, because of a (shrinkage) prior on the hidden cause. This means the hidden states are poor estimates

of the true values and the hidden cause is underestimated as a consequence. However, within the range imposed by prior expectations, the response scales linearly with the true hidden cause (velocity). However, we were unable to reproduce the increase in the normalized response to the outward sequence as a function of velocity. It is an open question at what level this mismatch arises; e.g., at the level of the generative model presented above, which would call for a further adaptation of the model, or in terms of measuring its responses (note we report absolute as opposed to normalized responses).

# **DISCUSSION**

We have described a scheme for synaptic regression and sampling that is consistent with the selectivity of dendritic responses of pyramidal cells to spatiotemporal input sequences (Branco et al., 2010); and is consistent with the principle of free-energy minimization (Friston et al., 2006). The scheme explains how a dendrite optimizes synaptic re-organization in response to presynaptic input to minimize its free energy and therefore produce Bayes-optimal responses. From a neurobiological perspective, the mechanism implied by the model is simple; intracellular states of a dendrite are

viewed as predicting their presynaptic inputs. Postsynaptic specializations, with imprecise predictions (over a period of time) are retracted and a new dendritic spine is elaborated elsewhere. Over time, the dendrite comes to sample what it expects to sample and this self-limiting process of synaptic reorganization converges on an optimum pattern of synaptic contacts. At this point, postsynaptic responses become selective for the expected sequence of inputs. In the model, synaptic reorganization is described as model optimization at a slow time-scale (Eq. 7), which is based on free-energy minimization schemes at two faster time-scales (Eqs 5 and 6). Using simulations, we showed that this scheme leads to self-organized synaptic reorganization of a simulated dendrite and replicated two key experimental findings reported by Branco et al. (2010); directional selectivity and velocity-dependent responses.

### **FREE-ENERGY MINIMIZATION AND INTRACELLULAR DYNAMICS**

Thefunctionalform of Eq. 5 has an interesting and straightforward interpretation in terms of transmembrane voltage differences and conductances. The first equality of Eq. 5 can be associated with the dynamics of transmembrane voltage in each segment. This

suggests that the prediction errors in Eq. 4 play the role of potential differences that drive changes in voltage <sup>μ</sup>˜ (*x*) according to Kirchhoff's law (where the accumulation of charge is proportional to current). In other words, each term in Eq. 5 corresponds to a current that is the product of a conductance and potential difference (prediction error). For example, in the second term, where, <sup>ε</sup>˜(*s*) <sup>=</sup> *s*˜ − ˜*g* (μ˜), the synaptic input *s*˜(*t*) corresponds to local depolarization, while the generalized prediction of this depolarization, *g*˜ = (*<sup>I</sup>* <sup>⊗</sup>**W**)μ˜ (*x*) plays the role of a dynamic reference or reversal potential. The associated conductance *<sup>g</sup>*˜*<sup>T</sup> <sup>x</sup>*˜ <sup>Π</sup>˜ (*s*) depends upon the presence of a synaptic connection *g*˜*x*˜ = (*I* ⊗**W**) and the synaptic precision (gain). The conductance of the third term is more complicated and can be thought of in terms of active membrane dynamics of the sort seen in Hodgkin Huxley formulations of gating variables (e.g., Cessac and Samuelides, 2007). In short, our formal (Bayesoptimal) model of dendritic dynamics can be regarded as a mixture of currents due to active conductances that are entrained by synaptic currents. This is important because it means that the present model is consistent with standard models of active dendritic function; see Gulledge et al. (2005) for a review. Interestingly, we did not make any such assumption in the generative model (Eq. 1) but these active dendritic dynamics emerged as a functional feature of self-organization from free-energy minimization. An exciting prospect for future research is that one can ask how the generative model could be refined to replicate other experimental findings, such as spike timing dependent plasticity (STDP).

Although slightly more speculative, the kinetics of the hidden cause ˙ <sup>μ</sup>˜ (*v*) may correspond to slow dynamics, such as the calcium ion concentration. Although the associations with membrane voltage and calcium dynamics are hypothetical at this stage, we note they can be tested by using the dynamics described by Eq. 5 as qualitative or even quantitative predictions of empirical dendritic responses (cf, dynamic causal modeling; Kiebel et al., 2009a).

Furthermore, to identify generative models from empirical observations, one can use the concept of generalized convolution kernels, which describe the mapping between dendritic input and output. The fact that neurons are selective for temporal sequences necessarily requires the kernels to have a long support and to be non-linear (a linear kernel would just average over time and not care about temporal order). Critically, one can derive these kernels analytically from the differential equations used in the present simulations (Eq. 5). It is possible to evaluate these kernels empirically by looking at their input–output characteristics (e.g., Pienkowski et al., 2009). This means, in principle, it is possible to infer the implicit generative models used by neurons and dendrites, given empirical estimates of their generalized (Volterra) kernels and use these models to test concrete predictions of what output should be observed given some defined input, e.g., provided by glutamate uncaging.

#### **DYNAMICAL CONSTRAINTS**

In our scheme, the intracellular dynamics of a dendrite encode the implicit expectation that input is sampled in a particular sequence. This is enforced by prescribing the form of intracellular dynamics (where the parameters governing these dynamics are fixed): the only variables that can change are estimates of the hidden states and the time-varying rate constant (Eq. 1). The only parameters that are optimized (Eq. 3b) are the connections to presynaptic inputs encoded by matrix **W**. This means that a dendrite can only stop re-organizing its synaptic connections when the postsynaptic effect of synaptic inputs are consistent with (predicted by) its intracellular dynamics. Intuitively, this behavior may be interpreted by an observer as if the dendrite is actively looking for a sequence in its input. This view is conceptually important because it suggests that single neurons cannot decode arbitrary synaptic input but implicitly expect specific spatiotemporal input patterns. This scheme may be considered slightly counter-intuitive: In the textbook view, the assumption is that neuronal networks should be decoders of arbitrary spatiotemporal input, thereby mimicking the generalization abilities of the brain (Hertz et al., 1991). In contrast, in the present scheme, a dendrite of a pyramidal cell is "cherry-picking" just those inputs that happen to form particular sequences. Input selectivity of this kind is not necessarily a surprise to neurophysiologists, because this hypothesis has been entertained for some time (Destexhe, 2010). It is reasonable to expect that neurons, whatever their apparent function, generally expect specific spatiotemporal patterns of synaptic input; where the morphology of a neuron (the dendritic infrastructure and ion channel distributions) place strong constraints on its function (Torben-Nielsen and Stiefel, 2009). The advantage of this selectivity may be that constraints simplify structural reconfiguration and learning because there are fewer free parameters to optimize (and fewer local minima to confound learning). In this paper, we provide some evidence that anatomic–dynamical constraints enable synaptic reorganization by self-organization of intracellular and synaptic dynamics.

### **ACTIVE INFERENCE**

In the model, the dendritic branch is optimizing its internal representation of presynaptic inputs at a fast time-scale (Eqs. 3a,b),

while the dendrite's implicit model of those inputs is itself optimized over longer time periods, using synaptic regression (Eq. 3c). Crucially, because model-optimization changes the way that presynaptic signals are sampled, this corresponds to a form of "active inference" or optimal sampling of the local environment. Conceptually, this sampling is based on model selection, which contrasts with the use of gradient descent schemes we have used in previous work (Friston et al., 2009). The model selection scheme used here is stochastic and necessarily slow due to the sampling of synapses that do not support a sequence. However, due to its stochastic nature, the scheme is more robust to local minima and may provide a useful metaphor for how real neuronal architectures are selected; cf, neuronal Darwinism (Edelman, 1987).

#### **ROBUSTNESS, SCALABILITY, AND CONVERGENCE BEHAVIOR**

In the present paper, we did not evaluate the proposed scheme with respect to robustness to noise or artifacts, its scalability or its convergence behavior. Our aim was to provide proof of principle that free-energy minimization is a sufficient explanation for recent empirical observations about dendritic behavior. However, the present model was robust to noise and showed good convergence behavior within 64 iterations. We did not explore scalability, due mainly to computational reasons: the current implementation of free-energy minimization [dynamic expectation maximization (DEM), see software note] is relatively fast on a modern desktop computer (∼10 min) for small numbers of segments (five) but becomes prohibitive (with runtimes of hours) for dendrite models with more than 10 segments. We are currently working on a more efficient implementation and will report a thorough evaluation of the proposed algorithm and extensions in future communications.

#### **RELATED MODELING WORK**

There are several computational treatments that share key features with the present modeling approach: Gutig and Sompolinsky (2006) have described a classification of input spike trains based on the membrane potential function of a point neuron. Although both their model and inference technique differ from the present approach, they share the idea that intracellular dynamics can be used to decode spatiotemporal input structure. We extend this notion and show that Bayesian inference for non-linear dynamical systems enables decoding based on dynamical generative models (such as Lotka–Volterra dynamics). A different idea is shared with the work by Deneve (2008) who considers single spiking neurons as Bayesian decoders of their input, where decoding dynamics map to neuronal and synaptic dynamics. This is exactly the view we take here but we use non-linear dynamical systems to describe the multi-dimensional internal state of a dendrite, as opposed to a single state representation of "the internal activation level." In other words, we share the view that neurons are Bayesian decoders of their input but assume that a single neuron (dendrite) can represent many more variables than a single state. This enables us to describe spatiotemporal Bayesian decoding at multiple time-scales.

Conceptually, there is a strong link with the work of Torben-Nielsen and Stiefel (2009) where the function of a neuron (detecting the order of two inputs) is specified first, followed by an optimization of the neuron's morphology and ion channel distribution, in relation to that function. This is similar to the free-energy formulation, where a generative model specifies the function by describing what input is expected. The subsequent free-energy minimization optimizes the neuronal system to perform this function using neurobiologically plausible intracellular dynamics.As in Torben-Nielsen and Stiefel (2009), the goal is to map the resulting inversion dynamics to the intracellular dynamics of real neurons.

# **FREE-ENERGY MINIMIZATION AT DIFFERENT SCALES**

We have exploited free-energy minimization over three temporal scales in the dendritic simulations (intracellular dynamics, synaptic dynamics, and synaptic regression) and have framed these as model inversion and optimization respectively. Free energy can be minimized consistently over spatial and temporal scales because the underlying imperative is to minimize the sum or integral of free energy over all parts of the system and over all times. Because the time-integral of energy is called *action*, we are basically appealing to the principle of least action (Friston et al., 2008). Action here is fundamental and, mathematically, is an upper bound on the entropy of presynaptic inputs. In short, by minimizing surprise (self-information) at fast temporal scales, systems can place upper bounds on their entropy and therefore resist a natural tendency to disorder; i.e., they resist the second law of thermodynamics.

In terms of network formulations of free-energy minimization; how does recognizing sequences of presynaptic inputs help at the neuronal network level? The answer to this question may rest on message-passing schemes in cortical hierarchies that can be understood in terms of free-energy (prediction error) minimization (Mumford, 1992; Rao and Ballard, 1999; Friston, 2005; Kiebel et al., 2008; Friston and Kiebel, 2009). A key aspect of these schemes is that they are based on prediction error units that report the generalized motion (local trajectory) of mismatches between bottom-up presynaptic inputs and top-down predictions (Friston et al., 2008). This necessarily entails a selective response, not to input patterns at any instant in time, but patterns over time. But how does a neuron learn what to respond to? In this paper, we have avoided this question and assumed that the neuron has a pre-ordained generative model (prior expectations) of its local presynaptic milieu. This model rests upon the form and parameters of internal dynamics; i.e., the form and parameters of Lotka–Volterra dynamics. Clearly, in the real brain, these parameters themselves have to be learned (optimized). Future research may show the utility of free-energy minimization at different spatial and temporal scales to relate learning at the single neuron and network level.

# **SOFTWARE NOTE**

The simulations described in this paper are available (in Matlab code) within the DEM Toolbox of the SPM Academic freeware (http://www.fil.ion.ucl.ac.uk/spm). To reproduce the figures in this paper, type "DEM\_demo" at the Matlab prompt and select "Synaptic selection" from the user interface.

### **ACKNOWLEDGMENTS**

Karl J. Friston was funded by the Wellcome Trust. Stefan J. Kiebel was funded by the Max Planck Society. We thank Katharina von Kriegstein and Jean Daunizeau for helpful discussions.

# **REFERENCES**

Branco, T., Clark, B. A., and Hausser, M. (2010). Dendritic discrimination of temporal input sequences in cortical neurons. *Science* 329, 1671–1675.

Cessac, B., and Samuelides, M. (2007). From neuron to neural networks dynamics. *Eur. Phys. J. Spec. Top.* 142, 7–88.

Daunizeau, J., Friston, K. J., and Kiebel, S. J. (2009). Variational Bayesian identification and prediction of stochastic nonlinear dynamic causal models. *Physica D* 238, 2089–2118.

Dayan, P., and Abbott, L. F. (2005). *Theoretical Neuroscience*. Cambridge, MA: The MIT Press.

Deneve, S. (2008). Bayesian spiking neurons I: inference. *Neural Comput.* 20, 91–117.

Destexhe, A. (2010). Dendrites do it in sequences. *Science* 329, 1611–1612.

Destexhe, A., and Huguenard, J. R. (2000). Nonlinear thermodynamic models of voltage-dependent currents. *J. Comput. Neurosci.* 9, 259–270.

Edelman, G. M. (1987). *The Theory of Neuronal Group Selection*. New York: Basic Books.

Feynman, R. P. (1972). *Statistical Mechanics; a Set of Lectures*. Reading, MA: W. A. Benjamin.

Friston, K. (2005). A theory of cortical responses. *Philos. Trans. R. Soc. Lond. B Biol. Sci.* 360, 815–836.

Friston, K. (2008). Hierarchical models in the brain. *PLoS Comput. Biol.* 4, e1000211. doi:10.1371/journal.pcbi.1000211

Friston, K. (2010). Is the free-energy principle neurocentric? *Nat. Rev. Neurosci.* 11, 607–608.

Friston, K., and Kiebel, S. (2009). Predictive coding under the free-energy principle. *Philos. Trans. R. Soc. Lond. B Biol. Sci.* 364, 1211–1221.

Friston, K., Kilner, J., and Harrison, L. (2006). A free energy principle for the brain.*J. Physiol. Paris* 100,70–87.

Friston, K., and Penny, W. (2011). Post hoc Bayesian model selection. *Neuroimage* 56, 2089–2099.

Friston, K., Stephan, K., Li, B. J., and Daunizeau, J. (2010). Generalised filtering. *Math. Probl. Eng.* 2010, Article ID: 621670.

Friston, K. J. (2009). Modalities, modes, and models in functional neuroimaging. *Science* 326, 399–403.

Friston, K. J., Daunizeau, J., and Kiebel, S. J. (2009). Reinforcement learning or active inference? *PLoS ONE* 4, e6421. doi:10.1371/journal.pone.0006421

Friston, K. J., and Dolan, R. J. (2010). Computational and dynamic models in neuroimaging. *Neuroimage* 52, 752–765.

Friston, K. J., Trujillo-Barreto, N., and Daunizeau, J. (2008). DEM: a variational treatment of dynamic systems. *Neuroimage* 41, 849–885.

Gulledge, A. T., Kampa, B. M., and Stuart, G. J. (2005). Synaptic integration in dendritic trees. *J. Neurobiol.* 64, 75–90.

Gutig, R., and Sompolinsky, H. (2006). The tempotron: a neuron that learns spike timing-based decisions. *Nat. Neurosci.* 9, 420–428.

Hertz, J., Krogh, A., and Palmer, R. G. (1991). *Introduction to the Theory of Neural Computation*. Redwood City, CA: Addison-Wesley Publishing Company.

Herz, A. V. M., Gollisch, T., Machens, C. K., and Jaeger, D. (2006). Modeling single-neuron dynamics and computations: a balance of detail and abstraction. *Science* 314, 80–85.

Hinton, G. E., and van Camp, D. (1993). "Keeping the neural networks simple by minimizing the description length of the weights," in *COLT '93* *Proceedings of the Sixth Annual Conference on Computational Learning Theory*, 5–13.

Katz, L. C., and Shatz, C. J. (1996). Synaptic activity and the construction of cortical circuits. *Science* 274, 1133–1138.

Kiebel, S. J., Daunizeau, J., and Friston, K. J. (2008). A hierarchy of timescales and the brain. *PLoS Comput. Biol.* 4, e1000209. doi:10.1371/journal.pcbi.1000209

Kiebel, S. J., Garrido, M. I., Moran, R., Chen, C. C., and Friston, K. J. (2009a). Dynamic causal modeling for EEG and MEG. *Hum. Brain Mapp.* 30, 1866–1876.

Kiebel, S. J., von Kriegstein, K., Daunizeau, J., and Friston, K. J. (2009b). Recognizing sequences of sequences. *PLoS Comput. Biol.* 5, e1000464. doi:10.1371/journal.pcbi.1000464

Lendvai, B., Stern, E. A., Chen, B., and Svoboda, K. (2000). Experiencedependent plasticity of dendritic spines in the developing rat barrel cortex in vivo. *Nature* 404, 876–881.

London, M., and Hausser, M. (2005). Dendritic computation. *Annu. Rev. Neurosci.* 28, 503–532.

Mel, B. W. (2008). "Why have dendrites? A computational perspective," in *Dendrites*, eds G. Stuart, N. Spruston, and M. Hausser (New York: Oxford University Press), 421–440.

Mumford, D. (1992). On the computational architecture of the neocortex.2*.* The role of corticocortical loops. *Biol. Cybern.* 66, 241–251.

Pienkowski, M., Shaw, G., and Eggermont, J. J. (2009). Wiener-Volterra characterization of neurons in primary auditory cortex using Poissondistributed impulse train inputs. *J. Neurophysiol.* 101, 3031–3041.

Rabinovich, M. I., Varona, P., Selverston, A. I., and Abarbanel, H. D. I. (2006). Dynamical principles in neuroscience. *Rev. Mod. Phys.* 78, 1213–1265.

Rao, R. P., and Ballard, D. H. (1999). Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptivefield effects. *Nat. Neurosci.* 2, 79–87.

Spruston, N. (2008). Pyramidal neurons: dendritic structure and synaptic integration. *Nat. Rev. Neurosci.* 9, 206–221.

Torben-Nielsen, B., and Stiefel, K. M. (2009). Systematic mapping between dendritic function and structure. *Network* 20, 69–105.

Vetter, P., Roth, A., and Hausser, M. (2001). Propagation of action potentials in dendrites depends on dendritic morphology. *J. Neurophysiol.* 85, 926–937.

**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 07 June 2011; paper pending published: 05 July 2011; accepted: 06 September 2011; published online: 11 October 2011.*

*Citation: Kiebel SJ and Friston KJ (2011) Free energy and dendritic selforganization. Front. Syst. Neurosci. 5:80. doi: 10.3389/fnsys.2011.00080*

*Copyright © 2011 Kiebel and Friston. This is an open-access article subject to a non-exclusive license between the authors and Frontiers Media SA, which permits use, distribution and reproduction in other forums, providedthe original authors and source are credited and other Frontiers conditions are complied with.*

# Fragmentation: loss of global coherence or breakdown of modularity in functional brain architecture?

# *Daan van den Berg1,2, Pulin Gong1,3,4, Michael Breakspear 5,6 and Cees van Leeuwen1,7\**

*<sup>1</sup> Laboratory for Perceptual Dynamics, Brain Science Institute RIKEN, Wako-shi, Saitama, Japan*


#### *Edited by:*

*Gustavo Deco, Universitat Pompeu Fabra, Spain*

#### *Reviewed by:*

*Gustavo Deco, Universitat Pompeu Fabra, Spain Junji Ito, Research Center Juelich, Germany*

#### *\*Correspondence:*

*Cees van Leeuwen, Laboratory for Perceptual Dynamics, Brain Science Institute RIKEN, 2-1 Hirosawa, Wako-shi, Saitama 351-0198, Japan. e-mail: cees.vanleeuwen@ppw. kuleuven.be*

Psychiatric illnesses characterized by disorganized cognition, such as schizophrenia, have been described in terms of fragmentation and hence understood as reduction in functional brain connectivity, particularly in prefrontal and parietal areas. However, as graph theory shows, relatively small numbers of nonlocal connections are sufficient to ensure global coherence in the modular small-world network structure of the brain. We reconsider fragmentation in this perspective. Computational studies have shown that for a given level of connectivity in a model of coupled nonlinear oscillators, modular small-world networks evolve from an initially random organization. Here we demonstrate that with decreasing connectivity, the probability of evolving into a modular small-world network breaks down at a critical point, which scales to the percolation function of random networks with a universal exponent of α = 1.17. Thus, according to the model, local modularity systematically breaks down before there is loss of global coherence in network connectivity. We, therefore, propose that fragmentation may involve, at least in its initial stages, the inability of a dynamically evolving network to sustain a modular small-world structure. The result is in a shift in the balance in schizophrenia from local to global functional connectivity.

**Keywords: small-world, connectivity, percolation, schizophrenia, computer simulation, complex system**

# **INTRODUCTION**

Connectivity is key to understanding activity in neural systems (Sporns et al., 2000). Network connectivity in science and in engineering fields as diverse as mechanics, communication technology, public health, geography and town planning, is studied mathematically using the concepts of graph theory (Bollobas, 1998). Recently, graph theory is being applied to brain connectivity (Sporns and Zwi, 2004; Bullmore and Sporns, 2009) and its pathologies in Alzheimer's disease (Stam, 2004; Stam et al., 2007), brain tumors (Bartolomei et al., 2006), epilepsy (Ponten et al., 2007) and, in particular, schizophrenia (Bleuler, 1911/1950; Friston and Frith, 1995; Andreasen, 1999; Micheloyannis et al., 2006; Rubinov et al., 2009a).

Applying graph-theoretic concepts to the brain sheds new light on the basic principles of integration and segregation underlying adaptive cognitive processes, and on their disruption in maladaptive states. Schizophrenia has been understood as a cognitive disorder (Bleuler, 1911/1950) based on the breakdown of largescale cortico-cerebellar-thalamic-cortical (Andreasen, 1999) or prefronto-temporal circuits (Friston and Frith, 1995; Goldman-Rakic and Selemon, 1997), or more generally the inability to integrate neural processes in different brain areas, a syndrome termed dysconnectivity (Stephan et al., 2006, 2009). The density of dendritic spines is reduced in the brains of subjects with schizophrenia. This condition may pare down, in particular, the input to pyramidal cells of the dorsolateral prefrontal and temporal cortex (Garey et al., 1998; Glantz and Lewis, 2000). These cells are glutamatergic and receive projections from the thalamus and widespread cortical areas, and hence are likely to be involved in higher-level cognition. Reduced connectivity may thus lead to *fragmentation*, a loss of coherence in cognitive activity.

The relevant graph-theoretical notion is the loss of network connectivity. Graph theory enables us to model the loss of connectivity in simulated neuronal networks and predict the time course of fragmentation. On the face of it, a crucial factor appears to be *percolation*. Percolation plays an important role in the evolution, growth, and maintenance of a large variety of natural, technological, and social systems (Ben Avraham and Havlin, 2000). It refers to the probability of existence of a path between every pair of nodes in a graph, or equivalently, the graph being "connected" (Bollobas, 1998; Kesten, 2006). Whilst many previous studies have examined cortical network connectivity in schizophrenia and other disorders, none to our knowledge have employed the concept of percolation, an issue that we presently redress.

The percolation function is the cumulative density function (CDF) of percolation as a function of connectivity. It is possible,

*<sup>2</sup> Vrije Universiteit, Amsterdam, Netherlands*

*<sup>3</sup> School of Physics, University of Sydney, Sydney, NSW, Australia*

in principle, to measure the percolation function in living neural tissue, by using progressive lesioning, for instance through the administration of inhibitory neurotransmitters (Breskin et al., 2006)—clearly something not feasible in humans. Observations on human brain functional connectivity may be compared to the theoretical percolation function for random networks (Erdös and Rényi, 1959). The percolation function *Cp*(*n*) of a random graph of *n* vertices and *E* edges is given by *Cp(n)* = *e* <sup>−</sup>*e*−*k*1*(E*−*k*2*)* (Erdös and Rényi, 1959), in which *k*<sup>1</sup> = 2*/n* and *k*<sup>2</sup> = Arand*(n)* = ½ × *n* × ln*(n)*representing the *anchor point* of the random graph. Arand(*n*) is where the sigmoidal curve *Cp*(*n*) shows the greatest inflection. This critical point indicates the *percolation threshold*: if the number of connections in the graph is gradually reduced, a sudden breakdown of percolation occurs, and the network decomposes into several disconnected fragments.

The presence of a critical threshold motivates us to revisit the notion of cortical dysconnectivity as a sudden breakdown of percolation. There are, however, reasons to assume that the percolation threshold is neither the first, nor the most predominant, critical transition in the development of schizophrenia: brains are not random networks. In both the structural (Sporns and Zwi, 2004) and functional (Salvador et al., 2005; Achard et al., 2006; Bassett et al., 2006) domains, the hallmarks of brain organization include local clustering as expressed in high values for the Clustering Coefficient (CC), high global connectedness as specified by a short Characteristic Path Length (CPL) (Watts and Strogatz, 1998), and modularity (Murre and Sturdy, 1995) a combination characteristic of modular small-world networks (He et al., 2009). Graph-theoretical studies (Murre and Sturdy, 1995; Watts and Strogatz, 1998) showed that small-world and modular networks can secure global connectivity with a small number of connections. For brains configured as modular smallworlds, a few connections will suffice to ensure percolation. Most likely, therefore, percolation is not *the* crucial bottleneck for brain pathologies such as schizophrenia.

We will propose as an alternative theoretical possibility that, instead, brain pathologies are associated with a breakdown in the local organization. In schizophrenia patients, functional connectivity in scalp EEG channels appears to reflect a loss of clustering after correcting for differences in the density of functional connections (Micheloyannis et al., 2006; Rubinov et al., 2009a). The question, therefore, arises, whether fragmentation can be understood as a critical breakdown in the ability of the brain to establish and maintain a modular small-world functional architecture. Here we show by numerical simulations that in neural activity networks, with loss of connectivity a self-organizing small-world neural network cannot sustain its local clustering, well before global connectivity breaks down.

#### **METHODS**

The iterated logistic map *<sup>f</sup>(x)* <sup>=</sup> <sup>1</sup> <sup>−</sup> *ax*<sup>2</sup> is unimodal on [−1;1] →[−1;1] and capable of both periodic and chaotic behavior depending on its control parameter *a*. In this study, we construct networks of coupled logistic maps, all with parameter *a* = 1*.*7 such that the dynamics of a single unit are chaotic under iteration of a randomly chosen initial activation value. A unit *xi* is coupled with coupling strength ε = 0*.*4 to any number *Mi* of other units in the network such that its activation value *xi <sup>n</sup>*+<sup>1</sup> at iteration *n* + 1 depends on the activation value of itself and all adjacent units at iteration *n*:

$$\mathbf{x}\_{n+1}^{i} = (1 - \varepsilon) f\left(\mathbf{x}\_{n}^{i}\right) + \frac{\varepsilon}{M\_{i}} \sum\_{j} j \in \mathcal{B}(i) \; f\left(\mathbf{x}\_{n}^{j}\right) \tag{1}$$

In this equation, *B(i)* is the set of units adjacent to unit *i*, *Mi* is the number of units adjacent to unit *i*. The coupling strength ε is divided by *Mi*, and has a compensation term (1 − ε) to make sure that logistic map of an individual unit retains its mapping [−1:1] → [−1:1], and thus functions properly for any numbers of adjacent units.

A network of this type can be used to study the buildup and breakdown of modularity resulting from Hebbian adaptive structural self-organization. It implements a simple rewiring rule based on synchronization of chaotic activity and rewires at most one connection per iteration, carefully keeping the network's total number of connections constant throughout the process. A single iteration of the network consists of four steps, to be repeated several times after an initial random inception:


Networks implementing these iterative steps exhibit development from an initial random configuration to modular smallworld configurations (Gong and van Leeuwen, 2003; van den Berg and van Leeuwen, 2004; Rubinov et al., 2009b) But as it turns out, both the consistent build-up of connective modularity on one hand, or the loss of structural coherence due to functional fragmentation on the other, are a result of changing dynamic activity depending critically on the number of connections in the network. The influence of these numbers shows a close relationship to the percolation function of random graphs.

### **RESULTS**

#### **EVOLVING NETWORKS**

A common principle for neural network evolution is preferential attachment (Barabási and Albert, 1999). This mechanism leads to networks that are scale-free, but not modular small-worlds. Only by combining preferential attachment with adaptive, Hebbian rewiring, does a network emerge that is scale-free and also has modular small-world network structure (Gong and van Leeuwen, 2003). An adaptive rewiring scenario for evolving networks allows networks with initially random or regular structures to develop into modular small-world structures (Gong and van Leeuwen, 2004; Rubinov et al., 2009b). The scenario requires network units (edges) that produce ongoing, non-random, non-periodic oscillatory activity. These could, for instance, be represented by spiking model neurons (Kwok et al., 2007) or by nonlinear maps as an extremely simplified model of neural mass activity (Breakspear et al., 2003a,b). With these simple units as edges, the vertices of the network represents the couplings of a coupled nonlinear map (Kaneko, 1989). Adaptive rewiring operates on this activity according to the general Hebbian principle of "what fires together wires together" (Paulsen and Sejnowski, 2000). At successive points during the systems ongoing spontaneous activity, connections are added between pairs of synchronously active but hitherto unconnected units, while connections between desynchronized units are removed (see *Methods*). Over time the network gradually assumes a modular, small-world structure (**Figure 1**).

Meanwhile a mixture of regular and irregular behavior is established in the network activity that is itself optimal for sustaining the small-world structure. Crucially, whilst low dimensional, ordered, and synchronized activity dominates within modular communities, high dimensional unsynchronized activity in connector hubs ensures that the system does not fragment (Rubinov et al., 2009b). The resulting systems can thus be thought of

as "attractors" in the space of possible systems (Gong and van Leeuwen, 2004), which offers a potential explanation for their ubiquity in biological neural networks at different scales, including the entire brain (Barabási and Albert, 1999).

In this scenario, connectivity constitutes a critical limit for the evolution to small-world structure (**Figure 2**). When the number of edges is large enough, adaptive rewiring guarantees a robust evolution from random to small-world connectivity. Below this limit, this evolution is frustrated, and fails to reach a stable asymptotic state. With reduced connectivity levels, we first encounter critical fluctuation: intermittently during some episodes, clusterings are formed intermittently, which are annihilated in other episodes. This may reflect the intermittent occurrence of certain symptoms (e.g., delusions) as the brain disease first becomes manifest. For still lower connectivity levels, adaptive rewiring becomes completely ineffective; this may reflect the advanced state of the disease.

# **PERCOLATION AND SELF-ORGANIZATION IN SMALL-WORLDS**

We compared the critical limit on the evolution to small-world structures to percolation thresholds of random networks with the same numbers of edges and vertices. **Figure 4** shows that the observed minimum CC can be modeled as a linear function of *Cp*(*n*), with *k*<sup>3</sup> for offset and *k*<sup>4</sup> for amplitude: CCpred = *k*<sup>3</sup> + *k*4*Cp(n)*. Parameter *k*<sup>3</sup> was in the range [0.107:0.196], parameter *k*<sup>4</sup> in [0.392:0.459] and parameter *k*<sup>1</sup> in [0.001:0.006]. The behavior of these parameters across network sizes was not monotonic (**Figure 4**). Parameter *k*<sup>2</sup> however, the horizontal position of the anchor point, showed a universal scaling law to the anchor point in the percolation function of random graphs, namely (**Table 1**): ASWN*(n)* <sup>=</sup> Arand*(n)*1*.*17.

# **DISCUSSION**

We propose that important insights into cortical activity and architecture can be obtained by modeling the activity-dependent rewiring of neural connections during development (Gong and van Leeuwen, 2003; Rubinov et al., 2009b). In our model, network connections evolve in accordance with the principle that the

**FIGURE 2 | Self-organization from random to small-world critically in a network of 700 vertices.** The self-organization occurs through adaptive rewiring. Whether a small-world emerges depends on the number of edges.

structure rewires in adaptation to spontaneous, on-going activity. Network structure thus evolves toward a modular small-world. This evolution, however, is only guaranteed if there are sufficiently many connections available. If connectivity is reduced below this number, the structure shifts toward randomness; in particular, local clustering is reduced.

Andreasen (1999) and Friston and Frith (1995) considered schizophrenia as fragmentation, understood as the breakdown of integration between widely distributed brain areas (Stephan et al., 2006, 2009). This breakdown can be associated with the loss of connectivity (Zalesky et al., 2011), in particular of input to Layer 3 pyramidal cells, an effect which is well-documented (e.g., Garey et al., 1998; Glantz and Lewis, 2000). Zalesky et al. (2011) observed widespread impairment in structural connectivity in schizophrenic patients, involving medial frontal, parietal/occipital and left frontal cortex. It should be observed that the loss of connectivity that may lead to the onset of schizophrenia can be relatively subtle. Across the population, inputs to layer 3 pyramidal cells are substantially reduced during late adolescence, the typical period for the onset of schizophrenia (Bourgeois et al., 1994). Given that brain connectivity is costly, it may well be that in normals, its density hovers just above the critical level (the anchor point in **Figure 4**), but in early schizophrenia it may fall just below this point.

The graph-theoretical concept of percolation tells us that a small decline in connectivity can lead to a sudden breakdown of *global* network coherence. Based on our results, however, we argue that fragmentation in brain pathologies such as schizophrenia may be considered theoretically as a breakdown in the *local* connectivity structure, prior to the loss of global coherence. The number minimally needed to secure local modularity, and hence to prevent it from shifting toward randomness in structure, is systematically related to, and greater than, that needed to secure global connectivity, even if the system has fallen into entirely random connectivity. This result is of potential importance for understanding the pathophysiological processes that give rise to this disorder.

The loss of local clustering in our model is in accordance with observations in schizophrenic patients by Micheloyannis et al. (2006) and Rubinov et al. (2009a). In Micheloyannis et al. (2006), the clinical group also showed longer path lengths than the controls, whereas in Rubinov et al. (2009a), the opposite was observed. We would have predicted path length to remain comparatively stable. Differences in methods limit the value of a direct comparison between these data. Nevertheless, we might attribute the discrepancy to the fact that in both studies comparisons were made, for statistical reasons, between networks that were thresholded to have identical connectivity. Whereas the above-mentioned effects of clustering remain relatively unaffected by threshold setting, the differences in path length rapidly disappear for lower thresholds (**Figure 1** in Micheloyannis et al., 2006). Rubinov et al. (2009a) observed larger, but looser clusters in their networks. Similarly, Breakspear et al. (2003b) reported that although there were no significant increases in the occurrence of nonlinear interdependence between pairs of electrodes in schizophrenia, there *was* an increase in the co-occurrence in multiple (widespread) instances of nonlinear interdependence. This means that a relatively large number of global connections will have survived thresholding in Rubinov's study, leading to their observation of path length shortening.

It cannot be concluded from Rubinov's study, therefore, that global connectivity is stronger in schizophrenics than in normals; it could, however, be concluded that the global connectivity becomes stronger in schizophrenia relatively to their local connectivity. Such a conclusion would entirely be in accordance with the modularity breakdown observed in our model. Along the lines set out here, a shift in the balance from local to global connectivity is perfectly consistent with an overall loss of connectivity in early schizophrenia. Lee et al. (2003) introduced the notion of "overbinding"—the formation of excessive connections that are effectively random—and, as such, do not enable distinguishing external from internal sources, thus providing conditions favorable for phenomena such as hallucination.

A possible objection to our findings is the specific choice of our rewiring algorithm. Note, however, that in the present paper we sought to establish the principled possibility using the simplest possible model, rather than to establish the empirical validity

through the most realistic model possible. Note that, as a consequence, the model contains only generic dynamical and adaptive principles. We have discussed elsewhere the robustness of this model (Gong and van Leeuwen, 2003, 2004; van den Berg and van Leeuwen, 2004; Kwok et al., 2007; Rubinov et al., 2009b).

An important limitation is that the model inevitably makes over-simplifying assumptions. In particular, it ignores the spatial embedding of the system. Inter-modular connections are physically of longer range than intra-modular ones and, therefore, have a higher metabolic cost and a greater vulnerability. They also originate from different cortical layers and involve different cell types. Preliminary analysis of models with more realistic constraints does not appear, however, to affect our conclusions. Clearly, a more differentiated model is needed to address empirical datasets such as (Rubinov et al., 2009a), an important goal of future work. However, it should also be noted that uncovering universal principles—such as those reported here—has the advantage of being "detail invariant"—that is, robust across a range of potential constraints, whereas findings arising in detailed models may not be robust to changes in those details.

We observed universal scaling behavior in adaptive selforganization of clustered small-world networks: the connectivity needed for these network properties to emerge under Hebbian rewiring scales with a universal power α = 1*.*17 to the percolation function in random networks. Note, first, that α *>* 1 might have been expected, given that the requirement to observe clustering and small-world structure are constraints additional to percolation. What is surprising is that these requirements are met with alpha very close to unity; near-linear scaling implies that these additional constraints can be realized with great efficiency.

In terms of Kolmogorov-complexity, small-worlds are compressible, whereas almost every possible network of *n* nodes and *E* edges (or equivalently a bit string of length *L* = ½*n(n* − 1*)* with

**Table 1 | Anchor points for random graphs and small-world networks.**


*Note: Anchor point Arand(n)* = *½* × *n* × ln*(n) for classic random graphs of n vertices; this anchor point indicates the percolation threshold, where the percolation function Cp(n) shows the greatest inflection. ASWN(n): anchor point for the small-world networks fitted according to Figure 4. p-values result from the fitting procedure using the Levenberg–Marquardt algorithm implemented in Fityk. Scaling power: the value of h in the equation ASWN (n)* <sup>=</sup> *Arand (n)h.*

*E* ones and *L-E* zeros) will be incompressible (Li and Vitányi, 1993). In this perspective, the ubiquity of small-world structure in real-world networks is quite astonishing: within human brains (Sporns and Zwi, 2004; Stam, 2004; Eguíluz et al., 2005; Salvador et al., 2005; Achard et al., 2006; Bartolomei et al., 2006; Micheloyannis et al., 2006; Ponten et al., 2007; Stam et al., 2007; Rubinov et al., 2009a; Bassett et al., 2010), as well as between them: networks of scientific co-authorship (Newman, 2001), collaborating movie actors (Watts and Strogatz, 1998; Amaral et al., 2000), social networks in general (Wasserman and Faust, 1994). Here we showed how such a network could arise with minimal connectivity close to random network percolation

What is the reason for the universality of the scaling exponent? We may wonder whether the same exponent found in other domains, could help us understand the principle. A study of class graphs in open-source, object-oriented software systems ranging from simple paint programs, peer-to-peer downloaders, racing games, database management software to a complete operating system, showed that the number of links between classes scales to the number of classes with an exact power α = 1*.*17. The authors found that class graphs are small-world networks at the critical threshold for the breakdown of modularity, which happens when developments to the system are widely dispersed and affect many unrelated classes in apparently distant modules (Valverde and Solé, 2007). The similarity of this finding to ours supports the view that the scaling exponent reflects a general feature in the emergence and breakdown of modular network structure.

The study of self-organizing modular small-world networks casts a new perspective on psychiatric illnesses characterized by disorganized cognition, such as schizophrenia, of which the expression has been attributed to fragmentation—a "subtle but pernicious disconnection" (Friston, 1996, p. 644). Rather than a breakdown in global connectivity, we propose that fragmentation is to be understood as a failure to organize the functional connectivity of the brain into a modular small-world structure. This is in accordance with the observed "random shift" in schizophrenic (Micheloyannis et al., 2006; Rubinov et al., 2009a) and Alzheimer patients (de Haan et al., 2009).

There are reasons why such a shift toward randomness is undesirable. Random networks are considered extremely uneconomical; in terms of cable length, an optimal configuration combines local modules with a limited number of large-scale connections (Murre and Sturdy, 1995). Even though our model does not consider distance, in terms of network topology it is still the case that information travels efficiently both within locally connected circuits of modular small-world graphs and between their circuits, which makes these networks efficient for transport or communication (Latora and Marchiori, 2001; Bassett et al., 2010).

The scaling observations tell us that fragmentation is a result of a breakdown in *local*, rather than global structure. With progressive loss of connectivity, the breakdown of modularity occurs before the breakdown of percolation. Ultimately, it may not matter which connections are lost first, the result may be a cascade of changes that lead to the network falling apart. For diagnosis, however, a proper understanding of the early stages of the disease is crucial; loss of modularity might offer a new perspective on the origins of the disease.

# **DATA**

One simulation consists of one network of *v* units and *e* connections, which is randomly initialized and then iterated exactly 4,000,000 times, simultaneously rearranging its connections and activity patterns, according to the adaptive rewiring scenario. The smallest simulation we adopt has *v* = 300 units and *e* = 2400 connections. During iteration, its CPL and its CC are taken every 1000th iteration (1000*,* 2000*,...,* 40*,*00*,*000) resulting in a 4000 point record, with a value for CC and a value for CPL at each point. Although the speed of convergence depends on the size of a network, 4, 000, 000 iterations prove to be enough to clearly discern asymptotic behavior for all simulations used in this investigation (**Figure 3**).

From the 4000 point record, the maximum, minimum, and average values for both CPL and CC are calculated from the last 2000 points. For statistical robustness, we do any single simulation five times, and average the five values over this simulationquintuple, resulting in a maximum, minimum, and average CC and CPL for the (*v* = 300, *e* = 2400) network.

We then start a new quintuple of simulations, increasing the number of connections *e* by 20, generating five networks with *v* = 300 units and *e* = 2420 connections, and calculate the maximum, minimum, and average CC and CPL values from these five new simulations. We keep starting new quintuples, repeatedly increasing *e* by 20, until *e* = 3300 and the batch of 300-quintuples is complete. From the entire batch, The six CC and CPL values of every (*v, e*) are taken to graph the asymptotic clustering and path-length behavior of networks of 300 units as it depends on the numbers of connections (**Figure 4**).

This process is then repeated for a batch of quintuples of networks with 400 units and numbers of connections 3600, 3620, ..., 5000 (see **Figure 4**, top-right box). We continue doing this for batches of networks with 500, 600, 700, 800, 900, and 1000 units, with connections increasing by 20, showing asymptotic clustering and path-length behavior depending on connectivity

for networks of different sizes (**Figure 4**). Note that although for 300 units, connections ranged from *e* = 2400 to *e* = 3300, these numbers are different for larger networks.

For each of the eight batches, a phase transition was witnessed for both the CC and the CPL. To pin down the exact location of the steepest inclination in the phase transition of the CC (its center, or "anchor point"), the percolation function for classic random graphs was function-matched to the Minimum CC of every batch. The minimum cluster coefficient was chosen over the average and the maximum cluster coefficient because it has the steepest inclination, which facilitates the fitting best.

The entire process of fitting was done in Fityk under Linux using Levenberg–Macquardt an iterative curve-fitting algorithm which operates by minimizing the summed squares of the residuals, in this case the difference between minimal CC-values of *n*-edge simulation quintuples on the one hand, and the (Erdös and Rényi, 1959) percolation function's value for *n* edges on the other. Both graphs are depicted in overlay in **Figure 4**.

# **REFERENCES**


functional networks. *Proc. Natl. Acad. Sci. U.S.A.* 103, 19518–19523.


The Levenberg–Macquardt algorithm is sensitive to local minima which makes it inefficient when using completely random initial values. Initial parameters were hand-guessed separately for each of the eight subgraphs in **Figure 4**, after which the algorithm was ran until convergence beyond the program's six-digit resolution, a procedure that was repeated three times with small differences in the hand-guessed initial parameters. The final values did not differ within the program's six-digit resolution over the three repetitions, and convergence was very fast (typically well before 100 iterations).

The fits show significant deviations from the data curve, due to intrinsic fluctuations in the data. Nevertheless the fits adequately track the data curve. We, therefore, considered reliable the estimates of the scaling power and other model parameters. Even more reliable estimates could, in principle, be obtained by scaling up the network size to 2000, 5000, and 10,000 vertices, resources permitting, as computation time and data grow nonlinearly with network size.

Functional neural network analysis in frontotemporal dementia and Alzheimer's disease using EEG and graph theory. *BMC Neurosci.* 10, 101.


Neurophysiological architecture of functional magnetic resonance images of human brain. *Cereb. Cortex* 15, 1322–1331.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 23 September 2011; paper pending published: 13 December 2011; accepted: 14 March 2012; published online: 30 March 2012.*

*Citation: van den Berg D, Gong P, Breakspear M and van Leeuwen C (2012) Fragmentation: loss of global coherence or breakdown of modularity in functional brain architecture?. Front. Syst. Neurosci. 6:20. doi: 10.3389/fnsys. 2012.00020*

*Copyright © 2012 van den Berg, Gong, Breakspear and van Leeuwen. This is an open-access article distributed under the terms of the Creative Commons Attribution Non Commercial License, which permits non-commercial use, distribution, and reproduction in other forums, provided the original authors and source are credited.*

# Organization of anti-phase synchronization pattern in neural networks: what are the key factors?

# *Dong Li and Changsong Zhou\**

*Department of Physics, Centre for Nonlinear Studies and The Beijing-Hong Kong-Singapore Joint Centre for Non-linear and Complex Systems (Hong Kong), Hong Kong Baptist University, Hong Kong, China*

#### *Edited by:*

*Claus Hilgetag, Jacobs University Bremen, Germany*

#### *Reviewed by:*

*Huo Lu, Philadelphia College of Osteopathic Medicine, USA Marc Huett, Jacobs University, Germany*

### *\*Correspondence:*

*Changsong Zhou, Department of Physics, Hong Kong Baptist University, Kowloon Tong, Hong Kong, China. e-mail: cszhou@hkbu.edu.hk*

Anti-phase oscillation has been widely observed in cortical neural network. Elucidating the mechanism underlying the organization of anti-phase pattern is of significance for better understanding more complicated pattern formations in brain networks. In dynamical systems theory, the organization of anti-phase oscillation pattern has usually been considered to relate to time delay in coupling. This is consistent to conduction delays in real neural networks in the brain due to finite propagation velocity of action potentials. However, other structural factors in cortical neural network, such as modular organization (connection density) and the coupling types (excitatory or inhibitory), could also play an important role. In this work, we investigate the anti-phase oscillation pattern organized on a two-module network of either neuronal cell model or neural mass model, and analyze the impact of the conduction delay times, the connection densities, and coupling types. Our results show that delay times and coupling types can play key roles in this organization. The connection densities may have an influence on the stability if an anti-phase pattern exists due to the other factors. Furthermore, we show that anti-phase synchronization of slow oscillations can be achieved with small delay times if there is interaction between slow and fast oscillations. These results are significant for further understanding more realistic spatiotemporal dynamics of cortico-cortical communications.

**Keywords: anti-phase, delay time, modular network, connection density, excitatory and inhibitory couplings**

# **1. INTRODUCTION**

Dynamical activity of the neural systems in the brain is characterized by collective oscillatory activity over a broad range of frequencies, showing complex spatiotemporal pattern formations (Gusnard and Raichle,2001;Corbetta and Shulman,2002;Buzsáki, 2006). These complicated patterns are related to cognitive process (Engel et al., 2001; Fries, 2005), so as to provide wonderful objects for research in order to properly understand the information processing in the brain. The mechanisms underlying these spatiotemporal patterns are consist of at least two components. One component refers to the background, which is a type of spontaneous pattern formation, not necessarily related to a state of cognition in evidence. This pattern is organized basing on the fact that in the absence of cognitive tasks, neurons still keep on firing and sending information to their neighbors. The other component refers to the pattern induced by some cognitive tasks, which is organized basing on the background one (Dosenbach et al., 2006; Ouyang et al., 2011). It is therefore of fundamental importance to gain insight into the organization of the background pattern formation for better understanding how brain realize its cognitive functioning.

The neuronal networks in the brain are very complex in topological structure (Hilgetag et al., 2000; Hilgetag and Kaiser, 2004; Sporns et al., 2004; Bassett and Bullmore, 2006; Hagmann et al., 2008; Bonifazi et al., 2009; Bullmore and Sporns, 2009). On such complex networks, even if in the absence of any external stimuli, the dynamical patterns could be very complicated under the

condition of spontaneous communications among the dynamical nodes (Newman, 2003; Boccaletti et al., 2006; Arenas et al., 2008; Yuan et al., 2008). The two simplest patterns are in-phase (zero time lag) and anti-phase oscillations. In-phase pattern means that the neurons fire simultaneously. Long-distance in-phase synchronization is believed to benefit for the integration of separated functions performed in different regions, which has been a topic of great interests (Engel et al., 1991b; Roelfsema et al., 1997; Rodriguez et al., 1999;Varela et al., 2001;Wang et al., 2006;Vicente et al., 2008). Differently, anti-phase pattern means that certain areas of the brain normally increase activity, when others decrease activity. Anti-phase pattern can be considered as the simplest case, where two regions can be distinguished from each other from the viewpoint of dynamics, so that the mechanism underlying the anti-phase patterns is quite useful for deeper understanding the formation of functional regions. In neuronal networks, such anti-phase patterns have been widely observed in experiments (Greicius et al., 2003; Fox et al., 2005; Fox and Raichle, 2007; Mantini et al., 2007; Shmueli et al., 2007; Vincent et al., 2007; Lewis et al., 2009). Actually, during attentional tasks, the pattern of different areas of functional networks can usually show two opposite types of responses that increase (Cabeza and Nyberg, 2000; Corbetta and Shulman, 2002) or decreases activity (Gusnard and Raichle, 2001; Simpson et al., 2001), so as to organize a type of anti-phase pattern. An anti-phase oscillation is also observed during the rest state (Greicius et al., 2003; Fox et al., 2005; Mantini et al., 2007). For example, it was demonstrated by

using functional magnetic resonance imaging (fMRI) method that spontaneous blood oxygen level-dependent signal during rest fluctuates slowly with a small frequency and is highly organized into anti-phase pattern (Fox et al., 2005). Even during sleep, similar phenomena have also been observed that some regions increase activities meanwhile some other regions decrease activities (Kaufmann et al., 2006; Horovitz et al., 2009). Anti-phase oscillation patterns have also been found in anesthetized monkeys (Vincent et al., 2007), meaning that they do not necessarily reflect a state of consciousness. These patterns are experimentally found to oscillate at various bands of frequencies, e.g., the cardiac rate (Shmueli et al., 2007) or even much slower (<0.1 Hz; Mantini et al., 2007). The anti-phase pattern in the absence of task is a type of background one organized by the spontaneous cortico-cortical communication dynamics, and the intrinsic dynamical reasons for the organization of the anti-phase pattern in the absence of overt task performance may have some relationship to that typically seen during attentional tasks.

In dynamical systems, the organization of anti-phase oscillation pattern has usually been considered relating to the time-delays in coupling, which is consistent with the real neuronal networks in the brain, where communication between neurons are carried out by propagation of action potentials from one neuron to the network neighbors through neuronal axons, with finite velocity (Swadlow, 1985, 2000). This finite velocity leads to conduction delays, which can reach up to many tens of milliseconds. Some of the unmyelinated axons can generate a delay time as large as 300 ms. Furthermore, cortical architectures are hierarchical modular networks with several characteristics: (1) The distances between neurons within a module are usually shorter than that in different modules (Achard and Bullmore, 2007; Bullmore and Sporns, 2009); Consequently, the delay time is most likely smaller within modules. (2) The connection densities are usually larger within modules (Hilgetag et al., 2000; Hilgetag and Kaiser, 2004; Achard and Bullmore, 2007; da F Costa et al., 2007), which may impact on synchronization within and between the modules. (3) Inhibitory couplings usually form local connections (Albus and Wahle, 1994; Bosking et al., 1997; Battaglia et al., 2007). As a result, inhibitory coupling likely exists more within modules.

These important structural factors may influence the pattern formation of neural networks significantly. In a recent theoretical study (Deco et al., 2009), it is shown that the anti-phase patterns can emerge from noise-driven transitions between different multistable cluster synchronization states, with a two-community network structure. In the study, impact of different factors of the time delay and the modularity is mixed. However, to elucidate which characteristics is crucial for the origin of anti-phase patterns is actually a significant problem.

In this work, we investigate the anti-phase oscillation pattern organized on a two-module network of either neuron cell model or neural mass model and study the key factors among the conduction delay times, the connection densities, and the coupling types (excitatory or inhibitory). Our results show that delay times and coupling types can play key roles in this organization, but connection densities cannot. In an excitatory coupled neuronal systems, the role of delay time is similar to a classical case of coupled phase oscillators. A delay time close to half-period can induce anti-phase

pattern between two neuronal modules. However, this is not a necessary condition, since we further reveal that if there exist an interaction between low and fast oscillations, small delay time can contribute to the organization of anti-phase pattern in slow oscillation, which could be especially relevant to those experimentally observed anti-phase synchronization of very small frequencies. The investigation is significant for understanding more complicated spatiotemporal pattern formations in the brains organized by cortico-cortical communication dynamics.

# **2. MATERIALS AND METHODS**

To study the key factors for the organization of an anti-phase pattern between cortices, we analyze the dynamical behaviors of both coupled neuron cells model and coupled neural mass model on a two-module network. As schematically demonstrated in **Figure 1A**, a part of cortex is modeled by a module, and the communication between cortices is represented by the time-delayed interaction between modules. By changing several important factors, we aim to elucidate the conditions for supporting an antiphase oscillation between modules, as illustrated in **Figure 1B**. Within each module, the collective behaviors of the oscillators should show a macroscopic rhythmic oscillation. In the simplest case, these oscillators within each module arrive at a relatively high degree of coherence with small phase shifts (approximate in-phase oscillation).

The factors we will study in the following include conduction delay times, connection densities, and coupling types (excitatory or inhibitory). In the brain, the nodes (nerve cells or cortices) are connected in different distances. Generally speaking,long-distance connections contribute to integrating the functions of different areas and short-distance connections can save the connection costs. According to experimental results, the nodes within-module usually have much shorter distance than those between modules (Achard and Bullmore, 2007; Bullmore and Sporns, 2009), so that

**organized on a two-module network. (A)** The network is divided into two modules. Several factors are different within-module and between modules, including conduction delays, connection densities, and coupling types (excitatory or inhibitory). **(B)** Illustration of anti-phase oscillation between modules. The lines stand for some kinds of collective behaviors within each module, e.g., total firing rate, total synaptic current, etc.

we can suppose that the conduction delays are usually smaller within modules. On the other hand, it has been reported that some long-distance conduction delays may not be too large in myelinated axons (Waxman, 1980), which is also believed to be significant for the integration of the functions. We will study systematically how the synchronization pattern changes when the delays within and between modules are varied.

The second factor is the connection densities. In the brain networks, there are much more short-distance connections than long-distance ones. In this work, the word *module* is used with broader meanings to allow us change various factors within and between modules, however,in the usual sense, the concept of module is based on the connection densities (Hilgetag et al., 2000; Clauset et al., 2004; Hilgetag and Kaiser, 2004; Newman and Girvan, 2004; Newman, 2006; Achard and Bullmore, 2007; da F Costa et al., 2007). Denser connections within a module and sparser connections between modules may be significant for performing some particular functions by the modules while maintaining communication between modules, allowing both functional segregation and integration. The modularity has been shown to impact on synchronization of oscillators without considering delays (Arenas et al., 2006, 2008; Gómez-Gardeñes et al., 2010; Zhao et al., 2011). However, its role in organizing the anti-phase pattern between modules in the presence of heterogeneous delays is not known. This issue is a very important open problem. In experiments,functional networks are usually detected by some coherent activities (Scannell et al., 1999; Hilgetag and Kaiser, 2004), and anti-phase pattern could be the simplest case that the coherent activity within one module can be distinguished from another. A deep understanding of the impact of connection densities in organizing anti-phase pattern is useful to elucidate the role of modularity in organizing functional networks in the brain.

The third factor refers to the coupling types. In the brain, there coexist excitatory and inhibitory neurons. They have different numbers, distributions, and connected distances in brains (Albus and Wahle, 1994; Bosking et al., 1997; Battaglia et al., 2007), so that the distributions of excitatory and inhibitory connections are also different within-module and between modules. The competition and balance of excitation and inhibition has profound effects on the collective dynamics of neural network, such as the emergence of slow oscillations (Wang et al., 2011). Therefore, the type of coupling may also contribute to the organization of anti-phase patterns in cortical communication.

When these factors are taken into consideration, we can simplify some other settings in our model. We consider a directed random network of *N*/2 nodes for each module. We assume that the connection probability *k*in and time-delays τ in are uniform for connections within the modules. Likewise, they are also uniform for the connections between modules and are denoted as *k*out and τ out, respectively. The ratios of the number of excitatory links to the number of all links within and between modules are represented as in and out, respectively. In the brain, the overall ratio is about = 0.8, while in our model we assume it is a tunable parameter.

After the analysis of this simplified model, we will also study the impact of distributed delay times using a realistic cortical network of visual system of macaque monkey, where the delay time is assumed to depend on the distance between the functional regions.

# **2.1. NEURON CELL MODEL**

We first study the dynamical behaviors with pulse-coupled integrate-and-fire (IAF) neuron model on each node. Each module may represent a local neuronal circuit from two distant, but connected cortical regions. The IAF model is described as

$$\frac{dV\_j}{dt} = I\_d - V\_j + \sum\_i K\_{\vec{\eta}i} \delta \left( t - t\_{i,k}^s - \tau\_{\vec{\eta}j} \right), \tag{1}$$

where *V*<sup>j</sup> is the membrane potential of the *j*th neuron. Whenever the membrane potential of a neuron crosses a spiking threshold *Vth*, an action potential is generated and the membrane potential is reset to the resting potential *Vr*. *t<sup>s</sup> <sup>i</sup>*,*<sup>k</sup>* is the time of the *k*th spike sent by neuron *i*. It takes a delay time τ *ij* for the spike to propagate to the neuron *j*. τ *ij* = τ *in* when the two neurons are from the same module and τ *ij* = τ *out* otherwise. *Kij* is the connectivity matrix, and *Kij* = 0 means there is no link from neuron *i* to neuron *j*. Otherwise, *Kij* are given a value common for all *i* and a positive (negative) *Kij* represents that the presynaptic neuron *i* is excitatory (inhibitory). The dc current *Id* is set that the neurons keep on spontaneous firing.

While investigating the aforementioned three factors, we respectively change the conduction delays (τ *in* -= τ *out*; *kin* = *kout*; and *in* = *out* = 100%), connection densities (*kin* -= *kout*; τ *in* = τ *out*; and *in* = *out* = 100%), and nerve type. In the third case, where we investigate the factor of coupling types, we change 20% neurons to inhibitory ones in each module. The output links of these inhibitory neurons only link to the neighbors within each module, so that we get *in* = 80% -= *out* = 100%, τ *in* = τ *out*, and *kin* = *kout*.

### **2.2. NEURAL MASS MODEL**

We can consider a model representing the cortical network of higher hierarchy compared to the neuronal networks in subsection 2.1. In almost all cortical regions, there is a basic neural circuit composed of a pyramidal cell receiving excitatory input from extrinsic afferent systems and spiny cells and inhibitory input from interneurons. Neural mass model (Wendling et al., 2000; Zhou et al., 2007) has been developed based on such basic microcircuits to describe mean activity of the cortical networks. The dynamics are described by average membrane potential *v* and spike density *S* of three subpopulations: excitatory pyramidal cells, excitatory and inhibitory interneurons. A network of coupled neural mass oscillators is described by the following equations,

$$\ddot{\boldsymbol{v}}\_{0}^{i}(t) = Aa\boldsymbol{S}\left[\boldsymbol{\nu}\_{1}^{i}(t) - \boldsymbol{\nu}\_{2}^{i}(t)\right] - 2a\dot{\boldsymbol{\nu}}\_{0}^{i}(t) - a^{2}\boldsymbol{\nu}\_{0}^{i}(t)\,,\tag{2}$$

$$\begin{split} \ddot{\boldsymbol{\nu}}\_{1}^{i}(t) &= Aa\left\{\boldsymbol{I}\_{0} + \sum\_{j=1,...,N} \text{g}\_{j}^{i}\boldsymbol{S}\left[\boldsymbol{\nu}\_{1}^{j}\left(t - \tau\_{ij}\right) - \boldsymbol{\nu}\_{2}^{j}\left(t - \tau\_{ij}\right)\right] \\ &+ C\_{2}\boldsymbol{\mathcal{S}}\left(\boldsymbol{C}\_{1}\boldsymbol{\nu}\_{0}^{i}(t)\right)\right\} - 2a\dot{\boldsymbol{\nu}}\_{1}^{i}(t) - a^{2}\boldsymbol{\nu}\_{1}^{i}(t) + \boldsymbol{\eta}\_{i}\left(t\right),\tag{3} \end{split}$$

$$\ddot{\boldsymbol{\nu}}\_{2}^{i}(t) = Bb \left\{ \mathbf{C}\_{4} \mathbf{S} \left[ \mathbf{C}\_{3} \boldsymbol{\nu}\_{0}^{i}(t) \right] \right\} - 2b \dot{\boldsymbol{\nu}}\_{2}^{i}(t) - b^{2} \boldsymbol{\nu}\_{2}^{i}(t) \,, \tag{4}$$

where the spike density is related to the potential by the sigmoid function

The anti-phase index ϑ*an* between modules is defined as

$$S\left(\nu\right) = 2e\_0 \Big/ \left(1 + e^{r(\nu\_0 - \nu)}\right),\tag{5}$$

where 2*e*<sup>0</sup> is the maximum firing rate, *r* <sup>0</sup> is the post-synaptic potential corresponding to a firing rate of *e*<sup>0</sup> and *r* is the steepness of the sigmoid. Here *C*<sup>1</sup> and *C*2, *C*<sup>3</sup> and *C*<sup>4</sup> are the average number of synaptic contacts,for the excitatory and inhibitory synapses, respectively. The superscript *i* is the serial number of the population. η*i*(*t*) represents independent background white noise. τ *ij* is the time delay from the population *j* to the population *i*. When the population *i* and *j* belong to the same module, τ *ij* = τ *in*; otherwise, τ *ij* = τ *out*. *gij* Is the coupling strength, where a positive *gij* is equivalent to an average excitatory effect and a negative *gij* to an average inhibitory effect. In numerical simulations, we use very small *gij*, so that the frequency of coupled oscillators does not have a big difference from that of a single oscillator.

This model can generate either delta or alpha oscillation, depending on the injected dc current *I* 0. The spatiotemporal patterns are very complicated, depending not only on system parameters but also on initial states, so that we use an anti-phase pattern with some noise perturbation as the initial state for numerical simulations in order to investigate the role of important factors in persisting this pattern.

We use a set of typical system parameters as in Wendling et al. (2000): *<sup>A</sup>* <sup>=</sup> 3.25 *mV*, *<sup>B</sup>* <sup>=</sup> <sup>22</sup> *mV*, *<sup>a</sup>* <sup>=</sup> <sup>100</sup> *<sup>s</sup>*−1, *<sup>b</sup>* <sup>=</sup> <sup>50</sup> *<sup>s</sup>*−1, *C*<sup>1</sup> = 135, *C*<sup>2</sup> = 108, *C*<sup>3</sup> = 33.75, *C*<sup>4</sup> = 33.75, *r0* = 6 *mV*, *e*<sup>0</sup> = 2.5 *<sup>s</sup>*−1, and *<sup>r</sup>* <sup>=</sup> 0.56 *mV* <sup>−</sup>1. In our simulation,we set*<sup>I</sup>* <sup>0</sup> <sup>=</sup> 115 mA for the delta oscillation and *I* <sup>0</sup> = 180 mA for the alpha oscillation.

#### **2.3. SYNCHRONIZATION INDEX**

In the following, we define the in-phase and anti-phase synchronization index in order to evaluate the anti-phase patterns. First we define the oscillatory phase ψ of a single neural mass oscillator *j*. Here we use a definition generalized from Pikovsky et al. (2001):

$$
\psi^{\dot{j}}\left(t\right) = 2\pi \frac{t - t\_0^{\dot{j}}}{T},
\tag{6}
$$

where *T* is the oscillation period and *t j* <sup>0</sup> is the first time that the output of the *j*th oscillator *v j* <sup>1</sup> − *v j* <sup>2</sup> arrives at its maximum value. Such a definition can also be applied to the oscillations in the neuron cell model. The order parameter within modules, Z1,2, can be defined as

$$\mathcal{Z}\_{1,2} = \left\langle e^{i\psi^\dagger(t)} \right\rangle\_{N/2},\tag{7}$$

where the subscripts 1,2 denote the two modules, and ·*N*/<sup>2</sup> means averaging within a module and over long time.

The in-phase index ϑ*in* within modules can be defined as

$$\vartheta\_{\rm in} = \frac{|\mathcal{Z}\_1| + |\mathcal{Z}\_2|}{2},\tag{8}$$

so that ϑ*in* = 1 when an ideal in-phase pattern is achieved within modules and ϑ*in* = 0 when it is a zero-coherence state.

$$\vartheta\_{an} = 1 - \frac{|\mathcal{Z}\_1 + \mathcal{Z}\_2|}{|\mathcal{Z}\_1| + |\mathcal{Z}\_2|},\tag{9}$$

so that ϑ*an* = 1 when a perfect anti-phase pattern is achieved between modules and ϑ*an* = 0 vise verse. If within a module, the oscillations are incoherence, ϑ*in* ≈ 0, and the collective behavior is merely some fluctuations. In order to avoid such irrelevant case, we only define ϑ*an* when ϑ*in* > 0.1.

# **3. RESULTS**

#### **3.1. EFFECT OF DELAY TIME**

In the neuron cell model, when we select reasonably different delay times for the connections within-module and between modules, anti-phase pattern can be achieved between the two modules. The precondition is that in-phase pattern has to be achieved within each module. An example can be seen in **Figure 2A**, where the delay time between module is τ *out* = 100 ms, and the delay time within each module is τ *in* = 2 ms. The other two parameters, the density and ratio of excitatory coupling, is the same within and between modules. The existence of an anti-phase synchronization is robust to these parameters. Therefore, the delay time can be a key factor in organizing the anti-phase pattern.

In the neural mass model, we can get similar conclusion. With *I* <sup>0</sup> = 115 mA, a single oscillator oscillates at about 1.5 Hz in the

**FIGURE 2 | Dynamics of two-module networks with neuron cell models. (A)** Delay times are different within-module τ *in* = 2 ms and between modules τ *out* = 100 ms; *kin* = *kout* = 0.6; and *in* = *out* = 100%. **(B)** Connection densities are different within-module *kin* = 1 and between modules *kout* = 0.2; τ *in* = τ *out* = 100 ms; and *in* = *out* = 100%. **(C)** 20% neurons are changed to be inhibitory and the outputs of them only link to the neighbors within modules; *in* = 80%, *out* = 100%, τ *in* = τ *out* = 150 ms, and *kin* = *kout* = 0.3.

delta band. We use excitatory couplings, i.e., positive *gij*. Numerical results show that conduction delays also play a key role in the persistence of the anti-phase patterns between modules and in-phase patterns within modules, as shown in **Figures 3A–H**, where the pronounced anti-phase pattern mainly persists in two regions: the delays within a module τ *in* ∼ 0 and the delays between modules τ *out* are around 200 ms, and τ *in* are around 500 ms and τ *out* are around 200 ms. The latter one has a larger region than the former one. This situation, i.e., the delay time between distant modules can be significantly smaller than those within the module, is unlikely to be realistic, though we can simulate it in the model. When we change the input current to *I* <sup>0</sup> = 180 mA, the frequency of a single oscillator comes to the alpha band, around 11 Hz. The numerical results are shown in **Figures 3I–P**, which are similar

to those in **Figures 3A–H**. The most prominent difference is that the time scale is only about 1/7 of the **Figures 3A–H**, noting the frequency in **Figures 3I–P**is about 7 times of that in **Figures 3A–H**.

# **3.2. EFFECT OF CONNECTION DENSITY**

In neuron cell model, no matter how we change the connection densities within-module and between modules, if other factors are homogeneous, the anti-phase pattern never emerges. An example is shown in **Figure 2B**. Therefore, the connection density is not a key factor in organizing the anti-phase pattern. However, in the neural mass model, we can see that, though anti-phase pattern cannot be found when delay times are homogeneous (τ *in* = τ *out*, the diagonal of each panel of **Figure 3**) the parameter region of τ *in* and τ *out* for anti-phase pattern are different, depending on

**FIGURE 3 | The in-phase synchronization index** *ϑin* **within a module and the anti-phase synchronization index** *ϑan* **between the modules as functions of the delay times** *τ in* **and** *τ out* **at different connection densities in networks of coupled neural mass oscillators (Eqs 2–4). (A–D)** and **(I–L)**: the in-phase synchronization index ϑ*in* in color scale. **(E–H)** and **(M–P)**: the anti-phase synchronization index ϑ*an* in color scale. ϑ*an* is not defined when

ϑ*in* < 0.1, represented by black region. *Upper panel* **(A–H)**: *I* <sup>0</sup> = 110 mA, the neural mass oscillators oscillate at about 1.5 Hz in the delta band. *Lower panel* **(I–P)**: *I*<sup>0</sup> = 180 mA, the neural mass models oscillate at about 11 Hz in the alpha band. The connections densities of the networks are: **(A,E,I,M)**, *kin* = *kout* = 0.3; **(B,F,J,N)**, *kin* = 0.6, *kout* = 0.3; **(C,G,K,O)**, *kin* = 0.9, *kout* = 0.3; **(D,H,L,P)**, *kin* = *kout* = 0.6.

connection densities. It means that the connection density also play an important role in maintaining the stability of the antiphase pattern in the presence of noise perturbation, if τ *in* and τ *out* are different.

Previously, it has been found that the connection densities have impacts on the synchronization properties of a network (Arenas et al., 2006, 2008; Gómez-Gardeñes et al., 2010; Zhao et al., 2011). The collective behaviors of nodes within a module are easier to be established since the interactions within a module is stronger (connection density is higher) than that between modules. Intuitively, the connection density probably has an impact on the organization of anti-phase pattern. In this work, we clarify that the connection density actually impact on the stability rather than the existence of an anti-phase pattern. With higher connection densities within or between the modules, the anti-phase attractors are stabilized in a broader parameter region of the delay time.

#### **3.3. EFFECT OF INHIBITORY COUPLING**

At last, when we change 20% neurons to inhibitory type in each module, anti-phase oscillation may also be observed if the inhibitory synaptic current is sufficient large, as shown in **Figure 2C**. This result also depends on the coupling strength (synaptic current). Under the condition τ *in* = τ *out* and *kin* = *kout*, in order to achieve the anti-phase pattern between modules, a good match of coupling strength with τ *in,out*, *kin,out* and *in,out* is required. In the example we show in **Figure 2C**, τ *in,out* = 150 ms, *kin,out* = 0.3,*in* = 80%,*out* = 100%, and the inhibitory coupling strength is ten times of the excitatory coupling strength. The balance of excitation and inhibition in neural network with large inhibitory coupling is realistic in biological neural networks, since inhibitory synapses are in general closer to the neuron cell body (soma; Buzsáki, 2006). These results also indicate that in the organization of an anti-phase oscillation pattern in cortex, the coupling types can be a key factor. The mechanism is that the inclusion of inhibitory coupling may generate slow oscillations within the module, which will be discussed in more detail later.

### **3.4. ANALYSIS**

Given a pair of oscillators φ1(*t*) and φ2(*t*), with time-delayed coupling, their phase difference is defined as φ = φ<sup>1</sup> − φ2. The stability of in-phase and anti-phase patterns versus delay time becomes clear when the evolution of φ can be approximately written in the following linearized form:

$$\frac{d\Delta\phi}{dt} = \begin{cases} G\left(2\pi\mathcal{Y}\tau\right)\Delta\phi, & \Delta\phi \sim 0\\ G\left(2\pi\mathcal{Y}\tau + \pi\right)\left(\Delta\phi - \pi\right), & \Delta\phi \sim \pi \end{cases},\tag{10}$$

where γ is the oscillatory frequency, τ is the delay time of the coupling and *G* is a periodic function with period 2π. In such a case, stability analysis theory can give the conclusion that when *G*(2πγτ ) < 0, the in-phase pattern ( φ = 0) is stable and when *G*(2πγτ + π) < 0, the anti-phase pattern ( φ = π) is stable. Furthermore, for some given values of delay time τ , if both *G*(2πγτ ) and *G*(2πγτ + π) are negative, the in-phase and the anti-phase pattern coexist in this parameter region; and whether the inphase or anti-phase pattern will be achieved depends on the initial conditions.

For example, in the case of classical coupled phase oscillators:

$$d\varphi\_{1,2}(\mathbf{t})/dt = 2\pi\chi + K\sin\left[\varphi\_{2,1}\left(t-\mathbf{r}\right) - \varphi\_{1,2}\left(\mathbf{t}\right)\right],\tag{11}$$

where *K* is the coupling strength, *G*(2πγτ ) takes the form *G*(2πγτ ) = −2*K* cos(2πγτ ). The function −2*K* cos(2πγτ ) (*K* > 0) versus 2πγτ is shown in **Figure 4A**. When 2πγτ <π/2 or 2πγτ > 3π/2, *G*(2πγτ ) < 0, and *G*(2πγτ + π) > 0, so that the in-phase pattern is stable. When π/2 < 2πγτ < 3π/2, *G*(2πγτ ) > 0, and *G*(2πγτ + π) < 0, the anti-phase pattern is stable.

In the following, we show that anti-phase synchronization in coupled neuron systems can be understood using coupled phase oscillators, with mathematical details presented in Appendix. The collective behaviors of coupled neuron cells are basis for the spatiotemporal pattern formations in neuron networks. In the simplest case where a pair of neuron cells are pulse-coupled together (*K*<sup>12</sup> = *K*<sup>21</sup> = *K*˜ ), the phase shift between them is determined by the delay time, coupling strength, and the initial conditions (Ernst et al., 1998). We can define the phase of an IAF cell as:

$$
\varphi^j(t) = 2\pi \frac{t - t\_0^j}{T},
\tag{12}
$$

where *t j* <sup>0</sup> is the time when the *j*th neuron fires for the first time. The membrane potential *V*(*t*) then can be expressed as a function of the phase *V*(ϕ). Actually, from Mirollo and Strogatz (1990), an IAF cell is usually described as a phase oscillator in this way. In the case of a pair of cells with small excitatory coupling, the phase difference | ϕ| = |ϕ<sup>1</sup> − ϕ2| equals to the time delay τ or 2π − τ (Ernst et al., 1998), so that a delay time approximated to half of the period can induce an apparent anti-phase oscillation pattern. This case in shown in **Figure 5A**. It looks different when compared to the classical phase oscillator in **Figure 5B**. However, there is similarity between a pair of pulse-coupled IAF model and a pair of coupled phase oscillators model: when τ approximates to 0 or 1/γ , the phase difference is small (in-phase or approximate in-phase),

whereas when τ approximates to 1/2γ , the phase difference is big (anti-phase or approximate anti-phase). This similarity actually has deeper significance in understanding key role of delay time in organizing the anti-phase pattern in neuron networks, as we will show shortly.

Let us consider a system with two populations of cells. Within each population, neurons are coupled together so that they fire orderly; and between the populations, the phase difference between pairs of neurons corresponds to the phase difference of the oscillations of the two populations. This is the simplest case where the interaction between neuron cells can map to the interaction between neuron populations. In this case, we can show (see Appendix) that the phase difference can be described by the same form as in Eq. 10, where the function *G*(2πγτ ) is piece-wise, satisfying:

$$G\left(2\pi\boldsymbol{\gamma}\boldsymbol{\tau}\right) \begin{cases} < 0, \ 2\pi\boldsymbol{\gamma}\boldsymbol{\tau} \in \textit{refract}\boldsymbol{\gamma}\text{ period} \\ > 0, \ 2\pi\boldsymbol{\gamma}\boldsymbol{\tau} \in \textit{integration}\boldsymbol{\text{period}} \\ < 0, \ 2\pi\boldsymbol{\gamma}\boldsymbol{\tau} \in \textit{fring}\boldsymbol{\text{period}} \end{cases},\tag{13}$$

as shown in **Figure 4B**. When the neuron receives a spike, if *V*(ϕ) increases to a new value *V* (ϕ)+*K*˜ , the neuron is in the integration period, (ϕ ∈ *integration period*); if *V*(ϕ) increases to *Vth* where the neuron fires a spike, it is in the firing period, (ϕ ∈ *firing period*); and if *V*(ϕ) remains at the value *Vr*, it is in the refractory period, (ϕ ∈ *refractory period*). More details about the function *G* can be obtained in the Appendix. These three periods are qualitatively

similar to the three regions I, II, and III in coupled phase oscillators with *K* > 0 in **Figure 4A**. The above analysis can therefore explains the relationship of the organizations of anti-phase pattern between coupled neuronal systems and coupled phase oscillators, so as to understand the key role of the delay time.

In a pair of coupled neural mass oscillators, the dynamical behaviors versus delay time will be more similar to the case of coupled phase oscillators. We show an example in **Figure 5C**. Compared to the coupled phase oscillators, the region for antiphase pattern is smaller and shifts to smaller τ in coupled neural mass oscillators. This result has influence on the organization of the anti-phase pattern on the two-module networks. For example, in each panel of **Figure 3**, the region with high values of the in-phase index ϑ*in* and anti-phase index ϑ*an* is smaller in size at small τ *in* than that of large τ *in*.

To organize the anti-phase oscillation pattern between modules, highly coherent oscillation (approximate in-phase synchronization) has to be achieved within modules, or otherwise there would not be macroscopic oscillations of the population except for some fluctuations. From the above analysis, we know that a suitable delay time between modules τ *out* is very important. The delay time within modules τ *in* should be small enough (or close an integer times of the period) to allow the oscillators within modules have small phase difference, so that the oscillators within each module may exhibit the approximate in-phase pattern. For pulsecoupled neuron networks, in the extreme case, all the neurons within a module fire simultaneously, and the interaction between modules is equal to the interaction between a pair of neurons as analyzed by Ernst et al. (1998). The delay time between modules approximating to half of the period can induce this anti-phase pattern. In network of neural mass models, the dynamics is more similar to the coupled phase oscillators, though the attractors are more complex, depending on the initial conditions. Therefore, different delay times within and between modules can be a key factor for anti-phase synchronization.

The connection density does not play an important role as the delay times. In the network of coupled neuron cells with homogeneous delay time τ and excitatory coupling, high-coherence collective oscillation can only be achieved when τ = *nT* where *T* is the firing period of the neurons (e.g., *n* = 4 in **Figure 2B**). Supposing that the neurons in the first module fire, after time τ , the signal transfer to the second module, inducing firing of the neurons in the second module. At the same time, the neurons in the first module also fire because of the relationship between the delay time and the firing period, so that only in-phase oscillation is observed. In this case, changing the connection density within and between modules cannot organize an anti-phase pattern. However, connection densities also have impact on the anti-phase pattern. Neuronal network is complicated in dynamics, where multiple attractors usually coexist, and network connection matrix is very important for the stability of an attractor (Memmesheimer and Timme, 2006). Connection densities play a very important role in the stability of an existing anti-phase pattern, though changing connection densities cannot influence the existence of an antiphase pattern. Specially, in the network of neuron cell model, under the condition of suitable delay times and other parameters for anti-phase pattern, if the connection density is not suitable

(17)

to stabilize this pattern robustly, this pattern is still difficult to achieve. For example, if *kin* is small, anti-phase pattern can be only achieved by using the initial condition very close to this attractor. Otherwise, with random initial conditions, the neurons will fire randomly without achieving in-phase and anti-phase synchronization. For the network of neural mass model, other attractors can also coexist with the anti-phase synchronization. Comparing the **Figures 3E–H** or the **Figures 3M–P**, one can notice that, even in these simulations we use the same initial conditions, under the noise perturbations, the parameter regions of anti-phase pattern are not the same with different *kin* or *kout*. In this case, the connection density can play an important role in preserving the anti-phase pattern.

The other key factor is the inhibitory coupling.When inhibitory couplings are added within modules, the firing patterns can be altered. Compared to the case of totally excitatory coupling, Brunel and Wang (2003) have ever shown that in the presence of inhibitory coupling, an oscillatory behavior at the population level can happen among the neurons. The period of the collective oscillations is determined by several parameters of the excitatory and inhibitory synapses. Given an oscillation period, a suitable delay time τ approximating to half of the period can induce an anti-phase pattern, similar to the case of a pair of coupled neuron cells when the oscillations within-module is highly coherent. In the example we show in **Figure 2C**, the period is about 320 ms, and the delay time is 150 ms. Therefore, the role of inhibitory couplings in anti-phase synchronization is that they induce emergent oscillations with periods fitting to delay times between modules.

#### **3.5. INTERACTION BETWEEN HIGH AND LOW FREQUENCIES**

The analysis in the previous section shows that the time delay approximating to 1/4γ <τ< 3/4γ contributes to the anti-phase patterns for oscillations with frequency γ for dominant excitatory couplings between modules. Collective activity in the brain oscillates in a quite wide range of frequencies. The conduction delays between neurons in different cortex are usually tens of milliseconds which can support anti-phase synchronization of oscillations with a period of hundreds milliseconds if the oscillations are narrowbanded. In reality, the anti-phase oscillations can be much slower (sometime slower than 0.1 Hz), corresponding to the delay times several orders of magnitude larger than the conduction delays. A crucial question is whether the relatively small delay times of neuronal communications can contribute to the slow anti-phase patterns? Our analysis using a simple mathematical model show that this is possible if there exists an interaction between slow and fast oscillations.

Supposing the signal measured from a cortical region is a function of two variables δ and α as *H*(δ,α), where δ and α respectively denote the phase of slow and fast oscillations. We describe the evolution of them as coupled phase oscillators

$$\begin{aligned} \dot{\delta}\_{1,2}\left(t\right) &= 2\pi \chi\_{\delta} \\ &+ I\_{\delta}\left[\delta\_{1,2}\left(t\right), \delta\_{2,1}\left(t-\tau\right), \alpha\_{1,2}\left(t\right), \alpha\_{2,1}\left(t-\tau\right)\right], \text{ (14)} \\ \dot{\alpha}\_{1,2}\left(t\right) &= 2\pi \chi\_{\delta} \\ &= \pi \chi\_{\delta} \end{aligned} \tag{14}$$

$$+I\_{\alpha} \left[ \delta\_{1,2} \left( t \right), \delta\_{2,1} \left( t - \tau \right), \alpha\_{1,2} \left( t \right), \alpha\_{2,1} \left( t - \tau \right) \right], \text{ (15)}$$

where γ <sup>δ</sup> and γ <sup>α</sup> represent their intrinsic frequencies and the subscripts 1 and 2 indicate two cortical regions. The question now can be specified as: can the anti-correlated population activities be observed between these two regions in δ band, if the conduction delay τ has the same order as 1/γ <sup>α</sup>, and τ < 1/4γ <sup>δ</sup> .

If there is no interaction between high-frequency and lowfrequency oscillations, the system will be reduced to two coupled oscillators in the δ and α bands separately, very similar to that of Eq. 11. In this case, anti-phase synchronization cannot be realized for the slow oscillations with τ < 1/4γ <sup>δ</sup> . Therefore, if the anti-correlation pattern organized in low-frequency band is induced by small time-delays, there must be an interaction between high-frequency and low-frequency oscillations.

We use an example in the following to demonstrate this mechanism, where we take the interaction terms as

$$I\_{\delta\_{1,2}} = K \sin \left( \delta\_{2,1}(t - \tau) - \delta\_{1,2}(t) \right) \cos \left( \alpha\_{2,1}(t - \tau) - \alpha\_{1,2}(t) \right), \tag{16}$$

$$I\_{\alpha\_{1,2}} = K \sin \left( \alpha\_{2,1}(t - \tau) - \alpha\_{1,2}(t) \right) \cos \left( \delta\_{2,1}(t - \tau) - \delta\_{1,2}(t) \right),$$

with positive coupling strength *K* > 0. This is one of the simplest cases where the interaction between high-frequency [cos(α2,1(*t* − τ ) − α1,2(*t*))] has influence on the interaction between low-frequency oscillations [sin(δ2,1(*t* − τ ) − δ1,2(*t*))], and vise versa. Using new variables *u* = δ + α and *v* = δ − α, we can separate the system into two pairs of coupled phase oscillators with frequencies γ <sup>δ</sup> + γ <sup>α</sup> and γ <sup>δ</sup> − γ <sup>α</sup>, respectively,

$$i\mu\_{1,2}(t) = 2\pi \left(\wp\_{\mathbb{R}} + \wp\_{\mathbb{a}}\right) + K \sin\left(\mu\_{2,1}(t-\tau) - \mu\_{1,2}(t)\right), \qquad (18)$$

$$\dot{\nu}\_{1,2}\left(t\right) = 2\pi \left(\wp\_{\delta} - \wp\_{a}\right) + K \sin\left(\wp\_{2,1}(t-\tau) - \wp\_{1,2}(t)\right). \tag{19}$$

Now the conditions for in-phase and anti-phase synchronization of these new oscillators become clear. Since δ = (*u* + *v*)/2 and α = (*u* − *v*)/2, we can obtain the phase differences of the original fast oscillations α1,2 and slow oscillations δ1,2. We can identify several important values of the delay τ . The results are summarized in **Table 1**. All of τ 0,1,2,3,4 have the same order of 1/γ <sup>α</sup>, and are smaller than 1/4γ <sup>δ</sup> when γ <sup>δ</sup> is much smaller than γ <sup>α</sup>. Notably, there is a region (τ 2, τ 3), the slow oscillations δ<sup>1</sup> and δ<sup>2</sup> can show antiphase oscillation. Examples of numerical simulation are shown in **Figure 6**.

In the real cortex system, the interactions between high and low-frequency oscillations will not be as simple as the model shown in Eqs. (16) and (17). Furthermore, some of the anti-phase patterns of cortical dynamics are investigated by using indirect methods, e.g., fMRI studies (Fox et al., 2005), where the underlying mechanism of slow oscillation is quite complicated and

#### **Table 1 | Phase difference of** *α***1,2 and** *δ***1,2.**


τ *<sup>0</sup>* = *0;* τ *<sup>1</sup>* = *1/4(*γ <sup>δ</sup> + γ <sup>α</sup> *);* τ *<sup>2</sup>* = *1/4(*γ <sup>δ</sup> − γ <sup>α</sup> *);* τ *<sup>3</sup>* = *3/4(*γ <sup>δ</sup> + γ <sup>α</sup> *);* τ *<sup>4</sup>* = *3/4(*γ <sup>δ</sup> + γ <sup>α</sup> *).*

is not yet clearly understood. Nevertheless, this part of discussion provides an understanding of principle that it is possible to obtain anti-phase pattern in very slow-frequency band with a relative smaller delay time if there is the interaction between high and low frequencies oscillations. Moreover, in the presence of these interactions, more types of patterns can emerge in addition to in-phase and anti-phase oscillations, e.g., the pattern of π/2 phase difference (see **Table 1**; **Figure 6**). It is therefore expected that more complex pattern formations can organize in the time delay environments when the interactions among different frequency bands are entangled in a complicated manner. Our research is of fundamental meaning for further understanding the complex pattern formations organized within cortico-cortical communications.

# **4. DISCUSSIONS**

# **4.1. IN-PHASE SYNCHRONIZATION WITHIN MODULES**

When we show the organization of an pattern which is anti-phase between modules and in-phase synchronization within modules, the analysis is made basing on that in-phase has been organized within modules. The dynamics between modules is then similar to

α<sup>1</sup> − α<sup>2</sup> = δ<sup>1</sup> − δ<sup>2</sup> = 0; **(B)** τ = 0.025 ∈ (τ 1, τ 2), α<sup>1</sup> − α<sup>2</sup> = δ<sup>1</sup> − δ<sup>2</sup> = π/2; **(C)** τ = 0.05 ∈ (τ 2, τ 3), α<sup>1</sup> − α<sup>2</sup> = 0, δ<sup>1</sup> − δ<sup>2</sup> = π; **(D)** τ = 0.075 ∈ (τ 3, τ 4),

a pair of coupled neuron cells or coupled phase oscillators (Eq. 11). In other words, in our analysis, in-phase pattern within-module is a precondition for the organization of anti-phase pattern between modules.

However, when two neurons without direct connections are coupled to the third intermediary one, each fires in anti-phase with the third one, the two neurons can achieve in-phase firing. Such a case can be also observed on a neuronal network. As shown in **Figure 7**, the network is divided into two populations; within each population, there is no connection among neurons, but between populations, neurons are randomly coupled with suitable delay times for anti-phase firings. On this network, neuron A and B can achieve in-phase because they have common input from neuron D and both of them are anti-phase to neuron D. The neurons D and E can achieve in-phase because of the common input from neuron B. This mechanism can finally make all the neurons within one population fire in-phase, even for those without a common input, like the neurons A and C. In such a case, anti-phase between the two populations forms a precondition for the organization of in-phase pattern within each population in the presence of delay times. This case is therefore different from what we have analyzed where the connection density is usually higher within modules than between modules. In a real neuronal network, the two populations presented in **Figure 7** will not be defined as a module in term of connectivity. Including interaction within such a population does not necessarily enhance in-phase synchronization, so that if there exist interactions within populations, especially when the delay time and coupling type are the same as those between the populations, the in-phase pattern could be destroyed.

Generally speaking, the mechanism, where anti-phase between populations is precondition for in-phase within populations, is more appropriate to describe the case where two populations are indirectly coupled by the third intermediary one. This situation is perhaps relevant for understanding in-phase synchronization between distant cortical regions (Vicente et al., 2008); the mechanism, where in-phase within modules is precondition

**FIGURE 7 | A network structure where in-phase pattern within population can be induced by anti-phase between populations when there is no interaction between neurons within the populations.**

α<sup>1</sup> − α<sup>2</sup> = − π/2, δ<sup>1</sup> − δ<sup>2</sup> = π/2.

for anti-phase between modules, is more relevant for understanding the organization of anti-phase pattern between two coupled cortical regions.

### **4.2. UNDER TASKS**

We have analyzed important factors in the spontaneous organization of anti-phase patterns without external stimuli. Actually, this kind of anti-phase pattern can also be organized under tasks. From the traditional viewpoint, the task-induced anti-phase pattern often referrs the coexistence of both the areas of task-increased activity and task-decreased activity (Broyd et al., 2009). Our analysis can also shed some lights on this task-induced anti-phase pattern phenomenon.

If we suppose that one of the modules is forced to be inphase synchronization by external signal, rather than spontaneous organized, a suitable delay time may also be a key factor in inducing an anti-phase pattern between modules. We give a simple example in **Figure 8**. A number (*N* = 1000) of IAF neurons are coupled together with excitatory synapses. In the absence of external input, these neurons fire randomly, with a set of parameters *kin* = *kout* = 0.05 and τ *in* = τ *out* = 100 ms. We add an external input to the first module (1→*N*/2), to force the neurons in this modules fire nearly simultaneously every 200 ms, mimicking a firing pattern induced by some tasks. Since the delay time τ *out* = 100 ms, after 100 ms, the membrane potentials of the neurons in the second module are increased, so that a large fraction of them fire simultaneously at that time. Even though the neurons in the second module still show a relatively random firing pattern, anti-phase oscillation can be observed between modules. However, if the delay time does not approximate to half of the period of the external input, such an anti-phase pattern cannot be observed.

This simple mathematical mechanism can help us understand some observations in neuron systems. Previous studies showed that approximate in-phase patterns (or long-distance spatial coherence) can be observed in slow-wave sleeping states, anesthetized states, or under tasks. Anti-phase patterns are also

found in these states. Regardless whether the in-phase pattern in each module is self-organized (e.g., resting state) or induced by external signals (under some tasks), the in-phase patterns within cortex make the cortico-cortical communication dynamics much similar to the time-delayed coupled phase oscillators, showing anti-phase pattern with suitable delay times. There is a precondition that the in-phase synchronization is achieved within modules, which may bridge the understanding of the common characteristics of cortical dynamics in slow-wave sleeping states, anesthetized states, or under tasks.

# **4.3. DIVERSITY IN NETWORK CONNECTIVITY AND DELAYS**

All the above analysis is made basing on the conditions that the delay times and connection densities are uniform within each module or between modules. However, in the real cortical networks, none of them is uniform. In the following, we discuss some results on network with diverse connectivity and delays, and show that delay time is still a key factor but the connected density is not as important in organizing in-phase and anti-phase patterns.

We use the cortico-cortical network among the visual areas of monkey (Kötter, 2004; Kaiser and Hilgetag, 2006). Basing on network connectivity, this network can be put into two modules (da F Costa et al., 2007), and the connection densities are much higher within each module than that between modules. The spatial position of the mass-center of these cortical areas (network nodes) are shown in **Figure 9A**, where the black and red colors represent different modules. We simulate the dynamics of the network by putting a neural mass oscillator on each node. The delay times are introduced by a conduction velocity assumed to be common for all the links. Therefore, the delay times are non-uniform, proportional to the Euclidean distances between different pairs of areas. In the simulations we cannot find anti-phase synchronization

**FIGURE 9 | Dynamical pattern on the cortical network of Macaque visual system. (A)** The spatial positions of the network nodes. **(B)** The distributions of the delay time τ when the conduction velocity is set as a biological reasonable value 2.4 mm/ms. **(C)** The average values of the output *v <sup>j</sup>* <sup>1</sup> <sup>−</sup> *<sup>v</sup><sup>j</sup>* <sup>2</sup> within each modules.

between the two modules. Even if we start with an anti-phase pattern with some noise perturbation as the initial conditions, this pattern cannot be preserved on this network. For very large velocity, the delay times approximate to zero, and the whole network can preserve a highly coherent (in-phase) state. On the other hand, when the velocity is in the biologically reasonable range, the delay times have a broad distribution from zero to a value comparable to the period of the neural mass oscillator (**Figure 9B**), the phase differences among oscillators distribute randomly in (−π, π). Even though the average values of the output from each module show some small macroscopic oscillation, they do not show anti-phase pattern between the two modules (**Figure 9C**). These results confirm again that the formation of densely connected modules cannot induce an anti-phase pattern between them. In the real visual cortex, cells tend to fire simultaneously when activated by related features of a visual stimulus (Gray et al., 1989; Engel et al., 1991a; Castelo-Branco et al., 2000; Tiesinga et al., 2008), though this network has two modules.

# **ANTI-PHASE INDUCED BY INHIBITORY COUPLING**

The above analysis is basing on the case that the coupling between modules is dominantly excitatory. If the coupling is inhibitory, the results of coupled neural mass oscillators can be roughly predicted from Eq. 11, where a negative *K* stands for a negative coupling, so that we can expect that anti-phase pattern emerges when the delay time τ is around zero. Our simulations of a pair of coupled neuron mass oscillators confirmed that, anti-phase pattern emerges with zero τ , whereas in-phase pattern emerges when the delay is close to half of the oscillation period.

In neuronal systems, the inhibitory neurons and the nonsymmetry of the inhibitory connections can induce pronounced competition of activity among neuron pools. Competition dynamics broadly exists in neuronal system (Laurent et al., 2001; Levi et al., 2005; Mazor and Laurent, 2005; Moldakarimov et al., 2005; Rollenhagen and Olson, 2005; Komarov et al., 2009a; Szücs et al.,2009),which may play important roles in neuronal functions, e.g., the integration between low-frequency and high-frequency oscillations (Rabinovich et al., 2006). This type of dynamics also has a relationship to anti-phase pattern from a generalized viewpoint. For example, in the well-known *n*-competitor neuronal system (Komarov et al., 2009a,b), each population activity may achieve its peak value sequently. If in this case, *n* can be degenerated to 2, the dynamics degenerates to the anti-phase oscillation pattern. Therefore, it is significant to get insight into

# **REFERENCES**


Synchronization in complex networks. *Phys. Rep.* 469, 93–153.


the organization of anti-phase pattern for better understanding the organization of more complicated competition dynamics in the brain.

To summarize this part of discussion, when the coupling is inhibitory between modules, delay time can also be a key factor in the organization of anti-phase oscillation. The difference from the case of excitatory coupling is that anti-phase between modules require a small delay (or close to a period) between modules.

# **CONCLUSION**

We investigate the anti-phase oscillation pattern organized on twomodule networks with both neuron cell model and neuron mass model in time delay environments, among other factors. The timedelays and the coupling types (excitatory and inhibitory) can be key factors for organizing the patterns of in-phase within modules and anti-phase between modules, but the connection densities are not as crucial though the stability is influenced by the network connectivity. Our analysis shows that important understanding of the anti-phase synchronization in neural networks can be obtained by the classical coupled phase oscillators. The anti-phase patterns organized in real cortical networks are more complicated. Furthermore, some of them are investigated by using indirect methods, e.g., fMRI studies. It is possible that anti-phase pattern in the signals of these indirect measurements can reflect some kinds of anti-phase pattern in cortical dynamics. However, it is still an open problem how they are related. Our analysis in this work provides a clear understanding of the key factors in the organization of the anti-phase pattern on a two-module neuronal network, which is of importance for gaining insight into the mechanisms underlying the dynamics, no matter how this anti-phase pattern is expressed in the measurement signals. These results are significant for further understanding the formation of more complex spatiotemporal patterns and functional networks in the brain, and the bridge between the anti-phase patten in the measurement signals and the anti-phase pattern organized on the neuronal network need to be meticulously constructed in the further studies.

# **ACKNOWLEDGMENTS**

The work described in this paper was supported by Hong Kong Baptist University (FRG2/10-11/004) and the Research Grants Council of Hong Kong (HKBU 202710), and conducted using the resources of the High Performance Cluster Computing Centre, Hong Kong Baptist University, which receives funding from Research Grant Council, University Grant Committee of the HKSAR, and Hong Kong Baptist University.

long-range excitation. *Phys. Rev. Lett.* 99, 238106.


mode network during deep sleep. *Proc. Natl. Acad. Sci. U.S.A.* 106, 11376.


biological oscillators. *SIAM J. Appl. Math.* 50, 1645–1662.


of spike timing in visual cortical circuits. *Nat. Rev. Neurosci.* 9, 97.


distribution in mutual synchronization of neural networks. *Phys. Rev. E Stat. Nonlin. Soft Matter Phys.* 74, 041915.


relationship in complex brain networks expressed by hierarchical synchronization. *New J. Phys.* 9, 178.

**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 14 September 2011; accepted: 19 November 2011; published online: 07 December 2011.*

*Citation: Li D and Zhou C (2011) Organization of anti-phase synchronization pattern in neural networks: what are the key factors? Front. Syst. Neurosci. 5:100. doi: 10.3389/fnsys.2011.00100*

*Copyright © 2011 Li and Zhou. This is an open-access article distributed under the terms of the Creative Commons Attribution Non Commercial License, which permits non-commercial use, distribution, and reproduction in other forums, provided the original authors and source are credited.*

#### **APPENDIX**

# **ANALYSIS OF PHASE DIFFERENCE IN PULSE-COUPLED NEURONAL NETWORK**

The phase of an IAF cell can be defined as:

$$
\varphi^j(t) = 2\pi \frac{t - t\_0^j}{T},\tag{A1}
$$

where *t j* <sup>0</sup> is the time when the *j*th neuron fires for the first time. The potential *V* then have the properties *V*(ϕ = 0) =*Vr*, *<sup>V</sup>*(<sup>ϕ</sup> <sup>=</sup> <sup>2</sup>π) <sup>=</sup>*Vth*, *dV/d*ϕ > 0, and *<sup>d</sup>2V/d*ϕ*<sup>2</sup>* <sup>&</sup>lt; 0. For two coupled neurons,when a neuron receives an action potential from the other one, its phase will has an increment ϕ˜,

$$
\tilde{\varphi} = \lg(\varphi\_{1,2}) - \varphi\_{1,2},
\tag{A2}
$$

where the function *g* (ϕ) is

$$\text{gg } (\varphi) = V^{-1} \left( \min \left[ V(\varphi) + \tilde{K}, V\_{th} \right] \right), \tag{A3}$$

and *V* <sup>−</sup><sup>1</sup> is the inverse function of *V*(ϕ). The phase difference of these two neurons ϕ = ϕ<sup>1</sup> − ϕ<sup>2</sup> therefore follows

$$\frac{d\Delta\varphi}{dt} = \begin{cases} g(2\pi\gamma\,\tau + \Delta\varphi) - 2\pi\gamma\,\tau - \Delta\varphi, & \varphi\_2 = 2\pi\gamma\,\tau \\ -g(2\pi\gamma\,\tau - \Delta\varphi) + 2\pi\gamma\,\tau - \Delta\varphi, & \varphi\_1 = 2\pi\gamma\,\tau \\ 0, & \text{otherwise} \end{cases} \tag{A4}$$

It means that only when ϕ<sup>2</sup> = 2πγτ or ϕ<sup>1</sup> = 2πγτ , its value is non-zero. In the following, we assume it has non-zero values all the time, which means that the coupling effect always takes place. We can write this evolution of ϕ in the following form:

$$\frac{d\Delta\varphi}{dt} = \text{g}\left(2\pi\gamma\tau + \Delta\varphi\right) - 2\pi\gamma\tau - \Delta\varphi - \text{g}\left(2\pi\gamma\tau - \Delta\varphi\right)$$

$$+ 2\pi\gamma\tau - \Delta\varphi. \tag{A5}$$

This situation describes such a system that the neuron system is divided into two populations; within each population, nerve cells are coupled together so that they fire orderly; and between these populations, the phase difference between the paired nerve cells corresponds to the phase difference of the oscillations of the two populations. In this case, the evolution of ϕ can be written as

$$\frac{d\Delta\varphi}{dt} \approx \begin{cases} 2\mathbf{g}'\left[\left(2\pi\boldsymbol{\chi}\tau\right) - 1\right] \Delta\varphi, & \Delta\varphi \sim 0\\ 2\mathbf{g}'\left[\left(2\pi\boldsymbol{\chi}\tau + \pi\right)\right) - 1\left(\Delta\varphi - \pi\right), & \Delta\varphi \sim \pi \end{cases}, \quad \text{(A6)}$$

where *g* (ϕ) = *dg* (ϕ)/*d*ϕ.We therefore get*G*(2πγτ ) = 2*g* [(2πγτ ) − 1], so that one can understand the role of delay time τ in coupled neuronal systems as in the coupled phase oscillators.

In the following we will show that the expression of *g* (ϕ) in Eq. A2 can generate a similar performance of G(2πγτ ) versus τ to the case in coupled phase oscillators when we consider another factor existing in real neuronal systems, the refractory period.

The properties *dV*/*d*ϕ > 0 and *d*2*V*/*d*ϕ<sup>2</sup> < 0 (**Figure A1A**) are the basisfor the analysis in thefollowing.When the neuron receives

time τ in the stability of in-phase pattern, the value of *G*(2πγτ ) is vital; when referring to anti-phase pattern the value of *G*(2πγτ + π) is vital. **(A)** In IAF oscillator V(ϕ) versus ϕ; **(B)** *min*[*V* (ϕ) + *K*˜ , *Vth* ] versus ϕ; **(C)** *g*(ϕ) versus ϕ; **(D)** 2[*g* (ϕ) − 1] versus ϕ; **(E)** 2[*g* (ϕ) − 1] versus ϕ when refractory period is defined (the same form as **Figure 4B**); **(F)** in coupled phase oscillators, −2*K* cosφ versus φ when *K* > 0 (the same form as **Figure 4A**).

a spike, if *V*(ϕ) increases to *V*(ϕ) + *K*˜ , we call that the neuron is located in the integration period; if *V*(ϕ) increases to *Vth*, it is located in the firing period, as shown in **Figure A1B**. Therefore, we get

$$\text{gg } (\varphi) = \begin{cases} V^{-1} \left[ V \left( \varphi \right) + \tilde{K} \right], & \text{integration period} \\ 2\pi, & \text{firing period} \end{cases}, \tag{A7}$$

as shown in **Figure A1C**. Since *K*˜ > 0, we get

$$\mathbf{g}^{\prime}(\varphi) \begin{cases} > 1, & \text{integration period} \\ = 0, & \text{firing period} \end{cases},\tag{A8}$$

as shown in **Figure A1D**. On the other hand, if the refractory period is taken into consideration, *V*(ϕ) + *K*˜ = *Vr* is a constant during this period, so that *g* (ϕ) = 0 and *g* (ϕ) = 0. We can therefore express 2[*g* (ϕ) − 1] as

$$2\left[\mathcal{g}'\left(\varphi\right)-1\right]\begin{cases}<0, & \text{refractor}\varphi\text{ period} \\ >0, & \text{integration period} \\ <0, & \text{fring period} \end{cases},\tag{A9}$$

as shown in **FigureA1E**. These three periods are similar to the three cases labeled as I, II, and III in **Figure A1F** for coupled phase oscillators with *K* > 0. The above analysis can therefore explain the relationship of the organizations of anti-phase pattern between coupled neuronal systems and coupled phase oscillators, so as to understand the key role of the delay time.

# Using large-scale neural models to interpret connectivity measures of cortico-cortical dynamics at millisecond temporal resolution

# *Arpan Banerjee\*, Ajay S. Pillai and Barry Horwitz*

*Brain Imaging and Modeling Section, National Institute on Deafness and Other Communication Disorders, National Institutes of Health (NIH), Bethesda, MD, USA*

#### *Edited by:*

*Gustavo Deco, Universitat Pompeu Fabra, Spain*

#### *Reviewed by:*

*Gustavo Deco, Universitat Pompeu Fabra, Spain Gregor Rainer, University of Fribourg, Switzerland*

#### *\*Correspondence:*

*Arpan Banerjee, Brain Imaging and Modeling Section, National Institute on Deafness and Other Communication Disorders, National Institutes of Health (NIH), 10 Center Drive, Room 5D39, Bethesda, MD 20892, USA. e-mail: Arpan.Banerjee@nih.gov*

Over the last two decades numerous functional imaging studies have shown that higher order cognitive functions are crucially dependent on the formation of distributed, large-scale neuronal assemblies (neurocognitive networks), often for very short durations. This has fueled the development of a vast number of functional connectivity measures that attempt to capture the spatiotemporal evolution of neurocognitive networks. Unfortunately, interpreting the neural basis of goal directed behavior using connectivity measures on neuroimaging data are highly dependent on the assumptions underlying the development of the measure, the nature of the task, and the modality of the neuroimaging technique that was used. This paper has two main purposes. The first is to provide an overview of some of the different measures of functional/effective connectivity that deal with high temporal resolution neuroimaging data. We will include some results that come from a recent approach that we have developed to identify the formation and extinction of task-specific, large-scale neuronal assemblies from electrophysiological recordings at a ms-by-ms temporal resolution. The second purpose of this paper is to indicate how to partially validate the interpretations drawn from this (or any other) connectivity technique by using simulated data from large-scale, neurobiologically realistic models. Specifically, we applied our recently developed method to realistic simulations of MEG data during a delayed match-to-sample (DMS) task condition and a passive viewing of stimuli condition using a large-scale neural model of the ventral visual processing pathway. Simulated MEG data using simple head models were generated from sources placed in V1, V4, IT, and prefrontal cortex (PFC) for the passive viewing condition. The results show how closely the conclusions obtained from the functional connectivity method match with what actually occurred at the neuronal network level.

**Keywords: EEG, MEG, large-scale networks, information processing, high resolution, timing, decoding, delayed match-to-sample (DMS)**

# **INTRODUCTION**

We try to relate present realizations with prior experiences in numerous daily life activities. For example, imagine a situation where you encounter a person during a train ride who appears to be vaguely familiar. Almost instinctively, you try to remember this person from the near or distant past. Within a short time, maybe in a few seconds, you come to a decision by matching the current face against a huge sample of faces you have known as to whether this is somebody you were familiar with from the past. Outcomes of such a matching endeavor may further decide whether you are going to start an informal social interaction with the person. Such seemingly simple tasks engage a complicated set of information processing stages in the brain. Starting with sensory processing of the facial and bodily features of the person, the human brain performs face and object recognition, recollection of short and long-term memories, and finally decision-making. All this may occur with varying levels of attention in each occurrence of the event across individuals. Brain dynamics underlying these processing stages operate at a millisecond scale to ensure that the final outcome (decision to interact) occurs as soon as possible, most likely within seconds. Existing research in sensory visual processing (Prechtl et al., 1997; Horwitz and Braun, 2004), face and object recognition (Haxby et al., 1991, 1995; DiCarlo and Cox, 2007), long-term memory retrieval (Frankland and Bontempi, 2005; Smith et al., 2010a) and decision-making (Siegel et al., 2011) suggests that large-scale brain networks are involved in each stage of neural information processing (Atkinson and Shiffrin, 1968). Hence, understanding goal directed behavior in humans will require characterization of network mechanisms at varying spatiotemporal scales from neuroimaging data recorded at high temporal resolution.

The idea of studying networks as substrates of higher order cognition has gained relatively recent popularity. Human brain mapping have traditionally relied on functional segregation studies (Felleman and van Essen, 1991) based on the fact that cortical areas get preferentially connected in terms of their functional relevance. Functional integration (Luria, 1980), the mechanism that refers to integration of information processed in distributed brain structures, can be studied with modern neuroimaging techniques. Based on these two mechanisms that govern cognitive function, it can be hypothesized that neurocognitive networks form coordinated configurations which vary in size and temporal characteristics (Kelso, 1995; Bressler and Kelso, 2001; McIntosh, 2004; Horwitz, 2005). Identifying the task-specific variations in size and temporal characteristics of large-scale neurocognitive networks is highly non-trivial. Overlapping of the network components, also known as neural degeneracy (Tononi et al., 1999), presents a significant challenge for neuroelectromagnetic data analysis and interpretation as well as quantification of spatiotemporal network boundaries. In order to unravel the temporal structure of a functionally relevant brain network, two vital questions need to be addressed simultaneously: (1) How can low dimensional functional brain networks be defined from high dimensional electromagnetic recordings? (2) How can the time scales of modulatory and compensatory network processing mechanisms be interpreted from neurodynamics? Answering these questions on a subject-by-subject basis will help us understand the neural basis of several higher order cognitive tasks.

The purpose of this article is two-fold. First, in the Overview section we review a set of existing methods that aids in inferring the existence of large-scale networks from imaging data at high temporal resolution. We also present some existing modeling approaches to simulate the dynamics of large-scale networks and indicate their importance for testing novel connectivity analysis methods. Second, we review one novel method in detail that detects the time scales of formation and extinction of large-scale neurocognitive networks from EEG/MEG data. We applied this method to simulated MEG data from a biophysically realistic large-scale neural model of a delayed match-to-sample (DMS) task to partially validate the time scales of network level processing. Only with simulated data does one have the knowledge of ground truth, and hence data from large-scale neural models encompassing neural assemblies in both hemispheres provide an ideal platform to test the efficacy and validity of novel methods that are being developed to interpret the presence of network mechanisms.

# **OVERVIEW OF METHODS TO ASSESS LARGE-SCALE NETWORK MECHANISMS**

Identifying network substrates of higher order cognition poses a number of significant challenges. The number of nodes in a functional network and their connections can undergo reorganization within a few milliseconds. At certain instants of time, new areas may get recruited while subsisting areas disengage. Added to this is the fact that the total number of possible functional configurations that might exist within a large-scale network is itself a large number. These configurations may result from the different temporal relationships each node has on the others and are defined as functional connectivity during a specific task (Friston et al., 1993b). Alternatively, a large-scale network may need to recruit additional brain areas for task-specific information processing (Meredith and Stein, 1983; Damasio, 1989; Calvert and Thesen, 2004). Understanding this orchestra of recruitment-modulation mechanisms can be formulated as data-driven frameworks for elucidating neuronal processes (Banerjee et al., 2008; Smith et al., 2010b). There are several ways to explore the spatiotemporal features of network evolution which gives us a deeper understanding of the circuit mechanisms underlying ongoing behavior and emerging brain states. In this section we classify some of the popular concepts which are applied to neuroimaging data at high temporal resolution. The first two sections are based on indirect measurements of network function and the last two are devoted to direct measures of extracting networks.

# **REDUCTION OF HIGH DIMENSIONAL DATA TO A LOW DIMENSIONAL SUBSPACE**

Motivated by empirical observation, it can be argued that dynamics of brain network activations following an externally presented stimulus becomes low dimensional. This simply means that a significant proportion of data can be captured by the dynamics of a few patterns using spatiotemporal mode decomposition techniques, such as principal component analysis (Friston et al., 1993a; McIntosh et al., 1996; Kelso et al., 1998), independent component analysis (ICA) (Bell and Sejnowski, 1995; Makeig et al., 1997; Onton et al., 2006; Kovacevic and McIntosh, 2007) etc. Hence, the goal of any dimensional reduction analyses is to explain the maximum possible variance in the data with the minimum number of modes (spatial patterns) and corresponding temporal projections. The spatial patterns can be interpreted as signatures of large-scale networks that constitute the substrate on which information processing occurs. Extracting such spatial patterns by combining temporal evolution of the corresponding behavioral task and brain recordings in a covariance matrix, allows one to study the relationship between behavior and brain signals at the network level (McIntosh et al., 1996). One common method for extracting such spatial patterns is principle components analysis (PCA). PCA (also known as singular value decomposition and Karhunen-Loève transform) involves a high dimensional rotation of the covariance matrix to rank the orthogonal dimensions (components) in the data in order of decreasing variance explained by each dimension. Hence, data distributions structured as a high dimensional ellipsoid rather than a sphere is suitable for PCA analysis. Ranking dimensions in order of variance becomes ambiguous for a spherically symmetric distribution (Jollife, 2002). Higher order statistics, e.g., kurtosis and skewness is ignored by PCA and thus any sharp or abrupt changes in variability wouldn't be extracted as an independent feature.

The orthogonality of principle components (PCs) is both an advantage and disadvantage of PCA. Orthogonal PCs provide an intuitive backdrop of data visualization. Choosing the number of PCs to use for dimensional reduction can follow quantitative approaches (Mitra and Pesaran, 1999) rather than handpicking them based on *a priori* hypothesis of brain areas involved. On the other hand, there is no biophysical motivation as to why patterns of activity spanned by orthogonal PCs have to be functionally relevant. This is the single biggest disadvantage of the PCA method. However, it is important to note that attributing functional significance to individual principal components is where the main disadvantage of PCA is apparent. Thus, PCA is more suited to identify the most general features in the signal, characterize the overall subspace of neural activity and disambiguate it from the noise space.

The success of relatively simpler approaches like PCA has led to the development of more complicated dimension reduction techniques based on explaining higher order statistics in the data (variance is only second-order). Commonly known as ICA (Bell and Sejnowski, 1995) these techniques come in various flavors (Makeig et al., 1997; Onton et al., 2006). Typically ICA is based on the assumption that the true underlying generators of the data are statistically independent (but not necessarily orthogonal). Hence, ICA maps do not have any preference or temporal order in which they appear in the neural time series. However, a "second-order blind identification" (SOBI) approach (Molgedey and Schuster, 1994) considers relationships between multiple time points using an autoregressive model in which sources are assumed to have both differing spatial distributions and stable power spectra. ICA algorithms can be used as a quantitative technique to reject physiological artifacts in the neural data (Delorme et al., 2007), classify electromagnetic brain activity from different population groups (Kovacevic and McIntosh, 2007) and characterize task-specific network activations from whole brain recordings (Makeig et al., 1999). The temporal structure of task-related network dynamics can also be studied using ICA (Hong et al., 2005; Grau et al., 2007). However, the challenge here lies in quantifying the right set of independent components (ICs) that capture the global properties of the network over a certain period of time. The major disadvantage of ICA is that a subset of relevant ICs for explaining a particular function is often chosen based on *a priori* hypothesis on the role of certain brain structures onto the task rather than a quantitative scheme.

There are other options for dimensional reduction methods that address some of the disadvantages listed here, but not yet applied consistently to EEG/MEG data analysis. Notable among these are factor analysis (Everitt, 1984), probabilistic PCA (Tipping and Bishop, 1999), locally linear embedding (Roweis and Saul, 2000). In summary, there exist several dimensional reduction techniques for developing ways to compare spatiotemporal network mechanisms underlying two tasks and identification of the onsets and offsets of task-specific information processing. Combining this with behavioral measures of task performance can give us a comprehensive understanding of the network mechanisms involved during higher order cognition.

#### **SPECTRAL REPRESENTATIONS OF RAW EEG/MEG DATA**

Spectral decomposition of neurophysiological time series has now become an essential pre-requisite for analysis of multivariate EEG/MEG signals. In 1924, Hans Berger observed that the most dominant contribution in EEG signals came from 8 to 12 Hz oscillations during wakeful relaxation with closed eyes (Niedermeyer, 1997). He named these "alpha" rhythms and correspondingly the smaller amplitude faster waves (16–25 Hz) during awake state with eyes open were denoted as "beta." Originating in the occipital areas, alpha rhythms have been proposed to reflect the electrical activity of large-scale networks incorporating neocortical visual areas and thalamus that have strong bi-directional excitatory connections (Llinas and Pare, 1991). "Beta" rhythms have been related to synchronized firing activity of long distant brain networks (such as those comprising frontal and parietal areas) during a perceptual or cognitive task (Brovelli et al., 2004). Finally, the "gamma" oscillations (*>*30 Hz) initially related to processing of complex visual stimuli in pyramidal cell layers of the visual cortex (Gray et al., 1989) are now considered as a crucial ingredient for consciousness (Crick and Koch, 2003), perceptual binding (Rodriguez et al., 1999) and other higher order cognitive functions (Buzsáki, 2006).

However, for EEG/MEG time series recorded over a considerable period of time with high temporal resolution, power spectra may not show clear peaks and indeed such spectra follow a 1*/freq* distribution (pink noise). This scale free nature of brain dynamics has been termed self-organized criticality (Kelso, 1995; Linkenkaer-Hansen et al., 2001; Stam and de Bruin, 2004). Theoretically, the scale free nature of EEG power spectra can be viewed as counterintuitive to the notion of orderliness in EEG signals explained by dominance of one or two frequency bands, but taken together these two notions offer an attractive explanation for empirically observed brain dynamics, such as spontaneous transitions of global modes of oscillation in the absence of external input (for more detailed discussions see Kelso, 1995; Buzsáki, 2006). In this theoretical framework the ordered brain dynamics constitutes a transiently stable behavior resulting from the presence of a sensory input or task-processing on top of the perpetually critical state on which the brain sits. The spatial localization of the power spectra reveals interesting features of such transient stability. For example, the "mu" rhythm (9– 11 Hz), which seems to originate from network activity involving thalamus and motor cortex, disappears when there is movement or movement intention (Steriade and Llinas, 1988). "Phi" rhythm, localized over parietal regions and comprising two subcomponents; Phi1(10–12 Hz) and Phi2(12 Hz) are enhanced during social interaction. Phi1 during independent and Phi2 during coordinated behavior respectively (Tognoli et al., 2007). We will return to the scale-free nature of brain dynamics and try to relate it with underlying anatomical connections in a latter section. However, the key concept here is that identifying the large-scale network underlying task-related processing may lie in characterizing the spatial and temporal structure of transiently stable brain oscillations.

To understand the temporal dynamics of spectral power, timefrequency spectrograms can be computed. Primarily, this involves windowing the raw time series in small time windows over which the power spectrum is computed. To obtain a continuously varying spectrogram (power spectrum across frequency and time) wavelet (Grossman and Morlet, 1984; Antonini et al., 1992), and multitaper (Thompson, 1982) analyses techniques have been employed. One striking result obtained from the wavelet analysis was the discovery of spontaneous bursts of EEG/MEG activity in "gamma" band when spectrograms were computed over single trials (induced power) but not when they were computed from the trial averaged evoked potential time series (Tallon-Baudry et al., 1998). In a recent review, different spectral profiles have been related to different cognitive functions (Donner and Siegel, 2011). According to this hypothesis, encoding functions (such as the encoding of sensory features or motor plans) involving local computations will be reflected in gamma oscillations, whereas integrative functions (such as perceptual inference and decision-making) involving long-range interactions among distant brain regions will be reflected in beta band modulation. Time-frequency spectrograms (Percival and Walden, 1993) also allow the development of effective connectivity measures that establish the presence of networks of brain regions which communicate during the task-specific information processing. We will continue this discussion briefly in the next section. However, it is important to note that very few methods exist that quantitatively capture the onset times of spectral changes (Bokil et al., 2006). How can the onset and offset times of transiently stable large-scale oscillatory networks be computed at millisecond resolution is an open question that will be the target of future research.

# **APPLYING DIRECT MEASURES OF CONNECTIVITY ONTO FUNCTIONAL IMAGING DATA**

Functional integration (Luria, 1980) is the cortical organizational principle by which two or more brain areas can simultaneously engage in information processing via modulations of their interactions. Integration of information across distributed brain areas is required by higher order cognitive tasks (Luria, 1980; Horwitz, 1989, 2003; Horwitz et al., 1992; Sporns et al., 2004; Bressler and Tognoli, 2006; Friston, 2009). Hence, functional integration as opposed to the mechanism of functional segregation (Penfield and Erickson, 1941) operates over a larger spatial scale. Non-invasive neuroimaging techniques EEG/MEG/fMRI are ideal tools to study functional integration because they can record from multiple brain areas simultaneously. One way to measure functional integration would entail computation of statistical relationships between brain activations across different brain areas. Comparing the strengths of such connections (functional connectivity) across the entire brain will reveal the functional brain network underlying a specific task (Horwitz et al., 1992; Friston, 1994). The spatiotemporal scale of functional integration is an elusive concept (a recurring theme of this review) as it depends on the context of the experimental design (Horwitz, 2003; Kim and Horwitz, 2008). Thus a more restricted measure of functional integration—"effective connectivity"—is widely used (Friston, 1994; McIntosh et al., 1996; Horwitz et al., 1995; Horwitz, 2003; Sporns et al., 2004). Effective connectivity essentially constrains the more general functional connectivity onto a network with specified anatomical connections (McIntosh et al., 1994) between a relatively small number of nodes based on a model of the behavior being studied (Friston, 1994; McIntosh and Gonzalez-Lima, 1995; Friston et al., 2003).

In the early days of neuroimaging research, functional and effective connectivity concepts were primarily developed on modalities with high spatial resolution but low temporal resolution such as PET and fMRI (Friston et al., 1993b, 2003; Horwitz et al., 1995; Horwitz and Braun, 2004). In principle, functional connectivity changes of the order of a few milliseconds can be measured with EEG and MEG (Breakspear, 2004; Schnitzler and Gross, 2005; Wendling et al., 2009; Stam, 2010). However, results obtained from a connectivity analysis are highly dependent on the algorithms used to quantify a network and task context (Horwitz, 2003).

Perhaps the simplest measure of connectivity can be defined in terms of the correlation between neural signals from different brain areas. If *X(t)* and *Y(t)* represent the simultaneous electromagnetic activity (EEG/MEG) from two sensors at time *t*, correlation *(CXY )* between them is defined as

$$C\_{XY} = \frac{\text{cov}(X, Y)}{\sigma\_{\text{x}}\sigma\_{\text{y}}} \tag{1}$$

where, *σ<sup>x</sup>* and *σ<sup>y</sup>* are standard deviations of *X(t)*, *X(t)* and cov stands for covariance between *X* and *Y* over the time window of recording. Thus, correlation evaluates the degree of statistical interdependency between two time series within a given time window. It is important to note that correlation does not tell us anything about causality. A correlation analysis can also be performed on latencies of neural events across different brain areas (DiCarlo and Maunsell, 2005; Banerjee et al., 2010) or between task variables and parameters of a large-scale neural model (Daunizeau et al., 2009).

One way to understand the intricate relationship between two correlated time series can be to explore how correlated they are at different frequencies. This can be evaluated using coherence (Bendat and Piersol, 1971; Bressler et al., 1993). Coherence *(ρXY )* is defined as

$$\rho\_{XY} = \frac{|\mathcal{S}\_{XY}|^2}{\mathcal{S}\_{XX}\mathcal{S}\_{YY}}\tag{2}$$

where, *SXX* and *SYY* are power spectral densities obtained from the Fourier transforms of *X* and *Y* respectively and *SXY* is the crossspectral density. Significance tests on correlation, coherence, correlation differences and coherence differences can be performed parametrically using Fisher's Z-transforms (Kleinbaum et al., 1998) as well as non-parametric tests (Maris et al., 2007) that involves creating null distributions from shuffled data sets. Coherence quantifies the degree of synchronization in the oscillatory responses from multiple brain areas. Functional brain networks can be extracted by setting thresholds on coherence values from whole brain analysis of EEG and MEG data. A potential problem with pair-wise correlation and coherence measures magnify when dealing with multivariate recordings such as EEG/MEG. Significant correlation between two time series does not overrule the fact that such a relationship may arise from a common input driving two regions, rather than two independent regions working harmoniously over time. To address this issue, both correlation and coherence can be computed invoking multivariate considerations and the corresponding quantities are called partial correlation and coherence (Fisher, 1924). Here, the inter-relationship between two time series can be conditioned on the activations of all other sensors. Several other measures of functional connectivity are used for EEG/MEG data analysis such as nonlinear coherence (Lopes da Silva et al., 1989), phase synchrony (Rodriguez et al., 1999; Pikovsky et al., 2001) and generalized synchronization index (Arnhold et al., 1999). Typically each method is tuned to detect a specific feature of inter-relationships, however, the simpler regression based methods (correlation and coherence) seem to be the most robust in detecting the underlying neural coupling (Wendling et al., 2009). To detect the dynamics of coherence, the method of wavelet coherence have been used (Lachaux et al., 2002). This method holds a lot of promise in detecting the temporal structure of network evolution during a task.

# **GRAPH THEORETIC METHODS FOR ASSESSING NETWORK DYNAMICS**

In previous sections we presented a brief overview of some of the widely used EEG/MEG-based tools to measure the strength of inter-relationships between different brain regions. Here, we briefly present an alternative direct way to construct a functional brain network using graph theoretical tools (Bullmore and Sporns, 2009; Rubinov and Sporns, 2010). A network is characterized essentially by two components: nodes (vertices) and connections (links). Nodes in large-scale networks represent brain regions or sensors whereas connections can be anatomical, functional or effective (Horwitz et al., 1992, 1995; Friston, 1994). EEG/MEG records brain activity from sensors located outside the head and "far away" from the neural sources. Often cortical source dynamics is estimated using inverse techniques (Geselowitz, 1967; Gross et al., 2001; Hillebrand and Barnes, 2005). Hence, graph theoretic tools can be applied to both sensor level (Bassett et al., 2006; Stam, 2010) and source level (Palva et al., 2010) data. An overwhelming number of these studies indicate that the dynamical state of the networks lie near the order/disorder transition point, a phenomenon known as self-organized criticality (Stam and de Bruin, 2004; Bassett et al., 2006; Kitzbichler et al., 2009). Bassett et al. (2006) showed that the highest-frequency gamma network had greater synchronizability, greater clustering of connections, and shorter path length than networks in lower frequencies. Even though the global topology or synchronizability was not strongly influenced by the ongoing behavioral state, motor task performance was associated with emergence of long-range connections in both beta and gamma networks. Since the task was over a long time window, this observation fits within the theoretical framework of self-organized criticality governing the ongoing background activity of the brain upon which the task-specific transiently stable dynamics emerges (Buzsáki, 2006). In the future, metrics with higher temporal resolution such as wavelet coherence (Lachaux et al., 2002) may be able to reveal the time scale of transiently stable oscillations. Hypotheses about spectral markers of cognitive functions (Donner and Siegel, 2011) can be tested using this framework.

In addition to the number of nodes and connections of a network, other measures such as modularity, clustering, path length, and efficiency are often meaningful (Bassett et al., 2009; Bullmore and Sporns, 2009). Such measures seem to be ideally tuned to reveal the spatial structure of a large-scale network. Often complex brain networks are densely connected within a module and have only a few nodes that have connections with different modules. Such modes of network organization can be extracted via the measures of clustering and modularity (Meunier et al., 2010). Recent studies have tried to address the dynamics of task-specific processing in large-scale networks using complexity measures (Palva et al., 2010).

# **TEMPORAL MICROSTRUCTURE OF LARGE-SCALE CORTICAL NETWORKS UNDERLYING TASK-RELATED DIFFERENCES**

The knowledge of time scales of network level processing has been used to identify the sequential steps in task processing via feedforward and feedback processes (Garrido et al., 2007; Liu et al., 2009). In neurophysiological studies on non-human primates, top-down and bottom-up influences on neural information processing during higher order cognitive tasks have been disambiguated using onset time detection (Hanes and Schall, 1996; Monosov et al., 2008; Liu et al., 2009). Nonetheless, there are two major limitations in extending these approaches directly to multivariate EEG/MEG data. First, task-specific network recruitment cannot be interpreted from a "pure insertion" based subtraction of brain activity during control from the task condition, because the possibility of temporal modulation via changes in the strength of functional connections cannot be easily ruled out (Friston et al., 1996). Second, the existing analysis methods are somewhat tuned in an either/or fashion to address the spatial or temporal aspects of network dynamics.

To circumvent some of these issues, we have recently developed a computational framework that decodes the temporal microstructure of spatiotemporal network mechanisms (Banerjee et al., submitted). Here, dimensional reduction techniques are used to define control subspaces from an experimental control dataset (EEG/MEG at the sensor level). Data from an experimental task condition can be reconstructed from their projections onto this control subspace. Banerjee et al. (2008) showed how the goodness of fit of such reconstructions can be used to interpret the underlying spatiotemporal network mechanisms: "temporal modulation" where the task relevant large-scale network is comprised of the network components identified for the baseline control versus "recruitment" where compensatory network involvement is required for specific aspects of task processing. In our current framework (Banerjee et al., submitted)the temporal evolution of the goodness of fit of such reconstructions is used for detection of the time scales of task-specific network recruitment at millisecond resolution. Decoding the temporal microstructure of task-specific large-scale networks based on quantitative definitions is of immense practical importance. One may be able to obtain a network level biomarker for a broad variety of higher order neural processes where information processing occurs in stages. Using our approach, quantitative statistical analysis can be performed on the resulting onset times of network recruitment at an individual subject level. In a subsequent section, we will discuss the details of our computational framework and use it to decode the temporal microstructure of network level processing on simulated MEG data for a DMS task (Atkinson and Shiffrin, 1968). We have earlier applied this analysis for a paired associate long-term memory recall task (Banerjee et al., submitted). There, we were able to decode the time scale (onset time and offset time) of task-specific networks required for retrieving an auditory object (a non-linguistic tonal pattern) paired with a visual stimuli from the long-term memory at an individual subject level. A comparison with visual-visual association indicated that the onset times for long-term memory retrieval networks appeared after an initial period of sensory processing of about 0–250 ms.

# **LARGE-SCALE NEURAL MODELS**

We have briefly reviewed a range of techniques that can be used to estimate the presence and functionality of brain networks (from EEG/MEG data at high temporal resolution) essential for task-specific processing. We also reviewed one such method in detail to illustrate how such methods can be applied to actual experimental data. The regression based techniques such as correlation and coherence have been shown to be the most robust in detecting changes in inter-relationships between functional units (Wendling et al., 2009). However, this is not an isolated result. Using biophysically motivated neural mass models, each method was shown to have different sensitivity profiles for functional connectivity estimation in EEG data (David et al., 2004). This, as well as several other studies (Srinivasan et al., 2007; Deco et al., 2008; Daunizeau et al., 2009; Coombes, 2010), indicate that largescale models can become an important tool to test the validity of signal processing tools that are being applied to decipher functional connectivity in brain networks at high temporal resolution.

An important step in understanding large-scale network mechanisms is to develop an understanding of empirical EEG and MEG observations at the macroscopic level (Kelso, 1995; Horwitz et al., 1999; Buzsáki, 2006). Neural field models of macroscopic EEG/MEG activity have been used primarily to explain the dynamics of spatial patterns that are observed at the sensor level (Wilson and Cowan, 1972; Nunez, 1974; Amari, 1977; Jirsa and Haken, 1996; Robinson et al., 1997; Coombes, 2010). The underlying neural circuit mechanisms producing the spatiotemporal patterns at the sensor level are harder to address. One can only rely on the knowledge gained from electrophysiological recordings in non-human primates. Understanding from animal recordings can then be used to develop biologically realistic simulation frameworks to understand the circuit mechanisms in humans (Tagamets and Horwitz, 1998; Deco et al., 2004, 2008; Horwitz and Husain, 2007). These are large-scale approaches that consider a basic unit of neural processing to consist of excitatory and inhibitory populations of neurons within one cortical column, and the connection topology across several such units shape the overall network dynamics and ultimately control behavior. The mean field approximation is often employed by these approaches (Wilson and Cowan, 1972), which essentially means that overall activity from a population of neurons in one cortical column in a functionally active area such as V1 for processing elementary visual stimuli can be lumped into one variable whose temporal dynamics can be studied using differential equations. This approximation is supported by empirical evidence from electrophysiology (Mountcastle, 1957; Hubel and Wiesel, 1963). Event related potentials (ERPs), or changes in the brain's electromagnetic potential difference (voltage response) following the presentation of a visual/auditory/tactile stimulus recorded from EEG, reflect the collective behavior of neuronal populations combined with spatial filtering through volume conduction. Event related fields (ERFs) on the other hand capture the minute changes in magnetic fields generated from varying intensities of current sources inside the brain from MEG recordings with minimal volume condition. A combination of collective behavior and spatial filtering makes the large changes in the ERPs/ERFs highly correlated across sensors. Mathematically this makes spatiotemporal analysis of the EEG/MEG signals simpler. A high dimensional recording can be captured in terms of the dynamics of few spatial patterns (Friston et al., 1993a; Kelso et al., 1998; Banerjee et al., 2008). Modeling the collective behavior of neural networks giving rise to these patterns is of immense practical importance. They provide a phenomenological understanding of the laws which govern neurobiological elements that produce emergent dynamics (Deco et al., 2004, 2008; Assisi et al., 2005; Stefanescu and Jirsa, 2008) and can be used to validate new techniques that estimate functional connectivity (Banerjee et al., 2008).

A major advantage of using simulated large-scale models of neural dynamics is one can test the effects of different connectivity topologies on task-specific information processing. A large number of studies have shown the role of local and global connectivity in shaping cortical oscillations. For example a simplistic approximation of the feed-forward connectivity between thalamus and visual areas has been used for explaining the generation of alpha rhythms (Lopes da Silva et al., 1974). Whereas, detailed models of excitatory and inhibitory neuronal populations can capture complex phenomena such as multistability (Freyer et al., 2011), and epileptic seizures (Breakspear et al., 2006). Synchronization among distant populations of neurons in the gamma frequency band has been shown to occur in a network of integrate and fire neurons with small world connection topology (Bazhenov et al., 2008). Large-scale models on the role of attention and working memory encoding in prefrontal cortex (PFC) have been highly useful for explaining fMRI and PET brain imaging results and unifying them with animal recordings (Tagamets and Horwitz, 1998; Deco et al., 2004; Horwitz et al., 2005). Similar models have been extended to understand the neural dynamics underlying perceptual and sequential decision-making (Deco and Rolls, 2005; Deco et al., 2010). Large-scale models have also been used to explain the role of neural connectivity in bimanual motor coordination (Jirsa et al., 1998; Daffertshofer et al., 2005; Banerjee and Jirsa, 2007).

Thus, large-scale models of functional brain networks provide a unique way to relate brain structure with function. Though most of the neuromodeling studies we have discussed are at the network level, numerous studies model the local circuits at the level of few neurons (Wang, 2008). A detailed overview of some of these local microcircuits can be found in Wang (2008). This also presents us an opportunity to unify different levels of neuronal dynamics in a combined theoretical framework (Deco et al., 2008). Neuroimaging data with high spatial and temporal resolution are highly complex and requires careful interpretation of brain organization often using multiple methods of data analysis if possible (Horwitz, 2003).

# **HOW CAN WE USE LARGE-SCALE NEURAL MODELS TO TEST NEW NETWORK LEVEL MEASURES?**

Often the goal of network level measures is to understand minute changes in brain network organization during a particular task or during neurological disorders. Comparisons are drawn between different populations to reveal the change in functional neural circuitry and the strength of the results are established based on

(3)

statistical significance tests. Using large-scale models of neural activity provides us with sample data sets where all underlying network organizations and parameters are completely understood. Here, one can validate the reliability and sensitivity of a novel network analysis technique. In the spirit of this approach, we performed a sensitivity analysis for the temporal microstructure analysis (Banerjee et al., submitted)presented earlier using simulated data from a large-scale neural model for a visual DMS task (Tagamets and Horwitz, 1998).

## **MODELING THE MEG ACTIVITY DURING A DELAYED MATCH-TO-SAMPLE (DMS) TASK**

DMS tasks involve presentation of two consecutive sensory stimuli, following which a subject has to respond whether second stimulus matches the first one (Haxby et al., 1995). They are used extensively to study working memory with neuroimaging techniques PET/fMRI in humans (Sergent et al., 1992; Haxby et al., 1995; Courtney et al., 1997; Husain et al., 2006; Schon et al., 2008) and in single unit electrophysiological recordings in monkeys (Fuster et al., 1982; Haenny et al., 1988; Fuster, 1990; Wilson et al., 1993; Miller et al., 1996). Tagamets and Horwitz (1998) proposed a large-scale neural model incorporating the major nodes of the ventral visual pathway comprising areas V1, V4, inferior temporal (IT) and the PFC to generate simulated data for a visual DMS task. These nodes were observed in PET and fMRI studies of the ventral visual stream in relation to face and object recognition (Corbetta et al., 1991; Haxby et al., 1991, 1995; Sergent et al., 1992; Courtney et al., 1997; Connor et al., 2007). The basic circuit used to represent each cortical column (the basic neuronal unit in each region) is shown in **Figure 1**. The local response and total synaptic activity within a cortical area depends on the interactions of the afferent connections, originating from other areas and local connectivity which shapes the response. Following earlier results (Douglas et al., 1995; Tagamets and Horwitz, 1998) we have (1) 85% of the synapses in cortex are excitatory, and (2) of those, 85% are to other excitatory neurons. This high percentage of excitatory connection has given rise to the notion of "amplification" of neuronal responses within a local circuit in response to small amount of afferents. Following Tagamets and Horwitz (1998) we chose the total excitatory to excitatory connectivity weight at 0.6, excitatory to inhibitory connectivity weight at 0.15 and inhibitory to excitatory connectivity weight at −0*.*15 (**Figure 1**). Each pair of excitatory and inhibitory units is the well-known Wilson-Cowan unit (Wilson and Cowan, 1972). In our large-scale model, each brain area is composed of 81 Wilson Cowan units in 9 × 9 configuration (in order to capture complex visual patterns). We model the MEG activity for the DMS task in three steps. In the first step, we define the activation equations of membrane currents for each Wilson-Cowan unit. In the second step, we sum the activation of excitatory neurons to compute the primary currents which are the main sources of MEG data. In the third stage, we obtain the magnetic fields generated by the current sources outside the brain on a unit hemisphere using a forward solution for a spherical head (Mosher et al., 1999).

The electrical activity of each of the excitatory (E) and inhibitory (I) units is governed by a sigmoidal function of the summed synaptic inputs that arrive at the unit. This corresponds to average spiking rates from single-cell recordings. The electrical activity of an E-I pair is mathematically expressed as:

$$\frac{dE\_i(t)}{dt} = \Delta \left(\frac{1}{1 + e^{-K\_E\left\{\nu\_{\rm EE}E\_i(t) + \nu\_{\rm IE}I\_i(t) + i n\_{i\rm E}(t) - \tau\_{\rm E} + N(t)\right\}}}\right) - \delta E\_i(t)$$

$$\frac{dI\_i(t)}{dt} = \Delta \left(\frac{1}{1 + e^{-K\_I[\text{wHE}\_i(t) + i n\_{iI}(t) - \text{tr} + N(t)}}\right) - \delta I\_i(t) \tag{4}$$

where, *Ei(t)* and *Ii(t)* represent the electrical activations of the *i*th excitatory and inhibitory elements at time *t* respectively. *KE* and *KI* are the gains or steepness of the sigmoid functions for excitatory and inhibitory units respectively, *τ<sup>E</sup>* and *τ<sup>I</sup>* are the input thresholds for the excitatory and inhibitory units, is the rate of change, *δ* is the decay rate, and *N(t)* is the added noise term. *wEE*, *wIE*, and *wEI* are the weights within a unit: excitatoryto-excitatory (value = 0*.*6), inhibitory-to-excitatory (value = −0*.*15) and excitatory-to-inhibitory (value = 0*.*15) respectively. *iniE(t)* and *iniI(t)* are the total inputs coming from other areas into the excitatory and inhibitory units at time *t.*

$$i n\_{i\bar{E}}(t) = \sum\_{j} w\_{ji}^{E} E\_{j}(t) + \sum\_{j} w\_{ji}^{I} I\_{j}(t)$$

$$i n\_{i\bar{I}}(t) = \sum\_{k} w\_{ki}^{E} E\_{k}(t) + \sum\_{k} w\_{ki}^{I} I\_{k}(t) \tag{5}$$

where, *w<sup>E</sup> ji* and *<sup>w</sup><sup>I</sup> ji* are weights coming from excitatory/inhibitory unit *j* in another area into the *i*th excitatory and inhibitory units respectively. Electrical activations in the model range between 0 and 1, and can be interpreted as reflecting the percentage of active units within a local population. For this article we chose to keep parameter values (*KE*, *KI*, τ, δ, ) identical to Tagamets and Horwitz (1998).

The source of MEG activity is the primary currents across pyramidal cell assemblies dominated by excitatory connections (Okada, 1983). To obtain the source activity of magnetic dipoles, the generators of ERFs, we sum over the total inputs to one excitatory unit.

$$Iprim(t) = \sum\_{i} \boldsymbol{w}\_{EE} \boldsymbol{E}\_{i}(t) + \sum\_{i} \boldsymbol{w}\_{EI} \boldsymbol{E}\_{i}(t) + \sum\_{k,i} \boldsymbol{w}\_{ki}^{F} \boldsymbol{E}\_{k}(t) \quad (6)$$

where, the first two terms on the right hand side of Equation (6) represent the contribution of excitatory inputs onto itself (no axonal delays are considered) and the third term represents the input from other excitatory units.

The DMS task involves remembering the first stimulus S1 and responding after a second stimulus S2 with a button press if S2 matches with S1. We have square patterns of light (S1 and S2) as external visual stimuli presented consecutively interspersed with a delay period. A large-scale model of the DMS task is created by including brain areas V1, V4, IT, and the PFC as the magnetic dipole sources (**Figure 1A**). A local short-term memory circuit is implemented in the PFC by incorporating different

are located for simulating MEG data. The 3-D Talairach coordinates have been projected to the nearest gray matter on the cortical surface within a window of 5 mm. The medial surface locations V1 and V4 are shaded in lighter color, pink whereas the lateral surface locations in inferior temporal (IT) and prefrontal cortex (PFC) in brighter red. **(B)** The basic Wilson-Cowan unit. *E* represents the excitatory population and *I* the inhibitory population in a local assembly such as a cortical column. Local synaptic activity is dominated by the local excitation and inhibition, while afferents account for the smallest proportion, as indicated by the synaptic weights shown.

shown in lighter lines. Individual units in the excitatory and inhibitory populations within a group are connected as shown in **(B)**. **(D)** The working memory circuit in the prefrontal area of the model. It is composed of different types of units, as identified in electrophysiological studies, and shown in **(C)**. Each element of the circuit shown is a basic unit, as shown in **(B)**. Inhibitory connections are affected by excitatory connections onto inhibitory units. These D2 units also are the source of feedback into earlier areas. **Figures B–D** are adapted with permission from (Tagamets and Horwitz, 1998).



sub-modules (**Figure 1D**). We placed dipoles at the Talairach coordinates corresponding to each area (**Table 1**) with electrical responses as follows:


Based on primate electrophysiological recordings (Funahashi et al., 1990), D1, D2, and C are taken to be located near one another and hence, share the same Talairach coordinates. The effect of attention in the model is implemented by a low-level, diffuse incoming activity to the D2 units as shown in **Figure 1D** (from the modulator). While we do not model the source of this modulation, our model makes it explicit that the D2 units are the recipients. When the attention level is low there is very little delay period activity in the D1 and D2 units (**Figure 2**). Hence, the prefrontal working memory network is only recruited for the DMS task during high attention. Two sources of trialby-trial variability were incorporated in the model: (1) additive random noise to each Wilson-Cowan unit, and (2) activity of the non-specific units at a background rate were added to the DMS task-specific network (see Horwitz et al., 2005 for details). The magnetic dipole source dynamics at different brain locations are plotted in **Figure 2**. Finally, MEG activity at the sensor level is computed by applying a forward solution with sources at the aforementioned locations. We chose a single-shell forward model of a spherical head with unit conductivity derived by Mosher et al. (1999). MEG data were generated for DMS task with low and high levels of attention. The scalp topography of the simulated data at two different time points (stimulation and delay period respectively) is shown in **Figure 2**.

#### **DECODING TEMPORAL MICROSTRUCTURE OF TASK-RELATED INFORMATION PROCESSING**

A key event in the DMS task is the recruitment of the local prefrontal circuit to engage the working memory network during the delay period. This is required for temporarily storing the visual stimulus S1 until S2 arrives following which a DMS response is made. The large-scale model is set up such that an effective recruitment of the prefrontal working memory network only occurs for high attention scenarios. By applying a recently developed method to identify the temporal microstructure of task processing (Banerjee et al., submitted), we sought to decode the onset times of recruitment of this network and validate if the decoded onset times correspond to those occurring in the underlying neural model. Thus, the main objective here is to illustrate the usage of large-scale models for providing face validity for connectivity analysis. The temporal microstructure analysis involves three steps:


In principle, any of the dimensional reduction techniques discussed in the overview section can be used to compute the control subspace. However, for the purpose of this example, we chose one of the simpler methods: principal component analysis. This can be mathematically expressed as

$$\text{DMS}\_{\text{low }att}(\mathbf{X}, t) = \sum\_{i=1}^{n} \lambda\_i \Phi\_i(\mathbf{X}) \xi\_i(t) \tag{7}$$

where, *DMSlow att* are the simulated MEG data at the sensor level for low attention scenario, *i*'s are the principal components, *λ<sup>i</sup>* is the eigenvalue that scales component *i*, and ξ*i(t)* is the temporal coefficient of the spatial pattern *i*. *X* is a column vector of all sensors, *n* is the number of sensors at which MEG data were simulated (*n* = 264, matches closely with the 275 channel CTF MEG system) and *t* is the instantaneous time. The first two modes <sup>1</sup> and <sup>2</sup> capture 99.98% of the total variance in data *DMSlow att*. Hence, we chose *m* = 2 to construct the control subspace.

$$\text{DMS}\_{\text{low}\,\,att}(\mathbf{X}, t) \approx \sum\_{i=1}^{2} \lambda\_i \Phi\_i(\mathbf{X}) \xi\_i(t) \tag{8}$$

The two modes (<sup>1</sup> and 2) are plotted in **Figure 3A** with corresponding normalized eigenvalues reflecting percentage contribution to the total variance. The MEG data for high attention *DMShigh att* can be reconstructed ms-by-ms by using their projections onto the vector space spanned by the set of orthogonal vectors <sup>1</sup> and 2.

$$\overline{DMS}\_{\text{high\\_att}}(\mathbf{X}, t) \approx \sum\_{i=1}^{2} \Phi\_i^\dagger \left\langle DMS\_{\text{high\\_att}}(\mathbf{X}, t) | \Phi \right\rangle \tag{9}$$

where, | indicates projection operation, and † represents transpose. The orthogonality of basis vectors is an important requirement that is guaranteed by PCA. However, non-orthogonal basis vectors can be employed in the control subspace by using the dual basis for reconstruction (Banerjee et al., 2008). Similarly, *DMSlow att* can be reconstructed ms-by-ms from the two control modes.

$$\overline{\text{DMS}}\_{\text{low\\_att}}(\mathbf{X}, t) \approx \sum\_{i=1}^{2} \Phi\_i^\dagger \langle \text{DMS}\_{\text{low\\_att}}(\mathbf{X}, t) | \Phi\_i \rangle \tag{10}$$

capture by these two patterns. **(B)** The mean goodness of fit of reconstruction (*Gof*) time series plotted as a function of time when the difference in attention levels is low. The error bars at 95% significance level are also plotted as a function of time (patches). In this scenario, lack of significant recruitment results in statistically equivalent *Gof* over time. **(C)** The mean goodness *Gof* time series from two conditions are plotted as a

Goodness of fit of the reconstruction is computed by the following expression

$$Gof\_i = \left(1 - \frac{\overline{DMS}\_i^\dagger \cdot \overline{DMS}\_i}{DMS\_i^\dagger \cdot DMS\_i}\right) \times 100\% \tag{11}$$

where, *i* = low att, high att. We obtained the *Gof* time series for low attention and high attention DMS tasks. Using bootstrapping techniques (Efron, 1979), distributions of these variables can be derived (**Figures 3B,C**) for different levels of attention in the DMS task.

Using the lines of reasoning outlined in Banerjee et al. (2008), a high goodness of fit occurrence indicates that no additional nodes compared to the control network is present in the task network, whereas any depreciation of *Gof* from 100% will indicate recruitment of additional subspaces. The regimes where the two *Gof* 's significantly differ are identified as the time scales of recruitment. For example if the zero attention scenario is used as control and high attention as task, then the time scale of network

function of time when the difference in attention levels is high. The error bars at 95% significance level are also plotted as a function of time (patches). The regime of difference in *Gof* distributions reflect the time scale of recruitment. **(D)** Sensitivity analysis for onset time detection: Onset time plotted as function of different attention levels. For all levels of attention (except the zero attention scenario) the same prefrontal network (D1, D2) is recruited in the delay period, with varying degrees of intensity. At low attention (gain *<* 0*.*09), onset time of this prefrontal network recruitment occurs twice by chance. However, after a threshold level of attention (0.09), onset time is consistently detected for all higher levels.

recruitment for storing the identity of S1 in working memory is captured by the onset and offset of significant differences in *Gof* (**Figure 3C**). On the other hand, if the DMS task is performed at a lower attention level, significant amount of network recruitment is absent and hence, the *Gof* distributions are statistically equivalent over time (**Figure 3B**). The onset time of recruitment was defined as the first point where the two *Gof* time series are significantly divergent with *p <* 0*.*05 and remains so for 50 consecutive time points. Additionally at least for one point within this regime *p* should reach a value of 0.001. Choosing a window size of 50 time points sets a lower limit on the time scale of recruitment decoded from this analysis. The choosing of the time window for control subspace construction is another constraint on the network to which recruitment is analyzed. For our example, we chose to run the PCA on the entire DMS data from start of S1 to end of S2. However, choosing such time segments can also be done *a posteriori* from the experimental design and stability of onset times may be tested by varying the length of such windows.

The value of onset time detected by the temporal microstructure analysis (in **Figure 3D**) closely matched with the time at which D1 and D2 units get activated for the delay period in the high attention scenarios (**Figure 2**). We performed a sensitivity analysis for the temporal microstructure method by varying the attention level for the task condition and correspondingly computing the onset time for each levels of attention (**Figure 3D**). For low levels of attention (*<*0.09) our method was unable to detect an onset time consistently. However, the onset time detected after a threshold attention level of 0.09 did not change significantly when the level was parametrically varied up to 0.3. Hence, we conclude that statistically significant levels of recruitment occur for this network at a certain attention level beyond which the DMS task can be actively executed. Once the threshold attention level is reached, recruitment of a large-scale network is detected and the onset time does not change with further increases in attention level. The detection of the onset time of task-specific recruitment amounts to decoding an important event in the underlying information processing at the largescale network level. Using similar statistical thresholds, we can also decode when the recruitment of additional nodes get disengaged. Changes in functional connectivity via interactions among overlapping components of control and task networks are not detected by the temporal microstructure analysis. Hence, the regimes where, *Gof* 's are statistically equivalent can be used as ideal candidates for applying other functional connectivity techniques described in the overview, such as correlation and coherence. We conclude that the large-scale model provided a partial validation for the use of temporal microstructure analysis as a tool to detect spatiotemporal network mechanisms at millisecond resolution.

An important aspect of the temporal microstructure analysis that needs immediate attention is that once presence of recruitment is inferred, how can we estimate the spatial localization of recruited brain areas? This issue will be addressed in future research. The challenge here is that the residual activity which carries the signatures of spatial localization has higher noise to signal ratios than the original signals. One way to circumvent this issue would be to design experiments with a larger number of trials when the localization question needs to be answered.

The time scales of recruitment decoded from the microstructure analysis can be used as constraints for building large-scale models of behavioral tasks. Such models provide a mechanistic understanding of neural information processing. From a complementary perspective, decoding the timing of information processing is a key to understanding network mechanisms underlying ongoing behavior in action-perception paradigms where brain dynamics at millisecond resolution controls task processing. Also, in practical scenarios such as in development of neural prosthetic tools where decoding between two alternative task conditions is the often the main goal, the temporal microstructure analysis might provide a simple but robust algorithmic framework.

# **SUMMARY**

In this article we have provided an extended overview of methods that can detect the presence of functional brain networks for

a wide variety of tasks in which information processing occurs at the network level with rich temporal behavior. EEG/MEG/iEEG provide high temporal resolution whole brain recordings that have the ability to record the elemental properties of a largescale network. A major challenge in analyzing such giant data sets is the fact that no consensus exists as to what constitutes a large-scale network. It often depends on the modality of the imaging technique and the idiosyncrasies of the particular task that are being studied that ultimately shapes the patterns observed in the data. In addition each network level analysis (some of which are discussed in the Overview), may yield different results, sometimes seemingly conflicting. A classic example is the observation of temporally ordered brain responses (Gray et al., 1989; Tallon-Baudry et al., 1998; Tognoli et al., 2007; Donner and Siegel, 2011) and self-organized criticality (Kelso, 1995; Linkenkaer-Hansen et al., 2001; Stam and de Bruin, 2004) being simultaneously present as mechanisms of neural information processing. Interpreting such results with theoretical frameworks that allows co-existence of different mechanisms (such as Kelso, 1995; Buzsáki, 2006) will be a pre-requisite for future signal processing tools. We have recently developed a computational framework (Banerjee et al., submitted)which decodes the time scales of network level processing at high temporal resolution via characterization of two distinct modes of information processing: modulation of functional connectivity and recruitment of task-specific networks.

Our second objective in this article was to show how development of network analysis can immensely benefit from using biologically realistic large-scale models of brain activity. It is relatively easy to implement candidate functional connectivity topologies in such models at various time scales mimicking the complexity associated in real neuronal processing, following which the face validity of a novel method for network analysis can be tested on simulated data generated by the model. This is important, because only in simulated data one has the knowledge of the ground truth. Using this approach we have tested the face validity of our recently developed method to decode the temporal microstructure of task-specific information processing during a DMS task. Such tasks or its variants are used in the literature to study a diverse set of questions in higher order sensory processing (Corbetta et al., 1991; Sergent et al., 1992), multisensory integration (Haenny et al., 1988; Maunsell et al., 1991; Colombo and Gross, 1994), working memory processing (Haxby et al., 1995; Courtney et al., 1997), long-term memory retrieval (Naya et al., 1996; Smith et al., 2010a) and perceptual and cognitive decision-making (Bechara et al., 2000; Lamar et al., 2004). Hence, we believe our simple illustration of decoding network mechanisms during a DMS task may lay out a strategic framework to answer long-standing questions in several brain systems. In the future a combined modelingdecoding approach may help in characterization of timing of task-specific processing in networks at the level of single trials.

# **ACKNOWLEDGEMENTS**

This research was funded by the NIDCD intramural research program.

# **REFERENCES**


approach to blind separation and blind deconvolution. *Neural. Comput.* 7, 1129–1159.


information in the ventral pathway. *Curr. Opin. Neurobiol.* 17, 140–147.


using higher-order statistics and independent component analysis. *Neuroimage* 34, 1443–1449.


Dolan, R. J. (1996). The trouble with cognitive subtraction. *Neuroimage* 4, 97–104.


*Proc. Natl. Acad. Sci. U.S.A.* 88, 1621–1625.


R. J. (2001). Long-range temporal correlations and scaling behavior in human brain oscillations. *J. Neurosci.* 21, 1370–1377.


human EEG dynamics using independent component analysis. *Neurosci. Biobehav. Rev.* 30, 808–822.


memory task in humans. *J. Neurosci.* 18, 4244–4254.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 26 August 2011; accepted: 16 December 2011; published online: 06 January 2012.*

*Citation: Banerjee A, Pillai AS and Horwitz B (2012) Using large-scale neural models to interpret connectivity measures of cortico-cortical dynamics at millisecond temporal resolution. Front. Syst. Neurosci. 5:102. doi: 10.3389/fnsys. 201 .00102 1*

*Copyright © 2012 Banerjee, Pillai and Horwitz. This is an open-access article distributed undertheterms oftheCreative Commons Attribution Non Commercial License, which permits non-commercial use, distribution, and reproduction in other forums, provided the original authors and source are credited.*

# Functional embedding predicts the variability of neural activity

# *Bratislav Miši´c 1,2\*, Vasily A. Vakorin1,Tomáš Paus 1,2,3 and Anthony R. McIntosh1,2*

*<sup>1</sup> Rotman Research Institute, Baycrest Centre, Toronto, ON, Canada*

*<sup>2</sup> Department of Psychology, University of Toronto, ON, Canada*

*<sup>3</sup> Cognitive Neuroscience Unit, Montreal Neurological Institute, McGill University, Montreal, QC, Canada*

#### *Edited by:*

*Claus Hilgetag, Jacobs University Bremen, Germany*

#### *Reviewed by:*

*Mika Rubinov, University of Cambridge, UK Guadalupe Clara Garcia, Jacobs University, Germany*

#### *\*Correspondence:*

*Bratislav Miši´c, Rotman Research Institute, Baycrest, 3560 Bathurst Street, Toronto, ON, Canada M6A 2E1. e-mail: bratislav.misic@utoronto.ca*

Neural activity is irregular and unpredictable, yet little is known about why this is the case and how this property relates to the functional architecture of the brain. Here we show that the variability of a region's activity systematically varies according to its topological role in functional networks. We recorded the resting-state electroencephalogram (EEG) and constructed undirected graphs of functional networks. We measured the centrality of each node in terms of the number of connections it makes (degree), the ease with which the node can be reached from other nodes in the network (efficiency) and the tendency of the node to occupy a position on the shortest paths between other pairs of nodes in the network (betweenness). As a proxy for variability, we estimated the information content of neural activity using multiscale entropy analysis. We found that the rate at which information was generated was largely predicted by centrality. Namely, nodes with greater degree, betweenness, and efficiency were more likely to have high information content, while peripheral nodes had relatively low information content. These results suggest that the variability of regional activity reflects functional embedding.

**Keywords: variability, entropy, degree, efficiency, connectivity, centrality, functional integration**

# **1. INTRODUCTION**

The functional architecture of the cerebral cortex is configured in a manner that balances local segregation and global integration, endowing the system with a high degree of complexity (Tononi et al., 1994). The complexity of the system allows for a diverse dynamic répertoire and is reflected in the information content and variability of neural activity. The variability inherent in neurophysiological recordings is now considered a fundamental dynamical property of the brain, allowing for spontaneous transitions among several metastable states (Ghosh et al., 2008; McIntosh et al., 2008, 2010; Deco et al., 2009, 2011; Jirsa et al., 2010).

What determines the variability of neural activity? If brief functional associations allow information to be integrated, then regions which participate in the greatest number of functional subnetworks will facilitate the flow of information by bridging and integrating other regions that would be otherwise disconnected and topologically distant (Sporns et al., 2000, 2004; Stam, 2004; Stam and Reijneveld, 2007). Therefore, information content should depend on connectivity and more specifically on the functional integration enabled by individual nodes.

Recent findings are consistent with this notion and suggest that the information content of regional activity may depend on the configuration of functional networks and on their participation in such networks. For example, information – measured using entropy-based metrics – increases during normal brain development (McIntosh et al., 2008, 2010; Lippé et al., 2009; Mišic et al., 2010). This may reflect intensified global integration relative to local segregation and a gradual shift from local, clustered information processing in children to distributed processing in adults (Fair

et al., 2009; Supekar et al., 2009; Hagmann et al., 2010). Indeed, the developmental increase in entropy is most robust in areas such as the precuneus and posterior cingulate (Mišic et al., 2010), which are known to be central to the topology of resting-state functional brain networks (Hagmann et al., 2008, 2010; Buckner et al., 2009; Tomasi and Volkow, 2010). Moreover, just as functional networks reconfigure in response to task-induced perturbations (Bassett et al., 2006), so too do spatial patterns of entropy (Lippé et al., 2009; Mišic et al., 2010).

In the present study we tested the notion that the functional embedding of a brain region is related to the information content of neural activityfrom that region.We addressed this hypothesis by recording the electroencephalogram (EEG) from 56 participants in the resting-state with eyes open and eyes closed. Whole-head functional networks were constructed by measuring the phase lag index (PLI; Stam et al., 2007, 2009) between all pairs of electrodes. Centrality of individual nodes was assessed in terms of node degree, betweenness, and efficiency. The degree of individual nodes was indexed by counting the number of connections they make with other nodes in the network. Betweenness was measured as the fraction of all shortest paths in the network that pass through the node (Freeman, 1977, 1978). Regional efficiency was calculated as the inverse of the minimum path length (Latora and Marchiori, 2001; Achard and Bullmore, 2007; Bassett et al., 2009). Thus, regions which have a short minimum path to all other regions will have high efficiency. We predicted that the centrality of a region (indexed by degree, betweenness, and efficiency) will be associated with greater information content.

We quantified the information content of regional activity using multiscale entropy (MSE) analysis (Costa et al., 2002, 2005). As an entropy-based measure,MSE is sensitive to the complexity of the signal and will assign low values to both completely regular and to uncorrelated random signals. Moreover, the multiscale nature of the index takes into account the fact that physiological dynamics underlying the expressed information are likely to unfold over multiple temporal scales (Honey et al., 2007). Although functional magnetic resonance imaging (fMRI) would have allowed us to delineate functional networks with greater precision, the number of data points required to estimate entropy from empirical time series is quite large and necessitated an imaging technique with fast sampling such as EEG.

# **2. MATERIALS AND METHODS**

# **2.1. EEG ACQUISITION**

Fifty-six (29 male) healthy children 10 years old (mean 10.0, SD 0.393 years) participated in the study (see Poulsen et al. (2009) for details). The protocol was approved by the Research Ethics Board of the Montreal Neurological Institute and Hospital. The participants were asked to keep their eyes open or closed in 8 alternating 30 s epochs (4 each). The electroencephalogram (EEG) was continuously recorded from 128 scalp locations using a HydroCel geodesic sensor net (Electrical Geodesics, Inc., Eugene, OR) referenced to the vertex (Cz). The signal was digitized at a rate of 500 Hz. Impedances did not exceed 60 k-. All offline signal processing and artifact correction was performed using the EEGLAB toolbox (Delorme and Makeig, 2004) for MATLAB (Mathworks, Inc.). Data were then average-referenced, digitally filtered (bandpass: 0.5–55 Hz; notch: 60 Hz) and epoched into 30 s segments. Only the middle 20 s of each epoch (5–25 s) were used in the analysis to avoid excessive contamination associated with opening and closing of the eyes. In the absence of a true baseline, the temporal mean was subtracted from each epoch. Ocular (blinks and lateral eye movements) and muscle artifacts were identified and subtracted on a subject-by-subject basis using the Infomax independent components analysis (ICA) algorithm (Bell and Sejnowski, 1995) implemented in EEGLAB. The analyses described below produced identical resultsfor the eyes closed and eyes open restingstate runs, so for brevity only data from the eyes closed runs are displayed in the rest of this report.

### **2.2. MULTISCALE ENTROPY (MSE)**

In multiscale entropy (MSE) analysis (Costa et al., 2002, 2005) each single trial time series is downsampled to multiple temporal scales and sample entropy (*SE*; Richman and Moorman, 2000) is calculated for each scale. For a given temporal scale τ , the corresponding time series is derived by averaging data points in non-overlapping windows of length τ from the original time series (τ = 1 corresponds to the original time series). The *SE* algorithm calculates the conditional probability that any two sequences of (*m* + 1) data points will be similar to each other given that they were similar for the first *m* points, which reflects the degree of regularity in a given time series. The *SE* metric is the negative of the natural logarithm of this quantity, so higher values of *SE* are associated with less regular and more variable time series. In the present study, pattern length was set to *m* = 2 and the similarity criterion to *r* = 0.5. The pattern length (otherwise known as the embedding dimension) was judged to be optimal following the method proposed by (Small and Tse, 2004). The similarity criterion (also known as the tolerance) was chosen following (Richman and Moorman, 2000). MSE was calculated for each of the 128 channels and averaged across epochs.

# **2.3. FUNCTIONAL CONNECTIVITY**

The biggest challenge in estimating functional associations between EEG or magnetoencephalogram (MEG) surface sensors is the confounding influence of volume conduction (Nunez et al., 1997). Electromagnetic activity originating from a single brain region may be observed instantaneously by several scalp electrodes and misinterpreted as a functional connection. This zero-lag"blurring" tends to inflate the strength of short-length, neighbor-toneighbor statistical interdependencies. Moreover,in EEG the effect is exacerbated if the montage involves an active reference (Nunez et al., 1997). One approach to address this problem has been to reformat EEG recorded with respect to an active reference to a reference-free montage, such as the bipolar (Rubinov et al., 2009a). Although the technique can be extended to include multiple bipolar orientations, it may still miss certain dipoles (Schiff, 2005). Another approach has been to estimate patterns of functional connectivity among reconstructed sources and to define the graph in source space (De Vico Fallani et al., 2007, 2008). However, source reconstruction techniques do not produce unique solutions and therefore different assumptions and models may lead to different results. Moreover, many common localization techniques such as beam forming operate by changing the covariance structure of the observed data.

To minimize the effect of volume conduction, we opted to use a measure of functional connectivity that is insensitive to spurious coherencies due to volume conduction, known as the Phase Lag Index (PLI; Stam et al., 2007, 2009). The measure takes advantage of the fact that functional connectivity between two channels due to volume conduction or an active reference cannot produce phase delays. Thus, PLI attempts to quantify the distribution of phase differences between two signals but is explicitly insensitive to differences that center around 0 mod π. As a result, PLI indexes synchronization between electrodes in a way that is minimally affected by volume conduction.

### **2.4. GRAPH EXTRACTION AND ANALYSIS**

As we sought to compare aspects of network structure with information, which was indexed by *SE* at multiple time scales, we extracted functional connectivity graphs at multiple time scales as well. Thus, each EEG epoch was downsampled using the same procedure as for MSE and functional connectivity was estimated for each of the resultant 20 coarse-grained time series. PLI was computed between all pairs of electrodes for each epoch, scale, subject, and condition, yielding a series of 128 × 128 matrices. For each time scale, these matrices were then averaged across the four epochs to yield subject- and condition-specific association matrices.

The importance of individual nodes was quantified by directly computing measures of centrality from each subject- and condition-specific weighted graph. All network measures except regional efficiency were computed using MATLAB routines implemented in the Brain Connectivity Toolbox (Rubinov and Sporns, 2010). The degree of each node was determined by taking the sum of all weighted connections to other nodes in the network. In order to calculate regional efficiency and betweenness for weighted graphs, we first defined an inverse mapping from weight to length. Thus, the length of a weighted edge between regions *i* and *j* (*di,j*) was calculated as the inverse of the edge weight between those two regions (*wi,j*)

$$d\_{i,j} = \frac{1}{w\_{i,j}}.\tag{1}$$

The path length between any two nodes in the network can then be calculated as the sum of the lengths of edges along the path. Regional efficiency was calculated by taking the inverse of the harmonic mean of the minimum path length between a given node and all nodes in the network (Latora and Marchiori, 2001, 2003; Achard and Bullmore, 2007). Thus, if the minimum path length between regions *i* and *j* (*i*, *j* = 1, 2,..., *N*, and *i* -= *j*) is *Li,j*, the efficiency of region *i* (*Ei*) is given by

$$E\_l = \frac{1}{N-1} \sum\_{i \neq j \in G} \frac{1}{L\_{i,j}}.\tag{2}$$

The normalized betweenness of a node *j* is the proportion of all shortest paths between all pairs of nodes *i* and *k* (ρ*ik*) that also pass through *j* (ρ*ijk*)

$$B\_{\vec{j}} = \frac{1}{(N-1)\ (N-2)} \sum\_{i \neq j, i \neq k, j \neq k} \frac{\rho\_{ik}}{\rho\_{ijk}}.\tag{3}$$

Together, the three measures offer complementary information about the topological role of a given node. Degree indexes the connectedness of a given node. Efficiency tells us how close a node is to other nodes in the network. Betweenness measures the tendency of a node to serve as a way station between other pairs of nodes in the network. Each of these regional measures was averaged across subjects and then correlated with *SE*, separately for the two conditions and for each temporal scale.

### **3. RESULTS**

To demonstrate and broadly summarize the regional variation in MSE, **Figure 1A** shows the spatial pattern of *SE* values that have been summed across all 20 scales. These patterns demonstrate that total MSE tends to be greatest at medial and mid-lateral posterior parietal channels, in concordance with previous studies (McIntosh et al., 2008). The complete MSE curves for two representative channels (marked by black dots in **Figure 1A**) are displayed in **Figure 1B**. The curves show increasing entropy with coarse-graining, similar to previous studies using electrophysiological (McIntosh et al., 2008) and neuromagnetic recordings (Mišic et al., 2010; **Figure 1**). Moreover, the curves show large differences in *SE* across most temporal scales, except the first two. This demonstrates that differences in variability occur at multiple time scales and suggests that network structure should likewise be considered at different levels of coarse-graining.

**FIGURE 1 | Multiscale entropy curves.** Values of *SE* are summed across temporal scales and the spatial distribution is shown in **(A)**. The complete multiscale entropy (MSE) curves for two representative channels (marked by black circles), showing *SE* at each level of coarse-graining, are displayed in **(B)**. Error bars indicate standard errors of the mean.

The overall correspondence between information entropy and network embedding is explored in **Figure 2**. Node degree, regional efficiency, and betweenness were positively associated with entropy (**Figure 2**, top row). The relationships were statistically significant and were observed at all temporal scales (*<sup>p</sup>* <sup>10</sup>−3). On average, node degree, efficiency, and betweenness accounted for approximately 44, 70, and 42% of the variance in *SE* across electrodes. We also observed an effect of time scale on the relationship between network embedding and MSE, whereby correlations were slightly weaker for fine time scales and increased with coarse-graining (**Figure 2**, middle row). This was expected given the fact that coarse-graining acts as a low-pass filter. Thus, the broad-band signal at fine time scales contains higher frequencies in addition to lower frequencies and estimates of phase synchronization are likely to be less robust. In contrast, the coarse-grained representations of the signal contain a narrower band of lower frequencies and therefore estimates of phase synchronization are more likely to be reliable.

The bottom row of **Figure 2** shows the scale-specific spatial distributions of node degree, regional efficiency, and betweenness, summed across all scales. Importantly, the pattern of gradations was similar for *SE* (**Figure 1A**) and the three network metrics. For all three measures, the highest values were observed at medial posterior channels and this peak typically extended around the vertex, forming a ring-like relief pattern over the superior aspect of the scalp, with a minute dip at the vertex. In some instances, the correspondence was not perfect and there were also minor differences. For example, some anterior channels had relatively high efficiency while the same was not true for *SE*. Likewise, the topographical map for betweenness did not perfectly match the map for *SE*, but this is not surprising since betweenness was not as good a predictor of *SE* as the other two measures of centrality. Overall, comparison of topographic maps for *SE* and centrality revealed the possibility of two separate effects: a posterior parietal peak in centrality that matched a similar peak in variability, as well as a fronto-central peak in centrality that was not consistently accompanied by a peak in *SE*.

In general, the effects associated with node degree and efficiency were similar and this is to be expected because the two measures are complementary. Namely, efficiency is the average minimum path length between a given node and all others, while node degree quantifies the total number of connections. In the case of weighted graphs, if direct connections are also the shortest, then the measures will index the same thing and may be redundant. We quantified this overlap and found that, on average, direct connections constituted 40% of the shortest paths. Therefore, there was considerable overlap but also significant divergence.Moreover, calculation of betweenness did not take into account this rather large proportion of direct connections and this may explain why the effects and the topography associated with betweenness was not as similar to degree and efficiency as they were to each other.

Although we took several steps to ensure that the effects of volume conduction would be minimized, we still sought to assess the extent to which this may have affected our estimates of synchronization. In practice, simultaneous activation due to spatial smearing will tend to be most pronounced between proximal electrodes. Therefore, for each possible pair of electrodes we calculated a vector of Euclidean distances as well as a vector of PLI values. We then estimated the degree to which physical distance between channels could predict the strength of their functional coupling, using both mutual information and simple linear regression. Both methods revealed that Euclidean distance typically accounted for 24% of the variance in functional connectivity (consistently across all time scales). This suggests a reasonable effect of spatial proximity on coupling strength and even compares favorably to previous studies using MEG (Bassett et al., 2006).

# **4. DISCUSSION**

The temporal evolution of neural activity is far from regular. This element of unpredictability is indicative of the information carried by the signal and represents an important facet of the brain's organization at multiple scales of time and space (McIntosh et al., 2008, 2010; Deco et al., 2011). We have shown that the centrality of network nodes strongly predict the information content of their physiological activity during the resting-state.

Ephemeral functional associations among distributed regions constitute the dominant patterns of information flow in the brain. Vertices with many connections as well as short or direct paths to the rest of the network are conduits for system-wide communication. These nodes engender the exchange of information both within and between communities (Sporns et al., 2007). The fact that highly central areas also generate information at a high rate is consistent with the notion that they facilitate functional integration. Similar results were uncovered in a recent theoretical study which explored the relationship between node dynamics and connectivity using a simple deterministic model (Rubinov et al., 2009b). The authors found that highly connected, central nodes were more likely to receive discordant inputs and as a result exhibited variable, high-dimensional dynamics. Conversely, peripheral nodes were more likely to receive homogeneous inputs and exhibited less variable, low-dimensional dynamics.

Our data support the idea that the variability observed in neural activity is an important facet of the functional organization of the brain. Variability may take many forms and can be measured in a multitude of ways, from information content of neurophysiological signals (McIntosh et al., 2008) to SD of hemodynamic responses (Garrett et al., 2010). Increased variability is associated with many benefits, both for behavioral performance and physiology. For example, greater brain signal variability is linked to accurate responses and stable response times (McIntosh et al., 2008; Mišic et al., 2010). Network reorganization during healthy development is associated with increased variability (McIntosh et al., 2008; Lippé et al., 2009; Mišic et al., 2010). Conversely, pathologically low variability is associated with tissue damage due to seizures (Protzner et al., 2010), as well as autism spectrum disorder (Bosl et al., 2011).

The inherent variability of neurophysiological signals represents the trajectory of a metastable dynamical system with nonlinearities (Ghosh et al., 2008; Deco et al., 2009, 2011; Jirsa et al., 2010). As the brain typically occupies a high-energy state, small intrinsic fluctuations (for example, due to low-fidelity cellular processes; Faisal et al., 2008) are sufficient to perturb the system and induce excursions to other states/configurations. Thus, the stochastic characteristics of local physiology cause the system to continuously traverse the state space and confer a high degree of flexibility to adapt to changing demands in the external environment (Breakspear et al., 2010). This fluid reconfiguration is reflected by the variable dynamics observed in gross neurophysiological recordings.

In the present study we measured the connectivity of all nodes with respect to the entire brain and then matched regional differences in connectivity with differences in variability. In future studies it would be interesting to focus on one select region and to manipulate the connectivity of that region using some form of stimulation or cognitive task. This complementary approach would allow us to study how the information generated in a given brain region dynamically changes under different external and internal conditions.

#### **4.1. METHODOLOGICAL CONSIDERATIONS**

Graphs are a natural model for anatomical networks, but their application to functional networks is subject to several methodological issues (Bullmore and Sporns, 2009; Rubinov and Sporns, 2010). First, graph theoretic measures of centrality tend to be based on the importance of shortest paths (such as betweenness and efficiency in the present study) and this may not be an accurate model for a system such as the brain where information is

#### **REFERENCES**


human brain functional networks. *Proc. Natl. Acad. Sci. U.S.A.* 103, 19518–19523.


likely to be transmitted along multiple parallel paths that are not necessarily the shortest. Second, it is difficult to infer the directionality of functional connections and this limits the range of local and global metrics that can be calculated for the network. In the present investigation we chose simple indices to capture connectedness, centrality, and functional integration of individual nodes that are not sensitive to directionality of edges in the network. Third, whole-brain functional networks can only be defined with limited precision using neurophysiological measurements over the scalp. This was a necessary compromise that allowed us to estimate information content over a broad-band, which would not have been feasible with high-resolution methods such as fMRI.

In the present investigation, the resting-state paradigm was used instead of some cognitive task with multiple trials for two reasons. First, in the no-task setting cognitive processing could not be biased by external demand. Second, the resting-state allowed many measurements (long time series) to be recorded such that information and functional connectivity could be reliably estimated. For these reasons resting-state paradigms are often used to estimate functional connectivity and centrality (De Vico Fallani et al., 2007; Rubinov et al., 2009a; Boersma et al., 2011), as well as entropy (Escudero et al., 2006; Park et al., 2007; Mizuno et al., 2010; Takahashi et al., 2010; Bosl et al., 2011).

### **5. CONCLUSION**

The dynamical properties of the brain give rise to a complex functional architecture that stays true to the underlying anatomy over long periods of time, but at short time scales reconfigures in a highly fluidfashion (Honey et al.,2007,2009), thereby exploring its functional répertoire (Ghosh et al., 2008). The fluid landscape of functional associations drives and determines the statistical properties of neural activity. The present study demonstrates a link between functional topology and variability of neurophysiological activity.

#### **ACKNOWLEDGMENTS**

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as competing financial interests. This research was supported by grants from the Canadian Institutes of Health Research (CIHR) and Santa Fe Institute Consortium to Tomáš Paus and a J. S. McDonnell Foundation grant to Anthony R. McIntosh.


stability, and relation to Alzheimer's disease. *J. Neurosci.* 29, 1860–1873.


cerebral cortex. *PLoS Biol.* 6, e159. doi:10.1371/journal.pbio.0060159


EEG coherency. I: statistics, reference electrode, volume conduction, Laplacians, cortical imaging, and interpretation at multiple scales. *Electroencephalogr. Clin. Neurophysiol.* 103, 499–515.


"small-world"network? *Neurosci. Lett.* 355, 25–28.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 05 June 2011; accepted: 21 October 2011; published online: 22 November 2011.*

*Citation: Miši´c B, Vakorin VA, Paus T and McIntosh AR (2011) Functional embedding predicts the variability of neural activity. Front. Syst. Neurosci. 5:90. doi: 10.3389/fnsys.2011.00090*

*Copyright © 2011 Miši´c, Vakorin, Paus and McIntosh. This is an open-access article subject to a non-exclusive license between the authors and Frontiers Media SA, which permits use, distribution and reproduction in other forums, provided the original authors and source are credited and other Frontiers conditions are complied with.*

**METHODS ARTICLE** published: 23 November 2011 doi: 10.3389/fnsys.2011.00096

# Empirical and theoretical aspects of generation and transfer of information in a neuromagnetic source network

# *Vasily A. Vakorin1\*, Bratislav Miši´c 1,2, Olga Krakovska3 and Anthony Randal McIntosh1,2*

*<sup>1</sup> Baycrest Centre, Rotman Research Institute of Baycrest, Toronto, ON, Canada*

*<sup>2</sup> Department of Psychology, University of Toronto, Toronto, ON, Canada*

*<sup>3</sup> Department of Chemistry, York University, Toronto, ON, Canada*

#### *Edited by:*

*Claus Hilgetag, Jacobs University Bremen, Germany*

#### *Reviewed by:*

*Viktor Jirsa, Movement Science Institute CNRS, France Michael Wibral, Goethe University, Germany*

#### *\*Correspondence:*

*Vasily A. Vakorin, Baycrest Centre, Rotman Research Institute, Toronto, ON, Canada M6A 2E1. e-mail: vasenka@gmail.com*

Variability in source dynamics across the sources in an activated network may be indicative of how the information is processed within a network. Information-theoretic tools allow one not only to characterize local brain dynamics but also to describe interactions between distributed brain activity. This study follows such a framework and explores the relations between signal variability and asymmetry in mutual interdependencies in a datadriven pipeline of non-linear analysis of neuromagnetic sources reconstructed from human magnetoencephalographic (MEG) data collected as a reaction to a face recognition task. Asymmetry in non-linear interdependencies in the network was analyzed using transfer entropy, which quantifies predictive information transfer between the sources. Variability of the source activity was estimated using multi-scale entropy, quantifying the rate of which information is generated. The empirical results are supported by an analysis of synthetic data based on the dynamics of coupled systems with time delay in coupling. We found that the amount of information transferred from one source to another was correlated with the difference in variability between the dynamics of these two sources, with the directionality of net information transfer depending on the time scale at which the sample entropy was computed. The results based on synthetic data suggest that both time delay and strength of coupling can contribute to the relations between variability of brain signals and information transfer between them. Our findings support the previous attempts to characterize functional organization of the activated brain, based on a combination of non-linear dynamics and temporal features of brain connectivity, such as time delay.

**Keywords:MEG, signal variability, non-linear systems, non-linear interdependence, conditional mutual information, transfer entropy, sample entropy, time delay**

# **1. INTRODUCTION**

Recently, significant progress has been made showing that cognitive operations result from the generation and transformation of cooperative modes of neural activity (Bressler, 1995, 2002; McIntosh, 1999). Specifically, the progress in this field was based on the principle that emphasizes the integrative capacity of the brain in terms of ensembles of coupled neural systems (Nunez, 1995; Jirsa and McIntosh, 2007). In turn, we have witnessed advances both in the modeling endeavors to explore brain integration and the collection of empirical evidence in support for this integration.

From the theoretical point of view, the neural ensembles can be represented by single oscillators (Haken, 1996). Further, different neural ensembles can be coupled with long-range connections, forming a large-scale network of coupled oscillators. Due to the separation of sources in the space and limited transmission speeds, communication between brain regions may include time delays. Thus, the coupling between two nodes in a brain network can be characterized by the connection strength, directionality, and time delay. In turn, time delays in coupling can influence the dynamical properties of coupled oscillatory models (Niebur et al., 1991). Encouraging results were obtained in modeling the resting state

network dynamics wherein time delays play a crucial role in generation of the realistic fluctuations in brain signals (Ghosh et al., 2008; Deco et al., 2009).

At the same time, from the perspective of empirical analysis, recently developed non-linear tools were able to characterize variability of local brain dynamics and interaction effects of distributed brain activity (see Stam, 2005 for a review). Informationtheoretic techniques provide a model-free non-linear approach to address both issues (Pereda et al., 2005; Vakorin et al., 2011).

First, such techniques can be used to characterize the variability in brain signals as a consequence of more complicated neural processing. A typical application includes a comparative analysis of different groups, for example, in brain development (McIntosh et al., 2008) or clinical versus normal populations (Stam, 2005), or different conditions within the same groups (Lippé et al., 2009). Traditionally, the analysis is performed at the level of electroencephalographic (EEG) or magnetoencephalographic (MEG) scalp measurements that do not directly represent localized brain regions in the vicinity of one electrode due to volume conduction (Nunez and Shrinivasan, 2005). The translation to source space would be a logical extension, and it has been recently shown that

entropy-based techniques are sensitive enough to discriminate the variability of neural activity within a network of sources (Miši´c et al., 2010; Vakorin et al., 2010b).

Second, a number of studies have explored methods of assessing linear and non-linear interactions between dynamics of neuronal sources, reconstructed using beamformers (Hadjipapas et al., 2005; Vakorin et al., 2010b; Wibral et al., 2011). Analyses of asymmetries in non-linear interdependency between different brain areas, both in normal and clinical populations, may provide an insight upon processing and integration of information in a neuronal network. The time course of one process may predict the time course of another process better than the other way around. This enhancement in predictive power can characterize the coupling between these two processes (Blinowska et al., 2004; Hlavackova-Schindler et al., 2007). This idea was originally proposed by Granger (1969), who used autoregressive models to describe the interaction between the processes as well as the time courses of the processes themselves. A non-linear extension of the framework of predicting a future of one system from the past and present of another one is based on estimating the information transfer, using information-theoretic tools. Two measures can be used, namely transfer entropy (Schreiber, 2000) or conditional mutual information (Palus et al., 2001), which are essentially equivalent to each other under certain conditions (Palus and Vejmelka, 2007). Transfer entropy has been applied in both EEG (Chavez et al., 2003; Vakorin et al., 2010a) and MEG data (Vicente et al., 2011; Wibral et al., 2011), as well in functional magnetic resonance imaging (Hinrichs et al., 2006).

Differences in signal variability among brain areas constituting an activated network as a reaction to a cognitive or perceptual task, can be indicative of how that task is being processed in the brain (Miši´c et al., 2010;Vakorin et al., 2010b). In this study, we explored empirical aspects of the relations between complexity of individual sources constituting a network and the exchange of information between them. The analysis was performed under the assumption that the neuronal ensembles activated in performing the task can be represented by non-linear dynamic systems interacting with each other.

The first part of this study presents a data-driven pipeline for non-linear analysis of neuromagnetic sources reconstructed from human MEG data collected in reaction to face recognition task. Specifically, we first computed the asymmetries in mutual interdependencies between the original MEG sources using the conditional mutual information as a measure of information transfer. We then estimated variability of the MEG sources using the measure of sample entropy. Sample entropy was designed in essence as an approximation to the Kolmogorov entropy (Richman and Moorman, 2000), which can be interpreted as the mean rate of information generated by a dynamic system (Kolmogorov, 1959). Sample entropy can be used to infer the presence of non-linear effects. In practice, however, sample entropy is sensitive not only to non-linear deterministic effects but also to the linear stochastic effects such as, for example, auto-correlation. A number of studies indicate that the information averaged over a larger time horizon can reflect non-linear determinism with higher confidence (Govindan et al., 2007; Kaffashi et al., 2008). Multi-scale entropy represents an approach when sample entropy is estimated at different time scales (Costa et al., 2002). In this study, we explored how the differences in variability of the source dynamics, estimated at fine and coarse time scales, can be explained, in a statistical sense, by an asymmetry in the amount of information transferred from one source to another. In the second part of this study, using synthetic data based on a model of coupled non-linear oscillators with time delay in coupling, we demonstrated how the effects found in the MEG data, may arise from time delayed interactions.

# **2. METHODS**

# **2.1. PARTICIPANTS**

Twenty-two healthy adults (20–41 years, mean = 25.7 year, 9 female) took part in the study. None of the participants wore any metallic implants or had metal in their dental work and all reported normal or corrected-to-normal vision. Experiments were performed with the informed consent of each individual and with the approval of the Research Ethics Board at the Hospital for Sick Children.

# **2.2. STIMULI AND TASK**

Participants performed a one-back task in which they judged whether the currently viewed stimulus was the same as the previous one. The stimulus set comprised 240 grayscale photographs of unfamiliar faces of young adults (2.4˚ × 3˚ visual angle) with neutral expressions. All faces were without glasses, earrings, jewelry, or other paraphernalia. Male and female faces were equiprobable. In each block of trials, one-third of the faces immediately repeated. Thus, there were 120 new faces that either did or did not repeat on the subsequent trial (N1 and N2, 60 trials each), as well as 60 repeated faces (R) per block (180 faces in total). Upright faces were presented in one block and inverted faces in the other, with the order of the two blocks counterbalanced across participants. For more information on stimulus control please see Taylor et al. (2008). The tasks will be coded as invN1, invN2, invR, upN1, upN2, and upR.

# **2.3. MEG SIGNAL ACQUISITION**

The MEG was acquired in a magnetically shielded room at the Hospital for Sick Children. Head position relative to the MEG sensor array was determined at the start and end of each block using three localization coils that were placed at the nasion and bilateral preauricular points prior to acquisition. Motion tolerance was set to 0.5 cm. Surface magnetic fields were recorded using a 151-channel whole-head CTF system (MEG International Services, Ltd., Coquitlam, BC) at a rate of 625 Hz, with a band pass of DC-100 Hz. Data were epoched into [−100 1500] ms segments time-locked to stimulus onset. Structural Magnetic Resonance Imaging (MRI) data were also acquired for each participant. Following the MEG recording session, the three localization coils were replaced by MRI-visible markers and 3D SPGR (T1-weighted) anatomical images were acquired using a 1.5-T Signa Advantage system (GE Medical Systems, Milwaukee, WI).

# **2.4. EXTRACTION OF NEUROMAGNETIC SOURCES**

Individual anatomical MR images were warped into a common Talairach space using a non-linear transform in SPM2. Latencies of interest were chosen from the group average event-related fields (ERFs). Source analysis was performed using event-related beamforming (ERB; Robinson and Vrba, 1999; Sekihara et al., 2001; Cheyne et al., 2007), a 3D spatial filtering technique which is used to estimate instantaneous source power at desired locations in the brain. To model the forward solution for the beamformer, multiple sphere models were fit to the inner skull surface of each participant's MRI using BrainSuite software (Shattuck and Leahy, 2002). Activity at each target source was estimated as a weighted sum of the surface field measurements. Weight parameters and the orientation of the source dipole were optimized in the least squares sense, such that the average power originating from all other locations was maximally attenuated without any change to the power of the forward solution associated with the target source. The weights were then used to compute single-trial time series for each source.

Two prominent peaks sensitive to facial orientation were observed at 100 ms and 150 ms following stimulus onset (**Figure 1A**) and were localized bilaterally to the primary visual cortex (**Figure 1B**, sources 1 and 2) and bilaterally to fusiform gyrus (**Figure 1B**, sources 3 and 4), respectively. A third, less prominent peak was observed at 220 ms (**Figure 1C**) and was most affected by the memory manipulation (i.e., it differed most between the first presentation of a face and its repeat). To avoid any confounding interaction between the effects of face inversion and working memory, the N2-R difference waves were computed and localized separately for Upright and Inverted faces (**Figure 1D**, sources 5 and 6, respectively). Both were localized to the anterior cingulate cortex. Thus, neuromagnetic activity was extracted from all six source locations, in all six conditions. For the purpose of this paper, the sources were coded as follows: (1) *VISL*; (2) *VISR*; (3) *FUSL*; (4) *FUSR*; (5) *ACCUP*; (6) *ACCINV*.

#### **2.5. INFORMATION GENERATED BY A SYSTEM**

Many complex biophysiological phenomena are due to non-linear effects. Recently there has been an increasing interest in studying complex neural networks in the brain, specifically by applying concepts and time series analysis techniques derived from nonlinear dynamics (see Stam, 2005 for a comprehensive review on non-linear dynamical analysis of EEG/MEG). Various statistics quantifying signal variability based on the presence of non-linear deterministic effects, were developed to compare and distinguish time series. Among others, sample entropy was developed as a measure of signal regularity (Richman and Moorman, 2000). The sample entropy was proposed as a refined version of approximate entropy (Pincus, 1991), compensating for self-matches in the signal patterns. In turn, approximate entropy was devised as an attempt to estimate Kolmogorov entropy (Grassberger and Procaccia, 1983), the rate of information generated by a dynamic system, from noisy and short time series of clinical data.

One approach to non-linear analysis consists of reconstructing the underlying dynamical systems underlying EEG or MEG time series through time delay embedding. Specifically, let *x<sup>t</sup>* denote the delay vectors, describing recent history of the observed process *xt*:

$$\mathbf{x}\_t = \begin{pmatrix} \mathbf{x}\_t, \mathbf{x}\_t - \mathbf{r}, \dots, \mathbf{x}\_{t-\tau(d-1)} \end{pmatrix}^T \tag{1}$$

where *d* is embedding dimension, and τ is embedding delay measured in multiples of the sampling interval.

For estimating sample entropy of time series *xt*, two multidimensional representations of *x<sup>t</sup>* are used, as defined by two sets of embedding parameters: {*d*, τ } and {*d* + 1, τ }. Typically, the values of the time embedding delay τ are kept equal to 1, measured in data points of a given time series for which sample entropy is to be estimated. Sample entropy can be estimated in terms of the average natural logarithm of conditional probability that two delay vectors (points in a multi-dimensional state-space), which are close in the *d*-dimensional space (meaning that the distance between them is less than the scale length *r*), will remain close in the (*d* + 1)-dimensional space. A greater likelihood of remaining close results in smaller values for the sample entropy statistic, indicating fewer irregularities. Conversely, higher values are associated with the signals having more variability and less regular patterns in their representations.

Specifically, let *x*(*d*) *<sup>t</sup>* <sup>=</sup> (*xt* , *xt*−1, ... , *xt*−*d*+1)*<sup>T</sup>* represent *d*-dimensional (*N* − *m* − 1) delay vectors reconstructed from a time series *xt* of length *N*. The function *B<sup>d</sup> <sup>i</sup>* (*r*) is defined as 1/(*<sup>N</sup>* <sup>−</sup> *<sup>d</sup>* <sup>−</sup> 1) multiplied by the number of state vectors *<sup>x</sup>*(*d*) *j* located within *r* of *x*(*d*) *<sup>j</sup>* :

$$B\_i^d(r) = \frac{1}{N - d - 1} \sum\_{\substack{j \text{ such that} \\ j \neq i}}^{N - d} \Theta \left( r - \left\| \mathbf{x}\_i^{(d)} - \mathbf{x}\_j^{(d)} \right\| \right) \tag{2}$$

where *j* goes from 1 to *N* − *d*, and ||·|| stands for the maximum norm distance between two state vectors. Then, averaging across (*N* − *d*) vectors, we have

$$B^d(r) = \frac{1}{N-d} \sum\_{i=1}^{N-d} B\_i^d(r) \tag{3}$$

Similarly, the equivalent of *B<sup>d</sup> <sup>i</sup>* (*r*)in a (*d* + 1)-dimensional representation of the original time series *x*(*t*), the function *A<sup>d</sup> <sup>i</sup>* (*r*) , is given by 1/(*<sup>N</sup>* <sup>−</sup> *<sup>d</sup>* <sup>−</sup> 1) times the number of state vectors *<sup>x</sup>*(*d*+1) *j* located within *r* of *x*(*d*+1) *<sup>j</sup>* :

$$A\_i^d(r) = \frac{1}{N - d - 1} \sum\_{\substack{j \text{ such that} \\ j \neq i}}^{N - d} \Theta \left( r - \left\| \mathbf{x}\_i^{(d+1)} - \mathbf{x}\_j^{(d+1)} \right\| \right) \tag{4}$$

which can be averaged across (*M* − *n*) points as

$$A^d(r) = \frac{1}{N-d} \sum\_{i=1}^{N-d} A\_i^d(r) \tag{5}$$

Sample entropy is defined as

$$\text{SampEn}(d, r) = \ln\left[B^d(r)\right] - \ln\left[A^d(r)\right] = -\ln\left[\frac{A^d(r)}{B^d(r)}\right] \quad \text{(6)}$$

Multi-scale entropy (MSE) was proposed to estimate sample entropy of finite time series at different time scales (Costa et al., 2002). First, multiple coarse-grained time series are constructed

from the original signal. This is performed by averaging the data points from the original time series within non-overlapping windows of increasing length. Specifically, the amplitude of the coarse-grained time series *y*(θ) (*t*) at time scale θ is calculated according to

$$\chi\_t^{(\theta)} = \frac{1}{\theta} \sum\_{i=(t-1)\theta+1}^{i=t\theta} \chi\_i, \quad 1 \le t \le N/\theta \tag{7}$$

wherein the fluctuations at scales smaller than θ are eliminated. The window length, measured in data points, represents the scale factor, θ = 1, 2, 3,.... Note that θ = 1 represents the original time series, whereas relatively large θ produces a smooth signal, containing basically low frequency components of the original signal. To obtained the MSE curve, sample entropy is computed for each coarse-grained time series.

#### **2.6. INFORMATION TRANSFER**

A number of studies have used information-theoretic tools to characterize coupled systems (see Pereda et al., 2005 for a comprehensive review). Within this approach, predictive information transfer is a key concept used to define asymmetries in mutual interdependence (Palus et al., 2001; Lizier and Prokopenko, 2010). Information transfer *Ik*(*x* →*y*) is defined as the conditional mutual information *I*(*xt*, *yt* <sup>+</sup> *<sup>k</sup>*|*yt*) between the past and present of one system, *xt*, and a future of another system, *yt* <sup>+</sup> *<sup>k</sup>*, provided that information about the past and present of the second system, *y<sup>t</sup>* is excluded (Palus et al., 2001). The subindex *k* is used to designate dependence of the conditional mutual information *I*(*xt*, *yt* <sup>+</sup> *<sup>k</sup>*|*yt*) on the latency *k*, which typically is measured in units of data points. Thus, *I*(*xt*, *yt* <sup>+</sup> *<sup>k</sup>*|*yt*) can be considered as a function of the latency between the past and present of the first system and the future of the second one.

The measure *I*(*xt*, *yt* <sup>+</sup> *<sup>k</sup>*|*yt*) can be expressed in terms of individual *H*(·) and joint entropies *H*(·,·) and *H*(·,·,·) as follows:

$$\begin{aligned} I\_k(\mathbf{x} \rightarrow \mathbf{y}) &= I\left(\mathbf{x}\_t, \mathbf{y}\_{t+k}|\mathbf{y}\_t\right) = H\left(\mathbf{y}\_{t+k}, \mathbf{y}\_t\right) + H\left(\mathbf{x}\_t, \mathbf{y}\_t\right) \\ &\quad - H\left(\mathbf{y}\_{t+k}, \mathbf{x}\_t, \mathbf{y}\_t\right) - H\left(\mathbf{y}\_t\right). \end{aligned} \tag{8}$$

In a similar way, we can define the transfer of information from the past and present of the second system, *yt*, to the future of the first one, *x<sup>t</sup>* <sup>+</sup> *<sup>k</sup>*:

$$\begin{split} I\_k(\mathbf{y}\rightarrow\mathbf{x}) &= I\left(\mathbf{y}\_t, \mathbf{x}\_{t+k}|\mathbf{x}\_t\right) = H\left(\mathbf{x}\_{t+k}, \mathbf{x}\_t\right) + H\left(\mathbf{y}\_t, \mathbf{x}\_t\right) \\ &\quad - H\left(\mathbf{x}\_{t+k}, \mathbf{y}\_t, \mathbf{x}\_t\right) - H\left(\mathbf{x}\_t\right) . \end{split} \tag{9}$$

*I*(*xt*, *yt* <sup>+</sup> *<sup>k</sup>*|*yt*) or *I*(*yt*, *xt* <sup>+</sup> *<sup>k</sup>*|*xt*) are closely related to the statistic termed transfer entropy, a measure of the deviation from the independence property for coupled systems evolving in time (Schreiber, 2000). It can be shown that under proper conditions the transfer entropy is equivalent to the conditional mutual information (Palus and Vejmelka, 2007): *Ik*(*x* →*y*) =*Tk*(*x* →*y*).

Net transfer entropy or information transfer, *T*(*x* →*y*) = *Tk*(*x* →*y*) −*Tk*(*y* →*x*), can be used to infer the directionality of the dominant transfer of information between coupled systems. Positive *T*(*x* →*y*) would imply that the system *x<sup>t</sup>* has a higher predictive power to explain the time course of the system *yt*, than *vice verse*.

In estimating transfer entropy, the key issue is estimation of the entropies themselves. The straightforward approach is to divide the state-space into bins, *i* = 1, 2, 3,..., of some size δ and calculate the entropy of the multi-dimensional dynamics through constructing a multi-dimensional histogram, estimating probabilities of being in the *i*th bin. This study took another approach, as proposed by Prichard and Theiler (1995) and tested using linear and linear models (Chavez et al., 2003; Gourévitch et al., 2007). Specifically, individual and joint entropies *H*(*x*) are approximated by estimating the corresponding correlation integral *Cq*(*x*, *r*) computed as

$$H(\mathbf{x}) \approx -\log\_2 C\_q \left(\mathbf{x}, r\right),\tag{10}$$

where

$$C\_{q}\left(\mathbf{x},r\right) = \frac{1}{N\left(N-1\right)^{q-1}} \times \sum\_{s=1}^{N} \left[ \sum\_{s \neq t} \left( \Theta\left(r - \|\mathbf{x}\_{s} - \mathbf{x}\_{t}\|\right) \right) \right],\tag{11}$$

*N* is the number of data points, and is the Heaviside function. Specifically, the correlation integral *Cq*(*x*,*r*) is a function of a scale parameter *r*, which in general, can be related to the bin size δ, and the integral order *q*. The second order (*q* = 2) correlation integral, as used in this study, is interpreted as the likelihood that the distance between two randomly chosen delay vectors (points in the multi-dimensional state-space) is smaller than *r*.

#### **3. ANALYSIS 3.1. PIPELINE OF THE ANALYSIS**

The dynamics of the networks consisting of six sources were identified for 22 participants in 6 conditions, as described in the Section 1. To determine the optimal embedding parameters for reconstructing the delay vector from the observed time series, we applied the information criterion proposed by Small and Tse (2004). For most of the time series, with a few exceptions, the embedding window was estimated to be equal to 2, which implies the embedding dimension *d* = 2 (a two-dimensional system) and the embedding delay τ = 1. For each subject and condition, sample entropy was computed for the scales 1–20 for all of the single trials. The information rate produced by a system underlying the observed signal was computed by averaging the sample entropy statistic across the trials, as well as over some range of scale factors. Specifically, the information rate at fine time scales was estimated by averaging the first five scale factors, whereas the information rate of coarsegrained time series was computed by averaging the time scales 16–20. Thus, for a network of six sources, each source was associated with two values: information rate at fine and coarse time scales. For the purpose of this study, we use the terms variability, sample entropy and information rate interchangeably.

For the same networks, transfer entropy was computed as a function of the latency between the past of dynamics of one source and a future of the dynamics of another source (*k* = 1, 2,...,50), for all possible pairs of the sources (30 connections in total) and for all single trials. Following Palus et al., 2001, the transfer entropy was averaged across the latency *k* with the idea to decrease the variability of estimated statistics and to increase

the robustness of the results. Note that as the MEG epochs were relatively short, the transfer entropy was computed only at time scale θ = 1, which corresponds to the original time series. For each trial and pathway, the information transfer was estimated in both directions: *Ik*(*x* →*y*) and *Ik*(*y* →*x*), as described in Section 6. The net information transfer was computed as the difference between two amounts of transfer entropy, averaged across trials. Thus, for a network of six sources, each pathway between two sources was associated with a value of the net information transfer, reflecting the asymmetry in the predictive power between the source activity.

# **3.2. MEG DATA**

In **Figure 2**, the relations between asymmetry in mutual interdependence and variability are shown across subjects, separately for all the conditions. Specifically,the figure shows the net information transfer between two sources as a function of difference in sample entropy computed at fine (**Figure 2A**) and coarse (**Figure 2B**)

between the two measures (significant for all the conditions with *p*-values less than 0.001). A positive correlation implies that the net information is transferred from a source with higher sample entropy to a source with lower sample entropy. Negative correlations imply that more information is transferred toward a system with a higher sample entropy.

time scales, separately for each condition. Each point is associated with one subject and a pair of sources. Correlations between the two variables are given at the top of corresponding plots. In all the cases, the correlations are relatively strong (on the order of 0.5–0.8), statistically significant with *p*-values less than 0.001. Positive correlations in **Figure 2A** imply that a system with higher variability can better predict the behavior of a system with lower variability, than the other way around. Conversely, negative correlations observed in **Figure 2B** support the conclusion that at coarse time scales more information is transferred from sources with lower variability to sources with higher variability, than *vice versa*.

In addition to the relations between information transfer and complexity, it may be important to explore the connectivity maps of the networks based on neuromagnetic sources, in the context of the latencies between the peaks of the event-related fields (ERF). **Figure 3** illustrates the measures of transfer entropy for a pair of sources, shown as functions of the latency *k* between the future of one signal and the past of the other signal. **Figure 4** shows the reconstructed connectivity patterns masked by the bootstrap ratio maps, computed separately for six conditions. The significance of the couplings was estimating by bootstrapping the subjects (selection with replacement). The bootstrap ratio threshold of 3.0,which corresponds roughly to a 95% confidence interval, was used to define the connections which were robust across the subjects.

Connections can essentially be divided into two groups. One group represents the connections between the brain regions with the asymmetry in predictive power leading from right to left. Those are VIS*<sup>R</sup>* →VIS*L*, FUS*<sup>R</sup>* →FUS*L*, and FUR*<sup>R</sup>* →VIS*L*. The other group unites the connections with the net information transfer directed from the sources with smaller latencies in the peaks of ERF to those with larger latencies, such as VIS*<sup>R</sup>* →FUS*L*, VIS*<sup>R</sup>* →ACC*UP*, or FUR*<sup>R</sup>* →ACC*UP*.

**FIGURE 3 | Transfer entropy as a function of the time lag between the future of one signal and the past of the other signal, illustrated for a pair of sources.** The sources are taken from the same network for a given subject and condition. The errorbars are specified by the mean and standard error of the estimated measures across trials.

# **3.3. SYNTHETIC DATA**

In the previous section, we considered some empirical aspects of the interplay between sample entropy (information rate) and transfer entropy (information transfer) in the pairwise relations between the neuromagnetic sources. In the following section, we propose that such an interplay might be explained by coupling parameters, such as time delays or coupling strength, characterizing coupled non-linear dynamic systems. Our objective would be to demonstrate the same pattern of relationships between variability computed at different time scales and asymmetry in mutual interdependence between the original time series, using a simple computational model of interacting sources. Specifically, we will consider a model of coupled oscillators with time delay in coupling. We will show that such a model has a potential to explain the peculiarities we observed in **Figure 2**. The model we simulate is based on unidirectionally coupled chaotic Rössler oscillators.

Hadjipapas et al. (2009) used coupled Rössler systems to study collective dynamics in oscillatory networks as a simple case of periodic systems perturbed by a noise that has a deterministic rather than stochastic nature. Such systems represent a relatively simple non-linear system able to generate self-sustained non-periodic oscillations. In turn, oscillatory behavior and rhythms of the brain have been extensively studied as a plausible mechanism for neuronal synchronization (Varela et al., 2001). Under this context, the coupled Rössler oscillators can be viewed as a prototypical example of oscillatory networks.

Explicitly, the model reads

$$\begin{aligned} \frac{\mathbf{d}\,\mathbf{x}\_{1}}{\mathbf{d}\,t} &= -\omega\_{1}y\_{1} - z\_{1} + \varepsilon\,\mathbf{x}\_{2}\,(t-T) & \frac{\mathbf{d}\,\mathbf{x}\_{2}}{\mathbf{d}\,t} &= -\omega\_{2}y\_{2} - z\_{2} \\ \frac{\mathbf{d}\,\mathbf{y}\_{1}}{\mathbf{d}\,t} &= \omega\_{1}\mathbf{x}\_{1} + 0.15y\_{1} & \frac{\mathbf{d}\,\mathbf{y}\_{2}}{\mathbf{d}\,t} &= \omega\_{2}\mathbf{x}\_{2} + 0.15y\_{2} \\ \frac{\mathbf{d}\,\mathbf{z}\_{1}}{\mathbf{d}\,t} &= 0.2 + z\_{1}\left(\mathbf{x}\_{1} - 10\right) & \frac{\mathbf{d}\,\mathbf{z}\_{2}}{\mathbf{d}\,t} &= 0.2 + z\_{2}\left(\mathbf{x}\_{2} - 10\right) \end{aligned} \tag{12}$$

whereω<sup>1</sup> = ω<sup>2</sup> = 0.99 are the natural frequencies of the oscillators, ∈ is the coupling strength, and *T* denotes the delay in coupling. In the model, the dynamics of the first system determined by a behavior of three variables (*x*1, *y*1,*z*1) is the response driven by the second system based on a behavior of (*x*2, *y*2,*z*2). Further analysis is based on an assumption that only the dynamics of the variables *x*1(*t*) and *x*2(*t*) can be observed. Our specific goal is threefold: (i) to reconstruct the directionality of coupling between *x*1(*t*) and *x*2(*t*), (ii) to analyze the complexity of these signals, and (iii) explore relations between the complexity and causal information.

Numerical solutions of Eqs. (12) were obtained using the dde23 Matlab function (the Mathworks, Natick, MA) with a subsequent resampling of the time series with a fixed step 0.1. The dynamics were solved on the interval [0, 600], subsequently discarding the interval [0, 300] to avoid transitory effects. Thus, each time series had 3000 data points.

For a given pair of parameters, ∈ and *T*, the signals were generated 20 times. Analyses of sample entropy and transfer entropy were performed similarly to the pipeline for the analysis of the MEG data, as described in Section 1. The only difference was that for synthetic data, we had a network consisting of two systems, and

realizations of the model as an equivalent to trials. Transfer entropy between the two systems was computed for all the realizations, as functions of the past of system #1 and the future of system #2. The latency varied from 1 to 100 data points, which corresponded to the interval [0, 10]s. To obtain a value of the net information transfer, the difference between two amounts of transfer entropy was averaged across realizations and latency range. For the same data, sample entropy was computed as a function of scale factors 1–20. As in the MEG data analysis, the variability at fine time scales was estimated by averaging the sample entropy values across the first five scale factors, whereas the variability of coarse-grained time series was computed by averaging the sample entropy across the time scales 16–20.

First, we considered the influence of the time delay, *T*, varied on the interval [1, 20], with the coupling parameter ∈ fixed. The effects of its variability on complexity and information exchange are shown in **Figure 5**. The figure shows net transfer entropy (**Figure 5A**), differences in sample entropy at fine (**Figure 5B**)

and coarse time scales (**Figure 5D**) as functions of the time delay *T*. Note that, when we deal with real data, such relations cannot be observed as typically the true values of *T* are not known (see, however, Prokhorov and Ponomarenko, 2005; Silchenko et al., 2010; Vicente et al., 2011 for the attempts in recovering time delays in coupling). What we can observe is the correlations between the net transfer entropy and the differences in sample entropy shown in **Figures 5C,E**. The results revealed the presence of a relatively strong and robust linear correlation between the two statistics, similar to what we saw for MEG data in **Figure 2A**. However, the correlation observed in **Figure 5E** is close to zero and statistically insignificant, contrary to **Figure 2B**.

Similar to the time delay, the coupling parameter ∈ turned out to be able to explain, to some degree, the results in **Figure 2**. As expected, the net transfer entropy was found to be a monotonically increasing function of the coupling strength ∈, as shown in **Figure 6A**. Also, the difference in coarse-grained sample entropy was, at first approximation, a linear function of ∈, as shown in **Figure 6D**. In turn, this led to the negative correlation between the

complexity difference and net transfer entropy for all the values of the coupling parameter, as plotted in **Figure 6E**, in a good accordance with the results observed in **Figure 2B**. The influence of ∈ on the fine-grained sample entropy was ambiguous, as shown in **Figures 6B,C**. It is worth noting that **Figures 2A,B** are consistent with **Figures 6C,E**, respectively, only for a weak coupling.

# **4. CONCLUSION AND DISCUSSION**

In this paper, we examined relations between signal variability and asymmetry in mutual interdependencies between activated neuromagnetic sources. Variability was quantified based on sample entropy (Richman and Moorman, 2000), which is ultimately interpreted as the average rate of information generated by a dynamic system (Grassberger and Procaccia, 1983; Pincus, 1995). Using the concept of multi-scale entropy (Costa et al., 2002), we examined variability at fine and coarse resolutions of the same time series. Interdependencies between source dynamics was estimated using conditional mutual information between the past and present of one signal and the future of another signal, provided that the knowledge about the past and present of the second signal is excluded (Palus et al., 2001). The asymmetry in information transfer represent the differences in predictive power between sources, i.e., to predict the activity of each other.

The analyses of signal variability and information transfer were performed under an assumptions that neuronal ensembles involved in performing a task can be described by coupled nonlinear dynamic systems (Haken, 1996). Noise can be present at different levels of the non-linear models describing the observed time series. For the purpose of this study, we differentiate three types of noise-like activity. First, there is internal noise, which is an inherent component of a model, and is a part of the input entering the non-linear deterministic system. Second, we distinguish the variability in the signal generated by non-linear dynamic system. Finally, observational noise can be mixed with the output of the system.

This study focuses on exploring the variability in non-linear dynamics and describes this variability in its relations to the transfer of information in functional networks. Typically, there is the assumption that one observes non-linear systems in different states, and the goal is to describe these differences. Although different, two initial conditions would not be differentiated with certain experimental precision. However, they may evolve into distinguishable states after some finite time. Thus, one could say that a system that is sensitive to initial conditions produces information (Eckmann and Ruelle, 1985).

Sample entropy, which was used as a measure of variability, is closely related to the mean rate of information generated by a dynamic system underlying the observed signals. In practice, however, both linear stochastic and non-linear deterministic effects can contribute to the measure of sample entropy. A number of studies indicate that averaging the information rate over a larger time horizon allows one to alleviate linear effects, in particular, those associated with observational noise, and to focus on the signal variability due to the underlying non-linear determinism (Govindan et al., 2007; Kaffashi et al., 2008). Down-sampling of the original time series, as used in the multi-scale entropy approach, can be viewed as a way to extend the period over which the information generated by a system is averaged.

The first part of our analysis was based on the dynamics of neuromagnetic sources reconstructed from MEG data collected during a face recognition task. In the second part, we extended our empirical findings with an analysis of synthetic data based on the dynamics of coupled non-linear oscillators with time delay in coupling. We found that relations between sample entropy of the activity of neuromagnetic sources and the net information transfer between them depends on time scales at which the sample entropy is computed. Specifically, we found that more information is transferred from a source with a higher sample entropy at coarse time scales, but with a lower sample entropy at fine time scales.

Under certain conditions, analysis of the synthetic data offered a potential explanation our empirical findings. Specifically, a study of the system of two coupled oscillators with time delay in coupling revealed the same relations between the difference in sample entropy and asymmetry in information transfer. In particular, we found that the interplay between sample entropy-based on finegrained signals and information transfer can be explained, in a statistical sense, by the variability in the time delay in coupling. On the contrary, correlations between information transfer and sample entropy computed at coarse time scales were insignificant. In addition, we found that the variability in the coupling strength can contribute to the observed relations between the sample entropy-based on the coarse-grained signals and the information transfer. Taking into account that the coarse scales would better reflect non-linear effects, these results indicate that the variability of the signals due to non-linear determinism become more diversified as a result of the propagation of information in the network. In other words, propagation of information in a network may be described as accumulation of complexity (variability) of the brain signals. Similar results were found in (Miši´c et al., 2011), who showed that the variability of a region's activity systematically varied according to its topological role in functional networks. Specifically, the rate at which information was generated was largely predicted by graph-theoretic measures characterizing the importance of a given node in a functional network, such as the node centrality or efficiency of information transfer.

It would be worth discussing the differences between an analysis of transfer entropy, as performed in this study, and an analysis of causal relationships between the source activity. Lizier and Prokopenko (2010) suggested to distinguish information transfer and causal effects. Information transfer is defined as the conditional mutual information, representing the averaged information contained in the future of one process about the past of another process, but not in the past of the first process itself. In contrast, causal effect can be viewed as information flow quantifying the deviation of one process from causal independence on another process, given a set of variables that may affect these two processes of interest. Along a similar line of reasoning, Valdes-Sosa et al. (2011) differentiate predictive capacity between temporally distinct events and the effects of controlled intervention on the target process. Observing activity at a network node may potentially indicate its effects at remote nodes. However, identification of a physical influence upon a node at a given network assumes that any other physical influence that this node receives should be excluded.

Under this context, it should be emphasized that this study focuses on predictive information transfer, rather than on information flow. Using bivariate variant of information transfer, compared to the multivariate version, imposes a few limitations. First, it is impossible to distinguish between direct and indirect connections (Gourévitch et al., 2007). Specifically, confounding effects of indirect connections on estimation of transfer entropy were considered in Vakorin et al. (2009). Second, bivariate estimates of directionality in case of mutually interdependent sources may produce spurious results (Blinowska et al., 2004). With regards to this study, it should be noted that the issue associated with common sources is less of a problem in MEG than in EEG, as neuromagnetic signals do not suffer from volume conduction (Hämäläinen et al., 1993). However, in general, choosing an optimal set of variables constituting a network to analyze in a multivariate way remains an open issue. For example, it was shown that information-theoretic measures (transfer entropy), which in general does not require a model of interactions between nodes of a network, in contrast to autoregressive models, remain sensitive to model misspecification, wherein excluding a node from the analysis or adding a node affects the estimation of transfer entropy and robustness of the results (Vakorin et al., 2009).

# **ACKNOWLEDGMENTS**

This research was supported by research grants from the J. S. McDonnell Foundation to Dr. Anthony R. McIntosh. We thank

# **REFERENCES**


macroscopic aggregate signals? *Neuroimage* 44, 1290–1303.


Dr. Margot J. Taylor for sharing the MEG data. Also, we would like to thank Maria Tassopoulos and Tanya Brown for their assistance in preparing this manuscript.

noise is task-dependent and regionspecific. *J. Neurophysiol.* 104, 2667–2676.


Kuriki, H. Karibe, and N. Nakasato (Sendai: Tohoku University Press), 302–306.


Effective connectivity: influence, causality and biophysical modeling. *Neuroimage* 58, 339–361.

Varela, F., Lachaux, J. P., Rodriguez, E., and Martinerie, J. (2001). The brainweb: phase synchronization and large-scale integration. *Nat. Rev. Neurosci.* 2, 229–239.

Vicente, R., Wibral, R., Lindner, M., and Pipa, G. (2011). Transfer entropy – a model-free measure of effective connectivity for the neurosciences. *J. Comput. Neurosci.* 30, 45–67.

Wibral, M., Rahm, B., Rieder, M., Lindner, M., Vicente, R., and Kaiser, J. (2011). Transfer entropy in magnetoencephalographic data: quantifying information flow in cortical and cerebellar networks. *Prog. Biophys. Mol. Biol.* 105, 80–97.

**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 29 June 2011; accepted: 03 November 2011; published online: 23 November 2011.*

*Citation: Vakorin VA, Miši´c B, Krakovska O and McIntosh AR (2011) Empirical and theoretical aspects of generation and transfer of information in* *a neuromagnetic source network. Front. Syst. Neurosci. 5:96. doi: 10.3389/fnsys.2011.00096*

*Copyright © 2011 Vakorin, Miši´c, Krakovska and McIntosh. This is an open-access article subject to a nonexclusive license between the authors and Frontiers Media SA, which permits use, distribution and reproduction in other forums, provided the original authors and source are credited and other Frontiers conditions are complied with.*

# Laminar firing and membrane dynamics in four visual areas exposed to two objects moving to occlusion

#### *M. A. Harvey1 \* and P. E. Roland2*

*<sup>1</sup> Laboratory of Brain Research, Department of Neuroscience, Karolinska Institute, Solna, Sweden*

*<sup>2</sup> Department of Neuroscience and Pharmacology, University of Copenhagen, Copenhagen, Denmark*

#### *Edited by:*

*Claus C. Hilgetag, University Medical Center Hamburg-Eppendorf, Germany*

#### *Reviewed by:*

*Claus C. Hilgetag, University Medical Center Hamburg-Eppendorf, Germany Christopher I. Moore, Brown University, USA*

#### *\*Correspondence:*

*M. A. Harvey, Department of Organismal Biology and Anatomy, University of Chicago, Anatomy Room 100, 1027 E. 57th Street, Chicago, IL 60637, USA e-mail: michaelharvey@uchicago.edu* It is not known how visual cortical neurons react to several moving objects and how their firing to the motion of one object is affected by neurons firing to another moving object. Here we combine imaging of voltage sensitive dye (VSD) signals, reflecting the population membrane potential from ferret visual areas 17, 18, 19, and 21, with laminar recordings of multiunit activity, (MUA), when two bars moved toward each other in the visual field, occluded one another, and continued on in opposite directions. Two zones of peak MUA, mapping the bars' motion, moved toward each other along the area 17/18 border, which in the ferret maps the vertical meridian of the field of view. This was reflected also in the VSD signal, at both the 17/18 border as well as at the 19/21 border with a short delay. After some 125 ms at the area 19/21 border, the VSD signal increased and became elongated in the direction of motion in front of both of the moving representations. This was directly followed by the phase of the signal reversing and travelling back from the 19/21 border toward the 17/18 border, seemingly without respect for retinotopic boundaries, where it arrived at 150 ms after stimulus onset. At this point the VSD signal in front of the moving bar representations along the 17/18 border also increased and became elongated in the direction of object motion; the signal now being the linear sum of what has been observed in response to single moving bars. When the neuronal populations representing the bars were some 600μm apart on the cortex, the dye signal and laminar MUA decreased strongly, with the MUA scaling to that of a single bar during occlusion. Despite a short rebound of the dye signal and MUA, the MUA after the occlusion was significantly depressed. The interactions between the neuronal populations mapping the bars' position, and the neurons in between these populations were, apart from 19/21 to 17/18 interaction, mainly lateral-horizontal; first excitatory and inducing firing at the site of future occlusion, then inhibitory just prior to occlusion. After occlusion the neurons that had fired already to the first bar showed delayed and prolonged inhibition in response to the second bar. Thus, the interactions that were particular to the occlusion condition in these experiments were local and inhibitory at short cortical range, and delayed and inhibitory after the occlusion when the bars moved further apart.

#### **Keywords: feedback, lateral interactions, visual cortex, visual motion, voltage sensitive dye**

When an object is moving into the visual field of view and the retina is still, the object is mapped in several visual areas as moving peak firing rates (Motter and Mountcastle, 1981; Harvey et al., 2009). This mapping is associated also with moving increases in the population membrane potential (Jancke et al., 2004; Yang et al., 2007; Harvey et al., 2009). Increases in the population membrane potential reflect the local dynamics of the neurons, but also dynamics influenced by activity in higher visual areas, (Harvey et al., 2009). Not only feed-forward input, but also lateral horizontal neuronal computations and action potentials from higher order areas influence visual perception and firing rates in the primary visual cortex (Gilbert and Wiesel, 1989; Lamme, 1995; Bosking et al., 1997; Bringuier et al., 1999; Buzas et al., 2006; Roland et al., 2006; Roland, 2010). The relative weights of these three inputs are still debated. Feed-forward input from the lateral geniculate nucleus affects the computations of neurons primarily located in layer 4 of primary visual cortex, (Maunsell and Gibson, 1992; Hirsch et al., 2002). Horizontal interactions within an area may take place in supra- and infra-granular layers, and these layers are also assumed to be the target of back projecting axons from higher areas. It is known that another object outside a receptive field occupied by an object can influence the spiking from that receptive field (Jones, 1970; Bishop et al., 1973; Allman et al., 1985). The other object can facilitate or inhibit the spiking from the receptive field, depending on its distance from the field (Jones, 1970; Bishop et al., 1973). The current picture is that long-range horizontal connections are excitatory

**Abbreviations:** MUA, multiunit activity; VSD, voltage sensitive dye; SRP, spatially restricted pre-depolarization.

for similar stimulus orientations whereas short-range axons have no orientation preference and are mainly inhibitory, (Tucker and Fitzpatrick, 2003). Visually moving objects add complexity to this picture. A moving object, as a stationary object, elicits a laterally spreading excitation in the supragranular layers, but this is soon superimposed upon by excitation and firing ahead of the object mapping in the direction of object motion, (Harvey et al., 2009).

When a single object moves in the field of view and the eyes don't move, the object is mapped in a retinotopic fashion in visual area 17 as increased multiunit activity (MUA) and increased membrane potential by one population of neurons (Jancke et al., 2004; Harvey et al., 2009). This population forms a path in the cortex, corresponding to the trajectory in the field of view. For each position of the object in the field of view, there is one sector of this population where the object is mapped as the peak of the MUA and membrane potential increase (Harvey et al., 2009). However, after some 130 ms, the neurons not yet having mapped the object, start to fire and increase their membrane potential as far as 8◦ ahead, thus marking the future trajectory of the peak activity across cortex. (Harvey et al., 2009). The present study is an extension of the Harvey et al. (2009) study with the purpose of revealing the cortical dynamics elicited by two bars moving in opposite directions.

When more than one object traverses the visual field simultaneously, neurons firing to one object might be affected by other neurons firing to another object in a way that is not possible to predict from the dynamics associated with a single moving object. This happens for example when two objects move to occlude or partially occlude one another (Adelson and Movshon, 1982). This situation is common in natural scenes. A moving object is mapped by spatially extended populations of neurons in all layers of the visual cortex (Harvey et al., 2009), but to our knowledge the spatio-temporal representations and interactions between two continuously moving objects has not been examined neurophysiologically. We used a simplified visual scene consisting of a gray background and two objects, white bars, moving in opposite directions to occlude one another (**Figure 1**). We examined the relative changes in population membrane potential in supragranular layers of areas 17, 18, 19, and 21 with voltage sensitive dyes (VSDs), as well as the laminar MUA in areas 17 and 18 of ferrets.

From the single bar dynamics, we predicted that four populations of neurons would map the two bars in the cortex, i.e., two representations, one of each bar, at the 17/18 border and two other representations at the 19/21 border. Based on previous work (Harvey et al., 2009), we expected to see evidence for communications between areas 19/21 and 17/18. Finally based on Harvey et al. (2009) we expected an increase in membrane voltage ahead of the representations along the future cortical trajectory of object motion. Our results confirmed these predictions, and open the possibility that populations of neurons could provide an advanced signal of the location of an upcoming occlusion.

We also tested the hypotheses that inter-area and cross-area interactions would be non-linear. To our surprise we found both linear and non-linear interactions. The dynamics of the cortical interactions prior to occlusion depended on the distance between the objects, but after the occlusion they were largely independent of the distance between objects. The interactions started early at a long range and initially they were net excitatory. Close to occlusion the population membrane potential showed strong net decreases of excitation, or alternatively net increases

left hemisphere of the ferret brain with visual areas 17, 18, 19, and 21. The cortex monitored by the hexagonal photodiode array is delimited in black. The cytoarchitectural borders between areas 17 and 18 and between areas 19 and 21 correspond to the mapping of the vertical meridian in the field of view on the cortex. The two white dots mark the expected mappings of the center of field of view. The relation

center of field of view mapping) varies somewhat between animals. Each diode picks up a signal from a cortical spot of 150 μm in diameter. MED, medial; LAT, lateral; and ANT, anterior direction. **(B)** The stimulus conditions and timing of the occlusion. All stimulus conditions were compared to a gray screen (baseline condition). All latencies relate to the appearance of the stimuli on the display screen.

in inhibition. After the occlusion, the previous mapping of the objects continued to affect the MUA.

# **MATERIALS AND METHODS**

## **ANIMALS**

All experimental procedures were approved by the Stockholm Regional Ethics Committee and were performed according to European Community guidelines for the care and use of animals in scientific experiments. Recordings were performed in 14 adult, female ferrets. Ferrets were initially anesthetized with Ketamin (15 mg kg−1) and Medetomidine (0.3 mg kg−1) supplemented with Atropine (0.15 mg kg−1). After the initial anesthesia ferrets received a tracheotomy and were ventilated with 1:1 N20:02 and 1% Isoflurane. The arterial pCO2 (partial pressure of CO2) was maintained between 3.5 and 4.3 KPa. A craniotomy was made exposing the left hemisphere visual areas 17, 18, 19, and 21 and was covered with a chamber affixed to the skull with dental acrylic. Animals were paralyzed with pancuronium bromide (0.6 mg kg−1), the left eye was occluded, and the right eye had its pupil dilated (1% atropine), nictating membrane retracted (10% Phenylephrine), and was then fitted with a zero power contact lens.

### **STIMULATION AND IMAGING**

A reverse ophthalmoscope was used to record the position of the optic disk and center a video monitor to the area centralis. Known cortical landmarks were then used to guide a single electrode penetration at the estimated crossing of the vertical and horizontal meridian. The receptive field (RF) at this area was then mapped using an m-sequence method, (Reid et al., 1997). The monitor position was then further adjusted so as to be precisely centered to this RF location. The cortex was stained for 2 h with the VSD RH1838 (0.53 mg ml−1; *<sup>n</sup>* <sup>=</sup> 3) or RH1691 (0.53 mg ml−1; *<sup>n</sup>* <sup>=</sup> 11) (Optical Imaging, Rehovot, Israel). After staining, the cortex was rinsed with artificial cerebral spinal fluid, the chamber was filled with silicon oil and sealed with a cover glass. Imaging was centered on the initial recording site and acquired using a 464-channel photodiode array, (H469-IV WuTech Instruments Gaithersburg, MD) through a macroscope fitted with a 5× objective (Red Shirt New Haven, CT). Images were acquired at a rate of 1.6 kHz, stimulus presentation was synchronized to the ECG signal, and respiration stopped during stimulus presentation. Stimuli were presented in a pseudorandom order on a video monitor with a refresh rate of 120 Hz located 57 cm in front of the animal. Stimuli were controlled using a VSG series IV system (Cambridge Research Systems, Kent UK). Stimuli consisted of 1 <sup>×</sup> <sup>2</sup>◦ horizontal bars (64.5 cd m−2) on a homogenous gray background (7.2 cd m−2). There were three stimulus conditions: (1) upward and (2) downward moving bars *originating* 10.5◦ below and 10.5◦ above the center of field of view (CFOV), respectively, and moving a total of 21◦ with a velocity of 25.4◦ s <sup>−</sup><sup>1</sup> for a period of 825 ms with start and end points equidistant from the screens center. (3) In the occlusion condition, upward and downward moving bars were presented simultaneously 10.5◦ below and 10.5◦ above the CFOV moving toward each other with a velocity of 25.4◦ s <sup>−</sup><sup>1</sup> for a period of 392 ms. At 392 ms the bars abutted one another, such that for a

short moment, 8 ms, they occupied 2 × 2◦ square in the CFOV (**Figure 1**). Then the bars began to occlude one another until there was, at 412 ms, only the image of a single bar at the center of the screen. From 412 to 432 ms this central bar grew until again the bars were at an abutting position. From 432 ms until 825 ms the bars moved away from each other until they reached their final positions *10.5*◦ *below and 10.5*◦ *above the CFOV* at which point they disappeared.

# **ELECTROPHYSIOLOGY**

Electrode penetrations were made perpendicular to the cortical surface along the estimated course of the vertical meridian using single shank, 16 channel, laminar probes (NeuroNexus, Ann Arbor, MI) with recording site resistances of ∼3 M*-*, and separated by 100μm. Signals were routed through an RA16AC head stage to an RA16PA Medusa preamplifier and amplified at 40 K using the RA16 Medusa Base station (Tucker-Davis Technologies, Alachua, FL). For multiunit recordings the signal was digitally band pass filtered between 100 Hz–10 KHz and for local field potential recordings between 1 and 10 KHz. Signals were acquired and written to a hard-drive using CED power 1401 AD-converter and Spike 2 Software (Cambridge Research Systems). All subsequent analysis was done using Matlab R13 (The MathWorks, Natick, MA). At each recording site receptive fields were first mapped using the methodology noted above.

### **CYTORARCHITECTONICS AND FUNCTIONAL RETINOTOPY**

At the end of the experiment three vertical needle marks were made around the recorded area, the animals were sacrificed (pentobarbital) and perfused transcardially with 4% paraformaldehyde. Brains were sectioned and alternate 50μm sections were stained for Nissl and cytochrome oxidase. Areal borders were then reconstructed using cytoarchitectonic landmarks (Innocenti et al., 2002), and these borders were mapped onto the image of the cortical surface in each animal. After reconstruction, the cytoarchitectural borders of individual animals were aligned by simultaneous standard affine transformations as described in Harvey et al. (2009) (see **Movies S1**, **S3**).

Although recordings of the optical intrinsic signal in many cases is helpful in showing the border between areas 17 and 18, this method has not consistently shown the borders between area 18 and 19, nor the borders between area 19 and 21 or between area 21 and the suprasylvian area in the ferret. We used the VSD signal, *V(t)*, to determine the location of the stimulusinduced peak of *V(t)* in all four areas according to the method devised by Kalatsky and Stryker (2003). In the control conditions the bar moved upwards in half the trials and downwards in the other half. Thus, the bar would reach an identical position on the screen from two different directions. As each position on the screen corresponds to one cortical position along the 17/18 border, we could estimate how the bar would be mapped on the cortex if there was no delay between its position on the screen and its mapping as the *V(t)* peak on the cortex (Kalatsky and Stryker, 2003). **Figure A1** illustrates this procedure and **Movie S1** panel **C** shows the locations of our peak *V(t)* estimate. This map, color coded in **Figure A1**, then serves as an additional independent mapping of the retinotopy obtained from a bar moving along the vertical meridian. The velocity of the moving representation over the cortex predicted from this mapping without delay was 25.43◦ s <sup>−</sup>1, (s.e.m. 1.1◦ s <sup>−</sup>1, *<sup>n</sup>* <sup>=</sup> 4), which might serve as a further validation of the recording procedures. It is apparent that the bar mappings in visual areas 17, 18, 19, and 21 always lag the position of the object mapped without delay. This non-delayed representation was on average 50 ms (s.e.m. = 4 ms; *n* = 4) ahead of the maximal *V(t)* and the maximal MUA firing rate. Note that this procedure uses information from all imaged cortical points. For further details see **Figure A1**.

# **DATA ANALYSIS**

# *Treatment of the VSD signal*

All VSD signals were analyzed in terms of fractional fluorescence, the details of which have been described elsewhere (Roland et al., 2006; Eriksson et al., 2008). In brief, the signal in the blank (background alone) condition is subtracted from the signal of the stimulus conditions and divided by the background fluorescence to yield the fractional fluorescence (*F/F*0) referred to here as *V(t)*.

Two types of normalization procedures were used, normalization to the maximum *V(t)* value in time, *V(t)*rel*(t)* = *V(t)/* max*t*[*V(t)*] and normalization the maximum *V(t)* value in space, *V(t)*rel*(s)* = *V(x, y, t)/* max*x,y*[*V(x, y, t)*]. For the spatial normalization this meant that for each frame of our VSD measurement (0.616 ms) the *V(t)* from the photodiode with the highest value would be set to 1 and the *V(t)* from all other diodes would be relative to that. Finally a specific additional normalization scheme was used. In this procedure the *V(t)* from each diode is made relative to itself within a 25 ms sliding window, such that for each time point *V(t)*rel = *V(x, y, t)/* max*t*{−12*.*5*,*12*.*5}[*V(x, y, t)*]. Using this scheme we can then monitor when the *V(t)* at each diode reaches its maximum relative to its self, rather than relative to the *V(t)* of the surrounding diodes.

We selected five sites along the 17/18 cytoarchitectural border and hence the path of the moving *V(t)* maximum at the 17/18 border, and three sites along the 19/21 cytoarchitectural border and the path of the moving *V(t)* maximum at the 19/21 border for detailed analysis. The purpose was to examine whether the *V(t)* and MUA added linearly until the time of occlusion, with some caveats. First we have no systematic multiunit recordings at the 19/21 border. Second, while the distance from the site mapping the CFOV could always be calculated precisely for the *V(t)*, the location of the individual electrode penetrations were subject to some variability. This is obvious in the average MUA recorded at the central location (Site 3 in **Figure 3** MUA), where the average time of arrival of the peaks in the downwards and upwards movement conditions are offset by some 72 ms. This would correspond to an average lateral displacement of our penetrations at this site of 180μm. Thus, in order to test whether the occlusion condition generated a larger amplitude of *V(t)* and MUA than that for the single moving bar conditions we compared the *V(t)* and MUA only to the larger of the two amplitudes generated by the control conditions for each temporal bin. Similarly when testing if the occlusion condition generated a weaker response than that for the control conditions, we compared it to the lesser of the two amplitudes of the control conditions for each bin. When comparing response onset and peak times, we always chose for statistical comparison the control condition that had the earliest onset or the earliest peak, respectively. As the positional error of the electrodes at the center was lateral, this meant that our comparisons at the center are biased toward the downward movement condition. However, data from individual animals for which there was no such discrepancy in the electrode positions do not contradict our main results as shown in **Figure A2**. Finally, primarily due to differences in the cortical vasculature, it was rare for two electrode penetrations in different animals to be at exactly the same distance from the cortical site mapping the CFOV, therefore the distances from center for the MUA are described as mean values in **Figure 3**.

### *Calculating significance*

Using the amplitude fluctuations in the pre-stimulus interval to define the noise level, the *V(t)* was thresholded at *p <* 0*.*01 of being noise. In this we assumed the amplitude fluctuations to be not significantly different from a Gaussian distribution. This significance threshold applied for single photodiode channels and small regions of interest consisting of three channels (see below and **Figures 3**–**6**). Once statistically significant epochs of *V(t)* changes were determined for a particular region of interest, the timing of the first significant frame in the first post-stimulus epoch *>* 10 ms was the onset latency, thus calculated from the start of the stimulus. The peak latency was simply the mean peak time calculated across animals from stimulus onset (in a statistically significant epoch).

Statistics for the whole photodiode array of 464 channels. A threshold of estimated *p <* 0*.*01 was set for each photodiode detector channel and divided by the number of channels (464) to give the Bonferroni corrected value of *p <* 0*.*01.

Statistical comparisons between conditions are corrected for mass significance with a false discovery rate of 0.01 (Benjamini and Hochberg, 1995). For the movies the pre-stimulus *V(t)* or d[*V(t)*]/dt was thresholded with a global *p <* 0*.*025 or *p <* 0*.*01; this threshold was used on the post-stimulus *V(t)* or d[*V(t)*]/dt. The results are movies and snapshot sequences of only statistically significant membrane events.

In order to calculate significant responding in the MUA, a Poisson distribution was fitted to the spike trains in the prestimulus period *and* spikes from the background trial. Spike trains passing both the criterion of having significantly increased discharge rate compared to the pre-stimulus period of *p <* 0*.*01 *and* increased rate compared to the background condition of *p <* 0*.*01, were considered statistically significant periods of firing.

Once statistically significant epochs of MUA were determined for a particular region of interest, for the number of animals in which the region was exposed, the timing of the first frame in the first post-stimulus significant epoch *>* 10 ms was the onset latency, thus calculated from the start of the stimulus. The peak latency was simply the mean peak time calculated across animals from the stimulus start (in a statistically significant epoch).

# **RESULTS**

VSDs are potentiometric dyes that bind non-specifically to all cell membranes, (Grinvald and Hildesheim, 2004). Changes in the fluorescence of these dyes has been shown to bear a near linear relationship to changes in membrane voltage, recorded intracellularly *in vivo* from cells in superficial cortical layers, (Petersen et al., 2003; Ferezou et al., 2006; Berger et al., 2007), as well as *in vitro*, (Cohen et al., 1974; Ross et al., 1977). We used VSDs in order to image the spatiotemporal evolution of the relative population membrane potential in the supragranular layers of ferret visual areas 17–19 and 21, when ferrets viewed two identical luminance bars moving toward each other along the vertical meridian of the field of view. In the ferret, the vertical meridian of the field of view is represented along both the cytoarchitectonic border separating visual areas 17 and 18, and along the border separating areas 19 and 21, (Manger et al., 2002), (**Figure 1**). Subsequently we used the results from the VSD imaging to guide the placement of a laminar electrode along the 17/18 border, and thus along the predicted path of activity evoked by the moving bars.

The VSD signal, *V(t)*, is the difference between the fluorescence recorded during a stimulus condition and that recorded during the baseline, gray screen, condition divided by the fluorescence obtained in darkness (Materials and Methods). According to a recent estimate, approximately 90–95% of this difference signal reflects the difference in synaptic activity (Berger et al., 2007). The spatio-temporal dynamics of the *V(t)* and the MUA associated with the motion of a single bar is relatively complex. For this reason, we shortly summarize the results of the control conditions [for further details see Harvey et al. (2009)].

# **SPATIO-TEMPORAL DYNAMICS OF THE MEMBRANE POTENTIAL AND MULTIUNIT ACTIVITY CHANGES ASSOCIATED WITH THE MOTION OF A SINGLE BAR**

There were two control conditions, (**Figure 1B**), in which a single bar moved upwards or downwards along the vertical meridian of the field of view. As the retina was always stationary, the MUA and *V(t)* maximum moved over the cortex. Due to the diverse dynamics of the *V(t)* when a single moving object enters the field of view, the peak MUA has been considered as the best estimate of the cortical position receiving the retinal signal of the moving bar (Harvey et al., 2009). When the bar moved upwards, the peaks of the MUA and *V(t)* moved laterally along the cytoarchitectural border between areas 17 and 18, with a weaker second *V(t)* peak moving laterally along the cytoarchitectural border separating areas 19 and 21 (Harvey et al., 2009). When the bar moved downwards, the peaks of the MUA and *V(t)* moved medially along these cytoarchitectural borders **Movie S1** panel **D**; **Figure 2**). The representation of the bar in visual areas 17, 18, 19, and 21 always lags the position of the object mapped without delay. This non-delayed representation was on average 50 ms (s.e.m. = 4 ms; *n* = 4) ahead of the maximal *V(t)* and the maximal MUA firing rate. See Materials and Methods, **Figure A1** and **Movie S1** panel **C**. We had no systematic electrode penetrations along the 19/21 cytoarchitectural border.

Thus the motion of a single bar is associated with MUA peaks moving in retinotopic cortical coordinates corresponding to the position of the bar in the field of view (**Figure 3**).

After the bar appears in the field of view the signal from the VSD increases at cortical sites representing the bars' position and then spreads out in all directions, as has been previously demonstrated (Grinvald et al., 1994; Slovin et al., 2002; Roland et al.,

between the representation sites of the two bars at the 19/21 border, this is the slender spatially restricted pre-depolarization, SRP and the beginning of a similar SRP is seen along the 17/18 border (150 ms). After 450 ms the amplitude of the *V (t)* decreases. Notably, we did not perform specific subtractions of the *V (t)* from bars with other orientations, and consequently no orientation specific domains can be seen in the figures or movies as the orientation specific part of the *V (t)* amounts to maximally 15% (Sharon et al., 2007). Data filtered with a σ = 20 frame temporal filter. The snapshots cannot reveal the full dynamics, for this the reader should look at the **Movies S1**–**S3**.

2006; Harvey et al., 2009; Polack and Contreras, 2012). At ∼ 120 ms, the neurons in areas 19 and 21 produce a *V(t)* increase extending in the direction of motion far ahead of the peak activity mapping the bars' position, (compare **Movie S1** panels **D,C)**. This, increase in the population relative membrane potential propagates back to reach the representation of the bar at the 17/18 area border (see Harvey et al., 2009 Figure 5 and Supplementary Movie 7 in that paper). At the 17/18 border a similar slender *V(t)* increase is then produced extending on average 8◦ ahead of the mapping site (**Movie S1** panel **D**, 170–205 ms). This slender increase was roughly restricted to the cytoarchitectural border between areas 17 and 18 and hence to the future trajectory of the bar representation. It was therefore referred to as a spatially restricted pre-depolarization, (SRP). Corresponding to the SRP, there was an increase in the MUA recorded from neurons in layers 5, 6, i.e., ahead of those neurons mapping the bar. (see Harvey et al., 2009, Supplementary Movie 7). This MUA increase is more moderate, such that it is easily distinguished from the peak MUA associated with the mapping of the bar. When the neurons spike ahead of those neurons mapping the retinal input, it is like a prediction of the direction the object will move (**Figure 2**; **Movie S1** panel **D**).

# **THE SPATIO-TEMPORAL DYNAMICS ASSOCIATED WITH THE TWO BARS MOVING TOWARDS EACH OTHER**

In the occlusion condition, the two bars moved toward each other along the vertical meridian, one from above and the other from below the center of the field of view (**Figure 1**). The spatiotemporal dynamics contained all the characteristics just described for single bar motion, but in double. First two cortical sites of initial increased MUA appeared at the cytoarchitectural border between area 17 and 18 corresponding to the introduction of the bars in the field of view (data not shown). In what follows, we, in accordance with Harvey et al. (2009), use the terms representation and mapping to mean the peak firing in the MUA. Then the *V(t)* increased at these two sites and the increase spread laterally. Thereafter two additional sites of *V(t)* increases appeared along the cytoarchitectural border between areas 19 and 21, most likely as the result of a feed-forward input from areas 17/18. From 115 ms and onwards, two SRP's extended toward each other along the 19/21 border between the two representations of the bars, (**Figure 2**, **Movie S1** panel **B**). From 115 to 160 ms post stimulus, the two SRP's merged along the 19/21 border. This was followed by an increase of the *V(t)*rel, (see Materials and Methods) that propagated from the 19/21 border toward the two representation sites at the 17/18 border, i.e., back propagating synaptic activity (*p <* 0*.*01, **Movie S2**). At ∼160 ms two similar SRP's appeared extending toward each other in between the moving representations of the bars at the 17/18 area border, (**Movie S1** panel **B**, **Figure 2**).

From 286 ms the MUA increased significantly above the spontaneous rates in the neuron population representing the center of field of view (*p <* 0*.*01; **Table 1**, **Figure 3**).

Thus the activity during the occlusion condition, while the neuronal populations representing the two bars were still well-separated, displayed all the dynamic characteristics associated with the motion of single bars. That is: moving peak MUA over the cortex, activation of areas 19 and 21 following activation of areas 17/18, SRPs first in areas 19/21 and then after synaptic activity moving from here to areas 17/18 similar SRPs and advanced MUA along the 17/18 border. The only difference was that the SRPs merged and fired neurons at the cortical site representing the center of field of view, i.e., the site of occlusion. This indicates that the brain at this moment could have knowledge about the upcoming clash or occlusion, 126 ms in advance of the occlusion on the display screen (**Table 1**).


# **THE APPROACHING PHASE UNTIL OCCLUSION STARTS**

Since the dynamics in visual areas 17, 18, 19, and 21 of the *V(t)* (population membrane potential) and MUA up to 370 ms had all the characteristics of the dynamics associated with single bar motion, the dynamics up to 370 ms of the two bars moving toward each other might be a simple combination of single bar dynamics. We therefore tested the hypothesis that the occlusion condition *V(t)* and MUA was the simple sum of the *V(t)* and MUA in the two control conditions.

We selected five sites along the 17/18 cytoarchitectural border, and hence the path of the moving *V(t)* maximum at the 17/18 border, and three sites along the 19/21 cytoarchitectural border for detailed analysis. We calculated the sum of the *V(t)* for the control conditions and the sum of the MUA for the control conditions and tested when the amplitude of the *V(t)* and MUA for the occlusion condition deviated significantly from these sums. The tests of significance were done using a two tailed *t*-test using a within subjects design. The time courses of the MUA and *V(t)* at these sites are shown in **Figure 3** for the three motion conditions as well as for the sum of the *V(t)* and the sum of MUA of the control conditions.

**Figure 4** shows the results of the statistical tests for the *V(t)* as well as for the MUA in the supragranular, granular, and infragranular layers. First, the *V(t)* in between the mapping populations of neurons added not significantly different from the sum of *V(t)*s in the two control conditions, from 80 ms and onwards to occlusion (392 ms). During most of this time interval, the *V(t)* in the occlusion condition was significantly larger than that of either of the single bar conditions. Thus, the

(I) cortical layers at five sites along the 17/18 border **(Top)** and for the *V (t)* at three sites along the 19/21 border, **(Bottom)** (Sites identical to those in

linear addition hypothesis could not be refuted. For the rate of the MUA, there were epochs of supra-linear summation during which the firing in the infragranular and granular layers exceeded that associated with the sum of the single bar conditions at a population of neurons 540μm from the retinotopic point of the center of field of view (**Figure 4**). Unlike the *V(t)*, the sum of MUA lateral and medial to the representation of the center of field of view was roughly equal to the MUA of a single bar. This was because the presence of the other bar contributed little to the total MUA at these most lateral and medial positions (**Figure 3**). In cortex where the center of field of view was represented, however, there was an increase of the MUA in the infragranular layers just prior to occlusion that was significantly larger than that associated with a single bar (**Figure 5**).

So, whereas one could not refute that the population membrane potentials in the supragranular layers in between the cortical mapping of the two bars added linearly, the MUA was either supra-linear in short epochs or not significantly different from that associated with the similar motion of a single bar, as also seen in **Figure 3**. In particular, the addition of the *V(t)* in supragranular layers did not lead to an increase in the MUA in these layers. It should be noted that the dye signal adds if a larger area of membranes become excited at any measuring point, or, conversely, if already excited dendrites undergo further excitation. Thus, the addition of the *V(t)* signal in itself does not imply that the membranes of the dendrites and neurons already excited from one side (say lateral) are identical to those excited from the other side (say medial).

If the *V(t)* adds at a given cortical point, the onset latency may diminish compared to the single bar condition. This

condition is significantly less (*p <* 0*.*01) than the amplitude of the sum of the *V (t)* or sum of MUA in the single bar conditions are shown in blue.

**Frontiers in Systems Neuroscience www.frontiersin.org** June 2013 | Volume 7 | Article 23 |

happened at three measurement points, the point mapping the center of field of view and those two points flanking the center of the field of view at 540μm (**Table 1**). Also the onset of MUA occurred earlier at the cortical site representing the center of field of view (**Table 1**). At this site, both the onset (286 ms) and peak time of the MUA came equally earlier in the occlusion condition than in the control conditions by 23 ms (see Materials and Methods for judgment of latencies). The onset and peak of the *V(t)* recorded from this site at the 17/18 border were also advanced, in this case by 73 and 80 ms, respectively. At the site mapping the center of field of view at the 19/21 border, the timing of the peak of the *V(t)* arrived an average of 113 ms earlier than for the control conditions in a pair-wise comparison in which the animal was its own control (**Table 1**). Thus, as the populations of neurons firing to the moving bars came

and for the *V (t)* at the corresponding cortical site along the 19/21 border **(Bottom)**. Epochs where the amplitude of the *V (t)* or the MUA during the occlusion condition is significantly greater (*p <* 0*.*01) than the amplitude of *V (t)* or MUA of either of the two single bar conditions are shown in red, and epochs where the amplitude of *V (t)* and MUA in the occlusion condition is significantly less than the amplitude of responding to the single bar conditions are shown in blue (*p <* 0*.*01 see Materials and Methods). Note that the *V (t)* and MUA in the time window 392 to 460 ms was not significantly different from that of a single bar (*p >* 0*.*2).

within 600μm of one another, the MUA and *V(t)* peaked earlier than in the control conditions. At the center of the field of view the derivative of the VSD signal, d[*V(t)*]/dt, also became significantly greater than that that for single bar conditions, and this occurred already at 85 ms post stimulus, (**Figure 6** arrows; **Movie S3**).

# **OCCLUSION DYNAMICS**

Eventually, the two peaks of MUA, representing the two moving bars, moved closer to the cortical point representing the

representing the center of field of view. Epochs where the amplitude of these variables during the occlusion condition is significantly greater than the amplitude of the variables to either of the two single bar conditions are shown in red, and epochs where the amplitude of the variables to the occlusion condition is significantly less than the amplitude of responding to either of the single bar conditions are shown in blue (*p <* 0*.*01). Note that when the first effect of the excitatory synaptic activity reaches the area 17/18 border at 120 ms, the d*V (t)*/dt becomes statistically significantly stronger than in the single bar conditions (black arrows **Top** and **Bottom**).

center of field of view. At 392 ms the bars abutted one another on the display screen and the occlusion became maximal at 412 ms (**Figure 1**, **Movie S1**). At 370 ms the d*V(t)*/dt started to decrease at the cortical site representing the center of field of view (**Figure 6**). At this point in time the bars' distance on the screen was 2.6◦, and the bar representations at the 17/18 border in retinotopic coordinates were further apart, given the retino-cortical delay.

*In vivo*, the derivative of *V(t)*, the d[*V(t)*]/dt is an indicator of net membrane excitation and net inhibition (Ferezou et al., 2006, 2007; Berger et al., 2007; Eriksson et al., 2008; Roland, 2010). If the d[*V(t)*]/dt increases significantly over the pre-stimulation baseline this indicates net membrane excitation, if the d[*V(t)*]/dt decreases significantly below baseline, this indicates reduction of net-excitation and most likely increase of net inhibition. The d[*V(t)*]/dt of the neurons mapping the center of field of view at the 17/18 border (and identical to those mapping the occlusion) started to decrease at 370 ms (**Figure 6**). Thereafter the *V(t)* peaked at 403 ms. The MUA peaked at 413 ms when the occlusion on the screen was maximal. The MUA in the supragranular, granular and infragranular layers at this cortical spot and at this moment scaled to that of a single bar (**Figure 5**), as did the total MUA (**Figure 3**). In the supragranular, granular and infragranular layers, the MUA was not statistically different from the MUA associated with a single bar from 415 to 460 ms (*p >* 0*.*2, **Figure 5**).

At the cortical point mapping the center of field of view, the d[*V(t)*]/dt, then the d(MUA)/dt and subsequently the MUA in all layers continued to decrease, such that the d[*V(t)*]/dt, *V(t)* and the MUA were significantly below the values associated with a single bar at 460–500 ms (*p <* 0*.*005) (**Figures 5**, **6**). This significant dip in the d[*V(t)*]/dt, far below the baseline, may be interpreted as a net inhibition of the population. Accordingly the MUA also diminished significantly following the d[*V(t)*]/dt decrease. This raised the question of what might have caused this. We therefore looked at the d[*V(t)*]/dt over all neurons in the supragranular layers of areas 17, 18, 19, and 21. From 370 ms there was a strong decrease in d[*V(t)*]/dt starting at the four zones of cortex where the bars were mapped initially (*p <* 0*.*025, **Figure 6**). The significant d[*V(t)*]/dt decrease propagated from these zones following the subsequent cortical trajectory across the neuron populations subsequently mapping the bars. At 473 ms the d[*V(t)*]/dt decrease reached its minimum almost simultaneously over all four areas. The decrease though remained the strongest at the cortical bar trajectory zones (**Movie S3**). The *V(t)* also decreased, because of the strong d[*V(t)*]/dt decrease (**Figure 3**).

Although net-inhibition appeared first in the populations of neurons that had already mapped the approaching bars, this cannot explain why the population of neurons in the supragranular layers at the cortical spot mapping the center of field of view decreased *prior* to the time this population actually mapped the bars and their subsequent occlusion. It is likely therefore that there might be more than one mechanism reducing the d[*V(t)*]/dt.

# **THE POPULATION MEMBRANE AND MUA DYNAMICS WHEN THE TWO BARS MOVED AWAY FROM EACH OTHER AFTER THE OCCLUSION**

After the occlusion was maximal on the screen at 412 ms, the bars at the display screen formed one growing rectangle until 435 ms, at which point they began to move away from one another in opposite directions.

As seen in **Figure 6**, the d(MUA)/dt started to increase significantly from 500 ms, first in the granular layer. Thereby the MUA and subsequently the d[*V(t)*]/dt also increased at the cortical spot mapping the center of field of view. The d[*V(t)*]/dt increase appeared also outside the bar representations in areas 17, 18, 19, and 21 in the time interval from 473 to 515 ms (**Movie S3**). As is also apparent from **Figure 6**, these increases in d(MUA)/dt and d[*V(t)*]/dt were transient, significant (*p <* 0*.*01), and unique to the occlusion condition (see also **Figure 3**). The MUA increase was especially strong in the granular layer (**Figure 6**) and followed by an increase of d[*V(t)*]/dt (**Figure 6**). This resembles the dynamics seen after excitation by thalamocortical afferents (Roland et al., 2006; Harvey et al., 2009). The increase of MUA at 525 ms, however, occurred only at the cortical spot mapping the bar occlusion (**Figure 3**).

Perhaps the most conspicuous finding was that the MUA never recovered fully at their subsequent trajectory across the populations of neurons that once already had mapped the bars. This decrease in MUA compared to the single bar MUA was strongly significant for the MUA across layers at the positions outside the retinotopic spot of the center of field of view (*p <* 0*.*01, **Figure 3**). In the control conditions, the populations of neurons mapping the single bar also significantly decreased their d[*V(t)*]/dt, resulting in a decrease in *V(t)* some 200 ms after the peak MUA and peak *V(t)* as seen in **Figure 2**. These decreases in d[*V(t)*]/dt and *V(t)* are relatively broad.

In summary, when two bars move toward each other along the vertical meridian in the field of view and the retinas are still, the bars are mapped by two continuously moving maximal laminar MUA increases moving toward each other in constantly changing populations of neurons located at the border between areas 17 and 18. Peak *V(t)* activity also moved toward each other over populations of neurons along the border between areas 19 and 21 and toward each other along the border between areas 17 and 18. Further, the population of neurons along the 19/21 border generated net excitatory (synaptic) membrane activity propagating toward the area 17/18 border where SRP's appeared in the population of neurons located in between the moving peak MUA and peak *V(t)*. Almost until the occlusion, these SRP's were double the amplitude of those generated by single bar motion. The neurons between the moving peaks in the MUA then started to fire, especially in the infragranular layers. The neurons representing the center of the field of view then fired, 126 ms prior to the occlusion. When the neuronal populations representing the bars in areas 17/18 were ∼600μm apart, the activity in the population of neurons between them showed strong decreases in d[*V(t)*]/dt, and subsequently the MUA and *V(t)*. At the time of occlusion the MUA and *V(t)* matched those of a single bar. The d[*V(t)*]/dt behind the moving mapping populations was now also strongly decreased. After a short transient increase in MUA and then in *V(t)* at the cortical site representing the center of the field of view, the *V(t)* recovered somewhat, but the MUA remained significantly reduced in the populations of neurons where the bars were mapped following occlusion.

# **DISCUSSION**

Whereas there is a rich psychophysical literature showing interactions between continuously moving objects, the neurophysiologic mechanisms of these interactions have not previously been examined. We examined two bars moving toward each other and then occluding one another at the center of field of view. There are at least five types of interactions between the populations of neurons in areas 17, 18, 19, and 21 relevant for the understanding of the observed dynamics: (1) feed-forward passage of action potentials between layers and areas, (2) synaptic activity from higher to lower areas, (3) interactions between the populations mapping the bars, (4) interactions between the populations of neurons representing the bars and the population of neurons in between them, and (5) net inhibitory effects from the population that once mapped one of the bars and subsequently mapped the other bar.

We examined only occlusion taking place in the center of field of view. Consequently we cannot generalize our findings to dynamic object occlusions elsewhere. Also the bars had identical contrasts, such that their leading edges were no longer apparent from the moment the occlusion started. Rather they appeared to merge and shrink to one single bar at the moment of maximal occlusion. Moreover, we had no systematic electrode penetrations along the cortical path of motion in areas 19 and 21 and cannot therefore with certainty state the position of the bar representations here.

# **INTER-AREA FEED-FORWARD AND BACK PROPAGATING ACTIVITY**

Initially, just after the bars appeared, there were no differences between the single bar and the occlusion condition. Subsequently, the bars were represented, presumably through feed-forward communications, to two populations at the area 19/21 border. This was verified in the few examples where we had the appropriate electrode penetrations [data not shown, but see also Harvey et al. (2009), Roland (2010)]. The feed-forward flow of action potentials from the lateral geniculate nucleus to areas 17/18 continued, with modulations, throughout the time course of bar motion, continuously moving the two MUA peaks closer to occlusion. After occlusion, feed-forward excitation from the lateral geniculate nucleus moved the peaks of the MUA away from each other.

After the neurons along the border between areas 19 and 21 had produced SRP's, the neurons of area 18 and 19 showed increases in *V(t)* and d[*V(t)*]/dt as an organized wave from 115 ms to 160 ms (**Movie S2**). As the *V(t)* signal reflects differences in synaptic activity (Berger et al., 2007) and the d[*V(t)*]/dt increase reflects net increase in membrane excitation, one could interpret this wave as a propagation of synaptic excitatory activity from the area 19/21 border toward the 17/18 border. **Movie S3** shows an 11 ms delay in d[*V(t)*]/dt increase between the mapping site at the 19/21 border and that of the 17/18 border 60–71 ms after stimulus onset. This could arise from other causes than transmission of action potentials between these areas. However, in **Movie S2** one can follow the propagation over cortex of the relative *V(t)* peak with fast velocity from 19/21 to 17/18. The cortical motion of this peak also includes retinotopic positions in areas 19 and 18 that are not supposed to be stimulated by the stimulus. This, however, is a characteristic of these (waves of) back propagating synaptic excitation (Roland et al., 2006; Ahmed et al., 2008; Harvey et al., 2009). They seem to have a course similar to the course of the feedback axons in the ferret (Cantone et al., 2005).

When the synaptic activity reached the 17/18-area border, it added to the net-excitation of the sub-population of neurons in between the populations of neurons representing the moving bars. This might have contributed to bring some of these neurons over their firing threshold. Harvey et al. (2009) measured a similar propagation of d[*V(t)*]/dt in the same time interval, elicited by the motion of a single bar. Such motion of (net excitatory) synaptic activity is also observed in other species and other visual stimulus conditions (Eriksson and Roland, 2006; Roland et al., 2006; Xu et al., 2007; Ahmed et al., 2008; Takagaki et al., 2008; Harvey et al., 2009; Roland, 2010; Ayzenshtat et al., 2010).

# **INTRA-AREA INTERACTIONS**

Interactions were observed in all four areas beginning 85 ms post stimulus. These interactions began in the population of neurons in between the moving representations of the bars. During the occlusion condition the *V(t)* in the supragranular layers was not significantly different from the sum of the single bar conditions. We cannot discern how much of this summation was due to the recruitment of independent, for example directionally tuned, neurons, or to the increased drive on neurons responding to both directions of bar motion. Since the MUA started to increase significantly earlier in the infragranular layers, this indicates that at least some neurons in these layers could be influenced by additive net excitations. Also in the infragranular layers, there were nonlinearly additive epochs of MUA. This was despite the fact that the moving bars were not collinear (Chisum et al., 2003), but in accordance with reports of firing ahead of the object mappings, (Guo et al., 2007; Harvey et al., 2009).

The synaptic net excitation between the moving representations could be mediated by horizontal connections extending from the bar representations in the lower supragranular layers. The reason why the *V(t)* sums along the future path in between the moving bar representations could be that the populations of neurons representing the bars, through excitatory horizontal connections (Bosking et al., 1997; Chisum et al., 2003; Buzas et al., 2006), increased the synaptic net excitation along the future path of the bars' motion. In addition action potentials from higher order areas 19/21, where the future path was already mapped, could further increase the *V(t)* along the future path in area 17/18, (Harvey et al., 2009).

After 180 ms the neurons, especially in the infragranular layers, started to fire in between the moving bar representations (**Figure 3** and **Table 1**). In the occlusion condition the premature firing also reached the cortical zone for the future occlusion (280 ms) indicating that the brain at this point had information to predict the occlusion. As the firing was strongest in infragranular neurons and as the vast majority of neurons in primary visual cortex projecting to superior colliculus are in layer 5, (Palmer and Rosenquist, 1974), one may speculate that this premature firing toward the cortical point of future occlusion could be useful for generating a saccade to the point in the field of view where the occlusion was expected in analogy with parietal cortical neurons (Duhamel et al., 1992).

# **INHIBITION PRIOR TO OCCLUSION**

The d[*V(t)*]/dt is related to the inward/outward currents of the cells in the upper layers of cortex. This follows from the near linear relation between the population membrane potentials in supragranular layers and the *V(t)*, (Petersen et al., 2003; Ferezou et al., 2006; Berger et al., 2007; Eriksson et al., 2008; Roland, 2010). When the distance between the moving bar representations at the 17/18 border were approximately 600μm and 20 ms prior to the start of the occlusion of the bars on the display screen, the d[*V(t)*]/dt went below baseline and continued to decrease. After the d[*V(t)*]/dt went below baseline, the dMUA/dt in all layers decreased almost simultaneously. Together this indicates a decrease in excitation, or alternatively an increase of inhibition of all layers or both, at the central position of the 17/18-area border. What is in favor of an increased net inhibition is that the d[*V(t)*]/dt went far below baseline and that the dMUA/dt followed this decrease (**Figure 6**). From the measurements depicted in **Movie S3** one can see that the d[*V(t)*]/dt decreased all along the path taken by the bar representations until occlusion. The spatial dynamics of d[*V(t)*]/dt in the interval 370–570 ms is complex. For this reason first the net inhibition of the population of neurons that subsequently mapped the occlusion is discussed.

The inhibition of the population of neurons mapping the occlusion/the center of field of view could depend on several mechanisms. When the bars on the screen came closer together, the likelihood increases that neurons located close to the cortical point representing the center of field of view where the occlusion is going to take place may react. One mechanism could be that the geniculo-striate afferents exciting the granular layer neurons also contact basket cells providing almost simultaneous inhibition (Ahmed et al., 1994; Liu et al., 2011). If such elicited extra inhibition reaches the upper layers, the d[*V(t)*]/dt might decrease. Contradicting the geniculo-striate mechanism of feedforward inhibition, is that the decrease started in supragranular layers prior to the time when the d(MUA)/dt decreased. Another possibility is that the lateral geniculate neurons might have been inhibited. However, the lateral geniculate neurons cannot have been very much inhibited, as the MUA in layer 4 of the cortex was, at the time of occlusion, equal to that of a single bar.

Another alternative is that the increased inhibition of the population of neurons mapping the center of field of view is elicited intra-cortically by the horizontal connections.

The majority of the neurons in area 17 decreases their firing rates to counter-phase gratings and oppositely moving bars (Baker and Emerson, 1983; Qian and Andersen, 1995). *In vitro* experiments on the ferret visual cortex supragranular layers also show that simultaneous stimulation of cortical points separated by 500μm or less generates net inhibition in neurons getting synaptic excitation from both stimulating points in these layers in between the stimulation sites (Tucker and Katz, 2003). If the inhibition was elicited by contrast edges approaching each other, theoretically the inhibition should cease when the cortex detected that occlusion was maximal. The contrast edges, both the leading and the following edges of the bars, then would move away from the cortical site of the center of field of view. This was what happened. One may accordingly describe the behavior of the neurons, at and close nearby the central field of view representation at the 17/18 border, as being net inhibited by the simultaneous and oppositely moving excitation associated with the bar representations. This mechanism might require increased firing of local inhibitory neurons in between the bar representations. The horizontal connectivity in area 17 is most pronounced in lower layer 3 and layer 5 (Gilbert and Wiesel, 1989; Buzas et al., 2006). The net inhibition in these layers may have helped in bringing the MUA in phase across all layers when the granular layer mapped just one bar at the midst of the occlusion (**Figures 3**, **5**). According to **Figure 6**, the net inhibition or depression of the firing ceased first in the granular layer at 462 ms, i.e., 50 ms after the maximal occlusion on the screen.

# **THE REDUCTION OF THE MULTIUNIT ACTIVITY AFTER THE OCCLUSION**

As seen in **Figure 5**, the MUA at the site mapping the center of field of view was for a short period below that associated with a single bar just after the occlusion. As the MUA started to increase in the granular layer, the increase spread to supra and infragranular layers and increased the d*V(t)*/dt to a temporary maximum in about 50 ms, which is the time it normally takes to increase the population d*V(t)*/dt when a stimulus appears (Harvey et al., 2009; Roland, 2010).

Although the inhibition at the cortical retinotopic site of the center of field of view may be explained by mutual horizontal inhibition in areas 17/18 from the bar edges moving toward each other, this cannot explain why the *V(t)* decreased *behind* the moving bar representations (**Movies S1**, **S3**, **Figure 4**). Neither can this explain why the populations of neurons representing the vertical meridian uniformly suppressed the MUA, after the occlusion. One major result was that the total MUA, across layers, after the occlusion, was significantly reduced at all cortical points where the bars had been mapped prior to the occlusion. As seen in **Figure 3**, and in Harvey et al. (2009), there is no reduction of the total MUA when single bars get mapped. The total MUA associated with single bar representation moving over cortex thus is symmetrical, no matter whether the motion is toward or away from the center of field of view. However, when the representation of a single bar moves over the cortex, the d*V(t)*/dt of the neurons mapping the moving bar turns negative with a delay of 130–150 ms (Roland, 2010). This significant negativity thus is a sign of net inhibition.

As seen in **Figure 3**, this reduction of MUA is relatively long lasting. One possibility is that the cortex remains in a generally inhibited state after the inhibition associated with the bars approaching occlusion. This is unlikely for several reasons. First, the decrease in d[*V(t)*]/dt started behind the mapping populations prior to the occlusion. Second, the inhibition was released in the population of neurons representing the occlusion, increasing the MUA to the level of a single bar (**Figure 3**). Third, the d[*V(t)*]/dt did not remain suppressed, but showed a clear rebound above that associated with single bars and stayed normal. However, at 570 ms, i.e., 150 ms after the mapping of the occlusion at the d[*V(t)*]/dt started to decrease again, but only in the zone mapping the center of field of view from where it spread slowly (**Movie S3)**.

The mapping of single bars, however, is also associated with a significant decrease in d[*V(t)*]/dt below baseline after some 100–150 ms (Roland, 2010). This affects also the *V(t)*that eventually becomes negative (**Figure 2**). One possibility is that the mapping of the bar is associated with a delayed inhibition (afterhyperpolarization?) that lasts longer. In the occlusion condition, this delayed inhibition then reduces the MUA when the retinal input reaches the population that mapped the other bar 150 ms ago or earlier.

# **CONCLUSIONS**

The spatio-temporal dynamics of membrane potential changes and laminar MUA associated with objects moving toward occlusion and continuing thereafter is complex. When the retina is still, a single bar moving in the field of view is mapped retinotopically as peak increases in firing rates across cortical layers and, after some 150 ms also by peak increases in membrane potentials by a populations of neurons in each of the four visual areas 17, 18, 19, and 21. If the bar moves up or down the vertical meridian, the laminar peak increases follow paths over the cortex that correspond to the retinotopic mapping of events located at the vertical meridian. In cortex one path is equal to the cytoarchitectural border between areas 17 and 18 and another path equal to the cytoarchitectural border between areas 19 and 21 and yet other paths at several locations in other areas that were not explored in our study. The neurons that map the object at each position in the field of view in each area form a path over the cortex corresponding to the trajectory in the field of view, the path population.

When two objects move exactly toward each other, the path population is identical for the two objects. The instantaneous mapping of the moving objects was done by the laminar peak firing of two constantly approaching sub-populations of neurons at the 17/18 border. At the border between areas 19 and 21 two net excitations of the population membrane potentials appeared approximately 50 ms after stimulus onset. This excitation presumably derives from feed-forward connections emanating from the neurons representing the bars at the 17/18 border. Early on, the neurons of the path population at the 19/21 border in the sector between the peak net membrane excitation also became netexcited. The second interaction between the path populations was a back propagation of net excitatory synaptic activity 115–160 ms after the start of motion from the 19/21 path population to the 17/18 path population.

The interactions expressed in the path population of neurons, in the sector between the neurons mapping the progression of the bars, started 85 ms post stimulus onset with the d[*V(t)*]/dt, net membrane excitation, propagating to the populations mapping the center of field of view from both sides. This continued with the formation of net membrane excitation and increased membrane potential of double the amplitude of both of the whole path population of neurons (17/18 and 19/21) in the upper layers. This behavior of the *V(t)* thus could be interpreted as a long range horizontal interaction combined with the effects of an excitatory back transmission from areas 19 and 21.

Despite the additive effect of the relative population membrane potentials in the supragranular layers of the 17/18 population, the MUA in these layers did not deviate from that associated with a single bar, suggesting that the *V(t)* effect was mainly subthreshold. However, in the infragranular layers, the MUA started earlier and far ahead of the peak activity mapping the bars. Already at 286 ms when the bars were 7◦ apart, the significant firing in infragranular layers reached the cortical zone mapping the future site of the occlusion. This indicates that the brain from this moment had information to predict a collision or an occlusion. This finding and the following findings were particular to the occlusion condition.

When the laminar MUA associated with the moving bars came closer, the d*V(t)*/dt turned negative and thereafter the MUA decreased simultaneously in all layers. This we interpret as a net inhibition of the membranes in the cortical zone of the occlusion. So far these interactions in the 17/18 path population between the mapping populations may be described as horizontal interactions. At the time of occlusion in the cortex at 413 ms (**Table 1**) the spiking population was one population of neurons spiking with a peak rate corresponding to that of a single bar. After the occlusion the sub-populations mapping the bars moving away from each other became identical to the neurons that had once already mapped the bar moving in the opposite direction. Despite a short rebound starting with firing in the granular layers, the MUA of the bar mapping populations did not recover to that prior to the occlusion. We attribute this relatively long lasting depression of the spiking occurring with a delay of 130–150 ms to a delayed inhibition/after hyperpolarization.

The feed-forward and back transmission (feedback) interactions between the path populations of neurons of different areas and the local (horizontal) excitatory interaction between the mapping subpopulation and the sector of the path population ahead occur in association with movement of single bars as well as two bars moving to occlusion. The local net-inhibitory interactions at short range, and the delayed and long lasting inhibition of the spiking of the mapping neurons when the neurons that once already mapped the approaching bars now again must map the departing bars, are specific for the occlusion condition in these experiments. It remains to be investigated whether the local short-range inhibition is a general phenomenon for any two bars approaching each other, no matter at which angle.

# **ACKNOWLEDGMENTS**

This work was supported by grants from the Swedish Science Council K 2010-62X-09456-20-3 and The Wallenberg Foundation to P. E. Roland (and by the Stockholm Brain Institute, a consortium for Cognitive and Computational Neuroscience in Stockholm, funded by The Swedish Research Council, The Swedish Governmental Agency for Innovation Systems (VINNOVA), and the Swedish Foundation for Strategic Research to M. A. Harvey.

# **SUPPLEMENTARY MATERIAL**

The Supplementary Movies for this article can be found online at: http://www.frontiersin.org/Systems\_Neuroscience/ 10.3389/fnsys.2013.00023/abstract

**Movie S1 | The** *-***V(t) of the occlusion condition and the estimate of the position of the cortical position of the representations of the bar without delay in one animal. (A)** Animation of the stimuli in the occlusion condition with a frame frequency of 200 Hz. Distance from the center of field of view shown on the sides of the monitor. Time in ms from the start of motion is shown in green. **(B)** The voltage sensitive dye signal, *V(t)*, in the occlusion condition. The scale to the right shows the absolute values. Yellow and colors above yellow signify significant changes (*p <* 0*.*01). Notice the two spatially restricted depolarizations starting along the 19/21 border (115 ms), and then shortly after along the 17/18 border. For location of cytoarchitectural borders in this animal see C and D. Notice also the almost simultaneous decrease in the *V(t)* starting at 445 ms. **(C)** Location of the bar representation without delay in response to single bars, here shown as downward moving, after (Kalatsky and Stryker, 2003) (see details in **Figure A1**). **(D)** Bonferoni corrected (*p <* 0*.*01) and normalized, *V(t)*rel(s), signal for a downward moving bar. This movie shows only the significant responding after the presentation of a bar moving downward along the vertical meridian.

# **REFERENCES**


**Movie S2 | The increased net membrane excitation traveling from areas 19/21 toward areas 17 and 18. (A)** Animation of the stimuli in the occlusion condition with a frame frequency of 200 Hz. Distance from the center of field of view shown on the sides of the monitor. Time in ms from the start of motion is shown in green. **(B)** The statistically significant part of *V(t)*rel (phase plot) (*p <* 0*.*01) averaged across animals in response to two bars moving to occlusion. The feedback began at 120ms after stimulus onset and traveled toward the 17/18 border with an average velocity of 0.12 mm ms−<sup>1</sup> where it arrived at 160 ms (standard error of mean 10 ms, *n* = 14). The signal then decreases as the bars approach one another. This decrease then surrounds the site mapping the occlusion (representing center of field of view), black circle.

#### **Movie S3 | The spatio-temporal increases, back-propagation from areas 21 and 19, and subsequent decrease of d[***-***V(t)]/dt averaged over three**

**animals.** Statistically significant increases are shown yellow and red; statistically significant decreases are shown light blue and blue [*p <* 0*.*025 false discovery rate with correction for multiple comparisons (Materials and Methods)]. The cytoarchitectural borders are aligned (Methods) across animals. Notice also the net inhibition of the membrane potentials starting bilaterally in the periphery. The scale values shown in the scale bar for the d[*V(t)*]/dt should be multiplied with 10−6.


eds L. M. Chalupa and J. S. Werner (Cambridge: MIT Press), 733–746.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 28 June 2012; accepted: 04 June 2013; published online: 25 June 2013. Citation: Harvey MA and Roland PE (2013) Laminar firing and membrane dynamics in four visual areas exposed to two objects moving to occlusion. Front. Syst. Neurosci. 7:23. doi: 10.3389/fnsys. 2013.00023*

*Copyright © 2013 Harvey and Roland. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in other forums, provided the original authors and source are credited and subject to any copyright notices concerning any third-party graphics etc.*

# **APPENDIX**

#### **FIGURE A1 | Continued**

below the snapshots. If one reverses the time axis for one of the conditions one will end up with the same stimulus as the other condition, i.e., *T* − *(T* − *t)* = *t*. However, the maximal response will instead come *t before* the arrival time of the stimulus at point vt on the monitor, **(B)**. Now the time when the stimulus passes the position vt on the monitor can be found by averaging *p*<sup>+</sup> and *T* − *p*, which in this case equals the position (*x*, *y*) on the cortex without any delay, **(C)**. We can now calculate the delay of the maximal response *t*, by subtracting *p*<sup>+</sup> and *T* − *p*, The time that the peak would have if there were no delay from the screen to the cortex is calculated to be 415 ms after the start of the motion of the bars = start of the bar stimuli **(D)**. Once we have the value derived in **(C)**

for *every* imaged point, (*x*, *y*), we can construct a map of the bar position without delay as shown in **(E)**. Note that the point *x*, *y* used in the example above is not marked in **(E)**. Instead the position of the *V (t)* maximum is mapped as a white line, exactly at the time when the excitation ahead of the bar mapping, the SRP, was maximal. Note that the colors now show time from the start of stimulation. The arrow shows the time when the *V (t)* at the white line was maximal. **(F)** For comparison, a snapshot of the Bonferroni corrected *V (t)* map is shown at a corresponding time point [i.e., when the (predictive) spatially restricted pre-depolarization (SRP) is maximally ahead of the bar mapping during *downwards* motion, and where the bar map would have been (white line) if there had been no delay from the display screen to the cortex].

**site representing the central field of view.** Note that in both cases the

activity generated during the three stimulus conditions.

# Auditory stimuli elicit hippocampal neuronal responses during sleep

# *Ekaterina Vinnik†‡, Sergey Antopolskiy‡, Pavel M. Itskov†‡ and Mathew E. Diamond\**

*Tactile Perception and Learning Laboratory, Scuola Internazionale Superiore di Studi Avanzati, Trieste, Italy*

#### *Edited by:*

*Per E. Roland, Karolinska Institute, Sweden*

#### *Reviewed by:*

*Gregor Rainer, University of Fribourg, Switzerland Heiko J. Luhmann, Institut für Physiologie und Pathophysiologie, Germany*

#### *\*Correspondence:*

*Mathew E. Diamond, Tactile Perception and Learning Laboratory, Scuola Internazionale Superiore di Studi Avanzati, via Bonomea, 265, Trieste 34136, Italy. e-mail: diamond@sissa.it*

#### *†Present address:*

*Ekaterina Vinnik and Pavel M. Itskov, Champalimaud Neuroscience Program, Champalimaud Centre for the Unknown, Champalimaud Foundation, Lisbon, Portugal.*

*‡These authors contributed equally to the work.*

To investigate how hippocampal neurons code behaviorally salient stimuli, we recorded from neurons in the CA1 region of hippocampus in rats while they learned to associate the presence of sound with water reward. Rats learned to alternate between two reward ports at which, in 50% of the trials, sound stimuli were presented followed by water reward after a 3-s delay. Sound at the water port predicted subsequent reward delivery in 100% of the trials and the absence of sound predicted reward omission. During this task, 40% of recorded neurons fired differently according to which of the two reward ports the rat was visiting. A smaller fraction of neurons demonstrated onset response to sound/nosepoke (19%) and reward delivery (24%). When the sounds were played during passive wakefulness, 8% of neurons responded with short latency onset responses; 25% of neurons responded to sounds when they were played during sleep. During sleep the short-latency responses in hippocampus are intermingled with long lasting responses which in the current experiment could last for 1–2 s. Based on the current findings and the results of previous experiments we described the existence of two types of hippocampal neuronal responses to sounds: sound-onset responses with very short latency and longer-lasting sound-specific responses that are likely to be present when the animal is actively engaged in the task.

**Keywords: auditory, sensory, stimulus, sleep, space, location, hippocampus**

# **INTRODUCTION**

Hippocampus is located on top of the cortical hierarchy (Felleman and van Essen, 1991; Burwell et al., 1995; Burwell and Amaral, 1998; Vertes, 2006; van Strien et al., 2009) and is necessary for the formation of new episodic memories (Scoville and Milner, 1957; Aggleton and Brown, 1999; Eacott and Norman, 2004; Cipolotti and Bird, 2006). It is also necessary for the formation of spatial memory and navigation (Morris et al., 1982). During the last 40 years of study of the firing properties of hippocampal neurons, knowledge about the spatial variables that drive hippocampal firing became extensive in both phenomenology and details (O'Keefe, 1976; O'Keefe and Nadel, 1978; Muller and Kubie, 1987; Kubie et al., 1990; O'Keefe and Recce, 1993; Wilson and McNaughton, 1993; O'Keefe and Burgess, 1996; Wood et al., 1999; Jensen and Lisman, 2000; Anderson and Jeffery, 2003; Wills et al., 2005; Leutgeb et al., 2005a,b; Colgin et al., 2008). Non-spatial variables are also represented by hippocampal neurons and studies using various sensory modalities demonstrate that hippocampal neurons respond to sounds (Berger et al., 1976; Christian and Deadwyler, 1986; Sakurai, 2002; Moita et al., 2003; Takahashi and Sakurai, 2009; Itskov et al., 2012), textures (Itskov et al., 2011), odors (Wood et al., 1999; Wiebe and Staubli, 1999, 2001; Komorowski et al., 2009), and gustatory cues (Ho et al., 2011). The majority of place unrelated sensory responses in hippocampus have been demonstrated in animals actively engaged in a behavioral task. We recently developed a categorization task (Itskov et al., 2011, 2012) which allowed us to demonstrate that hippocampal neurons discriminate between sounds. However, in those experiments hippocampal neurons did not respond to sounds when played to the animal not engaged in a discrimination task. These recordings were performed in highly over trained animals and it is possible that some of the features of neuronal responses were absent due to repetition of the same stimuli over many thousands of trials. In the current experiment we characterized responses of hippocampal neurons to behaviorally relevant sounds which were played during sleep and to awake passively listening rats. In the behavioral task, the sounds cued the upcoming release of a reward. Outside the task, the sounds were unrelated to the animal's behavior.

# **MATERIALS AND METHODS ETHICS STATEMENT**

All experiments were conducted in accordance with standards for the care and use of animals in research outlined in European Directive 2010/63/EU, and were supervised by a consulting veterinarian.

### **SUBJECTS**

Six male rats weighing about 350 g were housed individually and maintained on a 14 h/10 h light/dark cycle. Animals were placed on water-restricted diet 1 day prior to the beginning of the experiments. To ensure that the animals did not suffer dehydration as a consequence of water restriction, they were allowed to continue the behavioral testing to satiation and were given access to ad lib drinking water for 1 h after the end of each recording session. The animals' body weight and general state of health was monitored throughout the experiments. Out of the six animals trained to perform the behavioral task, four were implanted using microdrives for chronic recordings. Neuronal data suitable for analysis was collected from two animals.

# **STIMULI**

The stimuli were chosen so that they were short sounds, easily perceived by rats. We chose artificial vowels as a more "naturalistic" class of stimuli than pure tones. Artificial vowels are a simplified version of vocalization sounds used by many species of mammals, and have been used in studies of the ascending auditory pathway from the auditory nerve (Cariani and Delgutte, 1996; Holmberg and Hemmert, 2004) to the auditory cortex (Bizley et al., 2009, 2010). The spectra of natural vowels are characterized by "formant" peaks which result from resonances in the vocal tract of the vocalizing animal (Schnupp et al., 2011). Formant peaks therefore carry information about both the size and the configuration of the vocal tract, and human listeners readily categorize vowels according to vowel type (e.g., /a/ vs /o/, Peterson and Barney, 1952) as well as according to speaker type (e.g., male vs. female voice) or speaker identity (Gelfer and Mikos, 2005). Many species of animals, including rats (Eriksson and Villa, 2006), chinchillas (Burdick and Miller, 1975), cats (Dewson, 1964), monkeys (Kuhl, 1991), and many bird species (Kluender et al., 1987; Dooling and Brown, 1990) readily learn to discriminate synthetic vowels.

The stimuli were generated using binary click trains at the fundamental frequency of 330 Hz and 100 ms duration which were bandpass filtered with a bandwidth equal to 1/50th of the formant frequency using Malcolm Slaney's Auditory Toolbox. Formant center frequencies were 6.912 and 15.008 kHz, respectively. The spectra of the stimuli are shown in **Figure 1B**. The formant frequencies chosen here lie within rat and human sensitivity range (Heffner et al., 1994). As demonstrated in our previous experiments (Itskov et al., 2012) the rats can perceive and discriminate similar artificial vowels. A third, control sound used in passive recordings as a novel stimulus was a naturalistic noise-like sound (see the spectrum in **Figure 1B**). The stimuli were ramped on and off (supplementary audio files 1–2). They were presented at a sound level of 70 dB SPL, from a speaker located above the experimental apparatus.

Sounds were presented through Visaton FRS 8 speaker, which has a flat frequency response (*<*±2 dB) between 200 Hz and 10 kHz. The speaker was driven through a standard PC sound card controlled by LabView (National Instruments, Austin, TX, USA).

# **APPARATUS**

The arena for the behavioral task consisted of a dim-lit soundattenuated box (35 × 30 × 40 cm). One of the walls had two round holes (called "nose pokes" throughout the text, see **Figure 1A**). A water well was placed behind each of the nose pokes. Infrared sensors at the edge of the nose pokes monitored the rat's entry and exit from the holes. Outside the context of the behavioral task, passive exposures to the awake rat were done in an unfamiliar dark plastic bin (33 × 22 × 35 cm) which differed from the training environment by geometry, texture of the floor and the walls, the lighting, and the smell. For recordings during the sleep we used a tall box (26 × 20 × 60 cm) lined with thick cotton tissue. Light sensors, sound, and water delivery were controlled by a custom-written LABview script, operating

a National Instruments card (National Instruments Corporation, Austin, TX, USA).

# **SURGERY AND ELECTROPHYSIOLOGICAL RECORDINGS**

The rats were implanted with chronic recording electrodes. They were anesthetized with a mixture of Zoletil (30 mg/kg) and Xylazine (5 mg/kg) delivered i.p. A craniotomy was made above left dorsal hippocampus, centered 3.0 mm posterior to bregma and 2.5 mm lateral to the midline. A Neuralynx "Harlan 12" microdrive loaded with 12 tetrodes (25µm diameter Pl/Ir wire, California Fine Wire) was mounted over the craniotomy. Tetrodes were loaded perpendicular to the brain surface and individually advanced in small steps of 40–80µm, until they reached the CA1 area, indicated by the amplitude and the shape of the sharp wave/ripples. After surgery, animals were given antibiotic (Baytril; 5 mg/kg delivered through the water bottle) and the analgesic caprofen (Rimadyl; 2.5 mg/kg, subcutaneous injection) for postoperative analgesia and prophylaxis against infection, and were allowed to recover for one week to 10 days after surgery before training started.

After recovery from electrode implantation the animals were trained, during which neuronal responses were recorded from the tetrode array using TDT data acquisition equipment (RZ2, Tucker and Davis Technologies, Alachua, FL, USA). We usually recorded from 1 to 4 single units per tetrode. Spikes were presorted automatically using KlustaKwik (Harris et al., 2000), using waveform energy on each of the four channels of the tetrode as coordinates in a 4 dimensional feature space. The result of the automatic clustering was inspected visually after importing the data into MClust

(kindly provided by A. D. Redish). Each single unit was recorded only in one session.

# **BEHAVIORAL TASK**

The task was designed in a way that allowed us to examine neuronal responses to behaviorally relevant auditory stimuli. We took into account the richness of location-related neuronal signals in hippocampus that are known to influence sensory responses recorded in different spatial locations (Itskov et al., 2012). The animal initiated each trial by poking its snout into a nose poke. During the recording sessions (referred to as "lottery," see below) the sounds were presented with 50% probability for a given nose poke entry. Trials with and without sound presentation were compared, assuring that the animal was in exactly the same position and motivational state in both types of trials.

# *Pre-training*

The day prior to behavioral testing, animals were placed on a water restricted dry food diet. To accustom the animals to the setup and the task they were given four pre-training sessions with 100% reward probability (**Figure 2A**). During these four sessions the rats learned to alternate between two water spouts, collecting a reward at each site. After receiving water in one of the spouts, the rat had to go to the second spout to receive reward, and then back to the first spout. Attempts to collect water from the same spout twice were neither rewarded nor punished. Initially water reward was delivered immediately upon the nose poke. During the following sessions, reward was delayed by 1 s on each successive session. Thus, by the end of fourth pre-training session,

water reward by continuously alternating between the two nose pokes (right and left). During the first pre-training session water was delivered instantly upon nose poke entry. The delay between the nose poke and water delivery increases during subsequent sessions

only in 50% of the trials. This sequence of pictures outlines the case when the animal had a "lucky" trial in one of the nose poke. Other "unlucky" trials have neither sound nor reward, the animal proceeded to the opposite nose poke (see Materials and Methods).

delay was 3 s. At this point, the rat was introduced to the main "lottery" task (see below). The rats typically started to perform the alternation task at the end of the first or beginning of the second session. By the end of the fourth session, rats reached stable performance of ∼300 trials per session (206–510, mean 309). No sounds were played during the pre-training.

# *"Lottery"*

On the fifth day of the training, the reinforcement schedule was changed to increase the salience of the reward and, especially, to make the rat attend to novel sounds associated with reward. Now the water was delivered only on 50% (not 100%, as before) of nose pokes (**Figure 2B**). The rat still had to alternate between right and left nose pokes and wait for at least 3 s for the water reward delivery. Whether the current trial would turn out to be "lucky" or "unlucky" was determined randomly by software. On "lucky" trials, a sound was played immediately upon entry into the nose poke, predicting the upcoming reward. There were two sounds used in the behavioral task (two artificial vowels, **Figure 1B**), each of them associated with just one of the reward spouts.

This task was designed so that we could examine neuronal responses to sound presentation in awake behaving animals. Importantly, in the "unlucky" trials the animals decreased waiting time over the course of the session, suggesting that the occurrence of the sound became associated with the water reward and absence of sound became a cue to go to the alternate nose poke (see Results).

# **PASSIVE LISTENING**

To record hippocampal responses to sounds in a context unrelated to the behavioral task we placed the rat in a novel environment 2–3 h after completion of the behavioral session. Three types of sound stimuli (familiar vowel associated with the right well, familiar vowel associated with a left well, and a novel noise-like sound) were presented pseudo-randomly, so that no sound was repeated more than two times in a row. Inter-stimulus interval varied randomly from 25 to 30 s. The sounds were presented irrespectively of the animal's action and location. Each stimulus type was presented 43 times on average. The long inter-stimulus interval was intended to reduce the animal's adaptation to stimulus repetitions (Ulanovsky et al., 2004).

Passive sound presentation during sleep started 5 min after the animal had fallen asleep, as determined by total immobility. The animal was monitored continuously via remote video camera. As was the case during the session in which the rat remained awake in a novel environment, three types of sound stimuli (familiar vowel associated with the right spout, familiar vowel associated with a left spout, and a novel noise-like sound) were presented pseudorandomly, so that no sound repeated more than two times in a raw. Inter-stimulus interval varied randomly from 25 to 30 s. Each stimulus type was presented 49 times on average.

To ensure the proper timing of the responses, we recorded a copy of the signal sent to the loudspeaker along with the neuronal signals.

### **STATISTICAL PROCEDURES**

Firing rates of hippocampal neurons typically vary over the course of a single trial, since they tend to respond to different aspects of the task. The timing and the latency of the response can vary across individual cells. Taking this into account, to quantify the responses to sounds we used a method intended to (1) take into account multiple time points, (2) provide a single measure of statistical significance of the effect, (3) take into account variability in the timescale of the responses, fast in some cases (on the order of tens of milliseconds, Brankack and Buzsaki, 1986) and slow (hundreds of milliseconds) in others.

# **STATISTICAL SIGNIFICANCE OF NEURONAL RESPONSES IN THE BEHAVIORAL TASK**

To quantify firing rate changes related to animal's location in space we measured firing rate in non-overlapping 50 ms bins during the approach to the nose pokes (see **Figure 3A**). Firing rates in each time bin were compared using Wilcoxon ranksum test for two independent samples (Siegel and Castellan, 1988). Benjamini–Hochberg false discovery rate corrections were used to correct for multiple comparisons (FDR, Benjamini and Hochberg, 1995). A neuron was considered to distinguish between the two nose pokes and therefore to be a "locationencoding" neuron if any of the time bins showed a corrected *p*-value smaller than 0.05.

To quantify a neuron's responses to sound presentation and water reward delivery during the behavioral task, we compared firing rates in the "lucky" trials to firing rates in trials in which no sound was played and no water delivered. To take into account the variability of the responses (e.g., fast transient or sustained responses), we used four a-priori defined bin sizes (50 ms, 500 ms, 1 and 2 s). In case of 50 ms bins, we measured firing rate in six non-overapping bins (0–300 ms). Wilcoxon rank-sum test for two independent samples was used to compare the firing rates, bin by bin, in "lucky" trials versus not reward trials. Benjamini– Hochberg FDR correction were used to correct for testing multiple time points (*n* = 6 in 50 ms bins) and bin sizes (50 ms, 500 ms, 1 and 2 s, FDR, Benjamini and Hochberg, 1995). A neuron was considered a "sound or reward-encoding" neuron if any of the time bins showed a corrected *p*-value smaller than 0.05.

# **STATISTICAL SIGNIFICANCE OF SOUND-EVOKED RESPONSES IN PASSIVE LISTENING AND DURING SLEEP**

To quantify sound-evoked responses during passive presentation, we compared post-stimulus firing rate with the baseline. To take into account the variability of responses we again used four a-priori defined bin sizes (50 ms, 500 ms, 1 and 2 s). For each analysis, baseline firing rates were defined as the firing rate in the time immediately preceding the presentation of the sounds. We used baseline bin size equivalent to the time bin size used to measure evoked activity (50 ms, 500 ms, 1 or 2 s).

In the case of the smallest bin size, firing rate was calculated in 50 ms non-overlapping bins in first 300 ms after sound onset, yielding six individual time points. Firing rate values in each of them were compared to firing rate values in 50 ms baseline (sign test for dependent measurements, Siegel and Castellan, 1988). This yielded six tests for each neuron, which were corrected for using FDR. Latency of the responses was calculated as time of the first bin after sound onset with activity significantly different from baseline.

sound-responsive neuron. This neuron responded to the sound presentation. **(D)** Responses at the right nosepoke of sound-responsive neuron. This neuron responded to the nosepoke, but fired at higher frequency when the sound was presented. All of the demonstrated neurons showed very similar robust responses at the opposite nosepokes, but fired at smaller peak rates (data not shown). Bin size for peri-stimulus histograms is 200 ms with 190 ms overlap.

In the case of 500 ms bins, we took a mean firing rate in 500 ms baseline and compared it with mean firing rate in the 500 ms post-stimulus interval. In the case of 1 s bins we took a mean firing rate in 1 s baseline and compared it with mean firing rate in 1 s post-stimulus interval. In the case of 2 s bins we took a mean firing rate in 2 s baseline and compared it with mean firing rate in 2 s post-stimulus. This yielded four measurements for each neuron: *p*-value for 50 ms bins (already corrected for comparing multiple bins), and *p*-values for 500 ms, 1 and 2 s bins. These *p*-values were corrected for multiple testing using FDR. A neuron was considered sound-responsive if any of the time bins showed a corrected *p*-value smaller than 0.05.

**(A)** Location-responsive neuron. **(B)** Responses of reward-responsive neuron during nosepokes to the left nosepoke. This neuron responded upon water

For the purposes of illustration, the firing rate was calculated in a 50 ms wide window sliding in steps of 25 ms along the whole duration of the trial (see **Figures 3**–**4**).

# **HISTOLOGY**

After conclusion of the recording experiments, the animals were overdosed with intraperitoneal injection of the anesthetic urethane and transcardially perfused with 4% paraformaldehyde solution. Brains were sectioned in the coronal plane and stained with cresyl-violet. Electrode tracks were localized on the serial sections in CA1 field.

# **RESULTS**

#### **BEHAVIOR**

This study set out to test how neurons in rat hippocampus represent auditory stimuli in the context of a behavioral task, during passive exposure and during sleep. Familiar sounds that during the training preceded water reward were played in the novel context (new enclosure) or during sleep.

Rats were given four pre-training sessions, during which they learned to alternate between two water spouts to get water reward (see **Figure 2A** and Materials and Methods). In the main experimental session ("lottery", **Figure 2B**) the water reward contingencies were changed: now water was given only in 50% of the entries to the nosepokes. On these "lucky" trials one of the sounds was played immediately upon the nosepoke entry. Distinct sounds were used for each of the two nose pokes (see **Figure 1B** and Materials and Methods).

#### **NEURONAL REPRESENTATION OF THE BEHAVIORAL TASK**

In awake rats, hippocampal neurons exhibit strong spatial selectivity: they encode the animal's position, head direction, and other spatial variables (O'Keefe and Dostrovsky, 1971, reviewed in Moser et al., 2008). Consistent with the existing literature, the neurons recorded in this task exhibited significant sensitivity to animal's location in space. Forty-one percent of tested neurons (24 out of 58) discriminated between right and left nosepokes (see **Figure 3A**; *p <* 0*.*05, corrected, rank sum test on firing rates 400 ms before the nose pokes, see Materials and Methods).

On the "lucky" trials (50% of trials, selected randomly), upon entry to the nosepoke the rats heard a sound and received water reward after a 3 s delay. Two different sounds were used in two different locations (nose pokes). In "unlucky" trials (50% of trials) the rat heard no sound and received no water reward (see **Figure 2B** and Materials and Methods). The rats stayed still with their snouts in the nosepoke waiting for 3 s between the nosepoke entry/sound onset and the water reward delivery. The average time of waiting in no-sound-no-reward trials was 5*.*09 ± 0*.*19 s (SEM). Waiting time decreased over the course of the session (*p <* 0*.*001, paired-sample *t*-test). To examine responses to sound and reward, we aligned neuronal activity on the entry to the nosepoke and compared firing rate observed in the two types of trials, separately for the two water wells. The responses are illustrated on **Figures 3B–D**; the line at 3 s denotes reward delivery. Out of 58 cells recorded in the task, 11 cells responded in the first 300 ms upon sound presentation (19%, *p <* 0*.*05, rank sum tests done in 50 ms windows, corrected for testing six time points and also four different bin sizes, see Materials and Methods). Fourteen (24%) responded upon the reward delivery (**Figure 3B**). There were no cells with responses to both sound and reward.

# **PASSIVE SOUND EXPOSURE**

The aim of this experiment was to characterize the representation of sound identity during passive listening. We played three different sounds to the animals outside the context of the behavioral task, in a novel box. Two of the sounds were artificial vowel stimuli associated with water in the two different nosepokes and familiar to the animals and one of the sounds was a dissimilar novel band pass filtered noise stimulus. Our intention was to compare the responses to familiar and relevant sounds with the responses to novel and behaviorally irrelevant ones. The animals were placed in a novel context 24 h after they performed the "lottery" behavioral task. Sounds were triggered with no relation to the animal's posture or movement, in a random sequence lasting approximately 50 min with a random interstimulus interval ranging from 25 to 30 s. The rats did not respond to the sounds with any overt behavior.

Three out of 25 neurons recorded during the awake passive conditions were classified as sound-sensitive (see Materials and Methods, *p*-values were 0.0029, 0.0000016, and 0.01, corrected). The responses are illustrated in **Figure 4A**. The latencies of response to passive sound presentation were less than 150 ms in two cases and less than 50 ms in the third case (see **Figure 4A**). Two of the neurons showed a significantly increased firing rate in response to the sound, and one a decreased the firing rate. It would be interesting to know whether the neurons that responded to sounds were also responsive to events occurring within the task, such as sounds, reward, or locations. Although we were able to keep the same neurons across the recording sessions, as evidenced by the neuronal waveforms, autocorrelations and firing rates, we feel that the number of recorded neurons does not allow us to make any conclusive statements regarding a single neuron's properties across time and context. None of the three soundresponding neurons discriminated between different sounds i.e., their responses to different sounds were identical.

# **AUDITORY RESPONSES DURING SLEEP**

During sleep sessions, the rats were placed in a small familiar box in which they soon curled up and fell asleep. Animals' activity was monitored remotely via webcam, placed above the box. Five minutes after the animals curled up and fell asleep, the sound stimuli were turned on. The animal's sleep status was constantly monitored online by the experimenter and sound presentation did not cause the animals to wake up. The sounds were presented the entire time when the animals were asleep (117–180 times, 49 presentations per sound type on average) and were turned off when they awoke. Only the trials where sounds did not cause any visible behavioral response from the sleeping animal were analyzed further.

The responses to sounds during sleep are illustrated in **Figures 4B–D** and **Figure 5**. Twenty-four percent of recorded neurons (8 out of 33) exhibited significant sound-evoked responses. Across neurons, latency ranged from 50 to 150 ms. In five cases firing rate decreased after sound presentation, in three cases the firing rate increased.

**Figure 5** illustrates neurons with significant differences between baseline and post-stimulus activity lasing more than 1 s, even when considered in short 50 ms long bins (e.g., black line in **Figure 5C**). In some cases an initial time-locked response was followed by a long sustained activity response (**Figures 5A,B,C,D**).

In five cases we observed significant differences between mean firing rate in 2 s post-stimulus and the baseline (*p <* 0*.*05, corrected). A neuron in **Figure 5** (panels F-G-H-I-J) illustrates the case. A short-latency onset response was followed by a sustained, long lasting response (**Figure 5F**). Panel I plots average firing rate in 2 s pre-and post-stimulus windows. *p*-value is indicated at the outset (*p* = 0*.*0035, sign test for dependent measures). Both neurons depicted on this figure had narrow spikes (**Figures 5E** and **J**), 0.2 ms when measured on 25% of spike amplitude. Both of these neurons are likely to be putative interneurons because of the narrow spike width and the absence of the 3–4 ms peak in the autocorrelation plot (data not shown). Unfortunately, limited size of our dataset does not allow us to make any further conclusions on this issue.

Long lasting responses to sounds were unique to responses recorded during sleep (**Figure 6**). The responses to sounds during sleep were on average almost twice as long as the response to sounds presented during passive wakefulness (**Figure 6F**). Responses to sounds during sleep also appeared slightly earlier than the responses during passive wakefulness (**Figure 6E**).

The majority of cells did not discriminate between different sounds. We found only one neuron with significant sound identity-dependent responses (*p* = 0*.*0101, Kruskal–Wallis test) comparing responses to three sounds, two artificial vowels and the novel noise-like sound.

Taken together, these results indicate that hippocampal neurons showed sound-evoked responses during awake passive listening. One quarter of recorded neurons showed reliable sound-evoked responses during sleep. Short latencies of the responses (less than 50 ms) were observed alongside with longlasting sustained activity.

# **DISCUSSION**

In this study we show that a significant subset of hippocampal neurons responds to sounds presented passively to a rat during both quiet wakefulness and during sleep. During sleep the responses were characterized by a sharp, short latency onset, and in some cases by long-lasting sustained activity. These responses were as strong as the responses to the sounds are associated with water reward and the novel sounds that had never been played to the animal before.

The absence of neuronal responses to sound stimuli outside the categorization task (Itskov et al., 2012) seems at the first glance inconsistent with the results of our current experiment, where we have found that hippocampal neurons responded to the sound stimuli presented during passive wakefulness. We think that this discrepancy may be explained by the differences in the behavioral task and experimental conditions. In the former study highly familiar sounds (used during 4–5 weeks of training) were

#### **FIGURE 5 | Sound-evoked responses during sleep can last for over 1 s.** Panels **(A–E)** and **(F–J)** depict responses and results of statistical analysis for two individual neurons recorded from two rats. Panels **(A, F)** show peri-stimulus time histograms, PSTH (0 ms corresponds to sound onset; trials with three sounds are pooled together); **(B, G)** show raster plots on the same time scale as the PSTH. Panels **(C, H)** show *p*-value (sign-test for dependent measures, 50 ms windows). Box plots in panels **(D, I)** present comparison of 2 s long baseline and post-stimulus periods. On each box, the central mark is

the median, the edges of the box are the 25th and 75th percentiles, the whiskers extend to the most extreme data points not considering outliers; outliers are plotted individually as dots. *p*-values are indicated at the outset (paired-sample sign test). Panels **(E, J)** depict average waveforms on each of the four channels of the tetrode for the corresponding neuron. Spike width (at 25% max spike amplitude) and firing rate for neuron on panels **(A–E)** are 0.2 ms and 1.8 Hz, respectively; for the neuron in panels **(F, J)**, 0.2 ms and 1.8 Hz, respectively. Bin size for panels **(A, C, F, H)** is 50 ms with 40 ms overlap.

**FIGURE 6 | Time course of sound-evoked responses: transient and sustained responses. (A)** normalized PSTHs of neurons with significant responses on a fast time scale (window size 50 ms), recorded during sleep; **(B)** normalized PSTHs of neurons with significant responses on a slow time scale (window size 500 ms), recorded during sleep; **(C)** normalized PSTHs of neurons with significant responses on a fast time scale (window size 50 ms), recorded during awake passive listening; **(D)** normalized PSTHs of neurons

with significant responses on a slow time scale (window size 500 ms), recorded during passive listening. Out of 25 neurons only three showed small, but significant responses; **(E, F)** Absolute values of normalized PSHT (pulled data from neurons that significantly increase and decrease the firing rate). Black trace represents the average of neurons recorded during sleep (*n* = 8 out of 33), Gray trace is the mean activity of neurons responsive during awake conditions (*n* = 3 out of 25).

played in the familiar context after the animal had quenched its thirst and was quietly sitting in the apparatus. In the present study the animals were thirsty during passive presentation and were placed in a completely novel dim environment, which could have facilitated attention to the auditory stimuli. In this situation we observed strong sound evoked responses in passive listening conditions.

Thus, there are three possible differences that could have explained the difference between these two results: motivational state of the animal (thirsty/satiated), familiarity of the stimuli and familiarity of the context. Short latency responses to sounds in hippocampus have previously described in the literature (Brankack and Buzsaki, 1986). This study described responses of extremely short latency evoked potentials with first component around 27 ms after the click onset and neuronal responses with latency similar latency (see their **Figure 5**). Several reports have demonstrated that stimulus-elicited responses appear in hippocampal neurons with latency of around 80 ms after the stimulus onset (Christian and Deadwyler, 1986; Moita et al., 2003). What is probably even more important is the similarity of the behavioral paradigms that we and the above mentioned authors used in their studies: the use of naive animals (Moita et al., 2003) or the lack of overtraining due to fast learning in a simple behavioral task (Christian and Deadwyler, 1986). Unfortunately it is not possible to compare response specificity to individual sound stimuli in these studies since the authors did not consider this feature in their experimental design. In addition to the abovementioned facts, one study that used tactile stimuli in anesthetized and awake naive passively stimulated rats (Pereira et al., 2007) have found similar short latency responses to trigeminal nerve shock. In over-trained, behaving animals tactile responses have longer latency (Itskov et al., 2011). Furthermore, the stimulus-specific responses to behaviorally relevant sensory stimuli can persist for at least several seconds after the stimulus has disappeared, as demonstrated using tactile-guided task (Itskov et al., 2011).

Based on our observations we suggest that there might be two types of neuronal responses in the hippocampus to sensory stimuli:

1. Short-latency responses (from ∼27 ms latency to tens of milliseconds). These responses are not present in the over-trained and/or satiated animals suggesting that they might disappear when the animal expects the stimuli.

2. Long-latency responses. These responses are likely to be evoked by behaviorally relevant stimuli. This second type of hippocampal responses to sounds is likely to encode object identity e.g., are specific to particular behaviorally relevant sounds (Itskov et al., 2012) and textures (Itskov et al., 2011). We posit that these responses are associated with the behavioral "meaning" of the sound.

The evoked responses in the CA1 region may be generated by perforant path terminals, originating in the entorhinal cortex or polisynaptically via the Schaffer collaterals. The origin of this short-latency signal in the entorhinal cortex is not known; previous studies suggest that it is unlikely to be of cortical origin, because bilateral ablation of the somatosensory cortex had no effect on the amplitude or the depth distribution of tooth pulp evoked responses in the hippocampus (Brankack and Buzsaki, 1986).

The long-latency responses might be routed to hippocampus through the same entorhinal cortex pathway or perhaps even through medial prefrontal cortex and thalamus, contributions of which remain, to the best of our knowledge, unexplored. It is likely that long-latency responses originate from higher-level cortices with more elaborated "object-related" rather sensory feature processing. Stimulus-specific representations can persist in hippocampus for at least several seconds after the offset of the behaviorally relevant stimulus (Itskov et al., 2011). This memory trace could be stored by sustained activity, previously described in entorhinal cortex (Egorov et al., 2002), or by some other cellular or network mechanisms in higher order associative cortices or hippocampus itself.

# **HIPPOCAMPAL RESPONSES DURING SLEEP**

It has been recently demonstrated that re-exposure to olfactory and auditory stimuli during sleep can enhance memory for specific events, associated with those stimuli (Rasch et al., 2007; Rudoy et al., 2009). Rasch and co-authors (2007) cued new memories in humans during sleep by an odor that had been presented as context during prior learning, and showed that this manipulation enhanced subjects' performance. Reexposure to the odor during slow-wave sleep improved the

# **REFERENCES**


retention of hippocampus dependent declarative memories but not of hippocampus-independent procedural memories. Reexposure was ineffective during rapid eye movement sleep or wakefulness or when the odor had been omitted during prior learning. Concurring with these findings, functional magnetic resonance imaging revealed significant hippocampal activation in response to odor re-exposure during slow wave sleep. Rudoy and co-authors (2009) trained participants to associate each of 50 unique object images with a location on a computer screen before a nap. Each object was paired with its characteristic sound (e.g., cat with meow and kettle with whistle). Presentation of the sounds during sleep resulted in better memory performance. Interestingly, the memory enhancement was specific to the played sounds and did not generalize to the other stimuli; those corresponding sounds were not played during sleep. Average electroencephologram (EEG) amplitudes measured over the interval from 600 to 1000 ms after sound onset were larger when there was less forgetting.

In the present study we investigated the hippocampal responses to behaviorally relevant sounds during re-exposure to the sounds during awake state and sleep. We found short-latency responses to sound presentation during sleep, as well as longlasting (more than 1 s) sustained responses. The latter finding might be related to the sustained EEG response found in Rudoy et al. (2009) after presentation of auditory stimuli to humans during sleep. Rasch et al. (2007) reported an increased BOLD-signal in hippocampus after presentation of the odors. There results suggest that long-lasting changes in neuronal activity contribute to stimulus-evoked responses observed in previous studies, in addition to previously shown local field potential change and short-latency onset responses. It is hoped that further studies will clarify how exactly the presentation of the relevant sensory cues during sleep facilitates the hippocampus-dependent memory.

# **ACKNOWLEDGMENTS**

We are grateful to Fabrizio Manzino and Marco Gigante for technical support and to Francesca Pulecchi for help with histological procedures. This work was supported by the Human Frontier Science Program (contract RG0041/2009-C), the European Research Council Advanced grant CONCEPT, the European Union (contracts BIOTACT-21590 and CORONET), and the Compagnia San Paolo.


and postrhinal cortices of the rat: a review of the neuroanatomical literature and comparison with findings from the monkey brain. *Hippocampus* 5, 390–408.


in rats: persistent and independent traces of stimulus and reward location. *PLoS ONE* 6:e16462. doi: 10.1371/journal.pone.0016462


sample task. *Front. Neural Circuits* 3:9. doi: 10.3389/neuro.04.009.2009


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 15 March 2012; accepted: 29 May 2012; published online: 27 June 2012.*

*Citation: Vinnik E, Antopolskiy S, Itskov PM and Diamond ME (2012) Auditory stimuli elicit hippocampal neuronal responses during sleep. Front. Syst. Neurosci. 6:49. doi: 10.3389/fnsys. 2012.00049*

*Copyright © 2012 Vinnik, Antopolskiy, Itskov and Diamond. This is an openaccess article distributed under the terms of the Creative Commons Attribution Non Commercial License, which permits non-commercial use, distribution, and reproduction in other forums, provided the original authors and source are credited.*

# *Eugene F. Civillico and Diego Contreras\**

*Department of Neuroscience, University of Pennsylvania School of Medicine, Philadelphia, PA, USA*

#### *Edited by:*

*Per E. Roland, Karolinska Institute, Sweden*

#### *Reviewed by:*

*M. Gustavo Murer, Universidad de Buenos Aires, Argentina Frédéric Chavane, Centre National de la Recherche Scientifique, France*

#### *\*Correspondence:*

*Diego Contreras, Department of Neuroscience, University of Pennsylvania School of Medicine, 215 Stemmler Hall, Philadelphia, PA 19104-6074, USA. e-mail: diegoc@mail.med.upenn.edu* Sensory responses in neocortex are strongly modulated by changes in brain state, such as those observed between sleep stages or attentional levels. However, the specific effects of network state changes on the spatiotemporal properties of sensory responses are poorly understood. The slow oscillation, which is observed in neocortex under ketamine-xylazine anesthesia and is characterized by alternating depolarizing (up-states) and hyperpolarizing (down-states) phases, provides an opportunity to study the state-dependence of primary sensory responses in large networks. Here we used voltage sensitive dye (VSD) imaging to record the spatiotemporal properties of sensory responses and local field potential (LFP) and multiunit activity (MUA) recordings to monitor the ongoing brain state in which the sensory responses occurred. Despite a rich variability of slow oscillation patterns, sensory responses showed a consistent relationship with the ongoing oscillation and triggered a new up-state only after the termination of the refractory period that followed the preceding oscillatory cycle. We show that spatiotemporal properties of whisker-evoked responses are highly dependent on their timing with regard to the ongoing oscillation. In both the up- and down-states, responses spread across large portions of the barrel field, although the up-state responses were reduced in total area due to their sparseness. The depolarizing response in the up-state showed a tendency to propagate along the rows, with an amplitude and slope favoring the higher-numbered arcs. In the up-state, but not in the down-state, the depolarizing response was followed by a hyperpolarizing wave with a consistent spatial structure. We measured the suppression of whisker-evoked responses by a preceding response at 100 ms, and found that suppression showed the same spatial asymmetry as the depolarization. Because the resting level of cells in the up-state is likely to be closer to that in the awake animal, we suggest that the polarities in signal propagation which we observed in the up-state could be used as computational mechanisms in the behaving animal. These results demonstrate the critical importance of ongoing network activity on the dynamics of sensory responses and their integration.

**Keywords: barrel cortex, mouse,** *in vivo***, voltage sensitive dyes, ongoing activity, spatiotemporal dynamics, network state**

#### **INTRODUCTION**

Complex representations in sensory cortices rely on the integration of inputs that overlap temporally and spatially, particularly in supragranular layers, yet the spatiotemporal dynamics of this synaptic integration are largely unknown. It is a general principle of cortical organization that cortical columns are arranged on the two-dimensional axis of the cortex in such a way as to form continuous topological representations, which may be thought of as maps of particular parameter spaces in the environment of an organism. When navigating the world and making decisions, organisms make use of the complete range of input from each sensory modality, and are continually combining inputs across modalities in the context of the brain's ongoing activity. Between the entry of the relatively segregated components of a representation into the input layer 4 of primary sensory cortex, and the so-called "higher" or "association" cortices where neurons respond to more abstract stimuli, these partial representations, the results of columnar computation, must be combined in a hierarchical processing scheme. This combination is the essence of sensory integration and requires anatomical and physiological overlap of the output of cortical columns. In the rodent whisker-barrel system, easily visualized input pathways for single whiskers in layer 4 of primary somatosensory cortex give rise to overlapping anatomical and functional representations in layer 2/3, creating an excellent opportunity to study the spatiotemporal dynamics of sensory processing and its relationship to ongoing brain activity.

In the past few decades, voltage-sensitive dye (VSD) imaging studies in this model system (Arieli et al., 1996; Contreras and Llinas, 2001; Petersen and Sakmann, 2001; Ferezou et al., 2006) have revealed fundamental properties of the neocortex which are sometimes left implicit in the presentation of results; namely, that subthreshold responses are locally homogeneous, that neighboring regions tend to be excited together, and that activity moves in all directions once initiated. These features are expected given early field potential results and the dense recurrent connectivity of neocortical tissue (Douglas et al., 2004; DeFelipe and Jones, 2010). At the same time, intracellular (Carvell and Simons, 1988; Moore and Nelson, 1998; Higley and Contreras, 2003; Wilent and Contreras, 2004) and extracellular (Simons, 1983; Brumberg et al., 1996; Shimegi et al., 2000; Webber and Stanley, 2004; Ego-Stengel et al., 2005) recordings indicate that stimulus-evoked spiking may be quite sparse, particularly in the rodent barrel cortex. Thus, we may think of the computations performed by a local cortical circuit as the selection of a subset of cells to fire from an envelope of partially synchronous subthreshold activity. The representation of sensory information as spikes which will be transmitted to other areas must be contained within the subthreshold spatiotemporal envelope; therefore, to understand the computation performed by a cortex it is important to understand the spatiotemporal dynamics of this envelope. Imaging studies with VSDs are uniquely suited to advancing this understanding and have revealed much about sensory responses in various modalities and organisms.

With some exceptions (Arieli et al., 1996; Petersen et al., 2003a; Ferezou et al., 2006; Hasenstaub et al., 2007; Poulet and Petersen, 2008), sensory studies have been based on data acquired as the average of many trials, combined indiscriminately with regard to brain state. Nevertheless, evoked responses in intact primary neocortex do not occur in a vacuum, but rather impinge on networks which are continuously active. Even the first experiments demonstrating the existence of the electroencephalogram (EEG) produced the surprising finding that the brain was active in the absence of sensory stimulation (Caton, 1875; Brazier, 1961). Since then, many studies have illustrated the rich repertoire of rhythms in the EEG. The cellular basis of most EEG rhythms has only recently been worked out (Speckmann et al., 2011). At both the single cell and network level, it is clear that there is a high level of ongoing activity constantly providing a background for sensory input. During sleep and epilepsy, highly synchronized slow (*<*15 Hz) EEG rhythms preclude the formation of dynamic neural ensembles necessary for the processing of information and the generation of rapid adaptive behaviors characteristic of the awake brain (Steriade, 2000). During the waking state, faster rhythms (*>*40 Hz) provide context and allow coordination of activity between groups of neurons within and between networks. Consistent with these observations, evoked sensory responses have been shown to be significantly dependent on the state of ongoing activity at the time of the stimulus (Arieli et al., 1996; Sachdev et al., 2004; Hasenstaub et al., 2007; Hirata and Castro-Alamancos, 2011).

One particular form of ongoing activity which has been the subject of much recent interest is the slow oscillation between socalled up- and down-states which occurs in the neocortex and other brain regions (Steriade et al., 1993a,b). For some time it has been known that this slow (*<*1 Hz) oscillation is a hallmark of deep ketamine-xylazine anesthesia and may be observed intracellularly in nearly all thalamic and cortical cells and extracellularly in the local field potential (LFP). The dynamics of cellular relationships during the slow oscillation have been extensively characterized with intracellular recordings *in vivo* (Steriade et al., 1993a,b; Contreras and Steriade, 1995; Contreras et al., 1996; Steriade, 1997) and in cortical (Sanchez-Vives and McCormick, 2000) and thalamic (Blethyn et al., 2006) slices *in vitro*. Neurons in thalamocortical circuits remain hyperpolarized and quiescent during the down-state, and are depolarized by 10–20 mV during the up-state. Both experimental and modeling studies have shown that the input resistance of cortical cells is lowered by the increased synaptic drive during depolarized states (Destexhe et al., 2003; Shu et al., 2003b; Sachdev et al., 2004; but see Waters and Helmchen, 2006). Negative LFP waves recorded from the cortical depth are the extracellular correlate of synchronized neuronal depolarization; positive waves are the correlate of synchronized and sustained hyperpolarization. Intracellular studies in the cat neocortex determined that the hyperpolarization of cortical cells is produced by the removal of synaptic input in combination with a powerful leak conductance, with the extreme synchrony owing much to the dense recurrent connectivity of these networks (Contreras et al., 1996; Sanchez-Vives and McCormick, 2000).

The recent avalanche of interest in the slow oscillation notwithstanding, neither its up- nor its down-state should be considered to be analogous to a conscious activated brain state (Steriade, 2000), nor is there yet sufficient evidence to ascribe to the phenomenon the role of a computational mechanism in the intact, awake brain, despite the demonstrated importance of persistent neural activity in other systems (Aksay et al., 2003; Major and Tank, 2004). Nevertheless, because of the well-characterized differences in the resting membrane potential and input resistance of single neurons in the two states, the slow-oscillating brain presents an opportunity to address a fundamental question in neurophysiology: how are the properties of an excitable tissue *en masse* shaped by changing the responsiveness of the individual elements which comprise it?

In this study we used tungsten bipolar electrodes to monitor the cortical LFP and multiunit activity (MUA) while recording whisker-evoked responses over the entire barrel cortex of the mouse *in vivo* using VSD imaging with either RH795 or RH1691. LFP and MUA recordings allowed us to characterize in detail the spontaneously occurring state changes observed under ketaminexylazine anesthesia and classify the sensory responses into three categories according to their relationship to the spontaneous oscillation. We used single-whisker stimuli, paired simultaneous stimuli, and stimuli paired at a 100 ms interval. These stimulation conditions represent a sampling of common paradigms in the whisker sensory literature and in sensory physiology more generally. Our goal was to revisit each of these common conditions with attention to network context.

We observed that evoked sensory responses both altered and were altered by the ongoing network activity in important ways. Using barrel-column averaged VSD signals to examine response properties across the entire barrel field, we show that the properties of sensory responses are altered by the shift to the up-state in ways which go beyond what has been previously reported. We find that, in the up-state, whisker deflections produce an initial depolarization which is smaller in amplitude and area than that observed in the down-state and more likely to propagate asymmetrically across the barrel field. In the up-state, but notin the

down-state, this depolarization is followed by a hyperpolarizing wavewhichmovesin a consistent direction across cortical columns. We also show that, regardless of state, there is little difference between single- and multiple-whisker stimuli in the spatial extent of subthreshold evoked responses. Finally, we show that the degree of suppression of a second response depends more strongly on the second stimulated whisker's location than on the relative radial position of the first stimulated whisker, regardless of brain state. Because the resting level of cells in the up-state is likely to be closer to that in the awake animal, we suggest that the polarities in signal propagation which we observe in the up-state could be used as computational mechanisms in the behaving animal.

# **MATERIALS AND METHODS**

Results are based on 40 adult C57 mice (7–12 weeks old, 20–35 g). From these, 30 experiments were selected for analysis based on the following criteria: (1) homogeneous VSD staining of the barrel cortex as judged by visual inspection of the baseline image (14-bit) obtained at the beginning of each recording, (2) stability of the optical responses throughout the experimental session, (3) stability of the spontaneous LFP pattern recorded from the same electrodes used for electrical stimulation, (4) stability of the evoked LFP responses recorded by electrodes adjacent to the stimulating one, and (5) large signal-to-noise ratio (10:1) of VSD responses to deflections of at least two different vibrissae.

### *Surgery and preparation*

Mice were deeply anesthetized with ketamine-xylazine (100 mg/ kg i.p., 20 mg/kg i.p., respectively,) and mounted in a stereotaxic apparatus. Supplemental anesthesia (25 mg/kg, 5 mg/kg) was administered as necessary to maintain cortical slow oscillations and weak or absent foot withdrawal reflex. A craniotomy was made which extended 2 mm in the anterior-posterior direction starting from bregma, and 2–4 mm in the mediolateral direction starting from the midline. In most animals this was sufficient to expose most of the posteriomedial barrel subfield (PMBSF). The dura was resected over the entire craniotomy.

Once electrodes were inserted, hand stimulation of the whiskers with audio feedback from the cortical LFPs was used to determine the approximate location of the electrodes within the PMBSF. This information was used to determine the whiskers most suitable for VSD imaging.

### *Staining*

Following Kleinfeld and Delaney (1996), a 1 mm3 piece of gelfoam (Upjohn Pharmacia) was soaked in a warm solution of the VSD RH795 (Molecular Probes, Eugene, OR, 1 mg/mL) or RH1691 (Optical Imaging, Mountainside, NJ, 1 mg/mL) in 0.9% saline and placed on the exposed cortex. Additional dye was added to keep the gelfoam soaked for 1.5 h. After staining and before recording the exposed surface of the brain was generously washed with saline to remove unbound dye. Throughout the experiment the brain surface was rinsed with saline to prevent desiccation. RH795 (Grinvald et al., 1994; Obaid et al., 2004) and RH1691 (Shoham et al., 1999) are potentiometric styryl dyes which attach to cell membranes and show a decrease (RH795) or increase (RH1691) in fluorescence on a microsecond time scale in response to membrane depolarization. For consistency with convention all VSD responses shown here are oriented so that positive-going deflections indicate depolarization. When applied topically *in vivo*, the dyes stain the supragranular cortical layers most intensely (Kleinfeld and Delaney, 1996; Petersen et al., 2003a). Potentiometric dyes are linear indicators of *Vm* over physiological ranges (Cohen and Salzberg, 1978; Cohen et al., 1978). The dye is taken up preferentially by dendrites and cell bodies. Because layers 2/3 are primarily neuropil, the signal source *in vivo* is considered to be mostly from dendrites (Grinvald et al., 1994; Yuste et al., 1997; Contreras and Llinas, 2001), although a recent detailed stereological analysis revealed that far more axonal than dendritic membrane per unit of volume is present in the neuropil of layers 2 and 3. Some contribution also comes also from glial cells (Konnerth et al., 1987; Salzberg, 1989).

# *Optical recordings*

Recordings were made with a modified upright microscope (Olympus, BX50WI). Epi-illumination was provided by a 12V halogen lamp. Excitation light was bandpass filtered at 540 ± 20 nm; light emitted from the preparation was long-pass filtered below 600 nm. The optical signal was collected with a CCD camera (MiCam01 or Ultima, BrainVision, Japan) with a detector array of 87 × 60 (Micam01) or 100 × 100 (Ultima) imageable pixels, at frame rates of 250–500 Hz (2–4 ms/frame). Frame times given in figures and text refer to the end of acquisition of a frame; for example, a frame labeled 24 ms and acquired at 250 Hz is a measurement of light emitted from 20–24 ms poststimulus. The microscope objective was 4X (N.A. = 0.28, Olympus, Japan). With the Micam01, an Olympus U-TV0.35×C demagnifying camera adapter was used, resulting in a total magnification of 1.4×, imageable area of 1.5× 2 mm, and a pixel size of 22 by 22µm (484µm2). With the Micam Ultima, the camera adapter was not used, resulting in an imageable area of and 2.5 by 2.5 mm with a pixel size of 23 by 23 µm (529µm2). Optical recording was controlled by the BrainVision software.

The fractional fluorescence change received little contribution from intrinsic metabolic signals related to oxygen delivery. No data was analyzed beyond 250 ms after a stimulus, i.e., the time between the first whisker deflection and the peak response to a second whisker deflection 100 ms later. In contrast, hemoglobinassociated absorbance changes have been shown to begin several hundred milliseconds poststimulus. For example, in Devor et al. (2003), blood-flow related signals were recorded from 1.5 to 2.5 s after the stimulus whereas LFP and MUA were integrated from 0 to 300 ms after the stimulus. The "early signal" or "initial dip" corresponding to the increase in deoxyhemoglobin by oxygen delivery to neurons takes almost 1 full second to develop (Frostig et al., 1990; Kim et al., 2000). Additionally, in experiments with RH1691, which is not sensitive to hemoglobin changes due to its shifted absorbance spectrum, the kinetics of the responses were identical to those seen with RH795.

### *Electrophysiological recordings*

To record LFPs, we constructed arrays of 3 or 4 pairs of tungsten electrodes (FHC, Bowdoinham, ME), with vertical tip separation of 0.5 mm and horizontal separation of 0.75 mm between pairs in the array. For each experiment the array was advanced into the cortex at the lateral edge of the craniotomy, normal to the cortical surface, until the upper electrode just entered the pial surface. Recording and stimulation were in bipolar configuration. The signal from the electrodes was band-pass filtered between 0.1 and 300 Hz to obtain LFP recordings, and between 300 and 10000 Hz to obtain multiunit recordings (MUA). Measurements of MUA and LFP were made at the periphery of the stimulated area, to allow for imaging over the greatest possible cortical area.

# *Whisker stimulation*

A reliable optical response was first obtained from a single whisker in response to a 100 ms ramp-and-hold deflection (8 ms rise time, 1300◦/s, calibrated as for (Wilent and Contreras, 2004) in the ventral direction with a piezoelectric device (Simons, 1983). For simultaneous stimulation of two whiskers, two whiskers were threaded into the same piezo-controlled tube. For multiwhisker sequences, an air puff (Picospritzer, Intracel, Herts (UK), 10 ms, 2–5 PSI) was directed to deflect another single whisker or group of whiskers in the ventral direction using a 1 mm diameter glass capillary tube bent into an "L" shape. Great care was taken to ensure that no whiskers were unintentionally stimulated; when necessary, nearby whiskers were trimmed away. Once clear responses had been evoked from two whiskers, pairs of deflections were presented at an interstimulus interval (ISI) of 100 ms.

# *Cytochrome oxidase histology*

At the conclusion of an imaging experiment, two fiducial marks were made by advancing an electrode into two different locations in the imaged cortex. Reference images in register with the VSD recordings were taken with these new marks. Animals were perfused with 4% paraformaldehyde in 0.1 M sodium phosphate buffer (PBS). Brains were postfixed overnight in the same fixative and the cortex was flattened by pressing gently between two clean microscope slides submerged on PBS. One hundred micron thick tangential sections were cut in a vibratome (Vibratome 1000-plus). In order to reveal the barrels, tissue was treated with 3,3 -Diaminobenzidine (DBA, Sigma D-5905) and Cytochrome C from horse heart (Sigma C-2506) according to (Wong-Riley 1979) with some modifications. Briefly, sections were washed in 0.1M PBS (3×10 min) at room temperature and incubated in a mixture of 0.1 M PBS with 10% methanol (Fisher Scientific BP1105-1) and 1% hydrogen peroxide (Sigma H-1009) for 15 min at room temperature, washed again in PBS (3×10 min) and kept in the dark shaking overnight at 4◦C in 0.1 M PBS containing 4 g sucrose, 50 mg DBA (Sigma) and 30 mg of cytochrome oxidase per 100 ml of PBS. The following day, tissue was washed in PBS, mounted on subbed glass slides, dehydrated, and coverslipped.

# *Trial sorting*

Optical data was collected as single trials, with no online blank subtraction or online averaging. We identified transitions between states by visual inspection of the simultaneously and continuously recorded LFP from S1 cortex, and classified the single-trial optical recordings into three groups: trials occurring in the down-state, trials occurring in the up-state, and trials in which the stimulus caused a transition to the up-state.

# *Barrel binning*

The tracks left by the field potential electrodes, in combination with the additional fiducial marks made at the end of the experiment, were used to align the barrel outlines from histology with the fractional fluorescence images. This allowed binning of pixels into signals corresponding to the average activity within barrelcolumns. The margin of error for the alignment of barrels was the width of a single electrode, which is close to the width of septa.

# *Data analysis*

Optical data was collected as differential fluorescence and divided by a reference image acquired automatically at the start of each trial to produce fractional fluorescence (*-*F/F) data which was used for all analysis and figures. Post-processing consisted of averaging of single trials after screening (see below) followed by binning into barrels as described above. For clarity the images shown in **Figure 4** were filtered with a flat 3 × 3 kernel. All analysis was done with custom routines written in Igor Pro (Wavemetrics, Lake Oswego, OR).

# **QUANTIFICATION OF RESPONSES**

Area functions (**Figure 4B**, right) were computed by counting the number of pixels in each frame with values at least 2 SD above the prestimulus noise at that pixel. These values were then multiplied by the pixel size to produce areas in mm2. The number of barrel columns participating in a response (**Figures 5** and **6**) was measured by first projecting the histologically identified barrel map onto the optical recording using fiducial marks (see Civillico and Contreras, 2006 **Figure 1**), and binning the pixels into columnsized regions accordingly. Average time traces were then taken using these barrel-column regions of interest. The determination of a response for barrel column counts was performed by comparing the maximum amplitude of each barrel column signal in a 100 ms window following the stimulus with a threshold based on the prestimulus SD of the barrel column signal.

### **POLAR AMPLITUDE PLOTS**

Polar amplitude plots (**Figures 7**, **8**, **9**, **10**) were computed as follows: a grid schematic of the barrel field was divided into eight radial half-quadrant sectors with the stimulated whisker taken as the center. Each barrel column accessible to the VSD recording was mapped to a sector based on its angular position relative to the stimulated whisker on the schematized grid. The values of the mapped quantity (amplitude, slope, latency, etc.) were averaged for each radial sector and plotted using polar coordinates.

Polar amplitude plots in **Figures 7**, **8**, and **10** were normalized by maximum polarity as follows: each eight-point plot consisted of four pairs separated by 180◦, representing four possible axes of polarity. The largest polarity or maximum anisotropy was identified by comparing the members of each of these four pairs, and the smaller value of that pair was used to normalize the histogram.To facilitate comparison with the center hyperpolarization, polar amplitude plots in **Figure 9** were normalized according to the value of the center hyperpolarization.

### *Quantification of divisive suppression*

To quantify the divisive suppression of whisker responses as a function of radial position, the transient ratio (TR) was calculated

as the magnitude of the second whisker transient following a preceding whisker response, divided by the size of the second whisker response alone. TR polar amplitude plots for second whisker responses were calculated as for the single response measurements, with different choices of origin as described in results.

# **RESULTS**

# **LFP AND MUA WHISKER RESPONSES DURING THE SLOW OSCILLATION**

The slow oscillation under anesthesia is characterized by spontaneous and widespread changes in membrane potential and firing probability. Our goal was to examine the effect of these well-characterized rhythmic neuronal excitability changes on the spatiotemporal properties of sensory responses measured with voltage sensitive dyes (VSDs) *in vivo*. In order to understand the effect of network state on sensory responses it is critical to first understand the variability of the oscillation under anesthesia and, perhaps more importantly, the interaction between single stimuli and the spontaneous cycles of oscillation. We reasoned that, if down-state stimuli could trigger new up-states, such responses must be analyzed separately from the other down-state responses. Because the period of the slow oscillation is much longer than the optical recordings, to monitor the spontaneous electrical activity on which responses were superimposed, we recorded the LFPs and MUA from the primary somatosensory cortex. The recordings shown in **Figure 1** illustrate the rich variability of the anesthetized preparation. Under deep and stable anesthesia, the spontaneous LFP was characterized by a prominent and stable slow oscillation (*<*1 Hz) consisting of alternating positive and negative waves (**Figures 1A,C,E**, **2**). Deepening of anesthesia resulted in an increase in amplitude and decrease in frequency of the slow oscillation, which reached values as low as 0.3 Hz (**Figure 1E**). Conversely, lightening of anesthesia was characterized by a progressive reduction in the amplitude of the slow oscillation (**Figures 1B,D** and **Figures 2A,B**, "activated"). We measured the bistability of EEG patterns by computing the distribution of amplitude values over 10 min periods (middle column,

distributions). When the EEG alternated between two stable states, the distributions contained two separated peaks (A, E), or occasionally a peak with a prominent shoulder, indicating a greater proportion of intermediate values due to frequent spindles (C). We computed power spectra for 10 min EEG periods. During periods of deep anesthesia the power spectrum was dominated by a peak between 0 and 1 Hz; under lighter anesthesia this peak was absent. For the data presented below, we controlled the level of anesthesia as to maintain a stable slow oscillation (see below **Figure 2**). We will refer to the negative phase of the LFP during the slow oscillation, in which neurons are depolarized, as the up-state, and to the quiescent, depth-positive waves, in which neurons are hyperpolarized, as the down-state.

Whisker stimuli were delivered at a fixed frequency (0.1–0.2 Hz) and, therefore, occurred randomly with respect to the ongoing slow oscillation. Because the stimulus frequency was much slower than the oscillation frequency, we recorded many oscillation cycles which contained no stimulus. The shape of the whisker responses was strongly modulated by the slow oscillation and, reciprocally, whisker responses entrained slow oscillation cycles, both in a phase-dependent manner (**Figure 2**). To illustrate the phase dependence between whisker responses and the slow oscillation, we ordered the LFP responses according to the phase in which they occurred (**Figure 2A**). Stimuli occurring in the second half of the down-state (*>*420 ms after the termination of the preceding up-state in this example) triggered a new up-state (**Figures 2B** and **C**, blue trace). Stimuli delivered early during the down-state did not trigger a new up-state but generated a large monophasic response lasting approximately 200 ms (**Figures 2B** and **C**, green trace). When stimuli occurred during the up-state, responses consisted of a short negative wave, followed by a positive wave and a return to the steady level of the up-state (**Figure 2B**, brown trace). Stimuli delivered during occasional periods when the slow oscillation was absent (similar to an activated EEG) (**Figure 2A**, bottom trace, detail in **Figure 2B**) generated a smaller amplitude response not followed by a large hyperpolarization. We did not observe changes in duration of the down-state when the stimuli occurred during the up-state, nor did we observe premature termination of the up-state by the occurrence of a whisker response. A typical triggered up-state is shown in **Figure 2C** (blue trace) with respect to a spontaneous cycle (red trace) and with respect to a response that does not trigger an up-state (green trace).

We quantified the effect of whisker stimulation on the ongoing activity by plotting the distribution of the durations of the downstates (*n* = 16 mice, 3968 total oscillatory periods; **Figure 2D**). To compensate for the variability in oscillation period across experiments, we normalized the down-state durations within each experiment to the median spontaneous duration for that experiment. The normalized distributions were added to obtain population distributions (**Figure 2D**, left), which were integrated to obtain population cumulative histograms (**Figure 2D**, right). The distributions show that down-states that were terminated by a sensory triggered up-state (blue trace) were significantly shorter than those terminating spontaneously (red trace, *p <* 0*.*01, K–S test), demonstrating that responses to whisker stimulation effectively reduced the duration of the down-state by triggering a new cycle of the slow oscillation. Responses that failed to trigger an up-state (green trace) occurred significantly earlier (hence, closer to the termination of a previous oscillatory cycle) than those that succeeded (green histogram vs. blue histogram, **Figure 2D**, *p <* 0*.*01, K–S test), which shows that the slow oscillation has a refractory period, as shown previously *in vitro* (Sanchez-Vives and McCormick, 2000).

The phase-dependence and duration of the LFP response was paralleled by the MUA recorded on the same electrodes (**Figure 3**). Spontaneous spiking activity occurred exclusively during up-states and paralleled the duration of the up-states recorded in the LFP, demonstrating that up-states represent sustained depolarization (**Figure 3A**). We quantified the MUA by constructing peristimulus time histograms (PSTH) from spikes isolated using an arbitrary threshold (**Figure 3A**, red line). Stimuli arriving during established up-states caused a short-latency multiunit response followed by a trough indicating inhibition which was strong enough to completely suppress firing for at least 50 ms (**Figure 3B**, top panel). Stimuli arriving early in the downstate triggered a phasic response which was sometimes followed by a temporally broader, lower-amplitude rebound (**Figure 3B**, middle panel). As described for LFP responses, stimuli arriving later in the down-state triggered an up-state (**Figure 3B**, bottom panel) and the response was characterized by a large initial MUA peak, followed by a valley and then a resumption of the activity level seen in spontaneous up-states. Neither the initial peak nor the valley were present in the firing pattern during spontaneous up-states (**Figure 3B**, bottom panel, red open histograms), indicating that the sensory response was to a certain degree additive with the up-state which it triggered. Thus, the slow oscillation showed a refractory period after which sensory responses to whisker deflections successfully triggered a new upstate. However, sensory responses did not terminate or delay an up-state. Furthermore, hyperpolarizing inhibition was only present in the sensory responses during the up-state. This is likely due to several effects of the more depolarized membrane potential, including an increased driving force for *GABA*-mediated chloride currents in pyramidal cells, as well as the increased firing probability of inhibitory interneurons. Thus, our results demonstrate that sensory responses are not only modulated by spontaneous activity, but also entrain it. Furthermore, sensory responses engage local circuit inhibition in a state-dependent manner.

#### **RESPONSES RECORDED WITH VOLTAGE SENSITIVE DYES**

The LFP (**Figure 2**) and MUA (**Figure 3**) recordings illustrate the state-dependence of responses at single locations. To quantify this state-dependence over the entire surface of the barrel cortex, we recorded the population *Vm* response from the supragranular layers with VSDs (**Figure 4**). The simultaneous LFP and MUA recordings allowed determination of the state of the network at all times, providing an independent means to classify the VSD responses for analysis. Optical responses were sorted according to state based on the simultaneously recorded LFP and 10 trials per state were averaged together to produce three sub-averages (for consistency with intracellular recordings all VSD responses shown here are oriented so that net depolarization is indicated by upward deflections). In the example shown in **Figure 4A**, a deflection of whisker D2 during the up-state (top row) resulted in a low-amplitude depolarizing response, beginning above the D2 barrel at 22 ms and spreading to all other barrels, which was followed by a large and widespread hyperpolarization visible over the entire barrel field at 172 ms. Responses occurring early in the down-state (middle row) showed a larger depolarization and an absence of hyperpolarization, and failed to trigger an up-state. When the stimulus occurred later in the down-state and triggered an up-state (bottom row), the response was larger and of shorter latency than that occurring during the down-state. Consistent with the LFP recordings, the triggered up-state was the largest optical response and lead to a strong widespread depolarization with a long latency and long duration. The difference in response amplitude between states can be best appreciated in the fluorescence signal averaged over the principal barrel being activated (**Figure 4B**, left column, D2 in this example) but may also clearly be observed in the column-averaged signal in individual

trials (thin traces). That such differences are robust across the surface is shown by the fact that single pixel, single trial behavior is consistently similar to the average (**Figure 4B**, middle column). The peaks of the fluorescence responses during the down- (blue traces) and the up-(red traces) states were 49% and 15% of the peak of the triggered up-state (blue traces), respectively. Finally, consistent with its corresponding LFP response, the VSD response in the up-state returned to baseline more quickly than in the other states (∼50 ms).

Clear responses with these same basic features were visible in state-averaged recordings from single pixels (529µm2, **Figure 4B**, middle column). To quantify the differences in the spatial spread of the responses, we measured the total depolarized area (**Figure 4B**, right column, see "Methods") as a function of time. In the representative example of **Figure 4**, the depolarized area of the response during the down-state reached a maximum of 62% of the visible barrel cortex (1.4 mm2 at 56 ms (**Figure 4B**, right, black trace). In contrast, in the up-state the depolarized area was smaller (**Figure 4B**, right, red trace), consistent with a previous report (Petersen et al., 2003b), reaching a maximum of 24% of the visible barrel cortex 0.45 mm2 at 40 ms. This depolarizing response was followed by a hyperpolarization encompassing the entire visible barrel cortex by 150 ms (top row, rightmost frame). Finally, the depolarization of the triggered-up state reached 0.93 mm2 by 34 ms, and extended over 1.6 mm2 by 150 ms (**Figure 4B**, right, blue trace). Only the response in the up-state had a significant hyperpolarizing component. The average of the population (*n* = 22) measurement of depolarized area is shown in **Figure 4B**. Because the triggered up-state responses consist almost entirely of self-generated recurrent activity in cortex, their spatiotemporal

properties are not comparable with those of the other sensory responses. Therefore, we will not consider the triggered-up state responses further. Thus, the remaining analysis will be concerned only with responses during up-states and responses during downstates that did not trigger an up-state.

Inspection of the up-state responses in images such as those shown in **Figure 4A** suggested that despite being fewer in number, activated pixels in the up-state were distributed across a large portion of the visible barrel cortex, suggesting that the number of barrel columns activated may not significantly differ across states. Such an observation would imply that, rather than "sharpening" a simple sensory representation, depolarization of the cortical network would instead function to make the representation sparser while remaining distributed. In order to quantify and compare the number of responding barrel columns across states we averaged the fluorescent signal inside each barrel column area (see "Methods"). We counted as responding barrel columns (*n* = 30 whisker responses in 20 mice) those in which fluorescence transient values crossed an arbitrary threshold expressed in units of standard deviation (SD, *z*-scores) of the prestimulus baseline (**Figure 5**). Because varying threshold revealed a difference in the number of barrels showing a response, we systematically explored the effect of detection threshold in the comparison between upand down-states. As expected, the number of barrels included in the response decreased with increasing threshold (**Figure 5A**). More importantly, the effect of increasing threshold was parallel between up- and down-states (**Figure 5A**, red and black trace, respectively); for all thresholds in which there was a difference there were more barrels responding in the down-state. At a threshold of 3SD, all barrels included in the image were responding in both states. But even at 5SD most of the barrels responded during both down- and up-states. We quantified the difference between up- and down-states by measuring the threshold level at which the number of barrel-columns participating in the response crossed below half of its maximum. Because we converted response amplitudes into *z*-scores (response amplitude divided by prestimulus SD), we refer to this value as *z*1*/*2. In all experiments, *z*1*/*<sup>2</sup> was lower for the up-state than for the downstate. The population average of the ratio of *z*1*/*<sup>2</sup> between up and down-state was 0.40 ± 0.03 (**Figure 5B**, mean and SEM indicated by the red circle with error bars).

To compare the number of depolarizing columns between the up- and down-states for the population, we arbitrarily chose two thresholds, one low (*z* = 2) and one high (*z* = 10). (**Figure 5C**, left). When the threshold was 2SD above background noise (minimal response, *z* = 2 in **Figure 5C**), we observed that the majority of barrel columns participated in responses in both up and down states, and the difference between the two states was statistically significant (Wilcoxon Signed Rank Test, *p <* 0*.*01). The column counts at lower threshold values showed that some degree of depolarizing response propagated over most of the imaged area in both the up- and down-states. Thus, at the level of barrel columns, responses in both the up- and the down-states spread over almost the entire barrel cortex. By combining pixels into a barrel column signal, we decreased the noise and became able to resolve a smaller response. The number of responding columns in the up-state is greater than what has been implied by previous work (Petersen et al., 2003b), a difference we attribute to an increase in sensitivity over previous work due to (1) binning pixels over a columnar area and (2) recent improvements in CCD technology. Because the entire barrel field could not be imaged in all experiments, we normalized the number of responsive barrels by the total number of imaged barrels in order to better compare population data between states (**Figure 5C**, right); the effects were unchanged. When the threshold was increased to 10 SD above background noise, fewer columns responded in both up- and down-states, but the decrease was greater in the up-state (decrease was from 98% to 41% in the down-state, and from 74% to 6.3% in the up-state).

#### **RESPONSES TO TWO WHISKERS**

Because under natural conditions many whiskers are stimulated simultaneously, and because single whisker responses have been shown to influence one another, we extended our context-based analysis to multiwhisker responses. In a subset of experiments (8/30), we threaded two adjacent whiskers into the glass of the same piezoelectric stimulator. This had no effect on the average

**FIGURE 5 | Columnar extent of depolarizing responses in up and down-states. (A)** Count of participating barrels as a function of chosen fluorescence (*z*-score) threshold for down-(black) and up-(red) states in an example experiment. Vertical dotted line indicates a *z*1*/*<sup>2</sup> of 1. **(B)** Up-state *z*1*/*<sup>2</sup> values for all experiments (each normalized to the corresponding down-state); red dot indicates mean. **(C)** Raw (left) and normalized (right) column counts for minimal response (*z* = 2) and large response (*z* = 10) thresholds in up- and down- states. Statistically significant differences between up and down-state counts (Wilcoxon Signed Rank Test, *p <* 0*.*01) are indicated with the ∗.

columnar extent of responses in either the up- or the down-state. We mapped the peak response in units of SD (*z*-scores) at all pixel locations and superimposed the histologically determined barrel map onto these maps; this showed that in both states there were statistically significant responses in all columns (**Figure 6A**, color scale starts at 2SD). The same effect is seen in the barrel-column averaged pixel traces (**Figure 6A**, bottom panels). We averaged the extent of columnar responses as for single-whisker responses and found the same effect (**Figure 6B**). In all subsequent data analysis, single and multiple whisker responses are pooled. This strong sublinearity of the summation of responses from more than one whisker when activated at a negligible time interval is consistent with our earlier detailed study of multiwhisker response interactions (Civillico and Contreras, 2006).

#### **SPATIAL ASYMMETRY IN THE DEPOLARIZING RESPONSE**

We compared barrel-column averaged traces from different parts of the barrel field and detected a consistent asymmetry in spatial distribution of response amplitude in the up-state only (**Figure 7A**). In the example of **Figure 7A**, after stimulation of D2 during the up-state, the peak amplitude of the response in the D1 column was 50% less than that in the D2 column, whereas the amplitude was unchanged from D2 to D3. This effect was not seen in the down-state. There was a similar but less pronounced effect along the arc axis: in the down-state, the responses in C2 and E2 were the same size as in D2 (**Figure 7A**, left). However, in the upstate, the response in the E2 column was 5% larger, whereas that in the C2 column was 5% smaller. This result shows that in the up-state, responses were asymmetric with respect to direction of propagation.

We quantified this asymmetry of propagation by binning the barrel field into eight radial quadrants with the stimulated whisker at the center, and generating polar plots of response amplitude (**Figures 7B** and **C**). We normalized the polar plots in order to compare their shapes (see "Methods"). The bias toward higher numbered arcs seen in the example traces (**Figure 7A**) is clear in the normalized barrel column polarity plot for this example (**Figure 7B**, upper panel) and was also clear in the entire population of barrel column-averaged recordings (**Figure 7B**, lower panel). In the direction of maximum asymmetry (indicated by the arrow in 7B) the average response in the preferred propagation direction was 1.6 times greater than the anti-preferred direction in the up-state vs. 1.3 times greater in the anti-preferred direction in the down-state.

The asymmetry in amplitude of the barrel-averaged responses could be due to a larger number of responding pixels within the barrel or to a greater response from the same number of pixels. Thus, we measured the percentage of responding pixels (response density) within all imaged barrel-columns as a function of time and plotted the value as radial averages in the same way as with the response amplitude above (**Figure 7**). Up-state responses used a smaller percentage of the barrels (averaging 20% vs. 60% in the down-state), consistent with the more granular responses in the up-state images shown in **Figures 4** and **6**. There was no significant polarity to the peak densities of responses indicating that the asymmetry in the spatial distribution of barrel column-averaged signals during up-state responses (**Figure 7B**) is due to differences

in amplitude and not to differences in the number of responding pixels within each barrel-column.

are indicated with the <sup>∗</sup>. (Right) Comparison of *z*1*/*<sup>2</sup> values in down- and

up-states for single whisker and multiwhisker experiments.

We also examined the polarity of the response time course. There was no significant polarity to the onset latency (**Figure 8**, left). Rise times, however, were significantly longer in the downstate toward the A row and lower-numbered arcs. In the up-state the rise times were more evenly distributed (**Figure 8**, middle). The initial slopes of responses were significantly larger in the down-row direction in both up- and down-states, but the effect

was more pronounced in the up-state (**Figure 8**, right). In summary, the population depolarizing responses at the subthreshold level show greater asymmetry when the network is depolarized: the depolarizing response spreads preferentially along the rows toward higher-numbered arcs.

#### **SPATIAL ASYMMETRY OF THE RESPONSE HYPERPOLARIZATION**

We next studied how subsequent whisker deflections would be differentially affected by the state of the network. Previous studies have demonstrated strong suppression of whisker responses by a combination of well-known postsynaptic cellular mechanisms (Coombs et al., 1955; Llinas et al., 1974; Simons, 1985; Simons and Carvell, 1989; Holt and Koch, 1997; Moore and Nelson, 1998; Higley and Contreras, 2003, 2005; Civillico and Contreras, 2006). Subtractive effects are due to a reduction in the peak level of a subsequent response due to the lowering of the baseline caused by a preceding response. As shown by the two examples in **Figure 9A**, whisker responses during the up-state were followed by a sharp hyperpolarization below baseline. We measured this hyperpolarization at 100 ms after a single stimulus and found that it had a consistent spatial pattern across experiments (**Figure 9B**). It appeared first over the barrel columns corresponding to more ventral whiskers (the D and E row), where it also reached its largest amplitude (see barrel traces in **Figure 9A**).

This spatial pattern did not appear as so-called "surround inhibition" (Derdikman et al., 2003, their **Figure 6**), since the wave moved indiscriminately across the stimulated whisker's barrel. In other words, the stimulated whisker's barrel participated in the wave at the same time as its neighbors, rather than after a delay or not at all, as in what is often described as a "surround" response. To demonstrate this, the average values in **Figure 9B** were normalized to the level of the stimulated whisker's barrel. We note that many values in the surround have normalized values less than 1, indicating that the spatial pattern of inhibition was greater at the center than in many of the surround locations. Thus, whisker deflection-triggered inhibition resembles more a wave than an annulus around the principal barrel. As expected, the hyperpolarizing wave was not observed in the down-state, most likely due to the hyperpolarized membrane potential that characterizes this state (Steriade et al., 1993a,c).

Finally, to measure the effects of network state on input integration, we performed experiments in which a second whisker was deflected 100 ms after the first one. This interstimulus interval produces the strongest whisker-to-whisker suppression (Civillico and Contreras, 2006) and seems equivalent to the 20 ms interval in the anesthetized rat (Simons, 1985; Shimegi et al., 1999, 2000; Higley and Contreras, 2003). We quantified the suppression of the second response by calculating a TR, as the magnitude of the response following the first whisker, divided by the size of the response alone (**Figure 10**). A complete suppression of the second response gives a TR of 0 while an unchanged second response will result in a TR of 1. Values of TR above 1 indicate response facilitation. Unlike single-cell experiments, these experiments produce values of TR for the entire spatial extent of the barrel field. As in the previous analyses, we binned the barrel field into eight radial quadrants so as to compute averaged TR values as a function of angular position with respect to the second stimulated barrelcolumn. An example of this analysis for a single experiment is shown in **Figure 10A** (barrel maps at left, polar plot at right). The origin of the polar plot is the location of the second stimulated whisker's barrel. The angular location of whisker 1 relative to whisker 2 is indicated "w1" and in this example corresponded to 225◦ (positive angles moving counterclockwise, origin at right).

The outer radius of the polar plot corresponds to a TR of 1, indicating no suppression. All angular locations relative to the second stimulated whisker showed some degree of suppression (TR *<*1) in both the up- and down-states (consistent with our previous study (Civillico and Contreras, 2006) in which trials were not sub-averaged by state). Suppression was greater in the down-state. However, the spatial pattern of suppression shown in the barrel maps (**Figure 10A**, left and middle columns) was similar in the up- and down-states: suppression was greatest in the part of the barrel field corresponding to the ventral posterior whiskers (arcs 1,2, rows D,E), and decreased in a gradient toward the barrels corresponding to more dorsal and more anterior whiskers. In the example shown the direction of greatest suppression corresponds to the direction of the previously stimulated whisker's barrel. When many angular plots were averaged with the whisker 1 directions aligned (*n* = 10, **Figure 10B**, left), suppression was clearly seen to be biased in the direction of whisker 1 (indicated by "w1" and arrow on polar plot). However, when the radial TR plots were rotated and re-averaged to be oriented to the barrel field map (i.e., with w1 directions pointing in their original various directions), the directional bias of suppression, rather than disappearing, became even more pronounced (**Figure 10B**, right). This indicates that the effect was not entirely caused by the location of the previously stimulated whisker in either up- or down-states, but rather appeared to be yet another intrinsic polarity of the barrel field, possibly related to the row bias observed with the single-whisker responses.

# **DISCUSSION**

In this study, we examined the effect of ongoing network activity on the spatiotemporal properties of population responses measured with VSD imaging. We used LFP and MUA recordings in the anesthetized mouse to confirm the presence of two welldefined network states in our preparation and to characterize the phase dependence of the interaction between the sensory stimulus and the ongoing oscillation. We found important variability in the slow oscillation, for which we accounted in our analysis. We observed that down-state stimuli could trigger a transition to the up-state only after the refractory period following each cycle of the oscillation. We excluded responses triggering up-states from the analysis of down-state responses because triggered up-states mainly consist of self-generated recurrent cortical activity. After classifying VSD trials by state using the simultaneous LFP recordings, we compared the spatiotemporal properties of the sensory responses and their interactions across the entire barrel field between up- and down-states.

In the up-state, whisker deflections triggered a distributed depolarization which was smaller in amplitude and area than that observed in the down-state, but nonetheless involved most of the barrel field. The spatial distribution of whisker-driven excitation during the up-state was characterized by (1) fewer barrel columns, (2) lower response density (smaller number of pixels per responding barrel column) and (3) asymmetrical amplitude distribution across the barrel field with higher response amplitude along the row-axis toward higher numbered arcs, with no matching change in response density. Although fewer columns showed high-amplitude responses in the up-state, using barrel-column averaged VSD signals we found that most columns participated in the responses in both states. As expected, this meant that we observed little change in number of participating columns when comparing one and two-whisker responses. In the up-state, but not in the down-state, response depolarization was followed by a spatially distributed hyperpolarizing response, which propagated along the arc-axis, preferentially toward the A row. Finally, we examined the state-dependence of the extensively studied phenomenon of cross-whisker suppression. At a 100 ms deflection interval, we found that suppression showed an asymmetry similar to that of the up-state depolarizing response which was apparently independent of the relative locations of stimulated whiskers. This asymmetry was not dependent on network state.

### **THE SLOW OSCILLATION**

The dynamic cellular relationships in cortex and thalamus during the slow oscillation have been extensively characterized with single and dual intracellular recordings *in vivo* (Steriade et al., 1993a; Amzica and Steriade, 1995; Contreras and Steriade, 1995; Contreras et al., 1996) and in cortical (Sanchez-Vives and McCormick, 2000) and thalamic (Blethyn et al., 2006) slices *in vitro*. It is known that in the up-state most cortical and thalamic cells are depolarized by ∼10–20 mV relative to the down-state, that they receive barrages of synaptic input in which excitatory and inhibitory drive are balanced (Compte et al., 2003; Shu et al., 2003a; Haider et al., 2006), and that the power in the gamma band (20–50 Hz) of intracellular *Vm* and extracellularly recorded LFP activity is increased (Steriade et al., 1996). This high-frequency activity is synchronous only over short distances (Steriade and Amzica, 1996), in contrast to the long-range synchrony of the slow oscillations (Amzica and Steriade, 1995; Destexhe et al., 1999). Both experimental and modeling studies have shown that the input resistance of cortical cells is lowered by the increased synaptic drive during depolarized states (Destexhe et al., 2003; Shu et al., 2003b; Sachdev et al., 2004 but see Waters and Helmchen, 2006).

#### **THE SLOW OSCILLATION AND EVOKED RESPONSES**

The effect on evoked responses of ongoing transitions between up- and down-states has been studied with intracellular and whole-cell patch recordings and has led to conflicting results. Single cell responses to electrical stimulation in cat motor cortex (Timofeev et al., 1996; Rosanova and Timofeev, 2005), to visual stimulation in cat visual cortex (Azouz and Gray, 1999) and to electrical stimulation of white matter in slices *in vitro* (Shu et al., 2003b) have shown a higher spike output probability during up-states compared to down-states. Conversely, responses to whisker deflection in mouse (Petersen et al., 2003b) and rat (Sachdev et al., 2004; Hasenstaub et al., 2007) barrel cortex have shown a decreased firing probability. While depolarization during the up-state brings neurons closer to firing threshold, seemingly increasing the probability of spike firing, decreases in input resistance and increases in spike threshold (Sachdev et al., 2004) have the opposite effect, decreasing spike probability; thus, the issue remains unresolved. Sensory responsiveness of single cells has also been studied in the activated state that results from brainstem electrical stimulation (Timofeev et al., 1996; Castro-Alamancos, 2004; Aguilar and Castro-Alamancos, 2005). In this activated state, cortical cells are even more depolarized than during the up-state of the slow oscillation, and the decrease in input resistance caused by the barrage of synaptic input may be partially counterbalanced by an increase in input resistance resulting from the closing of potassium channels by muscarinic action of acetylcholine (Curro Dossi et al., 1991; Steriade et al., 1991; McCormick, 1992).

We observed that some down-state sensory responses appeared to trigger an up-state transition and demonstrated that this was in fact the case. For this analysis we assumed no causal link between sensory stimuli and state transitions, and simply quantified various interval lengths. Because of the variability across preparations, it was necessary to use normalized interval lengths (**Figure 2D**). We observed that, in a given preparation, considering all LFPs aligned on up-to-down transitions, as time moves forward the first events encountered are stimuli which are not associated with a down-to-up transition; these are followed later in time by stimuli which are associated with a down-to-up transition; and finally these are followed in time by spontaneous down-to-up transitions. In other words, with reference to the examples depicted in **Figure 2C**, the "green" intervals in a given preparation were consistently shorter than the "blue" intervals, which in turn were consistently shorter than the "red" intervals. Our interpretation of this is as follows: during a down-state, an

underlying time-dependent process proceeds which will eventually result in the next up-state. When this process has reached a certain point, it can be accelerated by a sensory stimulus. When the process reaches a certain further point, even in the absence of a stimulus, the up-state will occur spontaneously. Such a process has been shown in detail in slices of entorhinal cortex and further confirmed by a large scale network model (Cunningham et al., 2006).

### **STATE DEPENDENCE: DEPOLARIZING RESPONSES**

The responses during down-states were larger in amplitude and spatial extent than those during up-states. However, the responses during up-states were not limited to the activated barrel and spread over the majority of the barrel field. This result is in contrast with a previous VSD study *in vivo* comparing whisker responses during up- and down-states under anesthesia, in which responses during up-states did not spread and remained confined within the principal barrel (Petersen et al., 2003b). The large area of the whisker-evoked depolarization during up-states that we report here was nevertheless significantly smaller than during down-states. More importantly, the depolarizing responses during up-states were much sparser than during down-states, i.e., fewer pixels responded within each barrel column. Because the density of active pixels was higher in the principal barrel, it is likely that the use of strong spatial filtering may create an image of responses confined to the principal barrel. We also note the possibility that our single-trial responses may have been tightly focused in variable locations, resulting in an apparent sparse and distributed response in the multitrial average. The results obtained with other methods in this system including intracellular recordings (Moore and Nelson, 1998), and current source density probes (Roy et al., 2011) would seem to contradict this possibility, however.

An important methodological difference between our results and the study of Petersen et al. (2003b) is our separation of responses during down-states according to whether an up-state was triggered or not. Our data show that triggered up-states will slowly and consistently propagate throughout the barrel cortex creating a long-lasting response (see **Figure 4**) with sustained neuronal discharge (see **Figure 3**). In contrast, responses during down-states (that do not trigger an up-state), although similarly widespread in space, are more transient in time. This is an important distinction because the transitions between upand down-states are all-or-none and involve recurrent activity that is independent of the response to the whisker deflection. Thus, the most informative basis for comparison with the responses during the up-state is provided by the subset of down-state responses which are not associated with an up-state transition (see **Figure 4** for a comparison of the three types of responses).

A recent VSD imaging study in the mouse *in vivo* (Ferezou et al., 2006) extended our understanding of the state-dependence of sensory responses by including recordings during states of quiet waking and active exploration. Using a combination of VSD imaging and whole cell recording in mouse *in vivo*, the study of Ferezou et al. (2006) showed that, surprisingly, responses to a passive whisker deflection were longer-lasting and more spatially widespread during quiet waking than during isoflurane anesthesia (∼90 ms vs ∼40 ms on average). Furthermore, the same widespread pattern of activation was observed during active exploration only when the whisker deflection was caused by self-generated movement. In contrast, small passive deflections during active exploration triggered spatially restricted responses. In the 2006 study, Ferezou et al. did not distinguish up- and down-states during anesthesia; however, we note that isoflurane is less often associated with clear slow oscillations. Furthermore, the most important finding of that study was the widespread depolarization observed in whisker responses during active exploration. This corresponds well with the widespread distribution which we observed during up-states and strongly suggests that up-states bear some similarity to the active waking state.

In agreement with the Ferezou et al. (2006) study discussed above, extracellular studies in chronically implanted rats performing a discrimination task (Krupa et al., 2004) showed that the spike output due to active whisking during discrimination is prolonged (over 200 ms in all cortical layers) compared with the few millisecond brief discharges caused by a passive whisker deflection presented during the task when the rat is not moving its whiskers. However, in contrast with VSD results (Petersen et al., 2003a,b; Ferezou et al., 2006; our results) brief responses to passive whisker stimulation were similar to those obtained during anesthesia (Krupa et al., 2004). Discounting possible species differences (rat vs. mouse), these apparent discrepancies could be simply due to the fact that the VSD signal reflects subthreshold depolarization and does not provide information about which areas of cortex are actually spiking and for how long. Therefore, the studies of single neurons, together with the VSD results discussed above and our own results shown here and elsewhere (Civillico and Contreras, 2005, 2006), demonstrate that, regardless of brain state, sensory-driven spiking is superimposed on a larger basin of subthreshold depolarization. Studies using combinations of extracellular spike recordings over large areas of barrel cortex are necessary to fully understand the dynamic spatiotemporal transformation between synaptic inputs and intrinsic dendritic depolarization on one hand and spike output on the other.

An earlier study of state dependence of visual responses using VSDs in the anesthetized cat primary visual cortex (Arieli et al., 1996) found that most of the trial-to-trial variability was due to the initial state, which was defined as a single 3.5 ms frame of the optical recording, and that a "template" average response added to the initial state provided a good prediction of the whole response in each trial. Therefore, that study showed that if the sensory response is small enough the spatiotemporal dynamics of the brain is entirely dominated by the large fluctuations caused by anesthesia.

# **STATE DEPENDENCE: HYPERPOLARIZING RESPONSES**

There is a remarkable correspondence between the barrelcolumn-averaged VSD signals in the up- and down-states described here, and the averaged intracellular PSPs recorded at corresponding resting potentials in the supragranular and granular layers of rat barrel cortex (Wilent and Contreras, 2004, their **Figures 5** and **8**). This is expected given that the resting potential of all cortical cells is depolarized by 10–20 mV in the up-state (Steriade et al., 1993a). Derdikman et al. (2003) reported a hyperpolarizing response following a depolarizing response to whisker deflection and attributed this observation to the use of the blue dye RH1692. Contrary to their interpretation, we associate the same observation with the depolarization of cortical cells in the up-state. The shift to more depolarized *Vm*s decreases the driving force for excitatory inputs and increases the driving force for inhibitory inputs. Thus, we suggest that the presence of the hyperpolarizing "tail" in the VSD signal is dependent on the depolarization of cortical cells, and therefore on the *Vm* during the up- or down-state (or the general level of anesthesia in a less bistable state). Sachdev et al. (2004) demonstrated these changes in driving force using intracellular recordings, and we have characterized the population spatiotemporal effect on the supragranular network in the present work. Furthermore, we did not observe that the presence of hyperpolarization was associated with the nature of the stimulus, since it was observed in response to both piezoelectric whisker deflections, as well as deflections of single whiskers with directed air puffs. Thus visualization of hyperpolarization did and does not require the use of an RH169x dye, but simply that the preparation be in the right state, as determined during the experiment by concurrent LFP recordings.

An open question of great importance and related to the above discussion is whether such strong and widespread hyperpolarization occurs in the active whisking animal since the discharges of single neurons in all layers are very prolonged (*>*200 ms, Krupa et al., 2004) and the available VSD studies during active whisking did not report hyperpolarizing responses (Ferezou et al., 2006).

# **DISTRIBUTED REPRESENTATION OF SENSORY RESPONSES**

Responses during up-states had a smaller depolarizing footprint despite the fact that neurons were presumably closer to spiking threshold. If the spread of activation depends only on spike probability, defined as proximity to spike threshold, it is reasonable to predict a much more widespread activation during the up-state. Since the issue of neuronal responsiveness during up-states is not resolved (see above), we suggest that the large inhibition that accompanies responses in the up-state plays an important role in shaping the spatiotemporal properties of whisker responses.

Our data showing a large spread of responses during up-states agrees with the widespread activation recorded with VSDs in rodent barrel cortex under isoflurane anesthesia, during quiet waking and during active exploration (Ferezou et al., 2006, 2007). The observation that, even during active tactile exploration, sensory responses initiate in the corresponding principal barrel and then spread over large portions of the barrel cortex (Ferezou et al., 2006) is key to the concept that the information carried by whisker deflections is highly distributed and represented by complex spatiotemporal patterns of activation. How these distributed spatiotemporal patterns in cortical layers 2/3 are decoded into meaningful images of the world is a fundamental open question in neuroscience.

The first mapping of sensory responses in mammalian neocortex using VSDs was done in rat somatosensory and visual cortex and showed widespread activation throughout the cortex in response to small stimuli (Orbach et al., 1985). Noticeably, in that study, responses to two distant whiskers were clearly segregated on the surface of the barrel cortex as recorded by the photodiode array, but responses in visual cortex spread over the entire imaged area. Large spread of activation by small stimuli was subsequently shown using VSDs in the monkey primary visual cortex (Grinvald et al., 1994). In the primary visual cortex of the cat, waves of supra and subthreshold activity are evoked by local visual stimuli (Bringuier et al., 1999; Jancke et al., 2004; Benucci et al., 2007) and full screen drifting gratings generate widespread responses that contain information about both specific and nonspecific properties of the stimulus (Sharon and Grinvald, 2002). A recent VSD study in cat V1 (Sharon et al., 2007) showed that the response to small drifting gratings of varying orientation consisted of a large activated area, the cortical response field, characterized by an orientation-independent plateau of activity on top of which one to three orientation-dependent peaks of activity emerged. Importantly, spiking activity was only observed at the peaks of activation. Furthermore, such results have been extended to the orientation domain (Chavane et al., 2011). Thus, in the visual system, as in barrel cortex, sparse and distributed firing is superimposed on a widespread basin of depolarization. Furthermore, as in the present study, a large area of cortex is "biased" in its integrative properties due to the presence of this large subthreshold plateau of depolarization.

In visual cortex, imaging studies with intrinsic signals (Grinvald et al., 1986) or VSDs (Bonhoeffer and Grinvald, 1991; Bonhoeffer et al., 1995; Maldonado et al., 1997) revealed several superimposed maps including retinotopy, orientation, and spatial frequency selectivity. In contrast, imaging studies in barrel cortex have only focused on the spatial location of the stimulus; stimulus properties such as direction or frequency of whisker deflection have not been systematically studied. However, extracellular studies in rat barrel cortex have provided some evidence for the existence of such maps, although the spatial resolution of the resulting maps is a function of the number of cells in the study and is inherently low even with a large number of recordings. One study suggested the presence of direction-selective vertical domains within each barrel akin to orientation columns in visual cortex (Andermann and Moore, 2006), while another study provided evidence of a putative map of frequency preference with isofrequency columns extending along the representation of arcs (Andermann et al., 2004). In the auditory system, the very existence of tonotopic maps is a matter of debate (reviewed in Schreiner and Winer, 2007).

The distributed nature of responses to sensory stimuli complements the observed broad tuning of individual neurons to stimulus properties such as location. Together, these two classes of observations imply that large numbers of neurons respond to many stimulus parameters. However, response properties vary with cell type and laminar location, two parameters not accessible to VSD imaging. Therefore, generating models of information encoding by distributed activity which satisfy both optical and single-cell electrophysiological data is a phenomenally complex task.

The large spread of activation is in agreement with the large receptive fields (RFs) of single cells in L2/3 of barrel cortex (Moore and Nelson, 1998; Zhu and Connors, 1999; Kwegyir-Afful et al., 2005; Roy et al., 2011). However, while the widespread propagation of activation in L2/3 is attributed to the long horizontal connections in L2/3 (Laaris et al., 2000; Laaris and Keller, 2002; Petersen et al., 2003a), the large RFs in barrel cortex are in large part due to subthalamic mechanisms (Timofeeva et al., 2004). Thus, lesions of the spinal trigeminal nucleus (pars intermedia) render neurons in the principal nucleus and their targets in VPm monowhisker-responsive. How the RFs of individual cells relate to the spread of subthreshold activation demonstrated with VSDs is unknown and will require extremely difficult experimental studies, if not major advances in experimental technologies and analytical methods.

## **POOLING ALGORITHMS**

A key aspect of our study is that the spatial analysis is based on columnar counts rather than single pixels. We binned pixels over the outline of columns based on L4 cytochrome oxidase staining. Given the sparse yet widespread nature of the response during the up-state, we reasoned that a more accurate description of the response was to count the columns with activity above a significance threshold. In doing so, we emphasize the information carried by individual columns rather than individual cells. Neurons with similar selectivity are generally organized in vertical columns spanning the cortical depth (Rose and Mountcastle, 1954; Mountcastle, 1957, 1997). One appealing solution to the problem of decoding whisker position is the use of pooling of neuronal responses according to spatial and temporal rules (Darian-Smith et al., 1973; Zohary et al., 1994). Using an information-theoretic formalism to analyze single whisker responses recorded from many neurons simultaneously, Panzeri et al. (2003) concluded that pooling can be an effective strategy to decode stimulus location when restricted to individual columns. Pooling conserves all the information conveyed by an intracolumnar neuronal population (Panzeri et al., 2003) with very little redundancy due to the very low firing rates (Petersen et al., 2002) and allows for summation of activity at the postsynaptic target. In this schema, the most relevant attribute of the firing neuron is its columnar location. The present study is the first to provide a quantification of whisker responses based on column counts and demonstrate the largely distributed nature of responses both in up and down-states.

#### **ASYMMETRIES**

Excitation in barrel cortex has been shown to propagate preferentially along the rows for both spontaneous and sensory evoked activity, and was observed to be similar during anesthesia and during quiet waking (Petersen et al., 2003a). Here we report an additional asymmetry in the distribution of depolarization present only during the up-state, consisting of a preferential propagation along the rows toward higher numbered arcs. The asymmetry in whisker-evoked response propagation has been attributed to a bias in the large axonal arborizations of L2/3 neurons along the row axis (Petersen et al., 2003b). Such long range intrinsic connections characterize L2/3 in all sensory cortices, including visual (Fisken et al., 1975; Gilbert and Wiesel, 1979; Rockland and Lund, 1982; Martin and Whitteridge, 1984), auditory (Read et al., 2001; Atencio and Schreiner, 2010) and somatosensory (Keller and Carlson, 1999; Lubke and Feldmeyer, 2007). Biases in horizontal connectivity have been shown in visual cortex where they support feature-specific connectivity between distant neurons (Ts'o et al., 1986; Ts'o and Gilbert, 1988). Interestingly, in similarity with our results, in the visual cortex there are also clear asymmetries in the spread of activation caused by a small stimulus. For example, in cat visual cortex the spread of activation is approximately two-fold larger along the axis orthogonal to the ocular dominance columns (Grinvald et al.,

# **REFERENCES**


1994), though it is important to note that this effect may originate from anisotropic retinotopy (difference in horizontal and vertical scales), in which case it is not a property of cortical processing per se. The idea that increased inhibition in the up-state could sculpt cortical response properties finds further support in a study (Wörgötter et al., 1998) which found sharper RFs in cat V1 during desynchronized EEG epochs, analogous to the up-state of the slow oscillation.

Because the resting level of cells in the up-state is likely to be closer to that in the awake animal, we suggest that the polarities in signal propagation which we observe in the up-state could play a computational role in the behaving animal. We propose that such consistent asymmetry could participate in the processing of whisker-evoked activity by introducing a bias toward the smaller whiskers at the front of the snout. Our results demonstrate the critical importance of ongoing network activity on the dynamics of sensory responses and their integration. Finally, similarly to the asymmetric suppression shown in our results, in cat visual cortex VSD imaging has shown that suppression by surround stimuli is stronger for stimuli with matched orientation than for those with orthogonal orientation (Grinvald et al., 1994). The functional significance of such asymmetries for higher-level processing remains largely unknown.

# **ACKNOWLEDGMENTS**

The authors are grateful to Esther Garcia de Yebenes for histology, and to members of the Contreras lab for helpful discussions. This work was funded by NIH Conte Center grant P50-MH-064045 and R01 EY020765, and by The Human Frontier Science Program Organization and the David and Lucille Packard Foundation.

pinwheel-like patterns. *Nature* 353, 429–431.


cortical synaptic networks during information processing states. *Prog Neurobiol.* 74, 213–247.


methods for monitoring neuron activity. *Annu. Rev. Neurosci.* 1, 171–182.


revealed by optical imaging. *J. Neurosci.* 20, 1529–1537.


columns of rat somatosensory barrel cortex revealed with voltagesensitive dye imaging. *J. Neurosci.* 21, 8435–8446.


processes. *Annu. Rev. Physiol.* 51, 507–526.


Silva (Philadelphia, PA: Lippincott Williams and Wilkins), 17–32.


primate striate cortex. *J. Neurosci.* 8, 1712–1727.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 27 November 2011; accepted: 26 March 2012; published online: 13 April 2012.*

*Citation: Civillico EF and Contreras D (2012) Spatiotemporal properties of sensory responses in vivo are strongly dependent on network context. Front. Syst. Neurosci. 6:25. doi: 10.3389/fnsys. 2012.00025*

*Copyright © 2012 Civillico and Contreras. This is an open-access article distributed under the terms of the Creative Commons Attribution Non Commercial License, which permits noncommercial use, distribution, and reproduction in other forums, provided the original authors and source are credited.*

# Cortico-cortical communication dynamics

# **Per E. Roland<sup>1</sup>\*, Claus C. Hilgetag2,3 and Gustavo Deco<sup>4</sup>**

<sup>1</sup> Department of Neuroscience and Pharmacology, Faculty of Health Sciences, University of Copenhagen, Copenhagen, Denmark

<sup>2</sup> Department of Computational Neuroscience, University Medical Center Hamburg-Eppendorf, Hamburg, Germany

<sup>3</sup> Department of Health Sciences, Boston University, Boston, MA, USA

<sup>4</sup> Department of Technology, University of Pompeu Fabra, Barcelona, Spain

#### **Edited by:**

Maria V. Sanchez-Vives, ICREA-IDIBAPS, Spain

#### **Reviewed by:**

Robert Turner, Max Planck Institute for Human Cognitive and Brain Sciences, Germany Lionel G. Nowak, Université Toulouse - CNRS, France Miguel Maravall, Universidad Miguel Hernández, Spain

#### **\*Correspondence:**

Per E. Roland, Department of Neuroscience and Pharmacology, Faculty of Health Sciences, University of Copenhagen, Blegdamsvej 3B, DK 2200 Copenhagen N, Denmark e-mail: mgt875@ku.dk

In principle, cortico-cortical communication dynamics is simple: neurons in one cortical area communicate by sending action potentials that release glutamate and excite their target neurons in other cortical areas. In practice, knowledge about cortico-cortical communication dynamics is minute. One reason is that no current technique can capture the fast spatio-temporal cortico-cortical evolution of action potential transmission and membrane conductances with sufficient spatial resolution. A combination of optogenetics and monosynaptic tracing with virus can reveal the spatio-temporal cortico-cortical dynamics of specific neurons and their targets, but does not reveal how the dynamics evolves under natural conditions. Spontaneous ongoing action potentials also spread across cortical areas and are difficult to separate from structured evoked and intrinsic brain activity such as thinking. At a certain state of evolution, the dynamics may engage larger populations of neurons to drive the brain to decisions, percepts and behaviors. For example, successfully evolving dynamics to sensory transients can appear at the mesoscopic scale revealing how the transient is perceived. As a consequence of these methodological and conceptual difficulties, studies in this field comprise a wide range of computational models, large-scale measurements (e.g., by MEG, EEG), and a combination of invasive measurements in animal experiments. Further obstacles and challenges of studying cortico-cortical communication dynamics are outlined in this critical review.

**Keywords: spontaneous activity, synaptic transmission, membrane potential dynamics, spiking dynamics, cortical areas**

# **BACKGROUND AND SCOPE**

When one speaks of cortico-cortical connections, one usually means that axons start in one cortical area and end in another cortical area. These cortico-cortical axons are excitatory, releasing glutamate at their terminals (Ottersen and Storm-Mathisen, 1986). Neurons communicate by sending an action potential or a sequence of action potentials, *r*(*t*), down their axons. By *corticocortical communication*, we mean that the *r*(*t*) travels from one neuron in one cortical area, area A, to target neurons in another cortical area, area B. Area A often has different sets of output neurons, such that set 1 send the *r*(*t*)s produced to area B, set 2 to area C and so on. In this way the *r*(*t*)s produced by the neurons in area A are communicated to target neurons in several other areas (Felleman and van Essen, 1991; Scannell and Young, 1993). Each area has a unique pattern of connections (Passingham et al., 2002). The word communicate does not imply that the neurons in one area send coded messages to their target neurons; it simply means that the neurons send action potentials to the pre-synaptic terminals on the target neurons. As cortico-cortical neurons are excitatory, the glutamate release increases the currents flowing through the membranes of the target neurons, d*V*m(*t*)/d*t*, such that this term becomes net-positive, no matter whether the target neurons are excitatory or inhibitory. Thus

$$r(t)\_{\text{area}\,\text{A}} \Rightarrow \text{d}V\_{\text{m}}(t)\_{\text{area}\,\text{B}}/\text{d}t \uparrow\tag{1}$$

in which *V*m(*t*) is the membrane potential. Note that for each presynaptic site, the cortico-cortical communication is transmission over one synaptic cleft only. The increased excitation of the target neurons may or may not lead to action potentials in area B. The point is that the communication of excitation to target neurons in area B, as a minimum, changes the membrane dynamics of the target neurons in area B, which may influence the further spiking in area B.

As neurons in one area communicate *r*(*t*)s to several areas, one could imagine that d*V*m(*t*)/d*t* would increase in several cortical areas when the *r*(*t*)s are transmitted. Moreover, as some neurons in the target areas may fire *r*(*t*)s as a consequence of the communication, these neurons might excite other neurons within the target area, of which some might communicate to another set of target areas. This should evoke d*V*m(*t*)/d*t* increases in yet other areas. By *cortico-cortical communication dynamics* we mean the spatial and temporal evolution of *r*(*t*)s and d*V*m(*t*)/d*t* between neurons in different cortical areas. If we could measure how such cortico-cortical communications evolve, then we may understand the mechanisms that ultimately drive the cerebral cortex and the brain to particular percepts and behaviors. Thus we would have captured essential traits of how the brain works in a relevant time scale and relevant spatial scale.

Despite the theoretical simplicity, experimental studies of cortico-cortical communication dynamics meet many and complicated obstacles. First, as the relevant time scale is 0.5 ms or less, many methods based on slower brain signals are automatically excluded, for example blood oxygen level detection (BOLD) responses, intrinsic optical signals, regional cerebral blood flow and metabolism and other methods based on vascular kinetics and extracellular diffusion over larger distances positron emission tomography (PET). Second, the relevant spatial scale ranges from single dendrites to the whole cortex. Current methods with sufficient time resolution are in practical use limited to certain spatial scales. At a small spatial scale, voltage sensitive dye recordings can capture events at the single dendrite and single neuron scale (Canepari et al., 2010; Fisher and Salzberg, 2010). At a large spatial scale, magnetoencephalography (MEG) captures events over the whole cortex, albeit with some limitations. It is a major theoretical and practical challenge to combine these methods. Furthermore, *in vivo*, both methods are largely insensitive to action potentials (Hämäläinen et al., 1993; Petersen et al., 2003a; Grinvald and Hildseheim, 2004). Action potentials can be captured easily with electrodes near the axon hillock; but so far there is no method by which on can capture all action potentials in the brain. Ca2<sup>+</sup> sensitive dyes can be used to localize neurons that *had* communicated action potentials, but current dyes are too slow to capture the time when each potential is created (Grienberger and Konnerth, 2012). In a prominent recent proposal, neuroscientists describe new (nano-) technologies that may allow capturing every single action potential in the cortex of the mouse within the next 15 years (Alivisatos et al., 2012).

A test of cortico-cortical transmission of *r*(*t*) from one cortical area to another requires two electrodes. One electrode, very close to or into the transmitting neuron, recording the action potentials transmitted and one electrode into one of the target neurons in the receiving area to record the increase in d*V*m(*t*)/d*t* and eventual subsequent action potentials. This monosynaptic transmission then in most cases should take a few ms until the d*V*m(*t*)/d*t* increases. One problem with this strategy is that the transmitting axon most likely makes synapses on the dendrites of the target neurons. Depending on where on the dendrites the transmitter opens the ion channels, it may take up to 5–6 ms until the dendritic d*V*m(*t*)/d*t* increase is detected at the soma where the electrode is sampling. This is because the dendritic conduction velocities are around 0.1 mm ms−<sup>1</sup> (Nicoll et al., 1993; Stuart and Spruston, 1998). Actually there might not even be a detectable d*V*m(*t*)/d*t* increase, as this could be shunted out by prevailing or concomitant inhibitory conductances and conductances provided by the many other (in the order of 1000 or more) neurons that make synapses on the target neuron. Now, the chance of putting a patch electrode into precisely one of the dendrites that receive the glutamate from the transmitting neuron is very small indeed. One may object that subthreshold excitation of dendrites does not matter anyway, only if the target neurons spike they can change the dynamics. This does not seem to be the case, as sub-threshold d*V*m(*t*)/d*t* increases very well may influence the subsequent dynamics of a neuron population both in single cortical neurons and at the mesoscopic neuron network scale. Indeed such d*V*m(*t*)/d*t* increases can be induced by neurons in other cortical areas (Roland et al., 2006; Ahmed et al., 2008; Harvey et al., 2009; Niell and Stryker, 2010; Roland, 2010; Harvey and Roland, 2013; Zagha et al., 2013).

Electrical stimulation and later, cortical micro-stimulation has been used widely to examine cortico-cortical communications. However, unless the micro-stimulation is done intracellularly, a small population of neurons is usually excited. Furthermore, even moderate stimulation currents evoke inhibition in the target area, most likely from engaging inhibitory neurons in the target area (Kara et al., 2002; Logothetis et al., 2010). It is possible to detect monosynaptic transmission between two areas by antidromic electrical stimulation of axons, for example those axons running from the primary visual area 17 to area middle temporal lobe visual area (MT)/V5, for which the time of transmission is 2 ms (Movshon and Newsome, 1996). This is an elegant technique, in which the synaptic transmission is checked by colliding the antidromic action potential with a sensory evoked orthodromic action potential, giving undoubtedly valuable results. However, even this method does not give any further information on the evolving dynamics associated with *natural* use of this communication. Similarly, although there now are powerful methods to localize the group of neurons that connect monosynaptically to a neuron of interest (Wickersham et al., 2007; Wall et al., 2010), the mere proof of the monosynaptic connection cannot predict how the d*V*m(*t*)/d*t* and inter-area spike dynamics will evolve under natural circumstances. Furthermore, although it is possible to stimulate neurons electrically by micro-stimulation, and although it is possible to stimulate genetically modified neurons by laser beam pulses, it is the naturally evolving *r*(*t*) and membrane potential spatio-temporal dynamics that is in the focus when scientists want to understand how the cerebral cortex creates perception and behavior (Lim et al., 2012). Identification of target neurons, measurements of conduction velocities and other reductionist approaches still might be very helpful in constraining the interpretation of cortico-cortical dynamics under natural conditions.

The study of cortico-cortical communication dynamics would be so much easier if only a certain spatial scale mattered. As one could imagine, the dynamics must at a certain state of its evolution engage larger populations of cortical neurons, as only larger populations may drive the brain to a certain percept or towards a certain behavior. Consequently, all dynamics of the *r*(*t*) and d*V*m(*t*)/d*t* that matters may occur at the (mesoscopic) scale of neuron populations. Unfortunately, the *r*(*t*) of a single neuron may change also the *r*(*t*) and d*V*m(*t*)/d*t* dynamics of larger neuron populations. Consequently, it seems that one must keep track of every neuron to understand the evolution of cortico-cortical communication dynamics. This seems so in both experiments and reasonable realistic models of the brain (Houweling and Brecht, 2008; Izhikevich and Edelman, 2008; London et al., 2010).

# **SPONTANEOUS AND INTRINSIC COMMUNICATION DYNAMICS—EXPERIMENTS AND COMPUTATIONAL MODELING**

Neurons sending action potentials to another cortical area increase the d*V*m(*t*)/d*t* of the target neurons, no matter what caused the action potentials in the first place (Roland, 2010). For example, in the sleeping and anesthetized brain, an up-state in one area may spread to other areas (**Figure 1**). Up-states typically lasts 1 s or more, during which period the *V*m(*t*) is around −50 mV and accompanied by an increased number of action potentials (Steriade et al., 1993; Paré et al., 1998; Destexhe et al., 1999; Petersen et al., 2003b). In the anesthetized and the awake brain, many action potentials are not related to external sensory events (Destexhe, 2011). Traditionally, this is called "spontaneous ongoing activity", as the sources of this activity are not known. This intrinsic activity is also communicated between cortical areas (Arieli et al., 1995; Lippert et al., 2007; Xu et al., 2007).

To get a full understanding on how d*V*m(*t*)/d*t* and spiking dynamics evolve among the cortical areas, one must know the sources and the targets. In principle, this may be possible in studies of anesthetized brains, where it is sometimes possible to capture the population of neurons bifurcating into an up-state (**Figure 1**). Then one can follow how the up-state spreads to populations of neurons in adjacent areas (Lilly, 1954; Lippert et al., 2007; Xu et al., 2007; Huang et al., 2010; Gao et al., 2012; Zheng and Yao, 2012). In contrast to sensory evoked activity, the spontaneous up-states may spread from different origins in the cortex. The spread out from the initiation site is often in the form of wave fronts of net-excitation traveling over the cortex (0.001–0.2 mm ms−<sup>1</sup> ), sometimes the waves have spiral character (Huang et al., 2010). The wide range in the velocity of propagation indicates that the mechanisms behind the spread can be monosynaptic at times (**Figure 1**) and polysynaptic at other times, or combinations of mono- and polysynaptic progressions.

In the awake state, not surprisingly, there may be spontaneous cortico-cortical communications of *r*(*t*)s in sensory cortical areas and in motor areas although the animal remains relaxed, immobile, and does not receive any external sensory stimuli (Ferezou et al., 2006, 2007; Han et al., 2008; Zagha et al., 2013). Surprisingly even in the awake state, d*V*m(*t*)/d*t* increases may also move as wave fronts from sensory to motor areas or vice versa, or between visual areas similarly to the spontaneous up-states. Again the velocity of this cortical propagation is highly variable. The direction of propagation in some cases however mimicks that of sensory evoked d*V*m(*t*)/d*t* increases or motor associated (whisking) d*V*m(*t*)/d*t* increases (Ferezou et al., 2006, 2007). All examples of spontaneous propagating d*V*m(*t*)/d*t* increases between cortical areas were captured by simultaneous measurements of changes in the membrane potentials of populations of neurons in the upper layers of cortex with voltage sensitive dyes. The dye signal change has a near linear relationship to the change in membrane voltage, recorded intra-cellularly *in vivo* from cells in superficial cortical layers (Petersen et al., 2003a; Ferezou et al., 2006; Berger et al., 2007). Furthermore, according to a recent estimate, approximately 90–95% of the dye signal reflects changes in synaptic activity (Berger et al., 2007). Given these premises, Eriksson et al. (2008) showed that significant increases in the temporal derivative of the dye signal *in vivo*, d*V*SD(*t*)/d*t*, can be interpreted as net excitation of the stained membranes and significant deceases as net inhibitions. This means that the investigators in these studies most likely observed the spatio-temporal dynamics of net-excitations of membranes in the upper layers of cortex traveling between cortical areas. The net-excitations could be indirect indications that *r*(*t*)s from one area were communicated to the target area(s). However, the sources of these communications are not known, as the dye signal recordings were not paired with simultaneous *r*(*t*) recordings. Even in the case where one directly observes that the neurons bifurcate into an upstate at a particular spot from where the depolarization spreads out, one must have laminar electrodes at the spot to find the source of increased spiking (which of course could be in the spot itself). Finding the spiking source of spontaneous activity that propagates between cortical areas may in practice involve an electrode density that is unrealistic. See also Chicharro and Ledberg (2012) for theoretical limitations of interpreting causal influences in studies of temporal dynamics of cortico-cortical communications.

Faced with the practical problems, the fact that the cortex has a rich and diverse spontaneous and intrinsic activity, and the microscopic likelihood of finding the sources of the *r*(*t*)s, neuroscientists have thought of ways in which the sources of the dynamics can be controlled. There are basically two strategies:

**FIGURE 1 | Upstate in areas SSy and 21 spreading to lower visual areas 18 and 17 in the ferret**. The voltage sensitive dye signal, reflecting the membrane potential at the mesoscopic scale, propagates at time 249.8–262 ms and again 317.9–340 ms from SSY to the border between areas 17 and 18 (from Roland, 2010, by permission).

computational models, and experimenter-controlled natural sensory perturbations of the cortex network.

# **CURRENT STATE IN THE COMPUTATIONAL MODELING OF NEURAL SIGNAL PROPAGATION**

Tremendous advances in IT hardware have made it possible to model neural networks of a scale approaching that in the real brain. Realistic computational models of the cortical neuron networks have the great advantage that all sources, synapses, and target neurons are known. Consequently the fundamental variables *r*(*t*) and d*V*m(*t*)/d*t* can be observed in any neuron and hence a detailed description of the evolving communication dynamics should be possible. With an estimated average convergence and divergence rate of cortical neurons in the order of 104– 10<sup>5</sup> inputs and outputs (Braitenberg and Schüz, 1998), realistic models even of small cortical patches require the inclusion of several 10,000s of neurons (Potjans and Diesmann, 2012). Hardware progress has allowed modeling of such large populations with some degree of realism in the local dynamics, that is, as biophysical or spiking neurons. For example, Izhikevich and Edelman (2008) modeled a population of 10<sup>6</sup> phenomenological spiking neurons and linked them in a multi-scale fashion by almost half a billion synapses, combining long-range connections estimated from diffusion imaging of the human brain at the large-scale with the "canonical" microcircuit from cat visual cortex (Binzegger et al., 2004) at the local scale. After adjustment by spike-time dependent plasticity, the network showed selfsustained activity in the absence of external inputs, which activity was organized into different dominant frequencies within different regions and layers. Moreover, the model exhibited propagating waves of excitation and simulated fMRI signals showing slow oscillations with multiple anticorrelated modules, similar to real data. More recently, Potjans and Diesmann (2012) presented a full-scale model of the canonical cortical microcircuit, comprising 80,000 spiking neurons and 0.3 billion synapses, which produced spontaneous asynchronous irregular activity and cell-type specific firing rates in agreement with *in vivo* recordings in awake animals. On a larger scale, the Human Brain Project (Markram, 2012) 1 is now under way and aims to build a model of the whole brain based on biophysical neurons, that is, including channels characteristics and other features at the molecular scale. While the promise of this enormous modeling effort is that multi-faceted dynamic phenomena may be found at multiple scales, a deeper understanding of such phenomena may also be hampered by the model complexity.

Alternatively, if the main goal of a neural network model is to understand the fundamental relationship between network topologic features and propagation of excitation, smaller models and more simplified assumptions about the local nodes may suffice. For instance, it can be shown with multi-scale models as well as simple excitable nodes (akin to cellular automata) that topological features of brain networks strongly shape brain dynamics. For instance, modular and hub features of biological neural networks induce a modular and target wave-like propagation of excitation, respectively (Zhou et al., 2006; Müller-Linow et al., 2008;

The question of how the topology of structural connections shapes cortical communication dynamics is addressed by several papers of the Special Research Topic "Cortico-cortical communication dynamics" (Roland et al., 2014). The references to these contributions are underlined. For instance, Miši´c et al. (2011) demonstrate through the analysis of functional connectivity derived from EEG data, that the variability of signals at different network nodes (as assessed by the measure of multiscale entropy) depends on the placement of the nodes within the network. In biological neural networks, which have a nonregular and non-random organization (Sporns et al., 2004), not all nodes are created equal. In particular, some nodes possess more connections, turning them into so-called hub nodes, which are also more central in the network topology. From the observations by Miši´c et al. (2011), it also turns out that more central hub nodes have higher signal variability. This finding complements previous experimental and modeling observations that hub nodes also have higher activity than other nodes, which in turns makes them more liable to injury (Buckner et al., 2009). Based on the analysis of MEG data in a visual, face recognition task, Vakorin et al. (2011) showed that the amount of information transferred from one node (i.e., a MEG source) to another was correlated with the difference in variability between the dynamics of these two sources. These results and similar outcomes from analyses of synthetic data suggest that both time delay and strength of coupling can contribute to the relations between variability of brain signals and information transfer between sources. Delay times as well as density and type of coupling were also found to be essential factors by Li and Zhou (2011) who used computational modeling, based on integrate and fire neurons or a neural mass model, to explore factors resulting in anti-phase oscillations between two network modules. The modeling also showed that interactions between slow and fast oscillations may provide a basis for anti-phase synchronization of slow oscillations at small delay times. This work deepens the understanding provided by previous computational models attempting to reproduce functional connectivity during spontaneous activity of the brain (e.g., Deco et al., 2009).

In humans, the neuroanatomical network structure is typically inferred from variants of diffusion tensor imaging and tractography techniques (see Jones et al., 2013 for a sober evaluation). The resulting anatomical matrix expresses the likelihood or density with which two different brain areas are connected through white matter fiber tracts. The second component of the models is the type of dynamics that is assumed for the local nodes. Some neurodynamical models considered a simple oscillatory dynamics (Ghosh et al., 2008; Deco et al., 2009; Cabral et al., 2011), others a more realistic spontaneous state dynamics (Honey et al., 2009), and finally, even very detailed and realistic local networks considering excitatory and inhibitory populations of spiking neurons coupled through realistic NMDA, AMPA and GABA synaptic dynamics, have also been formulated (Deco and Jirsa, 2012).

Further, van den Berg et al. (2012) studied the evolution of random networks of interacting nonlinear dynamical systems in which the coupling between the local dynamical nodes follows

Lohmann et al., 2010). "Nodes" in these models correspond to neural elements ranging in scope from individual cells to largescale populations (e.g., cortical areas).

<sup>1</sup>www.humanbrainproject.eu

a rule of adaptive rewiring. For a large enough number of connections, the system evolves towards a small-world network architecture similar to the one observed in healthy brains after development. Nevertheless, if the number of connections is not larger than a critical value, the system evolves towards a random network. They relate this failure with the fragmentation hypothesis underlying schizophrenia. This study is a beautiful example of how computational and theoretical analysis of dynamical systems serves to deepen our understanding on the relationship between function (activity), structure (anatomy) and development (rewiring). Kiebel and Friston (2011) investigated the reorganization and pruning of synaptic connections in a neuropil stimulated by spatiotemporal input sequences. They demonstrated that the reorganization underlies an optimal Bayesian principle, namely the minimization of free-energy. They were able to show that following this reorganization optimal principle, dendrites self-organize and replicate two key experimental findings (Branco et al., 2010) on directional selectivity and velocitydependent responses. Banerjee et al. (2012), review different measures characterizing functional and effective connectivity, in particular in MEG data. Furthermore, they propose and show how MEG measurements could be validated by combining the empirical data analysis with simulations of large-scale neurobiological realistic modeling.

# **ATTEMPTS TO FOLLOW SENSORY EVOKED CORTICO-CORTICAL COMMUNICATION DYNAMICS. DEPENDENCE ON THE STATE OF THE TARGET NEURONS**

In later years scientists have become increasingly aware that the spontaneous and intrinsic ongoing fluctuations in the membrane potentials and firing of action potentials have a profound effect on sensory evoked activity when it arrives to primary sensory areas (Destexhe, 2011). For example, it has been debated whether sensory evoked *r*(*t*) and d*V*m(*t*)/d*t* increases are favored by upstates or down states (Steriade et al., 1993; Contreras et al., 1996; Paré et al., 1998; Destexhe et al., 1999; Petersen et al., 2003b; Crochet and Petersen, 2006; Haider et al., 2006; Luczak et al., 2007). Up-states are associated with high inhibitory and excitatory conductances; whereas in down-states the conductances are smaller, but often coupled to a leak conductance (Contreras et al., 1996; Haider et al., 2006). Civillico and Contreras (2012) induced oscillation between a down-state and an up-state with ketaminexylazine. They then examined how the phases of the up-state and down-state affected the arrivals of *r*(*t*)s from thalamus and the membrane potentials in the barrel cortex. They found that the local field potentials, the membrane potential changes and the multi-unit activity in the barrel cortex increased less to a whisker stimulus applied during the up-state, as compared to whisker stimulus applied in the later part of the down-state (**Figure 2**). When the whisker stimulus was given when the membrane was maximally hyperpolarized or when the hyperpolarization diminished in the oscillatory cycle, the whisker stimulus almost invariably triggered an up-state during which the amplitude of the local field potential, the membrane potential and the multiunit activity was strong (**Figure 2**). Also the spreading of the depolarization to the whole barrel field was much stronger.

Many cortical areas send (multi-synaptic) communications via the entorhinal cortex to the hippocampus (van Hoesen et al., 1972). In awake animals, novel sounds evoke 50 ms latency, short lasting spike trains in hippocampus (Christian and Deadwyler, 1986). Overlearned sounds, if task relevant, may also modulate spiking in hippocampus, albeit often with long latencies 150– 300 ms (Itskov et al., 2012; Vinnik et al., 2012), However if the sounds irrelevant for a task, they modulate the spiking in only a few percent of hippocampal neurons also with long 150–300 ms latencies in the awake animal (Vinnik et al., 2012; **Figure 3**). Surprisingly, if the animals are asleep, 25% of the hippocampal neurons react with short 50–70 ms latencies and long lasting *r*(*t*) increases or decreases even to task irrelevant sounds (**Figure 3**). These results show that the access to hippocampal neurons is state and context dependent. The sounds did not arouse the EEG, suggesting that the effect, at least partly, may be cortico-cortical, although it is not clear whether the sleep stage had any influence on the accessibility.

Finally, Harvey and Roland (2013), explore experimentally, by using voltage-sensitive dyes, the propagation of activations in the ferret visual system in response to colliding visual stimuli, and how the propagation may be shaped by cortical connections, in particular their direction from primary visual cortices to higherorder cortical areas or in the opposite direction (**Figure 4**). Anatomical projections proceeding in these two directions have well known orderly characteristics of laminar projection origin and termination (Felleman and van Essen, 1991), but it still remains a challenge to understand the impact of these anatomical features on cortical communication dynamics (Bastos et al., 2012).

# **EVOLUTION OF SENSORY EVOKED CORTICO-CORTICAL COMMUNICATION DYNAMICS**

In a classical approach to follow the cortico-cortical communication dynamics scientists stimulated the sensory apparatus with a very brief stimulus and recorded action potentials or multiunit activity with laminar electrodes in one or more cortical areas. Typically such an effort result in an ON response, a fast increase in the number of action potentials over some 20 ms, in the primary sensory area. If the stimulus is sufficiently strong, ON responses will spread to many (higher order) sensory areas. In general, however, these studies failed to reveal any clear order of the start of the ON *r*(*t*)s in most cortical areas. For example in the visual areas there were no significant latency differences between the primary visual area neurons in layer 4 and the neurons in areas MT/V5, middle superior temporal visual area (MST) and the frontal eye fields (Best et al., 1986; Schmolesky et al., 1998; Schroeder et al., 1998; Bullier, 2001; Chen et al., 2007). One exception are the progression of ON *r*(*t*)s in V1,V2,V4 and inferior temporal cortex, where the mean ON *r*(*t*)s are separated by approximately 10 ms (Nowak and Bullier, 1997; Schmolesky et al., 1998; Schroeder et al., 1998; Chen et al., 2007). However, the number of potential sources altering the cortical dynamics are many already at the time, approximately 45 ms after the stimulus onset, when the majority of the ON *r*(*t*)s leave the primary visual cortex.

up-state, in the first half of a down-state, and in the last part of the

oscillations.

There have been a number of interesting studies in which paired electrode recordings were made in two or more areas that were known to connect anatomically, for example visual areas V1 and V2, V1 and MT/V5, and auditory areas A1, A2 (Movshon and Newsome, 1996; Nowak et al., 1999; Eggermont, 2000; Valentine and Eggermont, 2001). These studies give some insight in the development of temporal dynamics between the two areas, and may reveal likely sources (Movshon and Newsome, 1996). However, the spatial dynamics, and the simultaneous temporal dynamics of the neurons in all the other cortical areas cannot be revealed by this method.

After staining the cerebral cortex with voltage sensitive dyes one can in practice observe some spatial evolution of cortical dynamics of the *V*m(*t*) and d*V*m(*t*)/d*t*, at least in the upper layers of cortex. This seemingly contradicts the results of the action potential studies just described. Part of the explanation might be that the dye signal *in vivo* reflects synaptic activity at the mesoscopic scale, whereas the action potential recordings capture the activity of single neurons (Lippert et al., 2007; Eriksson et al., 2008). Nevertheless, in several studies one can follow how net increases in the synaptic activity propagate over the cortical areas when the cortex is perturbed by a sensory transient (Senseman, 1996; Prechtl et al., 1997; Senseman and Robbins, 2002; Slovin et al., 2002; Grinvald and Hildseheim, 2004; Roland et al., 2006; Ferezou et al., 2007; Lippert et al., 2007; Xu et al., 2007; Ahmed et al., 2008; Han et al., 2008; Takagaki et al., 2008; Yoshida et al., 2008; Harvey et al., 2009; Ayzenshtat et al., 2010; Meirovithz et al., 2010; Ng et al., 2010; Polack and Contreras, 2012; Harvey and Roland, 2013). This synaptic dynamics may show some order in the feed-forward propagation of net-excitation for example between V1 and V2 in monkeys, rats and turtles, between the barrel field and the motor cortex in the mouse, and between visual areas 17, 18 and 19, 21 in the ferret. Typically the higher order area(s) lag the primary areas with some 8–15 ms depending on species.

Some of these studies contain observations of a reverse order of synaptic propagation, that is, from higher areas towards the primary sensory areas, some 40–50 ms later, i.e., 80–100 ms after the stimulus onset (Roland et al., 2006; Lippert et al., 2007; Xu et al., 2007; Ahmed et al., 2008; Takagaki et al., 2008; Yoshida et al., 2008; Harvey et al., 2009; Ayzenshtat et al., 2010; Ng et al., 2010; Lim et al., 2012; see also Zheng and Yao, 2012; Harvey and Roland, 2013). This mode of propagation has been named feedback. The sources of these feedbacks are not known (but see Zagha et al., 2013). As the synaptic net excitation during feedback propagates fast (0.15–0.25 mm ms−<sup>1</sup> ) over the cortex, it was suggested that feedback axons from higher order areas made synaptic contact during their way back from the higher order area. This propagation velocity, though, is slower than that of 1–3 mm ms−<sup>1</sup> measured in primate axons running from V2 to V1 (Girad et al., 2001), suggesting that higher areas may influence lower areas with different mechanisms.

One major finding from the voltage dye studies was that the dynamics of the d*V*m(*t*)/d*t* evolved to engage whole sensory

**FIGURE 4 | Eight phases of dynamics of net-excitation, net-inhibition (obtained with voltage sensitive dyes) and multi-unit activity in the cerebral cortex of the ferret exposed to two bars in the field of view moving towards each other**. Mean temporal derivative of the population membrane potential (related to dV <sup>m</sup>(t)/dt) in cytoarchitecturally defined cortical areas 17, 18 , 19 and 21 of the ferret. Mean of three animals shown. **(1)** The two bars have not yet entered the part of cortex monitored by the photodiode camera, but the net-excitation especially in areas 19 and 21 has. **(2)** The mapping of the bars in areas 19/21 has entered the part of cortex monitored. The net-excitation ahead of the spiking neurons at the area 17/18 border from the two sides meet at the cortical zone mapping the center of field of view. **(3)** Feedback from areas 19/21 to areas 18 and 17 begin. **(4)** The bars are now separated by 15◦ in the field of view and the neurons start to spike at the edge of in the cortex monitored (gray dots). **(5)** The bars are separated by 7.5◦ and the neurons at zone mapping the center of field of view start to fire. The positions with more than 90% of the maximal firing rate (the mapping sites) are marked with white dots. **(6)** There is now only one cortical mapping site at the 17/18 border, corresponding to the fact that the bars now occlude one another. Net-inhibition now dominates the cortex at the former mapping sites. **(7)** The net-inhibition is maximal 70 ms after the occlusion in the field of view. **(8)** The net excitation recovers somewhat at the sites of the mapping (bars now drifting apart by 9◦ ), but the spiking remains reduced. The color scale ranges from −6.5 10−<sup>6</sup> to 6.5 10−<sup>6</sup> (for laminar propagation see Harvey and Roland, 2013).

cortical areas within 100 ms after the sensory stimulus. In the barrel field of mice and rats this happened 16–36 ms after the start of stimulation of single whiskers (Derdikman et al., 2003; Petersen et al., 2003a; Civillico and Contreras, 2006, 2012; Ferezou et al., 2006, 2007; Lippert et al., 2007). The whole primary auditory cortex was engaged in 26–40 ms after stimulus start in guinea pigs (Horikawa et al., 1998; Kubota et al., 2012). The whole craniotomy exposed part of the primary visual cortex in ferrets, cats, and monkeys became engaged 48–70 ms after stimulus start, even with small stimuli (Slovin et al., 2002; Jancke et al., 2004; Eriksson and Roland, 2006; Roland et al., 2006; Sharon et al., 2007; Eriksson et al., 2008; Harvey et al., 2009; Ayzenshtat et al., 2010; Meirovithz et al., 2010; Roland, 2010; Chavane et al., 2011; Reynaud et al., 2012; Harvey and Roland, 2013). In mice and rats it took some 70–110 ms for the dynamics to engage the whole primary visual cortex (Xu et al., 2007; Han et al., 2008; Gao et al., 2012; but Lim et al., 2012: 46 ms; Polack and Contreras, 2012). The engagement of the whole area lasted some 60–70 ms, i.e., up to 140 ms after the start of the stimulus, even after very short stimuli (Eriksson et al., 2008). This is the relevant time scale for perceiving changes in the sensory environment (Thorpe et al., 1996).

# **CONCLUDING REMARKS**

To measure the evolution of cortico-cortical communications, first one must identify the neurons that communicate their action potentials between cortical areas. Then one must measure how these neurons spread their action potentials to neurons in other cortical areas under natural conditions. Finally one must measure the effect of this communication in the target neurons, i.e., measure the d*V*m(*t*)/d*t*, because the temporal evolution of the d*V*m(*t*)/d*t* affects the future dynamics of the target neurons. The experience, from experiments and large-scale models of the brain (cerebral cortex), is that the measurements should be done in different scales, from the single neuron scale to the mesoscopic scale (larger populations of neurons), because spiking from a few neurons can spread through cortical layers and evoke spiking in many cortical areas. Moreover, sensory evoked spiking in cortical areas tends to increase d*V*m(*t*)/d*t* in a large part or a whole cortical area. This means that the task is to measure the *spatio*temporal dynamics, at least of the fundamental variables *r*(*t*) and d*V*m(*t*)/d*t* from the single neuron to the large population of neurons scale during natural conditions. As discussed, neuroscience so far does not have efficient methods to do this.

In the case of sensory evoked *r*(*t*) one has a chance to identify the neurons in the primary mammalian sensory area starting to send their action potentials to other areas. But what about the neurons starting a thought or starting planning an action? To get insight into this type of cortico-cortical communication dynamics one must monitor neurons in all layers and all cortical areas with sufficient spatial density. The available experimental results show that already 20–30 ms after the start of sensory evoked spiking in cortex 10000's of neurons may be spiking and perhaps two orders of neurons in addition will have changed their membrane potentials. Furthermore, a fair proportion of these spiking neurons will mutually affect each other across area borders. At this point of time, causal relations of spiking, i.e., which neuron drives which neurons, are not so clear. This problem of understanding the corticocortical communication dynamics at the single neuron scale while the communications evolve, experimental neuroscience shares with large-scale computational models of the cerebral cortex and models of whole brains. One, speculative, solution of this conundrum would be if the collective dynamics of the *r*(*t*) and d*V*m(*t*)/d*t* of larger populations after the initial evolution reduced the importance single neuron *r*(*t*) dynamics. Thus by observing larger scale spatio-temporal dynamics of these variables one might hope to observe spatio-temporal patterns giving hints of what the brain will perceive or do (Roland, 2010). Such speculations notwithstanding, advances in experimental methods are prerequisites for understanding cortico-cortical communication dynamics.

Science is not there yet.

## **ACKNOWLEDGMENTS**

Claus C. Hilgetag is supported by DFG grants HI 1286/5-1 and SFB 936/A1. Per E. Roland is supported by The Danish Ministry for Science and Innovations through FSS, and the University of Copenhagen 2016 funds through the Dynamical Systems Interdisciplinary Network.

# **REFERENCES**


Grinvald, A., and Hildseheim, R. (2004). VSDI: a new era in functional imaging of cortical dynamics. *Nat. Rev. Neurosci.* 5, 873–884. doi: 10.1038/nrn1536


**Conflict of Interest Statement**: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 23 September 2013; accepted: 25 January 2014; published online: 05 May 2014.*

*Citation: Roland PE, Hilgetag CC and Deco G (2014) Cortico-cortical communication dynamics. Front. Syst. Neurosci. 8:19. doi: 10.3389/fnsys.2014.00019*

*This article was submitted to the journal Frontiers in Systems Neuroscience.*

*Copyright © 2014 Roland, Hilgetag and Deco. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*